5 years ago, I welcomed the new release of ISTQB CTFL new syllabus with an honest (if a bit hostile) reading. This may, ISTQB have released version 4.0 of the CTFL syllabus, and after hearing about it from a colleague, I thought to myself that it would only be fair to give it the same respect. After all, for reasons unbeknownst to me, there are some very good professionals who identify with this organization, so maybe they have managed to get something decent from under their hands.
There is one significant improvement in this version only 50 pages of non-skippable content (such as refrences, appendices, copyright and the like) compared to 76 in the previous one, which can explain how I managed to go over it in 1 hour and 55 minutes (this time, as I was reading in the train, I did not use pomodoro, but rather a time tracking software name "super productivity"). Amusingly, I ended up with a similar number of comments (59, compared with the 57 from last time), and my conclusions are still the same - it is a terrible piece of content, and I'm sure that each and every one of the people who were involved with it (let alone all of them together) could produce something better from under their hands.
as before, the underlying problems are the business model and the assumptions it forces on the document: Teaching something in less than 19 hours (which is still 2 hours and 10 minutes more than 2018), must either rely on previous knowledge or experience or be extremely shallow and inefficient. Some of the topics mentioned in the syllabus are multi-faceted and complex, grokking them takes time and practice. Narrowing it to a 40 multiple-choice questions for people without any prior knowledge? that's simply impossible to do.
Aiming to people with some experience (and adding this as requirement to the certification, in a similar manner to what ISC2 does with its CISSP certification) would enable much more focus to this document. As it is, though, the syllabus still tries to be an introduction to working in tech and a description of the heaviest test procedure fathomable (Heck, they even mention the notorious IEEE 29119). Mentioning that it's ok to tailor it to your needs? nah, why not pretend it's the golden standard? much more fun this way.
As this document is a major revamp of the previous, there are some new things in this version. The buzzword bingo is even more noticeable. With terms such as shift-left, devops, CI/CD sprinkled just about everywhere. Naturally, it's devoid of meaning.
Even worse is the homage to some other terms that are by now well established in the testing profession. Those would be the test-pyramid and the agile testing quadrants. both terms are oddly placed under the section of test planning. Why? There are more things in heaven and earth, Horatio.
Their description of the pyramid is probably the worst botch I've seen in trying to describe or use it. Its existence is mentioned, but we skip completely questions like "how to use it". Then comes this:
Tests in the bottom layer are small, isolated, fast, and check a small piece of functionality, so usually a lot of them are needed to achieve a reasonable coverage. The top layer represents complex, high-level, end-to-end tests. These high-level tests are generally slower than the tests from the lower layers, and they typically check a large piece of functionality, so usually just a few of them are needed to achieve a reasonable coverage.
while most of it is technically correct, have you noticed how they shift the whole issue to a comparison of "how many are needed to get full coverage" (one might also ask, coverage of what?) instead of mentioning that those are completing each other, or that this is an observation on what a healthy set of tests usually looks like, it's now hinting that you can get the same done with a few large tests as you can with a lot of smaller ones.
Treating the agile testing quadrants is even worse: instead of treating it as an observational, descriptive model that can expose some gaps in the way we think about tests or in what we actually implement, in the hands of the ISTQB authors, it has become a prescriptive model.
for instance:
Quadrant Q1 (technology facing, support the team). This quadrant contains component and
component integration tests. These tests should be automated and included in the CI process
Naturally, the ISTQB treatment of exploratory testing still represents an utter lack of understanding of it. It's still treated like an after-thought, not really planned testing, and it being placed under "experience based techniques" is probably the best sign of this misunderstanding - it's not a technique, it's a complete approach to testing, and it is not any more "experience based" than the approach where we try to write our test plans ahead of time. The main difference is that the exploratory testing approach admits that experienced people perform better. During exploration we employ all of the formal looking techniques that can be performed ahead of time, as well as other heuristics that are more readily available during test execution.
so, summing it up:
CTFL certification still has negative correlation to one's ability to function in a professional testing situation. The syllabus still promotes a lot of misconceptions about testing - that it should have some "level of independence" ("accelerate" has thrown this claim out of the window for most cases) , that finding bugs is a goal and that a slow and heavy process is the best way to go forward. It still focused on telling the "what" and very rarely the "how", the most important "why" is not present at all (The only place is when the question "why is testing necessary", and there the question should be "When is it needed, and when not "). It still does not help testers communicate with non-testers, and I would say that it even hinders such efforts by choosing to use terminology in a way different than what is common in the industry. It still focuses too much on the useless artefacts produced and not on the value we should aim to get when creating them and it still aimed to enable untrained, unskilled people hold a piece of worthless paper and present it to future employers.
Or, briefly - it's still a scam.
No comments:
Post a Comment