December 2, 2016
Is it possible that test cases are dead?
Are you shocked at the suggestion that test cases may be dead? Please hear me out. I am not calling for the abandonment of test cases. I am warning you that test cases are not alive by definition, and that’s a problem. By alive, I mean that they are connected to a test executor’s brain that is turned on, always learning and designing even as they execute, ready to re-engineer if needed.
Requirements, both stated and implied, must agree with the business context upon which they are built. From the requirements, QA interprets the requirements to build checklists of their own. All test cases can trace their origins back to these checklists. But if test case creation is spread across time and different people in different roles (requirements interpreter, test case designer, test case executer), the context may vary from the original intention, colored by guesswork, broken communication, and/or ignorance. What if QA and Development diverge in thought, and the developers’ new business logic or implementation changes never get back to QA? Or what if the test case executers are simply too far removed from the requirements interpreter, losing track of the original context?
History illustrates business disconnects well. The Charge of the Light Brigade has an attacking army massacred due to a target miscommunication. The Ariane 5 rocket was destroyed when a 64-bit number got translated to 16-bit without knowing how to handle the truncation, and Mariner 1 was destroyed because of a missed hyphen in the code. The “unsinkable” Titanic sank due to a series of thin cuts tore through too many compartments on the right side (affecting wrought-iron rivets that get brittle in freezing weather), and had too few lifeboats for the ship’s passengers. Faulty equipment caused the Space Shuttle Challenger to explode. The Chernobyl nuclear plant had a steam explosion, fire, and radiation leak after performing stress tests with multiple safeguards overridden. The Deepwater Horizon oil rig was lost and flooded the Gulf of Mexico with oil after a series of mistakes that a judge termed “reckless” and “negligent” (a $128,000 safety test would have taken under half a day and predicted the impending failure). Business demands success, not preventable, foreseeable failure.
These extreme historical examples show what can go wrong when the original context gets lost, work gets sloppy, and quality is ignored. Test cases are as dead as an outdated link if they become disconnected from the checklist and original business context. Test cases are ineffective if they fail to attack the right place the right way. Testers with divergent goals will use mistaken judgment and miss needed coverage. Imagine testing a query page where one of the SQL query’s elements is unknown to QA, absent from all tests. Or a textbox you thought was ASCII is defined as Unicode. Or a medical database that must have HIPAA-compliant audit trails even though the reqs may not list that. Or a typo in screen text reqs that no one questioned because it was in the reqs. Those who deploy must be connected to the original intent, to ensure that smart, proper, useful decisions are made before live execution finally occurs.
Our QualiTesters are valuable because they are always thinking, and always ready to ensure that the checklists and business context are not lost. QualiTesters are people who perpetually synergize their learning, designing, and executing. Because just as a regression set can go stale between different releases, a test case can accidentally go stale before it is ever executed. And accidents are avoidable.