Do not be fooled, the introduction of model-based testing (MBT), more precisely model-driven or model-centric testing, can be a huge investment financially and technically. Not to mention the paradigm shift in test design. It is imperative that you start out with a thorough analysis of all the strengths, weaknesses, opportunities, and threats surrounding MBT in general and your project in particular. But more often than not, we are asked: “Will the adoption of MBT benefit my testing?” So we present our five high-level criteria, which are designed to indicate whether the use of MBT will be successful —or not.
Test Process Maturity
In TPI NEXT®-speak there are four levels of test process maturity: Initial, Controlled, Efficient, and Optimizing obtainable in 16 key areas. We suggest that your overall test process should be at least in the state “Controlled”, which guarantees that basic test activities are recognised as being important. Two key areas – Methodology Practice and Test Case Design – should be evaluated more closely, as test design and test automation are essential parts of MBT.
Being Efficient in these two areas is recommended. This ensures that, “[the] techniques are described that explain how activities are performed and how tools automate certain activities” as well as “tests should be performed with a minimum of effort, finding the right defects and finding the most important defects first.”1 Also it is necessary to keep an eye on the closely related requirements management process.
Make sure major internal stakeholders are willing to try out new things. Secure management support, as MBT might require substantial investment in skills, resources, and licenses. Your test team should embrace changes. If you force MBT on them, you might end up buying another tool, which will never be used.
One of the main benefits of MBT is the theoretically total requirements- and test coverage. So if you have high test- and product quality standards (don’t you?) or (God forbid!) your product lacks quality, which for instance manifests itself through a significant number of customer reported defects, not missing a crucial test case is key.
How much is your System-under-Test (SuT) worth? The Return-on-Investment related to the introduction of MBT rises alongside the value of your SuT. Its value could be measured by the revenue it generates (online shop vs. company website); cost of non-quality, which includes lost revenue, warranty work as well as a decline of reputation in case of failures; or quantified risks and their potential liabilities (automotive vs. telco).
In general, a model is usually a lot more comprehensible than a written document. So you should always create one anyway, and be it solely drawn on a whiteboard. Are you obliged, internally or externally to create an UML-like system- or test model, to document the test case creation process and test cases, or to proof a certain level of requirements- and test coverage? Then a MBT tool relieves you of those tasks too.
The criteria above are in no particular order. If at least one of these applies to you, it warrants a more thorough evaluation.
brightONE offers test process and test automation consulting and services as well as test automation solutions in general and MBT in particular. Check out our Testing Productivity Solutions or contact me directly.
- TPI NEXT®, Sogeti, dpunkt.verlag, 2011 ↩