In my previous posts, I discussed the first two of three considerations in the Requirements Challenge: knowing the unknown, and properly representing all perspectives in requirements elicitation, management and architecture. The third and last consideration in this series is perhaps the most salient, yet often ignored: requirements quality. Namely, how should product development teams answer the question: Do the requirements contain a sufficient level of quality to proceed with subsequent product development activities?
Requirements Quality
Many IT business analysts working in waterfall environments recall horror stories of development and testing teams approaching the BA with defects or issues within the requirements specification, but either the BA who authored those requirements left the organization, or the author has forgotten the relevant details after working on so many intervening projects. Whether in a waterfall or Agile environment, the quality of the requirements themselves is a critical factor in the success of the product or system. If we think of the requirements as a product, we want to subject them to the same level of rigor and scrutiny which is applied to products and systems per quality standards. This approach mandates involving quality personnel in requirements elicitation and reviews (e.g., QA, QE and SDETs), as well as agreement on what constitutes an adequate level of quality for systems requirements.
Agile teams might define the Definition of Ready criteria for backlog items such that all quality checks are performed (e.g., peer reviews, using Bill Wake’s INVEST criteria for user stories), while more traditional plan-driven product development teams might leverage standards such as ISO 25010 and IEEE 29148 for requirements quality. (ISO 25010 provides a robust product quality and quality-in-use model, while the latter standard provides a checklist of sorts for objectively evaluating requirements quality.) Even other teams might be subjected to regulations that require them to perform certain activities to reach a benchmark quality level, such as formal requirements inspections in the Automotive industry where a system falls under a certain ASIL (Automotive Safety Integrity Level). The chosen requirements attribute scheme is a simple means to ensure the product team has provided all relevant details: such as priority, risk, source, tests, status, etc.
Establishing a consensus for requirements quality helps to minimize defects, gold-plated features, and can provide additional control on when and whether subsequent product development activities should commence.
But how Much is too Much?
Numerous studies across various industries and over time (e.g., Fagan 1976, Leffingwell 1997, NIST 2002, McConnell 2004, Boehm and Tuner 2004, Standish Group 1994-2015) have consistently arrived at the conclusion that not only do faulty, incomplete or ambiguous requirements beget subsequent faulty system artifacts, but that the cost associated with correcting those faults significantly rises the later it is addressed in the system development lifecycle. If we can agree RE practices are vital for success, is there a limit to how much is too much? In a word, yes. As with almost all human endeavors, achieving perfection (defect-free) is impossible. Furthermore, at some point our efforts will result in diminishing marginal returns. But where to draw the line is highly dependent upon industry, safety regulations, the customer, business, and product/system itself.
We have a surprisingly clear picture on where one organization thinks the line is between too little and too much RE. NASA performed a retrospective study of its space programs, and in the early 2010s, published its findings on the correlation of program cost overrun (vs. budget) and the amount of requirements engineering invested on the program (as a percent of overall). The results imply that somewhere around 8-14% invested in the requirements engineering approach correlate with less than 60% in cost overrun. If we think of this RE investment as insurance, then spending less than 5% is risky business, while investing more than 15% is likely not worth any marginal gains.
Clearly 8-14% is not necessarily the sweet spot for every product or service, but there is a point at which all teams will find that their requirements approach is sufficiently matured without being overbearing. Focusing on these three requirements considerations – thorough requirements elicitation, proper stakeholder representation, and maximizing requirements quality – will help guide product development teams amidst the challenges of today’s “digital” marketplace.
Cody May is Principal Business Analyst at SQS USA
