Bad Ass BA; Peer Review. Part 3.

Written by Cecilie Hoffman on November 10, 2009. Posted in Articles.

Co-authored by Rebecca Burgess

Use and Profit from Peer Reviews of Requirements Documents

This article is part 3 of a 4-part series on the topic of peer review of a requirements document. Before we dive into part 3, indulge me in a personal note.

My organization recently changed from having a peer BA review another BA’s BRD to a novel process in which four BA peers review one BRD. Each of the four peers has a different perspective based on their professional experience. Two weeks ago one of my BRDs went through peer review. In our process, first the reviewers provide their written comments. Then the BRD author has a full day to review the comments before conducting a structured walk-through of the BRD with all the peers in a conference call.

While reviewing the combined comments on my BRD, I experienced a series of emotions I haven’t felt in quite a while: initial horror, shame, indignation, despair and dismay. One of my peers wrote me a note, “Hey Cecilie, is this a test? I’m commenting on your BRD with words that I hear in my head in your voice.” I wish I could say that it was a test, but it wasn’t. I had submitted a BRD to the peer review process under duress (time pressure); the BRD wasn’t my best work but I honestly didn’t expect that scrub cloths would come back so dirty after my peers had examined my work.

I spent several hours cleaning up that BRD. I wasn’t looking forward to the conference call review where, in accordance with our internal process, I would walk the group through the comments that were marked “High” and “Showstopper” – I would have to reply to each such comment and offer a suggestion for remediation. All High and Showstoppers had to be resolved, otherwise my BRD would fail the peer review and fail to move to the next step of the BRD approval process.

My BRD received a conditional approval from my peers – I had to get a couple of questions answered from the stakeholders first. Fortunately the answers came back quickly. Another hour to tidy everything up, and the BRD was fully approved by the peer group to move forward. Most of the time I do leave my ego at the door but apparently I kept it with me during the review because it took a couple of hours of mentally pouring honey on my bruised ego for me to recover.

We’ve been running this peer-group review process for about two months. We are hearing from the downstream reviewers – all managers – that the quality of the BRDs has improved noticeably. I can attest that the quality of my BRD benefited from the scrutiny of my peers. The difficulty in a peer review is that, if the peer reviewer is also your friend, your friend may not be able to be as honest as you need him/her to be. The team of four gave me honest, constructive comments about scope and risk; they caught inconsistencies in the requirements, and questioned the testability of some of the requirements. An un-anticipated side effect of this peer-group review process is that we are learning from each other about how to address BRD authoring problems much faster than anyone expected.

So, that was a long note, but I hope it is helpful to anyone who is nervous about giving peer review a try. Embarrassment will not kill you and in the end you will be more proud of your work.

Let’s do a recap before we begin part 3.

One way to conduct a review is as a series of go/no go checkpoints.

Just to review, here are the four checks:

Business Case Synchronicity Check
Requirements Document Sanity Check
Requirements Statements Content Check (In this issue)
Requirements Document Housekeeping Check (December 2009)

And here are the typical excuses not to do a peer review, and why those excuses are bogus.

It’s scary! – They won’t say this out loud, but it’s true. You must separate your self esteem from the evaluation of your work by learning to accept criticism as constructive and valued, rather than destructive and to be avoided at all cost.
It takes time – Yes, but the payback is substantial, both in minimizing the impact of errors, and in maturing the BAs on the team.
We are already behind schedule – Would you rather let the customer catch the errors? Or worse yet, miss the errors?
I don’t know anything about that project – So you’ll limit your review to the more general, BA-level comments; that’s still valuable.
It’s hard to do – True. The more you practice, the less difficult it will be the next time.

In the first article we checked for Business Case Synchronicity. We found that the BRD synched up with the business case, that is, benefits for doing the project have been identified, there is an expectation of Return on Investment, and there is an expectation that the project will report on progress towards achieving the ROI claim. Moreover the customer has identified the means for validating project success.

In the second article we performed the Sanity Check. We verified that the Scope not only identifies what is in scope, but what is out of scope as well. The document has a business context diagram, a 30,000 foot view of what the project is about in terms of system functionality, the business entities that will interact with the system, and information that is exchanged between the entities and the system. Finally we verified that the categories of the requirements made sense for this particular BRD.

Now it is time to look at the requirements statements themselves, both individually and, as they relate to each other.

Requirements Statements Content Check

Take a deep breath, hold it, and let it out slowly. Make sure you aren’t going to be interrupted for at least half an hour. Depending on the length of the document, you’ll need chunks of uninterrupted time to do this part of the review.

Check the box next to the item if you can answer ‘yes’ for the document you are reviewing.

Clarity

Is each requirement relevant to the business problem?

Are all requirements actually requirements, not design or implementation statements?

Design and implementation statements are “how” statements; a requirements document has “what” statements that ideally do not imply a desired solution.

Do you understand each requirement? Does each requirement make sense to you?

If a requirement doesn’t pass your “sniff test”, it likely won’t make sense to anyone else.

Are the requirements “solution agnostic”, i.e., do they convey the business need without specifying a solution?

Are the statements constructed as proper statements or sentences?

If you think the requirement statement is a fragmented thought, other people will too. The BA author may need to request more information (not necessarily more detail).

Is the content in each statement just sufficient to convey the idea?

Statements can become too terse (fragments), or too verbose. “Verbose” can be any of these:

Too wordy, but the statement is still just one requirement
More than three lines long.

A statement that is longer than three lines of text is not always verbose, but verify!

Are the business requirements written in non-technical language?

Is the text understandable to a business person and understandable by an independent third-party? The problem isn’t just a missing term in the glossary; if the requirement is jargon infested, the BA author should revise it for the benefit of mere mortals.

If a requirement needs specific business knowledge to understand, will the people who implement and/or test the requirement have that knowledge?

If not, the BA author may need to provide more information in the requirement, or explanatory text along with the requirement.

Are the requirements constructed to reduce the possibility of more than one interpretation?

If not, does the requirement need more detail to limit it to a single interpretation, or should there be a set of related requirements to reduce the likelihood of ambiguity?

Does each requirement address a single concept, topic, element, or value?

It takes a clever BA to decompile a complex idea and turn it into a requirement. The result of the first attempt may be an overloaded requirement, i.e., a single requirement that is actually more than one requirement. Look for requirements that have more than one relative clause, more than one instance of a semicolon, or the frequent red flag, more than three lines of text. These attributes are often indicators that the requirement needs to be split up into a set of multiple-but-related requirements, each with their own identifier, e.g., Security-011a, Security-011b.

Are requirements statements distinguishable from the explanation that often follows a requirement?

Sometimes a requirement needs an explanatory note, that is, text that is supplementary to the core requirement. Be sure that the note is separated, even by a blank line, from the core requirement so that it is clear which statement is the requirement and which is the explanatory note. Notes should not have the telltale “shall” verb.
Sometimes the explanation is a design constraint. Design constraints need to be assembled in their own section such as an appendix section for design constraints, or, at the very least, listed in the Assumptions.

Testable, Verifiable

Are the requirements verifiable by testing, demonstration, review, or analysis?

Do the requirements have success criteria, and a metric that specifies the success threshold?

“I’ll know when I’m happy” is not an acceptable evaluation criterion!

Key, high priority requirements need explicit success criteria and/or metrics. Lower priority requirements or very detailed requirements may have implicit success criteria.

Are the criteria for a “pass/fail” evaluation clear and quantifiable?

An example of a clear evaluation criterion is a named process that a human or a machine can execute to verify the requirement has been met.

If the success criteria/metrics for a requirement are known, is that information associated directly with the requirement?

Sometimes the success criteria/metrics get separated from the requirement, e.g., in another section of the document. Use cross-references or some other means for the reader to put the two together.

Are the requirements free from weasel words? (see BA Times article http://www.batimes.com/component/content/article/106-articles/377-bad-ass-ba-caution.html)

Weasel words will be the ruin of the project.

Business-Critical Dates

Are timing constraints such as due dates, deadlines, and completion-date dependencies identified?

Are the “critical” dates truly critical?

There are many ways to define “critical to the business”. It is important to distinguish “critical” and “important”. In my humble opinion, there are four types of truly “critical” dates:

Beyond this date there will be an impact on revenue generation
Beyond this date we will lose the ability to conduct business
If we miss this date (which we announced to the media) we will be subject to ridicule and derision in the press which will drive our stock price into the ground
If we miss this date we will be subject to contractual or regulatory sanctions, such as a monetary fine. For example, being out-of-compliance, or a real estate lease date expiring, or a contractual completion date not being achieved.

Are the “important to customer” dates articulated?

Even if the date isn’t “critical” by the definition above, a date can be important to the customer and should be identified as such so that the project manager knows about both the date and the expectation.

3rd Parties and Partnerships

Are the major requirements identifying the need for information to be shared, processed, or stored by 3^rd party partners or hosted applications?

Compliance

Are compliance requirements identified?

Compliance requirements often first appear as an assumption. Assumptions can be overlooked during design and implementation. Calling out a compliance requirement ensures that the project test plan will have a line item for verification of compliance. Examples of compliance are data privacy (Personal, Identifying Information (PII)), payment card industry (PCI), legal, financial, and governing entity policy compliance.

If the project does not have any compliance requirements or impact, is that made explicit?

Adding the statement, “This project do not have any compliance requirements.” is better than just leaving the Compliance section of the document blank, which looks like the author forgot something.

Business Communication Processes

Are the major requirements related to supporting business communication processes included?

If the project is going to have an impact on how users do their work, in terms of what they see or what they do, then there needs to be a communications plan. The BA author may think that it is overkill to have a requirement for a standard project deliverable but the manager of the group that provides end user support may see things differently.

Internationalization and Localization

Are the major requirements related to internationalization and localization included?

Internationalization, known as I18n (meaning “I – eighteen letters -N”) to people who do this for a living, is the process of planning and implementing products and services so that they can easily be adapted to specific local languages and cultures. Doing the work of translating the text into local language and culture is called localization, aka “L10n”.

Timing / Response

If time-critical functionality is known when the requirement is written, is the time-criticality identified, and are timing criteria specified with metrics?

A requirement for “fast response” isn’t testable! “Fast” compared to what? Is less than 10 seconds acceptable? Is less than 2 seconds acceptable?

Reporting

Is there a requirement for creating a report to show progress towards achieving the projected return on investment (ROI) on the project/product?

According to best practices, a requirements document should have a business case. That business case identifies the expected ROI for the work that will be done. For the project to be truly successful, it needs to produce the evidence for delivering on the ROI claim.

Are the reports/dashboards/extracts that are expected from this project identified?

Are the audiences for those reports/dashboards/extracts identified?

Unknown, Missing, Incomplete Requirements

Are requirements where information is unknown, missing, incomplete or “to be determined” (tbd), annotated with a reason for the incompleteness?

Some BRD templates have an “Open Issues” section were unknown, missing, incomplete or “still under discussion” topics are called out. Ideally each issue has the following information to help the BA close the issue:

A unique identifier for the Issue, this could be as simple as “Issue.1”
A description of what the issue is
The date the issue was first identified
The name of the person responsible for resolving the issue and a target date for that resolution
A status for the issue, e.g., Open/Closed/Deferred. Seeing a column of “closed” issues is so satisfying.

Requirements Statement Style

Do the requirements statements use one verb consistently, e.g., shall or will?

Are the requirements articulated at a consistent level of detail?

Should any particular requirement be specified in more detail? In less detail? It is important that each requirement be stated at an appropriate level of detail, and that all the requirements in a given document be at the same level of detail.

Is level of information in the document consistent with commonly accepted practices for your company?

Each company has its own rules for the level of information that should be included in a requirements document. If your company doesn’t have a guideline, here is a possible starting point:

A Business Requirements Document has project-level requirements that describe the needs of the Stakeholders.
A Functional/Non-Functional Requirements document has requirements at the software system level and may include requirements at the subsystem-level.
A System Requirements document has even more detailed requirements.

Do the requirements statements use the “Trigger Actor Action Condition” syntax?

Requirements in the form of “Ability to …” are ambiguous unless there is a context-setting statement that all requirements describe who or what the actor is. Requirements must identify the actor as a human or an automated agent. “Ability to …” statements create problems because the reader has to guess at whom or what will have the ability.

Consistency of Requirements

Have inconsistencies, omissions or redundancy been identified and corrected?

Do requirements use consistent terminology to the same object?

It is normal for different groups within a company to have different terms for the same object. For example, in the United States what we call “regulations” may be called “rules” in Europe. It is important to identify both terms in the document’s glossary. Use one term consistently in the requirements.

Have requirements that describe two or more actions that conflict been identified?

The most common types of conflict are logical conflict and temporal conflict.

An example of logical conflict is when there is a general requirement for a “green” system, specifically, no printing, and, a conflicting requirement for the automatic generation of the quarterly sales results report to be faxed to the executives in Japan. An example of temporal conflict is when one requirement says that customer satisfaction data will be sent to an FTP site on a weekly basis and another requirement says that customer satisfaction data will be uploaded and processed on a daily basis.

Once such requirements have been identified, there are two possible ways to proceed. Either have the stakeholders come to an agreement, or, create an Open Issue to address the conflict as part of the design effort.

Requirement Traceability

Are all requirements traceable to at least one “in scope” objective?

Do all objectives/success criteria map to at least one requirement?

If not, there may be a gap in the body of requirements.

Common problems with Categories of Requirements

Do the requirements statements in the categories actually belong in those categories?

UI requirements are really good at hiding in a category other than a User Interface category.
System Interface requirements are often distributed throughout a BRD. Assembling interface requirements in one place makes it so much easier to understand what a project need is. Or, if there is a good reason for not moving the interface requirements from their functional category, give the technical team a reason to appreciate the BA by suggesting a table of Interface cross-references so that at least there is a logical assembly.
Be sure to distinguish internal interfaces from external interfaces. In some companies interface requirements are documented in a separate Interface Requirements Document (IRD) or Interface Control Document (ICD)

Are scope statements, assumptions, business rules and role/responsibility statements in the right category?

Sometimes a statement begins life as a requirement but over the course of revisions, it evolves into something different such as scope, assumption, business rule, or role/responsibility statement.

If there are interface and integration requirements, are they in separate categories?

Business people often use the terms interchangeably. The technical team knows there is a big difference!

Did you answer “yes” to the all the questions? If so, do you now believe in your heart that the requirements are good? If not, mark the requirements that need work and identify what your concern is.

If one or more of these items is not checked, there is some risk that the requirements are inadequate. Your mission now is to convey these “defects” to the BA author in a kind manner. If you are expected to return written feedback, take time to explain what is missing, and provide examples, if possible. If you will be participating in a structured walkthrough meeting, make notes of what you want to say so that your comments are thoughtful and constructive.

The document should not be seen by the customer or stakeholders until the problems are fixed. If the requirements are good, you can now move on to the last quality check, the good housekeeping check.

This is the third of four articles addressing Use and Profit from Peer Reviews of Requirements Documents

Business Case Synchronicity Check
Requirements Document Sanity Check
Requirements Statements Content Check
Requirements Document Housekeeping Check [December, 2009]

Don’t forget to leave your comments below

Cecilie Hoffman is a Senior Principal IT Business Analyst in the Business Analysis Center of Excellence, Symantec Services Group, Symantec Corporation. Cecilie’s professional passion is to educate technical and business teams about the role of the business analyst, and to empower the business analysts themselves with tools, methods, strategies and confidence. Cecilie is a founding member of the Silicon Valley chapter of the IIBA. She writes a blog on her personal passion, motorcycle riding, at balsamfir.com. She can be reached at [email protected].

Rebecca Burgess is the Business Process Methodology Analyst in the Commerce Lifecycle Transformation Office at Symantec and a Certified Six Sigma Black Belt. After many years of uncovering problems and determining root causes in software, she is now applying her BA skills to strategic process design and improvement. She can be reached at [email protected].