Testing Is A Risky Business

Some of the most important responsibilities we have as testers are identifying, reporting, and mitiating risks.  I’ll go as far as to say that if you aren’t regularly doing these things now, you aren’t reaching your full potential as a tester.  

Let’s take a look at each of these individually.

Identifying Risks

Risks are anything that could cause the project to go off schedule or result in lower than acceptable quality when finished.  Let’s consider a few examples and explore why they’re risky:

  • A seemingly endless stream of bugs found in a feature – Might indicate the feature’s code is too complex, its requirements were unclear, or it isn’t being tested in a methodical way.  Risky because it makes predicting the point by which the product can be released harder to estimate.
  • Features being implemented before design decisions are finalized – Risky because work might have to be thrown away and re-done late in the development cycle, causing late code churn and lots of extra testing needed with little time to stabilize.
  • Dependency teams missing deadlines – When teams you depend on can’t deliver high quality components to you in a predictable way, it adds uncertainty to your schedule. 
  • Features that used to work break frequently as code churns – Risky because deeper testing might get delayed while bugs are being fixed.  Also a sign that proper regression testing is not being done before code is committed.  Usually a smell of deeper problems in test coverage, team discipline, or code complexity.
  • A test plan review ends up with more questions about feature design than answers – Risky because it indicates the team doesn’t have a clear picture of what they’re trying to build, leading to confusion, inefficiency, and (usually) rework.

Focus on the biggest risks first

Any given project will have many risks; it’s just the nature of developing software.  The most effective testers will focus on the biggest risks first.

As testers, we’re accustomed to finding bugs, and each individual bug could be thought of as a risk.  But chances are, you’ll find more bugs than you care to track and report about on an individual basis.  Rather, it’s more helpful to step back and “look at the forest” instead of the trees (bugs).  

Sometimes the biggest risks will be obvious, but sometimes you’ll need to work with others on the team to determine which risk truly represents the biggest threat to the project’s success.

Reporting Risks

Once you’ve identified risks, one of the most effective ways to squash them is to keep them highly visible to project stakeholders.  A risk report could take many forms, but usually it includes:

  • a brief summary of the problem
  • a brief description of the planned mitigation
  • next steps
  • a date/time for when the risk should be mitigated
  • some indicator of whether the mitigation is on track or not (e.g. red/yellow/green)
You don’t need to write up an official memo or create a Powerpoint presentation for each risk.  Often, a simple email will do, or a few quick sentences in a team meeting.  The point is to raise visibility on the risks as soon as possible to give the team more time to mitigate the risks.  
 
You might be thinking to yourself, “Why would testers need to report this?  Isn’t this the project manager’s job?”  If your team is lucky enough to have a skilled project manager working with you, then yes, it’s certainly within the realm of what they might report.  That said, someone needs to inform the PM about the risks in the first place though.  As a tester, you are likely to have a unique view of the project that others don’t have, and therefore it is your responsibility to call out risks you see from the tester’s perspective.
 
Aside: Regardless of whether you have a PM or not, I’ve never been a fan of drawing hard lines around areas of responsibility.  As a team member, the team succeeds or fails together.  If you see a risk that nobody else is reporting, call it out!  Consider the 2012 security breach at Target.  Had someone raised the risk that their anti-malware software was detecting problems earlier, they might have been able to prevent any customer data from ever leaving their servers!

Mitigating Risks

This part should almost be obvious. Once we’ve identified risks, our job is to help make them go away.  I say “help” here because we can’t do it alone; we usually need others to work with us.  For example, once we’ve identified a risk that a particular feature has a seemingly endless stream of bugs, we might ask the developer team to revisit the code to see if it can be simplified or ask a designer or stakeholder to simplify the requirements.  For our part, we need to continue testing thoroughly and expand our thinking to find ways to test the product more efficiently or more effectively.  Once a fix has been made, we need to instill confidence among stakeholders that the risk is indeed mitigated.

To prevent risks from reappearing later, it helps to identify their root causes and then drive whatever changes are necessary to fix those problems.  The 5 Why’s technique can be helpful useful here.  Great testers know how to break problems down into smaller problems and debug them piece by piece.  This applies to debugging processes as well as software.

Other mitigation techniques for testers might include reviewing test plans with key members of the team to identify gaps in test coverage, building new test automation to catch bugs earlier in the process, or enlisting the help of other testers to bring “fresh eyes” to an area that was previously thought to be well tested.

Conclusion

Risks exist in every software project.  Successful software testers recognize this and regularly identify and focus on the biggest risks first, keep the team informed about risks, and help mitigate risks constantly.  Are you?

Author: Jason

I am currently a Senior Software Engineer with nearly 20 years experience devising test strategies and test plans, building test automation, managing software teams, and helping developer teams improve testability of their code. My thoughts on this blog are my own and do not necessarily reflect the opinions of any employer, past, present, or future.

Leave a Reply

Your email address will not be published. Required fields are marked *