Acceptance Criteria and Bug Threshold Services

Setting the Bar for Software Success

Software development projects succeed or fail based on the ability of a development team to meet their clients’ documented and perceived acceptance criteria. Acceptance criteria define the boundaries of a user story in Agile development, verify that the functionality meets user needs and expectations, and are used to confirm when a story is completed and working as intended.

Testing Acceptance CriteriaThey:

  • Define the system behavior
  • Ensure features work as expected
  • Help the team gauge the amount of work needed to complete a story
  • Guide development and QA testing

By including acceptance criteria as part of your requirements documentation, you greatly enhance the likelihood of a successful project.

Why Acceptance Criteria?

It’s common for stakeholders, project owners, and end users to have a hard time defining acceptance criteria at the start of a project. Conversely, once the solution is delivered, they seem to have no problem defining what is missing or wrong. Projects that kick off with incorrect or missing acceptance criteria often face low customer satisfaction levels, missed delivery dates, and development cost overruns as these criteria are uncovered throughout the development process. That’s why it’s critical to define criteria before any development work begins accurately.

Good acceptance criteria include:

  • Functional Acceptance Criteria – Identify specific user tasks, business processes or functions that must be in place at the end of the project.
  • Usability – Identify how to answer the question, is it easy to use? For any given feature, we may measure ease-of-use by looking at the rate of successful use, setting an acceptable threshold. Or we might set a threshold for time to completion. The key is to identify the right measure or measures and to make sure each measure is readily quantifiable.
  • Performance – Acceptance criteria should also define and include performance criteria from the perspective of an individual user. Does the page load quickly? Is the UI responsive? Do I have to wait for ads to load before I can view content? Expected performance should be clearly spelled out as a range such as “1-2 seconds for a screen to refresh.” And it may make sense to separate out and define performance for specific parts of the solution.
  • Error Handling – Identify what data you are expecting and how will you handle each instance of missing data. What happens if someone does something out of order? Or if they stop halfway through and then come back later? What happens if someone doesn’t have the data you are asking for? Can they move on or are they stuck? Enumerating error cases is a great opportunity to surface all of your assumptions, particularly around data needs, and how to resolve them if they are wrong.
  • Stress Tests – Stress tests measure what happens when you are inundated with lots of users, transactions, or queries. What happens when the system is under stress? Acceptance criteria should define acceptable thresholds for stress testing.

They are also written in simple language, just like the user story.

Our Approach

QAT Global customizes acceptance criteria to meet the needs and expectations of each client. We work closely with you to clearly define your needs and expectations, so your requirements documentation clearly outlines your acceptance criteria for each type of functionality, and your user stories are well defined.

Ultimately, testing is done using your acceptance criteria. Acceptance test-driven tests outline what the user should be able to do, define when acceptance criteria are “done,” and rely on the core principles of Agile by enabling communication between the business and engineering, and between dev and QA.

Bug Thresholds

A software bug is an error, flaw, failure, or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways. It is common practice for software to be released with known bugs that are considered “non-critical” as defined by the project owners and developers. While software products may, by definition, contain any number of unknown bugs, measurements during testing can provide an estimate of the number of likely bugs remaining. This becomes more reliable the longer a product is tested and developed.

Setting bug thresholds in software testingYou may decide not to fix a particular bug for a number of reasons, including:

  • A deadline must be met and priorities are such that all but those above a certain severity are fixed for the current software release
  • The changes to the code required to fix the bug are too expensive, will take too long for the current release, or affect too many other areas of the software
  • The problem is in an area which will be obsolete with an upcoming release; fixing it is unnecessary
  • The bug is already fixed in an upcoming release, and it is not serious enough to warrant an immediate update or patch
  • It’s not an actual software bug, but a misunderstanding between expected and perceived behavior

Our Agile team leads are to help you triage bugs to effectively identify which ones could present a risk to a project’s successful deployment and those that don’t need further attention. Using statistical data, test engineers keep projects focused on the intended outcome and do not waste time or budget chasing outliers that do not have an impact on project outcome.

Learn more about how our QA and Testing Services can solve your testing needs, contact us now >>

Let’s Talk