Acceptance Criteria and Bug Threshold Services
Setting the Bar for Software Success
Software development projects succeed or fail based on the ability of a development team to meet their clients’ documented and perceived acceptance criteria. Acceptance criteria define the boundaries of a user story in Agile development, verify that the functionality meets user needs and expectations, and are used to confirm when a story is completed and working as intended.
Download the Quick Sheet
They:
- Define the system behavior
- Ensure features work as expected
- Help the team gauge the amount of work needed to complete a story
- Guide development and QA testing
By including acceptance criteria as part of your requirements documentation, you greatly enhance the likelihood of a successful project.
Why Acceptance Criteria?
It’s common for stakeholders, project owners, and end users to have a hard time defining acceptance criteria at the start of a project. Conversely, once the solution is delivered, they seem to have no problem defining what is missing or wrong. Projects that kick off with incorrect or missing acceptance criteria often face low customer satisfaction levels, missed delivery dates, and development cost overruns as these criteria are uncovered throughout the development process. That’s why it’s critical to define criteria before any development work begins accurately.
Good acceptance criteria include:
Functional Acceptance Criteria – Identify specific user tasks, business processes, or functions that must be in place at the end of the project.
Usability – Identify how to answer the question, is it easy to use? For any given feature, we may measure ease-of-use by looking at the rate of successful use, setting an acceptable threshold. Or we might set a threshold for time to completion. The key is to identify the right measure or measures and make sure each measure is readily quantifiable.
Performance – Acceptance criteria should also define and include performance criteria from the perspective of an individual user. Does the page load quickly? Is the UI responsive? Do I have to wait for ads to load before I can view content? Expected performance should be clearly spelled out as a range such as “1-2 seconds for a screen to refresh.” And it may make sense to separate and define performance for specific parts of the solution.
Error Handling – Identify what data you are expecting and how you will handle each instance of missing data. What happens if someone does something out of order? Or if they stop halfway through and then come back later? What happens if someone doesn’t have the data you are asking for? Can they move on, or are they stuck? Enumerating error cases is a great opportunity to surface all of your assumptions, particularly around data needs and how to resolve them if they are wrong.
Stress Tests – Stress tests measure what happens when you are inundated with many users, transactions, or queries. What happens when the system is under stress? Acceptance criteria should define acceptable thresholds for stress testing.
Our Approach
QAT Global customizes acceptance criteria to meet the needs and expectations of each client. We work closely with you to clearly define your needs and expectations, so your requirements documentation clearly outlines your acceptance criteria for each type of functionality, and your user stories are well defined.
Ultimately, testing is done using your acceptance criteria. Acceptance test-driven tests outline what the user should be able to do, define when acceptance criteria are “done,” and rely on the core principles of Agile by enabling communication between the business and engineering and between dev and QA.
Bug Thresholds
A software bug is an error, flaw, failure, or fault in a computer program or system that causes it to produce an incorrect or unexpected result or behave in unintended ways. It is common practice for software to be released with known bugs that are considered “non-critical” as defined by the project owners and developers. While software products may, by definition, contain any number of unknown bugs, measurements during testing can provide an estimate of the number of likely bugs remaining. This becomes more reliable the longer a product is tested and developed.
You may decide not to fix a particular bug for a number of reasons, including:
- A deadline must be met, and priorities are such that all but those above a certain severity are fixed for the current software release.
The changes to the code required to fix the bug are too expensive, will take too long for the current release, or affect too many other areas of the software.
The problem is in an area which will be obsolete with an upcoming release; fixing it is unnecessary.
The bug is already fixed in an upcoming release, and it is not serious enough to warrant an immediate update or patch.
It’s not an actual software bug but a misunderstanding between expected and perceived behavior.
Our Agile team leads help you triage bugs to effectively identify which ones could present a risk to a project’s successful deployment and those that don’t need further attention. Using statistical data, test engineers keep projects focused on the intended outcome and do not waste time or budget chasing outliers that do not impact project outcomes.