business simulation testing

The Importance of Business Simulation Testing in Agile Development

By: Ben Barrell

Picture this: Two separate companies are each in the process of developing a new system to improve their customer’s shopping experience. Both receive the same demonstration of the system’s capabilities. Both have the same goal to grow sales through an improved experience. Both companies are using an Agile methodology. Yet, only one has discovered the benefits of using a Business Simulation Testing (BST) phase to minimize the likelihood of serious defects being detected during the User Acceptance Testing (UAT) phase.

Company one applies a typical approach to development and testing where the development team makes changes and demonstrates them incrementally to the business for approval. They perform system and functional testing, followed by UAT. The UAT phase is the first time the business users have hands-on access to the new system. Serious defects detected in UAT end up risking a delay in the release date. Even if it is released on time, this approach risks negatively impacting customers and revenue.

Customer two has leveraged an additional BST phase of testing, to allow the business to fully test the new system before moving into UAT. As a result, this company increases the likelihood of identifying serious defects earlier on, while minimizing the amount of time and money needed to patch the errors. Better yet, the chances of any customers experiencing defects decreases because they were resolved before the system rolled out. 

business simulation testing

The Goal of Business Simulation Testing

The BST is a testing phase in agile development where your team will test your end-to-end business processes. The goal of this testing phase is to uncover defects in everyday processes. The faster you find and identify defects in test scripts, the faster you can resolve them before more employees or customers feel the impact of those errors. 

Testers in the BST phase are encouraged to get creative when testing. Testers are encouraged to go deep in their workflows and span the entire surface that their job role covers so they can test as much as possible. While taking this deep and wide approach, testers are encouraged to break everything they can to find as many defects as possible before the system is rolled out to the entire team and, if it’s front-facing, to the customers. 

Looser Governance is Critical

In the UAT testing phase, you have stringent processes in place for documenting every move and approaching every decision compliantly. You dot every i and cross every t in the instructions given to the team. 

In BST, looser governance is critical — but there is governance. 

Management of successful BST phases uses a very specific plan to ensure a compliant approach. The testing focuses on capturing only the evidence necessary to recreate and resolve defective behaviors, which means there’s typically less governance in how the testers approach their work. It’s not until later, during the User Acceptance Testing (UAT) phase, that companies focus on screen capturing and having multiple layers of a compliance review. 

Governance in BST comes into play before testing begins. Each satellite system must be ready and in place. A team must be assembled and committed to testing. Your processes must be completed and available for testing. As your dedicated BST team enters a testing phase, they will receive entry and exit criteria to follow. This is where rigid compliance ends. 

Once your team is set loose, you can loosen the governance reins. Having a less rigid approach and directions allows the team the flexibility to try processes that you might not have outlined in the scope. While that might sound like you’re unleashing the Wild West, there is a reason for inspiring a little less rigidity to the testing process.

Advantages of less governance in BST 

There are distinct advantages to having looser governance in your BST testing system. Because the goal of this testing phase is to find what’s broken, you want your team to try anything and everything in the system. That horizontal movement allows the testers to throw more into the system and break it in ways you might not have anticipated.

Having less rigidity in the BST phase is a good thing. The test team has more of a lateral ability to break the system in every conceivable way possible. Without rigid testing constraints, your team is able to get a lot more creative. 

For example, rather than restricting testers to upload a specific file, your testing team might try uploading a file with an incompatible extension. That lack of compatibility might not have been found had your team been operating under specific guidelines and not given the ability to be creative in their approach. 

Compliance is important, no doubt. Still, we suggest leaving those strict steps to the UAT process. Instead, letting your team loose will help you find defects you might not have known to look for in a clearly defined scope. 

Putting Together Your Business Simulation Testing Team

Because you want your BST team to get creative, it’s important that you assemble a wide variety of team players. When choosing your team members, it’s important to pull dedicated members from each core business unit. 

Despite your team members having different job descriptions, each team member chosen will be following the same end-to-end process. But, because their job roles are different, they will each approach the problem a little differently, which will help shine a light on a wider range of defects. For example, if the end-to-end process was to ensure the order was delivered accurately and on time to the end consumer, each team member would approach that process in the same way they would in their job role. In casting a wider responsibility net, your BST team would ideally identify that problem where a customer wouldn’t end up getting their delivery and identify it before the process rolled out externally. 

When assembling a BST team, diversity is critical to stopping issues like these before the project unfolds to ensure adoption and reduce costly problems. 

The Business Simulation Testing Process

When the IT team first presents a demo to the stakeholders, they’re simply showing their plans. The stakeholders then give a thumbs up to the idea or a thumbs down. It’s not until the BST process begins that this team gets their hands dirty in the new system. It’s that process that allows teams to dig in deep and find defects.

You may be reading this and thinking that the creative approach turns the BST process into a free for all. Unfortunately, that’s not quite the case. While testers are encouraged to get creative in their approach, there are processes that are integral to the BST phase. 

Procedurally, once the BST team uncovers a defect, a defect management process is applied. 

In addition, you must have a test management plan to track all the many moving parts. It’s important to understand the scope, breadth, and depth of how those parts fit together, so you can better identify the defects and resolve them before a wider deployment. 

BST testing is a temporal development process. As you go through the process iteratively, the team is bound to uncover more defects, which they can then report and repair before deployment to the masses.

Putting the Tech In Motion When Starting the Business Simulation Process

To have a successful BST process, each satellite system must be in place. When we say “satellite system” we’re simply referring to the system that feeds in and out data. 

When deploying a very large program for testing, you will have numerous satellite systems for reporting, connecting to the various departments, gathering consumer data, and generating reports. 

Without these systems coded and updated before you deploy your BST, you’ll have a much harder time processing and supporting changes to the core system. Think of this as a domino effect. Once you plug each system into the end-to-end satellite system and begin following the new business process or scope, you’ll see the ripple effects of defective behaviors. For example, if you place a purchase order, you might find that it gets sent back in a different way, leading to shipping delays or missed shipments. Having the end-to-end technology in place first is crucial to uncovering the dominos dropping due to a defect.  

Maintaining Data Integrity Throughout the Business Simulation Testing Process

Everything in the BST phase is a dress rehearsal, including how you move your data over into the system. It’s crucial that, as you populate your newly changed system, you maintain data integrity. 

The BST phase allows you to see how your data will be impacted as you cut over into production. The data team will load all of the data into the system, just as they will before they go live, which allows you to see where there are leaks in your new system’s bucket and where you could get dirty data out as an unintentional consequence of this system update. 

You Can’t Afford to Ignore Business Simulation Testing

Ultimately, the BST phase is crucial to deploy because it saves you massive amounts of headaches and expensive repair dollars by finding defects in the new system early on. With each new phase in your project, fixing defects becomes more expensive in dollars and time. Worse yet, if defects make it to the end consumer, you could erode the customer experience. Implementing a BST phase within your agile testing methodology provides your project team with confidence that the best possible version of the system will be tested in UAT and released to the customer.