Shift Right - The Value of Testing in Production
How to monitor real time Customer eXperience (CX).
DevOps and CI/CD has changed the way we develop software and testing has had to embrace automation to keep pace with demand. And these new test automation platforms are offering more than just a stamp of approval.
We once understood testing simply as Quality Assurance, a manual, time-consuming, box-checking activity that is at odds with the automated, highly integrated approach to software development most teams now practice.
DevOps organisations have begun to develop a “Quality Assistance” attitude towards testing that keeps the development pipeline flowing through automation. What was once a bottleneck is now a facilitator of speed and quality. But the sophistication of these new tools continues to add value in post-production.
The importance of testing live production systems
Today, customers and users of digital services and products now expect to receive a rich, faultless experience anywhere, on any platform, and at any time. Our tolerance for faulty software or a poor user experience has rapidly gone from low to zero.
And that’s not the only set of demands we face. The arrival of CI/CD and DevOps has contributed to a huge increase in software releases. There’s more to test and typically we’re never more than 2 weeks from making a release.
Taken together, this means QA has to facilitate pipelines that are releasing software faster and faster to end users who are increasingly impatient with downtime and poor performance. The value of ensuring consistently positive customer experience and site reliability has never been higher.
To actually understand production trends you need to put yourself in the customer’s shoes. What do they feel when accessing your application? How much time do they have to wait to complete a journey? What is their experience at a given point in time? Having this knowledge means having control.
How to effectively test in production
Performing quality control by monitoring your production environment throughout the day allows you to measure application performance during peak hours. As well as noting any application downtime you can keep a close eye on functional health and application performance.
Being able to call upon quick responses to failures means you really have to understand how the system performs in response to real user behaviour. Seeing what your customers’ preferences and engagements are when applying a different application flow, or important design changes, will give you valuable knowledge
Traditionally, we build confidence by executing as many tests as possible prior to the release, but this approach has many flaws. It provides no assurance that the product will not fail in production because all the tests have been done within dedicated test environments.
These are some of the testing strategies that will help you to achieve effective CX monitoring.
Reacting to issues in production.
In DevOps organisations particular attention is paid to the ability to continuously shift right by testing in production. And when failures happen they act by deploying only small changes, released incrementally while the health of the application is closely monitored. It isn’t just about performance monitoring either. New, intelligent test automation platforms allow us to do much more in terms of functional testing and CX than is being done today.
Above all, it is important to be able to react quickly throughout the testing process and to maintain control of live situations. Testing in production makes it possible to understand real user journeys and analyse real use cases that are extremely hard to predict.
In the end, the only way to prove that software is ready for production is to release it into production! Automation in testing is giving us the freedom to do so with confidence.