Intelligent Production: Quality Assurance practices that save Money & Effort
The common vision that most organizations have of QA is that their investment in terms of money and effort will be repaid through the improvement in the quality of the end product. They hope that in the long run, the maintenance cost will be lowered, which in turn would lead to greater profitability. That’s a good plan. But, spending more effort than what’s necessary, on things that are not as important as they might seem, causes teams to deviate from this plan and eventually affects the organization’s net profit. Anticipatory anxiety — that’s the syndrome where you worry about bad things that could happen, things that cannot be predicted and are out of your control. Such a deviation due to unneeded bouts of anticipatory anxiety is unacceptable especially at a time when QA is being viewed as a key proponent that drives business outcomes.
QA in production is a set of practices that focuses on nurturing awareness about the real-time issues that can occur in production, that are worthy of being analyzed and fixed. These practices compliment, or, in some cases replace the traditional pre-production QA practices by providing quick feedback and crucial diagnostic data. So how do you move forward without worrying about things that have slender chances of occurring in production when the real users handle your software with different networks, varied devices, and a mix of rational and irrational actions? Below are some techniques following which your team can save a lot of production QA effort.
Set Alerts for Corner Case Scenarios
Uncertainty within a software team costs a lot. Let’s explain this statement further. Many a time, developers and testers worry about how the real-time user’s action might cause the application to misbehave. This uncertainty is natural among all the software teams. But what many don’t realize is that it silently eats up money as developers spend time on writing additional code to cover these unexpected scenarios. Any new developers who come into the project would then have to spend extra time to maintain these additional lines of code. Such abruptly included code has high chances of not adhering to the design of the code base and might lead to unnecessary abstraction and code complexity. Testers, in the name of extending the test coverage, continue to test further, and if issues arise from such scenarios, those will be deliberated, some will be fixed and retested: the cycle is endless. This greatly affects the schedule.
The best thing sometimes is to do nothing. Seems stupid, but it holds good in this situation. An alternate arrangement to deal with these unexpected scenarios is to do none of the above mistakes of addressing edge scenarios but to account for them via an alerting mechanism. For example:
Now that the error log is in place, developers and testers can forget about this corner case scenario until and unless it shows up in the logs. If it occurs, the frequency of it needs to be analyzed along with its business impact and most importantly, whether it’s worth the cost of being fixed. If all these factors score low, you can take a deliberate and if all are in agreement you can choose to not fix it. Many such corner case scenarios may never happen, but this error log is a safety net that ensures you get the information if and when they do happen. By implementing such an alerting mechanism in the logs, the team can focus on other important things at hand. Out of the ten unexpected scenarios that you could think of, just one might occur with regular frequency and might be concluded to be fixed. Just think of the time and effort you would be saving.
Maintain an Audit Log
Data storage has become dead cheap in the modern age. It would be a crime if software teams didn’t use this to their advantage. It’s important to store crucial metadata information that would act as the key for answering these critical questions:
Has something actually gone wrong in production or is it a misconception?
If something has indeed gone wrong, when and how?
Without an audit log, teams would be lost and spend exponentially more effort. But when there is a lot they can look into a specific issue with focus. Without an audit log, in most cases, even if it is actually an issue that has occurred in production, the exaggerated amount of effort spent might go in vain as the actual scenario to replicate the issue may never be found because it’s akin to searching for a needle in a haystack. In the case of a false negative being misconceived as an issue, the efforts spent would entirely prove to be a waste. But with an audit log in place, you have the means to clear the doubt of it being a false negative and proceed with analysis only if it’s not.
Let’s take the example of an eCommerce system workflow after the placement of an order. An invoice gets generated and loaded. It should then be approved by the manager before it gets scheduled for payment. The amount is finally paid to the seller on the scheduled date. This process is a lot more complex than it sounds and there are chances that something might go wrong somewhere. So consider this scenario where a seller calls your customer service and demands to know why hasn’t the payment been processed for a specific invoice ID for which the products were dispatched a couple of days ago and the scheduled date of payment was yesterday. With an audit log, there is no need to fret. You can search for the specified invoice ID and see what has gone wrong.
Based on the above information, we can see that the invoice has been scheduled for payment, but did not change to the next status ‘Payment Processed’ as the other invoice IDs normally do. Now check what happened differently with this particular ID and save a lot of unwanted effort which would have otherwise been required had there not been an audit log.
Had the audit log stopped after the status ‘loaded,’ we can identify some problems in sending the approval request email. In some applications, developers set up an automatic self-healing mechanism, where, if there is no response within a particular period or if there is an error in sending the email, the system would automatically send a reminder or notify the user who loaded the invoice that the email reminder wasn’t sent to the approver.
While storing metadata, it’s important to account for GDPR and other data privacy regulations. Store what’s required for your analysis and no more. Implementing an audit trail reduces the time to decipher and fix production issues and tremendously improves customer service ratings. Inquiring customers get clear answers within a few moments.
Reduce Time Spent on Low-Value Tests
Not all of the automated test scripts are worth their weight. Teams spend several hours fine-tuning automated scripts that never find any real issues. Automated tests confirm that all is well, but spending time creating scripts for corner cases is not a good idea. Instead, setting up the right production monitoring system that continuously observes Production API behavior and alerts the team if something goes wrong is a much better route. In such a corner case, APIs that aren’t automated include specific error messages, which get returned if a real-time user indeed replicates that scenario. The production monitoring system detects the misbehavior of the API and returns the error message configured. Do the same steps of analyzing the frequency pattern, business impact, and the worth of creating a test script for the scenario and conclude based on the evaluations. An accurately configured production monitoring system increases the team’s awareness of what happens in the production environment.
Production QA practices are just a safety net. But the need for this safety net can never be questioned. By following the above techniques, you can reduce the effort into knitting this essential safety net. Free your software teams from anxiety rooted in the uncertainty about how real-time users would use your software and whether it would misbehave in edge scenarios. These techniques will convey to you instantly if something has gone wrong, and you will have adequate information to analyze why it happened. This eventually will lead to huge savings in the effort spent on writing code, testing, and thinking of approaches to resolving issues that may never happen. Less time spent on non-important, non-occurring issues means less cash burnt, which means more new features shipped to market in a much shorter period.
We believe that intelligent QA automation solutions and strategies will be infused into all aspects of QA in the future, beginning to transform how QA automation is performed. Payoda Technologies is a global IT solution provider that can help an organization implement the best QA practices into production. Talk to us now!
Authored by Mohan Bharati
