©1996- Aspire Systems
In a post-pandemic world, the expectations and demands of customers availing services has witnessed an exponential shift. For software developers, the challenge is to keep such customers satisfied with the smoothest user experience, or risk losing them to a plethora of competitors who have adapted to the changing demand.
“Speed of applications has changed; earlier it was monthly, now it has moved to daily and can also be on-demand. There is an evolution in various software processes. New processes and tools are driving faster releases for organizations,” said an expert and senior manager at Sauce Labs, the American company offering cloud-hosted automated testing platform.
This is exactly why adopting Quality Engineering (QE) practices for developing your product can provide that edge and also help build customer loyalty. With QE, quality is checked rigorously at every level of software development. This is an inversion of the age-old system of evaluating quality during final production stages. According to experts, early detection of defects in the software is beneficial as resolving them later in the software development life cycle (SDLC) is costlier and cumbersome.
Why you need Quality Engineering
Quality is no longer a typical concern of the testing team; indeed, quality engineering is a discipline of its own, the Sauce Labs expert said during the webinar ‘Embedding Quality Engineering in DevOps pipeline’ hosted by Aspire systems on September 17, 2018.
In this, the developer becomes a voice of the customer by testing on the development side. This is how it differs from the traditional Quality Assurance (QA). Actionable data is provided on a regular basis and complete test automation ensures that requirements, architecture and design are reviewed. On the whole, quality is built in from the start of the DevOps cycle.
How does QE achieve this? QE envisions incorporating continuous testing as a catalyst or a sandwich layer between continuous integration and delivery. This is a shift-left approach which is at the heart of the DevOps cycle.
For the customer to have a better experience, it is imperative that functional and non-functional testing is done and certified before the software is sent into production.
“So, during continuous testing, all aspects from functional to performance, security, API and web services is looked into. It has a shift left approach,” the Sauce Labs expert said.
Importance of Quality Engineering in DevOps
QE, which envisages a system of precise quality checks during the SDLC, is similar to the continuous integration and continuous delivery (CI/CD) that is at the heart of DevOps. Continuous testing is also part of the process.
Tools used for this have evolved from proprietary single stack available on the premises of the company and costing annual license and maintenance fees to the latest cloud-based and open-source ones.
In testing too, legacy methods of a single team running tests at the end of a development process and a lot of back-end code on expensive proprietary frameworks are no match for continuous testing processes which run through the development process.
With the new-age retail or insurance consumer operating applications from a multitude of platforms like android and IOS phones and different browsers, tests have to be run to ensure the software is compatible on every one of them. Continuous testing enables testing across platforms. Free, open-source frameworks are also available for the same.
Continuous testing also real-time analytics which can be scaled up to predictive analysis with incorporation of artificial intelligence and machine learning. However, implementing this would necessitate rigorous training of the teams involved.
Different dimensions of Quality Engineering:
Quality engineering broadly can be broken down into three facets: Functional testing, tools and processes and Non-functional testing.
Functional testing covers aspects like the application itself, data, API automation, resolution tests and mobile/other device testing.
Non-functional testing looks into the performance, load, security and how the app works on multiple platforms.
Tools and processes of QE include cloud based test automation, test automation framework, continuous integration and development and DevOps.
Like discussed earlier, apps in today’s day and age are released in different platforms.
“Earlier it was either mobile development or the web. It was like this even five years ago. But now mobile is a must, especially with responsive design and hybrid applications,” the Sauce Labs expert said.
This explains the need for several types of functional tests.
The expert pointed out that cloud-based test automation is now essential as the application needs to be scalable. For this, various test automation frameworks like Selenium and Appium and for mobile, Espresso and XUI tests are used.
However, the key is in non-functional testing areas, which, the expert said, organizations might struggle with.
This is mainly due to the ability to conduct different types of tests across multiple platforms.
Cloud based test-automation
Automated testing on cloud has helped companies scale down their infrastructure and instead focus on scaling up testing across areas. The need to have an internal lab or data center is eliminated. “It is a highly cost-effective solution, even for smaller organizations,” the expert said.
Cloud based automated testing also helps manage features and trigger appropriate action items across all different areas. It will also help build pipelines that span across various functions of development. The goal, the expert says, is how cloud based test automation can implement CI/CD within your organization.
Tools for continuous testing
How continuous testing can make or break a DevOps environment:
The crux of DevOps is to ensure that the processes are made quicker and efficient, but quality can take a hit, unless QE is integrated.
“The DevOps process might be complete, but how successful it is, is primarily determined by continuous testing which makes a key decision-making role by providing a certification or an assurance of what is the percentage of applications that are stable, fit and ready to be released to the production for further rounds and real-time usage,” said Jayanthi Jayaraman, test automation consultant in Aspire systems, during the webinar.
In continuous testing, the engagement starts early in the software’s development lifecycle. It also has a repeatability or repetitive execution feature, which would be cumbersome for on-demand testing. The convenience of continuous testing was underscored during the pandemic; the process allows teams which are geographically distributed. For on-demand testing, this would be a difficult scenario.
“Continuous testing gives us a great helping hand in terms of coordinating with multiple stakeholders and technical functional teams and giving them another level of confidence, on whatever has been introduced, what is existing and how it has not been impacted as far as the portion of whatever is developed is working fine,” Jayanthi said.
During a scenario of continuous delivery, frequent code checking is mandatory. If multiple teams are split up across the world, there may be no control over who is doing what.
This would not make business sense in today’s times where the challenge is addition of on-demand functionalities on a daily basis to keep pace with the market’s requirements.
So, this is where continuous testing helps in the identification of all possibilities for test automation which helps in the early detection of defects and ultimately minimizing them.
“When you automate the system, it triggers the test in terms of parallel threads across machines and browsers and we get a chance to cover as many user platforms as possible,” Jayanthi said.
Continuous testing best practices: How using cloud can be advantageous for DevOps
For continuous testing, a central code repository is necessary. It might be proprietary or popular ones like git lab or git hub.
This is where using cloud to implement DevOps can be advantageous. “You can spin up actual environments that mimic the actual production environment that allow you to test against something that’s actually real,” the Sauce Labs expert said.
He further added that cloud helps usage of other best practices like building push code on a CI server.
Cloud will also help get as much test coverage as possible with a quick release as well. This prevents testing from becoming a bottleneck for the organization.
With a cloud-based solution like Saucelabs, live test bug reproduction can be done on any platform. “It is a necessity to have a cloud-based solution to implement test automation at a high level,” the expert said.
Benefits of cloud-based continuous testing
A case study where continuous testing using Sauce labs integration was used showed that an exercise which earlier took seven desktops, seven devices and 12 hours, was finished in 15 minutes. Similar results were seen in regression and integration.
Automation has enabled faster execution. An integrated test run across multiple windows and platforms like android and IOS gave a clear functional sync of the output.
“What was happening for hours together in the pre-automation phase took only minutes in post-automation, (less than 1/10th of the time)” Jayanthi said.
Additionally, work was completed with the same team strength; no expansion was required to get into newer functionalities.
Complete QA focus moved on to parallel strains of new functionality testing, Jayanthi said. Parallel automation, integration, supporting the continuous testing and continuous delivery in a DevOps mission.; this was accomplished with an optimal team size by bringing in this integrated solution of Aspire Framework for Test Automation (AFTA) integrated with Saucelabs cloud.
Whatever we had in terms of duration, the initial development effort was to setup the framework and just bring in the solution, she said. It later became a complete reusable piece of code with minimal maintenance effort, Jayanthi said.
The major costs involved were in procurement of the infrastructure, maintenance and scaling it up or down. “There will be major releases, minor releases and patch deployments, depending on which the number of fragments that we test also varies. A cloud infra like Saucelabs will help us to scale up or down based on number of platforms that really need to be tested.
Diversified testing requirements were accomplished with such an integration, Jayanthi added.
In conclusion, quality engineering can accomplish challenges of diversified testing requirements. Moving to an integrated, cloud-based testing solution helps in better defect management which ultimately brings a higher quality product which can run across multiple platforms like mobile and different browsers. The system also generates data on the go which can be used for predictive analytics. The Return on Investment (ROI) associated with adopting such solutions is also better than traditional testing methods.
Aspire's framework for Test Automation, AFTA 3.0 has been an enabler for many Enterprise DevOps teams across the globe with their continuous testing process. Click below to schedule a demo of Aspire's AI and ML powered Selenium based Test Automation Framework, AFTA 3.0 for multi-layer testing.