The Evolution of Software Testing
The summer of 2004, I graduated with a bachelor’s degree, majoring in the coveted “Computer Science and Engineering” field. I was hired by a multi-national company on campus, and started my software career soon after. My first role was that of a Software Integration and Test Engineer which I immensely enjoyed, before being bit by the programming bug. I attribute my good coding skills to my brief experience as a QA engineer, since consciously I started thinking from the perspective of a tester and wrote different test cases to break the code I have built. Software testing, which is an integral part of software development process, has evolved over the decades.
Software testing is the process of running a program with the intention of finding errors — Glenford J. Myers.
In 1957, Charles L. Baker distinguished program testing from debugging in his review of the book Digital Computer Programming by Dan McCracken. The review is published in the journal Mathematical Tables and Other Aids to Computation. In 1979, Glenford J. Myers introduced the separation of debugging from testing. Although his attention was on breakage testing, it illustrated the desire of the software engineering community to separate fundamental development activities, such as debugging, from that of verification.
In this article, I have presented the various best practices we’ve adopted and upcoming trends in software testing.
Waterfall Model
In most of the teams I’ve worked with, software development and testing were treated as mutually independent activities. For the most part, we adopted the traditional waterfall model for development and testing activities. Testing activities begun only after the software was fully developed and all the features were implemented. QA engineers were not very actively involved during the requirement analysis phase and had very limited interactions with business stakeholders and product management team. They were heavily dependent on the design documents and information gathered from developers who wrote the code. The lack of insight into the requirements and customer expectations led to limited testing strategies and the testing plan was not comprehensive. Often times, issues were raised by support during the UAT phase since the team wasn’t aware of customer specific use case scenarios. Also, most of the testing activities were manual and simulation tools were used only for performance and stress testing phases. This resulted in a long QA testing cycle for any major release/project. Furthermore, we ended up having additional release cycles, just to incorporate customer’s feedback on the software we had built.
Agile Model
The need of the hour was to move towards agile principles in order to support adaptive planning, evolutionary development, early delivery, continual improvement and flexible responses to change. By adopting agile culture, we were able to create self-organizing and cross-functional teams, and product owners became an integral part of our scrum team. With this agile approach, QA engineers were more involved during the requirements validation phase and when developers were coding, QA were writing the test cases. We were able to implement and test the new features in a continuous and iterative approach.
Best Practices for Continuous Testing in Agile
As we evolved as a mature scrum team, we established and implemented the following best practices.
Static Code Analysis: Static code analysis is the continuous inspection of code quality to detect bugs, code smells, duplicate code, code complexity and security issues. This type of analysis addresses weaknesses in source code that might lead to vulnerabilities. It is done by analyzing a set of code against a set of coding rules. This may also be achieved through manual code reviews, but using automated tools is much more effective.
Unit Tests: Unit tests are automated tests written and executed by developers to ensure that a section of an application meets its design and behaves as intended. Unit testing finds problems early in the development cycle. The process of writing a thorough set of tests forces the author to think through inputs, outputs, and error conditions, and thus more crisply define the unit’s desired behavior. Code can be difficult to unit test if poorly written, thus unit testing force developers to structure functions and objects in better ways.
Test Suite Automation: Iterative way of testing paved way for automating the tests that were repetitive in nature. Test automation makes use of special software tools and frameworks to control the execution of tests and then compares actual test results with predicted or expected results. The QA engineers were further empowered to carry out their tasks with improved efficiency. Automated testing not only saved us time and money by making testing more efficient but also improved testing accuracy and increased test coverage. By reusing our automated test suites for Sanity and Regression testing we were able to further reduce the non-functional testing cycles for every product release. We also made progress in automating our Performance tests.
Automated Builds: As part of our test automation efforts we were able to automate our smoke tests. Smoke testing is used to to determine the stability of your build. This led to automating our software builds. Build automation is the process of automating the creation of a software build and the associated processes which includes compiling source code, packaging binary code and running automated smoke tests. We achieved our goal to release a build as fast as possible and automatically validate its stability.
Penetration and Security testing: A penetration test or pen test, is an authorized simulated cyberattack on a computer system, performed to evaluate its security. The tests are performed to identify both vulnerabilities, including the potential for unauthorized parties to gain access to the system’s features and data, as well as strengths, enabling a full risk assessment to be completed.
Continuous Integration and Continuous Deployment Pipeline
Build and test automation is considered as the first step in moving toward implementing a CI/CD pipeline. A CI/CD pipeline is a series of steps that must be performed in order to deliver a new version of the software which typically includes build, test, release, deploy, validation and compliance checks. The CI/CD pipeline introduces monitoring and automation to improve the software development process, particularly at the integration and testing phases, as well as delivery and deployment phases. Although, it is possible to manually execute each of the steps of a CI/CD pipeline, the true value of CI/CD pipelines is realized through automation. The CI/CD pipeline forms the backbone of modern day DevOps & Site Reliability Engineering (SRE) approaches.
Doing QA without a QA Team
When people talk about software testing, they incorrectly assume that every organization has a dedicated team of QA engineers whose main job is to test software quality. The reality is that not all organizations have a QA team. Some companies just don’t see the value in hiring QA engineers, others might believe that in a DevOps world, you no longer need to have a dedicated QA team and some companies are just too small to have the resources to hire full-time QA. However, lacking full-time QA team does not mean your organization has to compromise on software quality.
I have worked with development teams where we have filled the QA gap with developers, business analysts and IT staff we had at hand. When we don’t have a dedicated QA team, we expect our developers, product owners and IT engineers to play an active role and be more hands-on with software testing and software quality optimization. Hence, it becomes critical to make quality assurance the responsibility of everyone involved in software engineering.
- Developers are expected to write automated unit, integration and end-to-end tests. They need to set-up test environments and collaborate with product owners to test the new features before launch.
- IT engineers need to contribute to “shift-right” testing. Shift right testing means testing in production. Along with monitoring applications, IT engineers can play a more hands-on role in QA by using production monitoring insights to improve overall software quality.
- From a cultural perspective, every developer, product owner and IT engineer should share the ownership of software quality. Ideally, this should happen even if you have a full-time QA team.
- It’s difficult to expect developers and IT engineers to own QA if they think it’s more work and time. Hence, it is critical to empower them with tools to make QA efficient, such as test automation, extended debugging and parallel testing.
- Developers and IT engineers need to establish and master a CI/CD pipeline.
Future Trends in Software Testing
The exponential and unprecedented changes in technology affect the way we develop, validate and deliver the software. Software testing roughly accounts for 30% of the total project efforts. Hence, we must consistently innovate and revamp our testing practices and tools to address the challenges of achieving quality at speed amid the increasing complexity of systems, environments and data. I have listed below the top trends in software testing, many of which have already emerged over the past few years.
Artificial Intelligence: Simply put, Artificial intelligence (AI)is the science behind computers performing tasks that were traditionally performed by humans. AI is the best choice for developers looking for faster deployments with insufficient infrastructure, since the arduous work is left to the AI powered automation which leaves only 20% of the testing work to human ingenious and cognitive ability. AI tools can help QA teams design tests from scratch with little-to-no human supervision. AI and machine learning can also help to update existing testing suites, removing redundant cases.
API and Services Test Automation: As organizations are increasingly migrating from a monolithic to a microservices architecture the use of APIs (application programming interfaces) is growing day by day. API and services are being reused in more than one application or component. These changes, in turn, require the QA teams to test API and services independent from the application using them. When API and services are used across client applications and components, testing them is more effective and efficient than testing the client. We need to ensure that these APIs are communicating well with each other, as well as functioning independently. To make this process more efficient, test automation on the API and service level will definitely increase.
Robotic Process Automation (RPA): While automation was definitely on the rise in software testing space, another growing trend was RPA. RPA is a form of business process automation technology based on metaphorical software robots (bots). In traditional automation tools, a software engineer produces a list of actions to automate a task and interface to the back-end system using internal APIs or scripting language. In contrast, RPA systems develop the action list by watching the user perform that task in the application’s graphical user interface (GUI), and then perform the automation by repeating those tasks directly in the GUI. The typical benefits of robotic automation include reduced costs; increased speed, accuracy, and consistency; improved quality and scalability of production.
QAOps: Since QA engineers are now involved in every aspect of software development, their role is not limited to testing activities. IT firms which are inclined towards DevOps, are now leaning towards QAOps. QAOps is a practice that brings developers, QA engineers and operation teams together where continuous testing is combined with DevOps practices.
Script-less Test Automation: Script-less test automation is being practiced by leading industry software IT firms to maximize the scalability of their tests. It helps QA engineers to run their test cases automatically without worrying much about the coding and results in faster turn around.
Internet of Things (IOT) Testing: The IoT describes the network of physical objects that are embedded with sensors, software, and other technologies for the purpose of connecting and exchanging data with other devices and systems over the Internet. With IoT, QA engineers will have to be aware of ways to test and assess all the risks and threats releasing an IoT based tool. Easier processes should be developed to replicate the issues and mitigate them in a timely manner. Security, usability, device compatibility, data integrity, performance and scalability will be the core aspects IoT testing.
Assessing your own experience
- Do you have dedicated QA engineers on your team? What about other teams in your organization?
- What are the activities performed by your QA engineers? What are the best practices adopted by your team?
- Have you established a CI/CD pipeline? What are some of the tools and technologies used for the same?
- Do you expect developers on your team to perform all the testing activities? Do you have business analysts or product owners who own testing activities?
