There are typically 3 types of automated tests that are run in a CI/CD pipeline
1. Sprint Level Tests
- These are new automated tests that have been written to test the functionality of the Sprint. They should be contained in a
separate test suite that runs once a day to ensure the newly implemented functionalities are working as expected. - They are later merged into the regression tests suite.
In the acceptance testing phase, usually the teams have a high level acceptance test plan that ensure the critical functionality of the systems are still working as expected afer the new
features have been merged into the main code branch. Also, the smoke and regression tests are run again in parallel.
2. High Level Smoke Tests
- These high level automated tests run on every code check in to ensure the critical functionality of the system is still working as expected. This could be a mixture of UI, API and Unit Tests. The point of this test is to get quick feedback about the system and it usually should finish running within 5 – 10 minutes.
3. Daily Regression Tests
- These tests are run to ensure the new code added to the system did not break existing functionalities. They are more detailed than smoke tests as they cover end-to-end flows through the system. They are usually run at least on a daily basis and probably multiple times before a release.
- Throughout this process, teams may continue to do manual scripted testing, exploratory testing and/or risk-based testing depending on their level of continuous testing maturity, application complexity and risk tolerance.
Source: Getting started with Automation (testIM E-Book). Click here to download.