Posted on

Understanding the Continuous Testing in DevOps

Continuous testing was originally proposed as a way of reducing waiting time for feedback to developers by introducing development environment-triggered tests as well as more traditional developer/tester-triggered tests. Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

[[Also Read: Challenges with Traditional Testing and the adoption of Continuous Testing]]

Following Key Points will help in understanding the Continuous Testing:

  • Continuous Testing’s primary goal is assessing business risk coverage
  • Continuous Testing provides instant insight on whether a release candidate is too risky to proceed through the delivery pipeline
  • Continuous Testing establishes a safety net that helps the team protect the user experience in accelerated development processes and avoid software failure headlines
  • Continuous Testing expects testing to be embedded within the development process, not tacked on at the end
  • Continuous Testing is seamlessly integrated into the software delivery pipeline and DevOps toolchain
  • Continuous Testing expects a stable test environment with valid test data to be available for each and every test run
  • Continuous Testing embraces everything from “shift left” (unit, component, coverage…) to “shift right” (monitoring/ APM, Testing in Production)
  • Continuous Testing involves executing the right set of tests at the right stage of the delivery pipeline—without creating a bottleneck
  • Continuous Testing delivers actionable feedback appropriate for each stage of the delivery pipeline
  • Continuous Testing evaluates each layer of a modern architecture at the appropriate stage of the delivery pipeline
  • Continuous Testing includes end-to-end tests that realistically assess the end-user experience across all associated technologies (front-end and back-end)
  • Continuous Testing’s tests must be broad enough to detect when an application change inadvertently impacts functionality that users have come to rely on
  • Continuous Testing reduces false positives by prioritizing robust, flexible modern test frameworks over brittle scripts
  • Continuous Testing involves continuously reviewing and optimizing the test suite to eliminate redundancy and maximize business risk coverage

source

Posted on

Continuous Testing Practices

  • Testing should be a collaboration of Development, QA, and Operations—aligned with the priorities of the line of business—within a coordinated, end-to-end quality process.
  • Tests should be logically-componentized, incremental, and repeatable; results must be deterministic and meaningful.
  • All tests need to be run at some point in the build pipeline, but not all tests need be run all the time.
  • Eliminate test data and environment constraints so that tests can run constantly and consistently in production-like environments.
  • To minimize false positives, minimize test maintenance, and more effectively validate use cases across modern systems with multi-tier architectures, teams should emphasize API testing over GUI testing.
Posted on

The 9 Best Tips to boost your Retail Mobile App Testing

In this post:

1. Test your User Experience application performance

2. Improve QA Environment availability

3. Test data complexity –

The key to solving this, based on our experience with large retail customers, is in applying these two methods:

– Implementation of service virtualization can help to address test data challenges.
– Use Rest API interfaces to configure and collect the current relevant test data.

These two solutions will help assure that test data is always valid, and minimize the time invested in investigating testing issues.

4. Combine API in your UI functionality tests – highly recommended that you combine an API with functional UI testing. Implementing this will eliminate setup overhead, and allow for your functional tests to stay short, atomic, and focused.

5. Test for Retail mobile app specific scenarios

6. Build a fast feedback pipeline – Building a fast feedback pipeline is important for every industry but it is especially important in a retail mobile app testing environment.

7. Handle device fragmentation

8. Test for accessibility

9. Test in the cloud

Read the complete article: https://experitest.com/mobile/the-9-best-tips-to-boost-your-retail-mobile-app-testing/

Posted on

Foundation for Effective Test Automation

Here are 5 best practices from @testcraft.io to consider when it comes to designing tests effectively:

Test Design Rule #1 – Keep Your Tests Short and Simple

When thinking of how to design your tests, make sure that they abide by the KISS principle: Keep It Simple, Stupid.

No matter what you’re testing, we recommend keeping your automated tests between 5-15 steps so that they’re simple and easy to understand

Test Design Rule #2 – Build Your Tests From Reusable Components

One great way to keep your tests simple is by reusing the same components when building them.

Test Design Rule #3 – Every Test Should Be Independent

Keeping your tests independent also prevents unnecessary test breakage. If feature B changes in an application under test, there should be no need to update the tests for feature A to ensure that it continues to run.

Test Design Rule #4 – All Tests Should Have a Purpose

Defining a purpose at the start of the design process and identifying that as a clear goal throughout the design will make sure that your tests continue to stay relevant and helpful.

Test Design Rule #5 – Don’t Forget the End User

Keeping your end users in mind will also help when mitigating risk. When designing your tests, focus on different ways a user can break it. 

Read detailed tips at https://www.testcraft.io/

Posted on

Prerequisites for Payment Gateway Testing

You need to do the following before you begin testing.

  1. Gather proper data for the dummy cards, including visa, master, etc.
  2. Gather payment gateway information, like PayPal.
  3. Gather error codes related to the payment gateway.
  4. Know the parameters that are passed through the payment gateway.
  5. Know the information that are passed through query string.
  6. Check the language of the payment gateway and the application.

References – https://www.360logica.com/blog/

Posted on

Payment Gateway Testing – Sample Test Cases

You can consider the following sample test cases for payment gateway testing.

  1. Change the language of the payment gateway during the payment process.
  2. Check whether the important components are retrieved or not after the payment.
  3. Check the response if the payment gateway stops responding.
  4. Check what happens in the back end and if the session ends.
  5. Check what happens if the payment fails.
  6. Check if the database stores the card details or not.
  7. Check the errors during the payment process.
  8. Check the response after turning the pop up blocker on and off.
  9. Check the buffer page between the payment gateway and application.
  10. Check what happens after successful payment.
  11. Verify the transaction process.
  12. Check the messages and verify it after successful payment process.
  13. Check the currency format of the payment.
  14. Check if the payment option opens the respective payment gateway after selection.
  15. Verify the default debit card option.

References – https://www.360logica.com/blog/

Posted on

[Tip] “Building Quality In” – Automation points to consider in while you prepare for the test strategy.

Building quality in means writing automated tests at multiple levels (unit, component, and acceptance) and running them as part of the deployment pipeline, which is triggered every time a change is made to your application, its configuration, or the environment and software stack that it runs on. Manual testing is also an essential part of building quality in: Showcases, usability testing, and exploratory testing need to be done continuously throughout the project.

[Reference: Continuous Delivery: Reliable Software Releases.. – Humble, Jez. ]

[bctt tweet=”Building quality in also means constantly working to improve your automated testing strategy.” url=”yes”]

While preparing the Test Strategy for any feature, consider the following points from automation perspective:

  • The automation approach for each layer – Unit, Integration, Regression etc.
  • Technical Debt (Automation Maintenance / Debt)
  • Running automated tests after each commit
  • Running automated tests as a part of deployment pipeline


Posted on

[Tip] How to Speed up the UI Automation tests?

When we ran our scenarios through the UI using Selenium it made our test suite slow.
What could you do to speed it up?
Often, the best way to speed up a UI test is to remove it, or rather, refactor it to a domain layer test that doesn’t go through the UI, but talks directly to the domain layer.
If you’re unable to do that, there are still things you can do to speed up UI tests. Start by
identifying what steps and hooks are slow.

Are you starting your web app in a Before hook for every scenario? Try starting your app only once, for all scenarios.

Are you launching a new browser for every scenario? Try reusing the browser instance.
Do your scenarios need to go through a lot of pages before you get to the UI you really want to test? Try to go directly to the page you need to test, and set up the data you need “underneath” the UI.

Many of these optimisations introduce a new problem – you might have state leaking between scenarios. When state leaks between scenarios they might pass when run in a certain order, but they might fail when run in a different order. This can make it very difficult and time consuming to fix failing scenarios. To avoid this we recommend you use Before hooks to explicitly clear state that might have been left over from the previous scenario.

Source/References: https://cucumber.io/