Posted on

Performance Testing Interview Questions – Part 1 (Scenario Based included)

Q: What is Endurance Testing & Spike Testing?

  • Endurance Testing – It is a type of performance testing where the testing is conducted to evaluate the behavior of the system when a significant workload is given continuously.
  • Spike Testing – It is a type of performance testing that is performed to analyze the behavior of the system when the load is increased substantially.

Q: What are the top 3 Response KPIs for Performance Testing?

  • Average response time – time to first byte or last byte
  • Peak response time – tells your longest cycle
  • Error rate – percentage of problems compared to all requests

Q: What Is Distributed Load Testing?

  • Distributed load testing is the process of simulating a very high workload of an enormous number of users using multiple systems. As a single system cannot generate a large number of threads (users), multiple systems are used for load testing. It helps to distribute the tests and the load.

Q: What is the difference between front-end and back-end performance testing? Which one is more important?

  • Front-end performance is concerned with how quickly text, images, and other page elements are displayed on a user’s browser page.
  • Back-end performance is concerned with how quickly these elements are processed by the site’s servers and sent to the user’s machine upon request. Remember: Front-end performance is the part of the iceberg above the water line, and back-end performance is everything underneath that you can’t see.
  • Both are important because they can both determine whether a user continues to use your application.

Q: You run a load test against a server with 4GB RAM and the results show an average response time of 30 seconds for a particular request. The production server has been allocated 8GB RAM. What would you expect the average response time of the same request to be in production given the same load?

  • While you may be tempted to answer that the response time would be halved to 15 seconds, reality is rarely that convenient. Response times are a factor of so much more than memory. Things like CPU utilisation, network throughput, latency, load balancing configuration, and just application logic are always going to influence load tests. You can’t assume a linear progression in response time just because you’ve upgraded one part of the hardware. This is why it’s important to load test against an environment that is as production-like as possible.

Q: After running a load test, the results show that 300 transactions were executed with a PASSED status and returning responses of HTTP 200. However, the DBA tells you that there were no records written to the database as a result of the transactions. How would you troubleshoot this?

  • Its important to add various counters and monitors DB servers too. + Its very important to add assertions to make sure a request is successful. In the above mentioned case, it can be a bug or it can be due to poor scripting too. Adding right assertions, and counters will help in troubleshooting.

Q: Have you heard about New Relic or Datadog?

  • These are the monitoring tools.

Q: What are the benefits of monitoring tools like – New Relic , Datadog, etc?

  • Monitoring tools lets you monitor all your infrastructure in one place by creating monitors that will continuously check the health of your infrastructure. Access everything you need in one dashboard and see the data as it comes in straight from your systems.
  • Establishes end-to-end application performance and stability, by identifying and leveraging preventative tasks on-demand.
  • Creation of active metrics associated with data response times, CPU usage, load-balance alarms, exceptions etc.

Stay Tuned For Part 2..

Posted on

Challenges with Traditional Testing and the adoption of Continuous Testing

Organizations now expect software development teams to deliver more, and more innovative, software within shorter delivery cycles. To meet these demands, teams have turned to lean approaches, such as Agile, DevOps, and Continuous Delivery, to try to speed up the systems development life cycle (SDLC). After accelerating other aspects of the delivery pipeline, teams typically find that their testing process is preventing them from achieving the expected benefits of their SDLC acceleration initiative. Testing and the overall quality process remain problematic for several key reasons:

[[Also Read: Understanding the Continuous Testing in DevOps]]

  • Traditional testing processes are too slow. Iteration length has changed from months to weeks or days with the rising popularity of Agile, DevOps, and Continuous Delivery. Traditional methods of testing, which rely heavily on manual testing and automated GUI tests that require frequent updating, cannot keep pace. At this point, organizations tend to recognize the need to extend their test automation efforts.
  • Even after more automation is added to the existing test process, managers still lack adequate insight into the level of risk associated with an application at any given point in time. Understanding these risks is critical for making the rapid go/no go decisions involved in Continuous Delivery processes. If tests are developed without an understanding of what the business considers to be an acceptable level of risk, it is possible to have a release candidate that passes all the available tests, but which the business leaders would not consider to be ready for release. For the test results to accurately indicate whether each release candidate meets business expectations, the approach to designing tests must be based on the business’s tolerance for risks related to security, performance, reliability, and compliance. In addition to having unit tests that check code at a very granular bottom-up level, there is a need for a broader suite of tests to provide a top-down assessment of the release candidate’s business risk.
  • Even if testing is automated and tests effectively measure the level of business risk, teams without a coordinated end-to-end quality process tend to have trouble satisfying the business expectations within today’s compressed delivery cycles. Trying to remove risks at the end of each iteration has been shown to be significantly slower and more resource-intensive than building quality into the product through defect prevention strategies such as development testing.
  • Organizations adopt Continuous Testing because they recognize that these problems are preventing them from delivering quality software at the desired speed. They recognize the growing importance of software as well as the rising cost of software failure, and they are no longer willing to make a tradeoff between time, scope, and quality.

Continuous testing has evolved to become an important phase in modern application development and delivery. Its very important for today’s testers to understand the Shift Left and Continuous Testing practices. It is not possible achieve Shift Left / Continuous Testing without enhancing the skill set to SDET/Full Stack QE

Posted on

Understanding the Continuous Testing in DevOps

Continuous testing was originally proposed as a way of reducing waiting time for feedback to developers by introducing development environment-triggered tests as well as more traditional developer/tester-triggered tests. Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

[[Also Read: Challenges with Traditional Testing and the adoption of Continuous Testing]]

Following Key Points will help in understanding the Continuous Testing:

  • Continuous Testing’s primary goal is assessing business risk coverage
  • Continuous Testing provides instant insight on whether a release candidate is too risky to proceed through the delivery pipeline
  • Continuous Testing establishes a safety net that helps the team protect the user experience in accelerated development processes and avoid software failure headlines
  • Continuous Testing expects testing to be embedded within the development process, not tacked on at the end
  • Continuous Testing is seamlessly integrated into the software delivery pipeline and DevOps toolchain
  • Continuous Testing expects a stable test environment with valid test data to be available for each and every test run
  • Continuous Testing embraces everything from “shift left” (unit, component, coverage…) to “shift right” (monitoring/ APM, Testing in Production)
  • Continuous Testing involves executing the right set of tests at the right stage of the delivery pipeline—without creating a bottleneck
  • Continuous Testing delivers actionable feedback appropriate for each stage of the delivery pipeline
  • Continuous Testing evaluates each layer of a modern architecture at the appropriate stage of the delivery pipeline
  • Continuous Testing includes end-to-end tests that realistically assess the end-user experience across all associated technologies (front-end and back-end)
  • Continuous Testing’s tests must be broad enough to detect when an application change inadvertently impacts functionality that users have come to rely on
  • Continuous Testing reduces false positives by prioritizing robust, flexible modern test frameworks over brittle scripts
  • Continuous Testing involves continuously reviewing and optimizing the test suite to eliminate redundancy and maximize business risk coverage

source

Posted on

Mutation testing with Stryker

Test your tests with mutation testing.

What is mutation testing?

Bugs, or mutants, are automatically inserted into your production code. Your tests are run for each mutant. If your tests fail then the mutant is killed. If your tests passed, the mutant survived. The higher the percentage of mutants killed, the more effective your tests are.

It’s that simple.

Are you still confused? Why not take a look at our example page and try it out yourself?

But wait, what about code coverage?

Well… code coverage doesn’t tell you everything about the effectiveness of your tests. Think about it, when was the last time you saw a test without an assertion, purely to increase the code coverage?

Imagine a sandwich covered with paste. Code coverage would tell you the bread is 80% covered with paste. Mutation testing, on the other hand, would tell you it is chocolate paste and not… well… something else.

Meet: Stryker

Sounds complicated? Don’t worry! Stryker has your back. It uses one design mentality to implement mutation testing on three platforms. It’s easy to use and fast to run. Stryker will only mutate your source code, making sure there are no false positives.

Checkout this example which demonstrates why code coverage is not sufficient and why there is a need for tools like Stryker – https://stryker-mutator.io/example

Stryker is an open source tool. Give this a try: https://stryker-mutator.io/

Posted on

Continuous Testing Practices

  • Testing should be a collaboration of Development, QA, and Operations—aligned with the priorities of the line of business—within a coordinated, end-to-end quality process.
  • Tests should be logically-componentized, incremental, and repeatable; results must be deterministic and meaningful.
  • All tests need to be run at some point in the build pipeline, but not all tests need be run all the time.
  • Eliminate test data and environment constraints so that tests can run constantly and consistently in production-like environments.
  • To minimize false positives, minimize test maintenance, and more effectively validate use cases across modern systems with multi-tier architectures, teams should emphasize API testing over GUI testing.
Posted on

Free – Automation Tutorials – BrowserStack Summer of Learning

 BrowserStack Summer of Learning, a free 5-part webinar series designed to help QA and engineering teams of all sizes learn and scale test automation. Whether you are a beginner QA analyst dabbling with exploratory testing or a 50 releases-a-day veteran, Summer of Learning is a go-to for everybody.

The series begins with an introduction, first to Selenium and then to BrowserStack, and gradually takes you through the process of moving from manual to automated testing. It covers industry trends in testing and features testing stalwarts such as The Weather Channel, who release to millions of users each day.

BrowserStack Summer of Learning has the following episodes:

Episode 1 — The Basics: Getting started with Selenium: An introduction to Selenium, how to set up/write your first test scripts, and how to pick the right framework. This is a great introductory session for those looking to learn test automation in 60 minutes.

Episode 2 — Introduction to BrowserStack Automate: In this episode, you’ll learn how to set up and run your first test with Automate, how to test on various real devices and browsers on the BrowserStack Real Device cloud, how to test your local instance on the cloud, and how to collaborate and debug better.

Episode 3 — Continuous testing at scale: You’ll learn how to build an efficient, well-integrated CI pipeline that helps release quality software at speed. You’ll also learn how to use BrowserStack to deploy faster and listen to stories from great companies like The Weather Channel, who release to millions of users every day.

Episode 4 — Selenium + BrowserStack at scale: In Episode 4, David Burns, core contributor to Selenium will explain how to plan parallelization more effectively to achieve faster build times, the best ways to maintain test hygiene while scaling your team or automation suite, and how to monitor test feedback effectively.

Episode 5 — Testing for a mobile-first market: There are 9,000 distinct mobile devices in the market—and you most definitely can’t test on them all. But with this episode, you’ll learn the best strategy to pick the right devices for testing your website or mobile app.

Posted on

The 9 Best Tips to boost your Retail Mobile App Testing

In this post:

1. Test your User Experience application performance

2. Improve QA Environment availability

3. Test data complexity –

The key to solving this, based on our experience with large retail customers, is in applying these two methods:

– Implementation of service virtualization can help to address test data challenges.
– Use Rest API interfaces to configure and collect the current relevant test data.

These two solutions will help assure that test data is always valid, and minimize the time invested in investigating testing issues.

4. Combine API in your UI functionality tests – highly recommended that you combine an API with functional UI testing. Implementing this will eliminate setup overhead, and allow for your functional tests to stay short, atomic, and focused.

5. Test for Retail mobile app specific scenarios

6. Build a fast feedback pipeline – Building a fast feedback pipeline is important for every industry but it is especially important in a retail mobile app testing environment.

7. Handle device fragmentation

8. Test for accessibility

9. Test in the cloud

Read the complete article: https://experitest.com/mobile/the-9-best-tips-to-boost-your-retail-mobile-app-testing/