Believe me Performance Testing can be 100% waste of time & money if performance tests are not realistic.
Always remember that Performance testing tool is a just a enabler to generate multiple user load traffic on the application under test. But if the traffic created by the load testing tool is not realistic, the performance test results cannot be representative of performance perceived by actual end users. The performance testing tool will not have intelligence to alert you if the workload used is wrong. Hence Performance tester has to be very careful while doing workload characterization before running performance tests.
It’s the job of Performance tester to rightly strategize Performance testing activities.
Here are some simple tips to run right & realistic performance tests
- Understand & capture max 5 to 6 key business critical use cases and/or traffic prone use cases to simulate peak hour traffic.
- Distribute the load across use cases / transactions as per realistic access pattern. Do not miss to analyze production user traffic logs (Adobe Omniture logs / Google Analytics logs / Web server access logs/etc) & mathematical validate your workload.
- If you are very serious about bringing in very realistic workload for a high critical application, incase of availability of production traffic logs, try comparing the traffic generated by actual users versus performance testing tool for the finalized workload model.The Inter-arrival time between requests for a internet / intranet web application will be Poisson or Exponential or Self-Similar distribution. So analyze the traffic generated by the performance testing tool to compare it with actual traffic pattern & tweak the workload model to create realistic traffic intensity.
- Configure randomized think times between transactions & iterations. This is important to make virtual users to behave realistic. Is your 1 virtual user equivalent to 1 real user? Ask this question before running your tests & report accordingly.
- Configure appropriate settings provided by the performance testing tool in order to simulate real browser behavior (like to do browser caching of static resources, request headers & cookie management). The tool settings should closely resemble the real end user browser in order to bring in realistic access from browser. Remember your performance testing tool does not use any Browser to render the web pages of application under test.
- Use client-side performance analysis tools to capture client-side browser performance separately. Use Performance testing tools (like JMeter) for performing server side performance analysis only.
- Include Business projections in your workload. If the application is expected to support 500 peak hour users, understand & include what is the business projection on user load for next 1 year / 2 years, etc based on business requirement.
- The workload should appropriately have logic implemented to perform login / logout transactions only once (if that’s the way actual end users access the application under test) , perform some of the transactions for 2-3 times (as per real time user behavior), some users might need to access a web page randomly, etc.
- Do not run 5 or 10 minutes load tests to make judgments. Run a load test with atleast 10-15 minutes of steady state load (constant load) excluding ramp up & ramp down test duration to conclude system supports a specific load level.
- Do not certify the application just by running few load tests. Run right type of tests to certify the system performance & scalability characteristics as per the realistic usage patterns.
- Use right Test data while carrying performance tests. Analyze what is the test data size on DB & projected DB volumes before planning tests in test environment,
- If the test environment is of smaller capacity than production, do not try to extrapolate the results for production environment without doing enough analysis on hardware differences (# number of CPUs used, CPU model, CPU speed, server type, RAM, etc) & deployment of architectural components.
- Do not guess your test results extrapolation strategy without referring to industry benchmarks (like TPC / SPEC) or using Queuing Models. Use appropriate techniques knowing the Pros / Cons of it.
Happy Performance Testing !