Educational Blog

How to Do Performance Testing

Learn how to plan, run, and analyze practical performance tests for apps and APIs.

Performance testing is the part of quality engineering that tells you whether a system can survive real demand. Functional tests prove that a checkout works, a search endpoint returns results, or a dashboard loads correctly. Performance tests answer a different question: will those same things still work when 10 users become 10,000, when latency spikes, when a database slows down, or when the cloud autoscaling policy reacts too late?

If you want a practical way to learn the process, the video above is a good starting point because it focuses on the mechanics of running performance tests with k6. This article expands that idea into a complete workflow you can use for web apps, APIs, and services of different sizes.

What performance testing actually measures

Performance testing is not one test. It is a family of tests that measure how software behaves under load, over time, and under stress. The most useful metrics usually include:

  • Response time or latency
  • Throughput, often requests per second or transactions per second
  • Error rate
  • Resource consumption such as CPU, memory, disk, and network
  • Concurrency and saturation thresholds

The important thing is that these metrics are interpreted together. A system can have good average latency and still be failing if the tail latency is terrible. It can also have high throughput while silently dropping requests. Good performance testing looks at the whole picture.

Start with a clear goal

Before you open any tool, define the business or technical question you are trying to answer. This prevents random load testing that produces charts but no decisions.

A few common goals:

  1. Verify the current production-like capacity of a service.
  2. Find the breaking point of a new feature before release.
  3. Compare two versions of an endpoint or query.
  4. Validate that a recent infrastructure change did not introduce a regression.
  5. Establish a baseline for future performance work.

A good goal is specific enough to drive the test design. For example, ?Can the login API sustain 500 requests per second with p95 latency under 300 ms and error rate under 1% for 30 minutes?? is much better than ?Test login performance.?

Choose the right type of test

Different questions require different test shapes. The table below gives a quick map.

Test typeMain questionTypical shape
Load testCan the system handle expected traffic?Ramp to expected load and hold
Stress testWhere does the system fail?Push past expected load until it breaks
Spike testWhat happens when traffic jumps suddenly?Fast increases and decreases
Soak testDoes it stay healthy over time?Moderate load for a long duration
Baseline testWhat is normal performance today?Repeated controlled runs

If you only do one test, make it a load test against the most important user path. If you can do two, add a soak test, because slow memory leaks, cache churn, and connection pool problems often appear only after the system has been under pressure for a while.

Build the test around real user behavior

The best performance test is shaped like production traffic, not like a lab fantasy. That means your scenarios should reflect how people actually use the system.

For a typical web application, identify a small set of critical journeys:

  • Home page or landing page
  • Login or session creation
  • Search or browse
  • Key write action such as checkout, submit, save, or publish
  • Reporting or dashboard views

Then decide what percentage of traffic each journey gets. If 70% of users only browse and 30% perform writes, your load should reflect that mix. Also include realistic think time. Real users pause between actions. A test that fires one request every 200 ms from the same virtual user is usually too artificial and can distort the results.

A practical workflow

A solid workflow keeps the test honest and repeatable.

1. Establish a baseline

Run the system under light, controlled traffic first. Capture normal latency, throughput, and CPU usage. This gives you a comparison point later.

2. Prepare the environment

Make sure you know what is being tested:

  • Which version of the code is deployed
  • Which configuration is active
  • Whether caches are warm or cold
  • What databases, queues, and third-party services are involved
  • Whether monitoring and logging are enabled

If the test environment is too different from production, document the differences clearly. Otherwise the results may not be useful.

3. Define pass and fail criteria

Decide in advance what success looks like. Common criteria include:

  • p95 latency below a threshold
  • Error rate below a threshold
  • No resource exhaustion
  • No unbounded queue growth
  • Recovery within a defined period after a spike

4. Run the test in stages

A ramp pattern is often safer and more informative than an immediate jump to full load. For example:

  • 5 minutes at 10% of target
  • 10 minutes at 50% of target
  • 20 minutes at 100% of target
  • 10 minutes at 120% of target for margin testing

This lets you see where degradation begins instead of only seeing the final failure state.

5. Correlate app metrics with infrastructure metrics

Do not rely on a single chart. When latency rises, ask whether the bottleneck is CPU, thread contention, database connections, GC pauses, lock contention, I/O wait, or an upstream dependency.

The fastest way to get value from a test is to connect the load generator output with application traces and host-level telemetry.

What to watch during a run

During the test, keep an eye on the patterns, not just the absolute numbers.

  • Flat throughput with climbing latency often means saturation.
  • Rising error rate can indicate connection pool exhaustion, timeout cascades, or rate limits.
  • CPU near 100% for long periods suggests compute pressure.
  • Memory growth without recovery may hint at leaks or cache bloat.
  • DB connection counts that never fall can reveal pool misconfiguration.

A useful habit is to write down the exact moment degradation begins. That timestamp makes it easier to inspect logs, traces, and metrics from the same window.

Tooling: k6, JMeter, or something else?

You do not need the perfect tool to start. You need a tool that can model your traffic and produce trustworthy results.

ToolBest forTradeoff
k6Scriptable API and web testingRequires comfortable scripting
JMeterBroad protocol support and UI-driven setupHeavier and easier to misconfigure
GatlingCode-based scenarios and strong reportingScala-centric workflow
LocustPython-based distributed testingNeeds careful worker coordination

The video focus on k6 makes sense for many teams because it is lightweight and easy to automate in CI. That said, the right tool is the one your team will actually maintain. A simple, repeatable test in a less fashionable tool is more useful than an abandoned benchmark in a trendy one.

Example structure for a basic test plan

A simple performance test plan might look like this:

  1. Document the target endpoint or user flow.
  2. Define the expected traffic profile.
  3. Select the environment and validate observability.
  4. Create one or two realistic scenarios.
  5. Run a baseline test.
  6. Run a load test with a ramp and hold.
  7. Capture bottlenecks and verify the root cause.
  8. Tune the system.
  9. Repeat the test until the improvement is measurable.

This loop is where performance work becomes engineering work. The goal is not to collect one dramatic graph. The goal is to improve a system and prove the improvement with data.

Common mistakes to avoid

Many first-time performance tests fail to answer the real question because of a few predictable errors.

  • Testing only one endpoint while ignoring the full user journey
  • Using unrealistic data distributions
  • Running without monitoring the server
  • Ignoring warm-up effects
  • Comparing tests with different datasets or cache states
  • Letting the load generator become the bottleneck
  • Using average latency instead of percentiles

If your test runner saturates before the application does, the result is invalid. Always check the generator itself, especially if you are running from a single machine.

How to analyze the results

After the test, use a structured review:

Step 1: Look at the user-facing metrics

Start with latency, throughput, and errors. Those are the metrics customers feel first.

Step 2: Check the server-side signals

Compare the timing of latency spikes with CPU, memory, GC, I/O, database saturation, queue depth, and upstream dependencies.

Step 3: Identify the bottleneck

Ask what resource became constrained first. The answer should guide the fix.

Step 4: Validate the fix with a repeat run

After tuning, rerun the same test shape. If the result improved, keep the comparison artifact. That becomes your baseline for later work.

A simple improvement loop

Performance work is usually iterative. A typical cycle looks like this:

  • Run a test
  • Find the bottleneck
  • Change one thing
  • Re-test
  • Compare the before and after numbers

One change at a time is important. If you change the database, cache settings, and application code all at once, you may see improvement but not know why.

When to stop tuning

You do not need infinite optimization. Stop when the system meets the current business requirement with a reasonable safety margin and no obvious instability. In many teams, the right question is not ?Can we make it faster?? but ?Is it fast enough, stable enough, and cheap enough for the expected demand??

That framing keeps performance testing connected to product reality.

A concise checklist

Use this before every meaningful test:

  • Define the goal and success criteria
  • Match the traffic mix to real usage
  • Confirm monitoring is ready
  • Make the environment details explicit
  • Include a ramp and a hold period
  • Watch percentiles, not just averages
  • Correlate app and infrastructure data
  • Repeat after each fix

Final takeaway

Performance testing is a practical discipline, not a ceremonial one. The best results come from realistic traffic, clear thresholds, and disciplined analysis. Start with one important user journey, measure it under controlled load, and use the evidence to improve the system. If you keep repeating that loop, your performance tests become a reliable engineering tool instead of a one-off benchmark exercise.

Written by

sasqag.org Editorial Team

Editorial team

sasqag.org publishes practical how-to guides and educational articles with clear steps and useful context.