Quantum programs are not tested like ordinary deterministic functions: a correct circuit can produce different samples on every run. This guide gives you a reusable workflow for testing quantum programs with statevector assertions, probability-distribution comparisons, expectation-value tolerances, controlled randomness, shot planning, regression tests, and ideal-versus-noisy validation.
Overview
The central testing question is not always, “Did the circuit return the expected bit string?” Instead, ask which observable should remain stable. Depending on the circuit, that may be an exact statevector, a measurement distribution, an expectation value, an energy estimate, or a performance metric such as success probability.
A practical quantum testing strategy separates three layers:
- Construction: Did the program create the intended circuit, gates, measurements, parameters, and classical-register layout?
- Semantics: Does the circuit represent the expected quantum operation or output distribution?
- Execution: Does the chosen simulator or hardware produce results consistent with the expected behavior under its noise, shot, and compilation settings?
Test the first two layers with small, controlled cases before investigating hardware results. A statevector simulator can often expose a wrong gate, qubit index, or phase convention quickly. Measurement-based tests then verify what users actually observe. Hardware tests should be treated as statistical experiments, not exact equality checks.
For a broader view of circuit depth and fidelity, see How to Benchmark a Quantum Circuit. When a test fails after compilation, compare the original and transpiled circuits using Qiskit Transpilation Explained.
A useful test record
For each test, record the circuit or circuit-building function, SDK and simulator versions, number of qubits, parameter values, transpilation settings, random seeds where supported, shot count, backend, noise model, and comparison tolerance. Without this context, a failure may be impossible to reproduce.
Checklist by scenario
Scenario 1: Testing a small circuit with a known state
Use a statevector check when the circuit is small and the expected quantum state can be calculated or constructed directly. This is appropriate for basic gates, Bell-state preparation, basis changes, and small algorithm components.
- Build the circuit without measurements, or keep a measurement-free copy for analysis.
- Simulate the statevector using the framework’s current statevector interface.
- Compare amplitudes with a numerical tolerance rather than exact equality.
- Account for global phase when the physical state is unchanged by a common phase factor.
- Check normalization and, where relevant, expected marginal probabilities.
A conceptual assertion might look like this:
actual = simulate_statevector(circuit)
expected = expected_state(parameters)
assert equivalent_up_to_global_phase(actual, expected, atol=1e-8)The exact helper names differ between SDK releases. The testing principle is stable: compare numerical arrays with an absolute tolerance and, for values that vary in scale, a relative tolerance as well. Do not use string representations of statevectors as test fixtures.
Scenario 2: Testing measured output distributions
Use a shot-based test when the circuit’s public behavior is a collection of measurement outcomes. Prepare the circuit, add measurements in the same order used by the application, run enough shots for the intended confidence, and convert counts into frequencies.
- Define the expected probability for each outcome, including outcomes expected to have probability zero.
- Run the circuit with a documented shot count.
- Normalize counts to frequencies.
- Compare observed and expected distributions with a distance or per-outcome tolerance.
- Report the largest deviation and the outcomes responsible for it.
For example, an ideal Bell-state circuit should concentrate probability on two correlated outcomes. A test should not require the two counts to be identical. It should check that the forbidden outcomes remain sufficiently rare and that the observed proportions are compatible with the expected distribution.
For low-probability outcomes, absolute thresholds can be misleading. A distribution-distance measure, such as total variation distance, can provide a more useful overall check. For individual outcomes, use a tolerance that reflects the shot count and the risk of false alarms.
Scenario 3: Testing expectation values
Variational algorithms, chemistry workflows, and quantum machine-learning circuits often return expectation values rather than bit strings. Test the value against a reference with a tolerance:
assert abs(actual_energy - reference_energy) <= energy_toleranceChoose the tolerance from the application, not from convenience. A tolerance for a debugging test on an ideal simulator may be tight, while a hardware experiment may need to allow sampling error, device noise, and measurement error. Keep separate thresholds for ideal simulation, noisy simulation, and hardware validation.
Scenario 4: Reproducing a failure
Control every source of randomness that the framework exposes. Set seeds for circuit generation, parameter initialization, simulator sampling, transpilation, and noise-model construction where supported. Treat a seed as diagnostic metadata, not as proof that a program is deterministic across machines or SDK versions.
Also pin dependency versions in the test environment and save the serialized circuit or a stable circuit description. A seed may produce a different result after a simulator or compiler changes its internal implementation.
Scenario 5: Testing after a circuit or compiler change
Maintain a layered regression suite:
- Smoke tests: Small circuits that run quickly and verify imports, circuit construction, and basic execution.
- Semantic tests: Known states, distributions, and expectation values that verify algorithm behavior.
- Compilation tests: Checks that required gates, measurements, qubit counts, and connectivity assumptions survive transpilation.
- Statistical tests: Repeated shot-based checks against accepted baselines.
- System tests: End-to-end tests covering parameter binding, job submission, result parsing, and application output.
Store compact reference data, such as expected probabilities or expectation values, rather than large raw result files unless raw results are needed for auditability.
Scenario 6: Comparing ideal, noisy, and hardware execution
Run the same logical circuit through at least two validation paths: an ideal simulator for semantic correctness and a noisy simulator or hardware backend for execution realism. Keep the comparison fair by documenting shots, measurement mapping, transpilation, and noise assumptions.
If hardware results differ from the ideal baseline, do not immediately conclude that the circuit is incorrect. First compare it with a noise model or calibration-aware baseline. Then inspect circuit depth, two-qubit operations, measurement assignment, queue-time changes, and backend connectivity. The guide to Quantum Error Mitigation can help when the circuit is logically sound but execution quality is limited.
What to double-check
Qubit and bit ordering
Different frameworks and result formats can display qubits and classical bits in different orders. Test a simple basis-state circuit to confirm how the backend labels outcomes. A reversed bit string can make a correct circuit appear broken.
Measurement placement
Measurements can change what is being tested. Keep a measurement-free circuit for statevector validation and a measured copy for shot-based validation. Confirm that measurement operations do not accidentally overwrite or omit the classical bits consumed by your result parser.
Global phase
Two statevectors that differ only by a common global phase represent the same physical state. A direct element-by-element comparison may flag a false failure. If phase matters to a later interference calculation, compare the complete operation or the downstream observable rather than assuming every phase difference is irrelevant.
Shot count and statistical thresholds
More shots reduce sampling uncertainty but increase runtime and cost, especially on cloud hardware. Use a small count for fast smoke tests and a larger, documented count for release or benchmark tests. Avoid choosing a threshold solely because it makes the current test pass. Define the acceptable error before examining the new result when possible.
Transpilation and backend constraints
A logical circuit and its hardware-compatible form are not the same artifact. Optimization passes may change gate count, depth, routing, and measurement behavior while preserving the intended operation. If a test is meant to validate hardware execution, validate the transpiled circuit as well as the source circuit. See Quantum Circuit Transpilation Explained for the distinction.
Common mistakes
- Expecting identical shot counts: Random sampling naturally produces variation. Compare distributions or confidence-aware ranges.
- Testing only one successful output: A circuit can produce the expected answer occasionally while having the wrong probability distribution.
- Using exact floating-point equality: Numerical simulation and parameterized calculations require tolerances.
- Ignoring zero-probability outcomes: Unexpected outcomes often reveal measurement-order, initialization, or gate errors.
- Mixing ideal and noisy acceptance criteria: A hardware test should not inherit the same threshold as an exact statevector test.
- Changing too many variables at once: Alter the circuit, compiler settings, backend, and shot count separately when diagnosing a regression.
- Relying on an undocumented seed: A seed without versions, circuit data, and configuration is not a reproducible experiment.
- Overfitting to one device run: Hardware behavior can vary. Repeat important checks and record the execution context.
- Testing only the quantum kernel: Parameter binding, result parsing, classical post-processing, and retry logic can introduce failures outside the circuit itself.
When to revisit
Revisit this testing checklist whenever you add a circuit, change an optimization pass, alter qubit or classical-bit mapping, move between simulators, upgrade an SDK, or submit work to a different hardware backend. These changes can affect APIs, random-number behavior, transpiled circuits, result formats, and numerical tolerances.
At least once per year, review the test harness and confirm that its simulator interfaces, seed controls, statistical comparisons, and backend adapters still match the versions you support. Also revisit it before a seasonal planning or release cycle if quantum workloads are part of a larger software system.
Before acting, use this compact release checklist:
- Run deterministic construction and statevector tests.
- Run distribution tests with documented shots and tolerances.
- Check expectation values against application-specific thresholds.
- Repeat important tests with fixed metadata and, where useful, multiple seeds.
- Compare ideal, noisy, and hardware results separately.
- Inspect transpiled depth, mapping, measurements, and backend configuration.
- Save the environment, circuit version, parameters, and result summary.
- Record whether a failure indicates a code defect, a statistical fluctuation, or expected device noise.
This workflow makes quantum testing closer to disciplined scientific software engineering: define the observable, control the experiment, compare with an appropriate tolerance, and preserve enough context to reproduce the result.