Introduction

Benchmarking quantum processors ensures we understand their capabilities and limitations by quantifying factors like gate fidelity, circuit depth, and execution speed. Up to 2023, several complementary metrics have emerged: Quantum Volume for overall performance, Randomized Benchmarking for average error rates, Cross-Entropy Benchmarking for fidelity on random circuits, Cycle Benchmarking for local and global error characterization, and CLOPS for speed. Each metric addresses distinct aspects of near-term devices.

Quantum Volume

Quantum Volume (QV) is a single-number metric designed by IBM to capture a device’s ability to run square circuits of width and depth equal to m, where larger QV indicates more complex circuits can be reliably executed. QV accounts for qubit count, connectivity, gate and measurement errors, and crosstalk [1].

Randomized Benchmarking

Randomized Benchmarking (RB) measures the average error rate of gate operations by applying sequences of randomly sampled Clifford gates and fitting the decay of survival probability as sequence length increases [2]. RB is robust to state preparation and measurement (SPAM) errors and scales efficiently to multi-qubit systems.

Cross-Entropy Benchmarking (XEB)

Cross-Entropy Benchmarking evaluates processor fidelity by comparing the output distribution of random circuits to the ideal quantum distribution. Google’s Sycamore experiments used linear XEB on 53-qubit circuits to demonstrate quantum supremacy, estimating fidelity from the cross-entropy difference [3].

Cycle Benchmarking

Cycle Benchmarking (CB) offers a scalable protocol to characterize local and global errors per cycle (a set of parallel native operations) across multi-qubit processors. Erhard et al. experimentally demonstrated CB on an ion-trap system up to ten qubits, reporting process fidelities from 99.6% (2-qubit cycles) to 86% (10-qubit cycles) [4].

Circuit Layer Operations Per Second (CLOPS)

CLOPS measures how many QV-style layers a processor can execute per second. IBM introduced CLOPS to capture device speed alongside fidelity, revealing trade-offs between volume and throughput [5].

Algorithmic Qubits

Algorithmic Qubits (#AQ) is an application-oriented metric from IonQ that aggregates performance across common quantum algorithms (e.g., VQE, QAOA) to estimate the number of effectively perfect qubits [6].

Reliable Quantum Operations Per Second (rQOPS)

rQOPS is a throughput metric proposed by Microsoft that weights operation count by error correction overhead and fault-tolerance requirements, measuring how many reliable logical operations a system can execute per second [7].

Volumetric Benchmarking

Volumetric benchmarks generalize QV to rectangular circuits of varying width (qubits) and depth, plotting performance trade-offs across width vs. depth and providing a detailed device profile [8].

References

[1] Cross, A. W., Bishop, L. S., Sheldon, S., Nation, P. D., & Gambetta, J. M. (2019). Validating quantum computers using randomized model circuits. Physical Review A, 100(3), 032328.

[2] Magesan, E., Gambetta, J. M., & Emerson, J. (2011). Scalable and robust randomized benchmarking of quantum processes. Physical Review Letters, 106(18), 180504.

[3] Arute, F., et al. (2019). Quantum supremacy using a programmable superconducting processor. Nature, 574(7779), 505-510.

[4] Erhard, A., et al. (2019). Characterizing large-scale quantum computers via cycle benchmarking. Nature Communications, 10(1), 5347.

[5] IBM Quantum. (2021). Updating how we measure quantum quality and speed. IBM Quantum Blog.

[6] IonQ. (2023). Algorithmic Qubits: A New Way to Measure Quantum Computers. IonQ Technical Report.

[7] Microsoft Quantum. (2023). Reliable Quantum Operations Per Second (rQOPS): A New Metric for Quantum Computing Performance. Microsoft Quantum Documentation.

[8] Proctor, T., et al. (2020). A volumetric framework for quantum computer benchmarks. Quantum, 4, 362.

[9] QED-C. (2023). Quantum Algorithm Exploration using Application-Oriented Performance Benchmarks. Quantum Economic Development Consortium Technical Report.

[10] Blume-Kohout, R., et al. (2017). Demonstration of qubit operations below a rigorous fault tolerance threshold with gate set tomography. Nature Communications, 8(1), 14485.