Why Is IBM’s Notion of Quantum Volume Only Valid up to About 50 Qubits?

28 min readOct 14, 2020

The IBM paper which introduced the notion of quantum volume as a metric for the power of a quantum computer has the odd caveat that it applies only to quantum computers of “modest size”, up to approximately 50 qubits. Why this odd limitation? Simple: because IBM’s method requires classical simulation of randomly-generated quantum circuits, which is exponential in the number of qubits, so 2⁵⁰, which is roughly one quadrillion (1,000,000,000,000,000 — a million billion) is considered the limit of the number of quantum states which can be represented and simulated on a current classical computer. This informal paper explores the nature of this 50-qubit limit.

To state it more succinctly:

Measurement of quantum volume is limited to the capacity of available quantum simulators running on classical computers.
At present, 50 qubits is viewed as the current practical limit for classical simulation of a quantum computer.

Caveat:

The 50-qubit limit appears to be the number of qubits which are simulated and executed by the generated circuits, not the total number of qubits on the system.
For example, IBM recently announced a quantum volume of 32 for a 65-qubit system. Only five (or so) qubits out of 65 were used in that test. And six (or so) qubits were used to calculate the quantum volume of 64 for their 29-qubit Falcon processor.

Topics to be discussed in the remainder of this informal paper:

The IBM protocol in a nutshell
Simulation of randomly-generated circuits
Square circuits
Measuring connectivity
Any-to-any connectivity
Swap networks
Calculation of quantum volume
QV vs. VQ
Limits of quantum simulators
50 is the base 2 logarithm of one quadrillion
One quadrillion quantum states pushes the limits of classical simulation
The IBM paper
Sorry, but this paper won’t dive into all details of the IBM protocol or all aspects of quantum volume
Quantum volume in the Qiskit Textbook
Unclear whether the 50-qubit limit is for the entire system or only for the circuit being tested
What if your quantum computer has more than 50 qubits?
How might logical qubits impact the 50-qubit limit?
Possible need for an artificial limit on quantum volume
What does the IBM paper say about lower error rates?
So what is the real limit?
What is the practical limit?
How much storage is needed for quantum state?
What are the prospects for a quantum performance metric that doesn’t require classical simulation?
Potential for application-specific benchmarks
Benchmarks based on verifiable algorithms
How many qubits are likely to be used for production-scale applications?
Quantum volume is a deadend
Why doesn’t IBM refer to quantum volume in their quantum roadmap?
Why doesn’t IBM publish the quantum volume of their 53-qubit machine?
But IBM does say that their 65-qubit system has a quantum volume of 32
How does one configure a simulator for a system with more than 50 qubits?
Did IonQ achieve a quantum volume of 4 million for 32 qubits?
Other related topics
The future

The IBM protocol in a nutshell

This informal paper won’t go very deep into the gory technical details of the IBM protocol for measuring quantum volume, beyond this brief summary.

In a nutshell, the “protocol” proposed by IBM requires generation of random quantum circuits and then executing those circuits on both real hardware and in a quantum simulator to ascertain the degree to which the results from real hardware agree with the simulated results — a reasonably high-degree of agreement is considered success, lower is considered failure. The largest randomly-generated circuit which has the best agreement between actual and simulated results will determine the quantum volume metric.

Some additional details will be described in subsequent sections.

Simulation of randomly-generated circuits

There is nothing special about the results derived from executing randomly-generated circuits, just that the results should be the same when executed on a real quantum computer as when executed on a quantum simulator.

Not exactly the same, both because quantum computers are probabilistic by nature, and some degree of errors are to be expected on current quantum computers.

This paper won’t go into the details, but the algorithm compares the degree of agreement of the actual machine results with the simulated circuit results.

The point here is that simulation of the randomly-generated circuits is required. And if the circuit uses n qubits, there may be 2^n quantum states, so larger values of n will require more classical resources to simulate larger circuits.

Square circuits

The essence of quantum volume is that it measures the largest square quantum circuit that can be executed on a real quantum computer which produces reasonably correct results. This informal paper won’t go into the deep details — see the IBM paper or the Qiskit Textbook description.

The squareness of the circuit is based on the width and depth of the circuit. Width measures the number of qubits. Depth measures the length of the quantum circuit executed for each qubit.

Measuring connectivity

The gates of the randomly-generated circuit will entangle qubits as well to test the degree of connectivity which the machine supports. Some machine architectures such as trapped-ion machines support any-to-any connectivity, while other architectures such as superconducting transmon qubits require swap networks to move the quantum states of two qubits to adjacent qubits since only nearest-neighbor connectivity is supported.

Measuring connectivity is really measuring the degree of errors that are introduced to connect or entangle two qubits.

Any-to-any connectivity

A quantum computer architecture such as a trapped-ion machine has the flexibility to entangle any two qubits with a single gate even if they are not physically adjacent.

This is the fastest, most reliable, and best form of connectivity.

A machine with any-to-any connectivity will tend to have a higher quantum volume for a larger number of qubits.

Swap networks

If the architecture of a quantum computer, such as superconducting transmon qubits, only permits nearest neighboring qubits to be directly entangled, a swap network is needed to physically move the quantum state of one or both qubits to two qubits which are physically adjacent.

A single SWAP gate is actually three CNOT gates, each of which has some potential for error. Executing a significant number of SWAP gates to entangle two qubits multiples that error potential, which reduces the probability that the overall randomly-generated circuit will match the ideal results from a classical simulation of that same circuit.

Calculation of quantum volume

The specific math for calculating quantum volume is:

log2(VQ) = argmax min(m, d(m))

Where VQ is the quantum volume, m is the number of qubits used by the circuit, and d is the depth of the circuit. d(m) refers to the fact that the circuit references m qubits. Technically d is the number of layers of the circuit.

The exact number of quantum logic gates for each qubit in each layer may not be one — for example, extra SWAP gates may be needed to move the quantum state of a qubit to perform entanglement between qubits which are not physically adjacent.

“argmax min(m, d(m))” simply determines which is smaller, m or d, for all randomly-generated circuits which return the closest to the correct results. “argmax” simply finds the maximum of m and d (actually, the maximum value of the minimum of m and d across all generated circuits) which generates a random circuit which returns the closest to the correct results.

The effect of d is to filter out values of m which cause the generated circuit to fail to produce results which are close to the simulated results.

The math calculates the exponent of the power of 2 of the quantum volume, so the actual quantum volume is 2 raised to that exponent.

Some examples:

m = 3, VQ = 2³ = 8.
m = 4, VQ = 2⁴ = 16.
m = 5, VQ = 2⁵ = 32.
m = 6, VQ = 2⁶ = 64.
m = 7, VQ = 2⁷ = 128.
m = 20, VQ = 2²⁰ = 1,048,576.

QV vs. VQ

QV is the marketing shorthand for quantum volume.

VQ, which is actually V with Q as a subscript (read as V sub Q), is the technical shorthand from the IBM paper.

The two mean the same thing.

I’d recommend using QV for general audiences, although spelling out quantum volume will generally be preferable.

I only use VQ here since I am directly referencing the IBM paper — and Medium doesn’t have support for subscript notation. If subscript notation is available in your writing platform, use it.

Limits of quantum simulators

The problem is that quantum simulators are incapable of simulating circuits using more than around 50 qubits since the number of quantum states grows exponentially as the number of qubits grows.

The exact limit of any given quantum simulator will depend on its implementation and the resources available of the classical computer on which it is hosted.

50 is the base 2 logarithm of one quadrillion

From the preceding discussion, it is not the 50 itself which causes a limitation, but the number of quantum states required to execute a circuit with 50 entangled qubits. Specifically,

50 qubits = 2⁵⁰ = 1,125,899,906,842,624 (one quadrillion) quantum states.
Or, log2(1,125,899,906,842,624 quantum states) = 50 qubits.

One quadrillion quantum states pushes the limits of classical simulation

One quadrillion — one thousand trillion or one million billion — is a lot of quantum states for a simulator to keep track of.

Experience with quantum simulators has led to the belief that 50 qubits is really about the limit for classical simulation of a quantum computer at the present time, both in terms of storage and in terms of time to run.

The IBM paper

The IBM paper which proposed the quantum volume metric:

Validating quantum computers using randomized model circuits
Andrew W. Cross, Lev S. Bishop, Sarah Sheldon, Paul D. Nation, Jay M. Gambetta
https://arxiv.org/abs/1811.12926

The abstract:

We introduce a single-number metric, quantum volume, that can be measured using a concrete protocol on near-term quantum computers of modest size (n≲50), and measure it on several state-of-the-art transmon devices, finding values as high as 16. The quantum volume is linked to system error rates, and is empirically reduced by uncontrolled interactions within the system. It quantifies the largest random circuit of equal width and depth that the computer successfully implements. Quantum computing systems with high-fidelity operations, high connectivity, large calibrated gate sets, and circuit rewriting toolchains are expected to have higher quantum volumes. The quantum volume is a pragmatic way to measure and compare progress toward improved system-wide gate error rates for near-term quantum computation and error-correction experiments.

The key phrase there:

near-term quantum computers of modest size (n≲50)

Where n refers to the number of qubits.

The paper doesn’t mention that aspect of the topic in the body of the paper, and never justifies the limit nor explains it in any way.

Sorry, but this paper won’t dive into all details of the IBM protocol or all aspects of quantum volume

Quite a few high-level details and even some low-level details from the IBM protocol for quantum volume are included in this informal paper, but there is no intention to include all details of the IBM protocol, just enough to discuss and support the headline topic of what the 50-qubit limit is all about.

As already mentioned, the IBM paper has all the details (except for the rationale for the 50-qubit limit!):

Validating quantum computers using randomized model circuits

Nor does this informal paper intend to provide a full treatment of all aspects of quantum volume in any general sense. A subsequent paper may pursue that larger objective, beyond the nature of the purported 50-qubit limit.

Quantum volume in the Qiskit Textbook

For additional reference, and an example, see the Qiskit Textbook:

No mention of the 50-qubit “limit” there either.

What if your quantum computer has more than 50 qubits?

First, it’s not exactly clear what the precise limit is for simulation. It could be a few qubits more than 50 or even a few qubits less depending on pragmatic considerations for the simulation software and hardware.

But if the number of qubits is well above 50, such as 64, 72, 128, etc., simulation of circuits with that number of qubits absolutely won’t be possible. What happens then?

Actually, it turns out that, by definition, quantum volume is based on a square circuit — an m x m grid of qubits and gates. It’s unlikely that a quantum circuit that uses 64 qubits in a circuit of depth 64 would execute properly and return valid results on near-term machines. The method for computing quantum volume starts with smaller circuits, such as 8 x 8, 16 x 16, and 32 x 32, and will stop once a circuit of such complexity no longer returns valid results when executed on real hardware compared to the classical simulation of the circuit.

My guess is that somewhere between 32 x 32 and 50 x 50 even a quantum computer with a very low error rate will begin failing, long before it would ever get up to trying circuits at 55 x 55.

Unclear whether the 50-qubit limit is for the entire system or only for the circuit being tested

Reading the IBM paper by itself, one might get the impression that the 50-qubit limit applied to the entire system — that quantum volume does not apply to quantum computers with more than about 50 qubits in the entire system.

But there is another reasonable interpretation — for quantum computers with limited connectivity and limiter coherence, it’s not possible to achieve a successful quantum volume test for a circuit using 50 or more qubits.

Actually, at present, it’s not even possible to come close to a quantum volume for 50 qubits with current technology.

Although IonQ recently claimed that their “new hardware features 32 perfect qubits with low gate errors, giving it an expected quantum volume greater than 4,000,000”, the wording as “expected quantum volume” suggests that they didn’t actually test and verify achieving that quantum volume.

Separate from IonQ’s claim, Honeywell’s recent claim of quantum volume of 128 suggests that only seven qubits were used (2⁷ = QV 128.) That may have been all of the trapped-ion qubits in their system, but it’s hard to say for sure since they published next to nothing in terms of technical details.

Even if we accept IonQ’s claim, they still have a long way to go to hit the limit of 50 qubits.

IBM recently announced (via tweet) that they had a new 65-qubit system (Hummingbird R2 — Hummingbird R1 was their 53-qubit system) with a quantum volume of 32, indicating that only five of the 65 qubits (2⁵ = QV 32) were used in the quantum volume test. More qubits may have been used, but they likely resulted in test failures due to accumulated errors and decoherence. This would be a perfect example of applying the 50-qubit limit to the actual circuits used by the quantum volume tests rather than to the total qubits available on the hardware system.

Over time, we can expect that connectivity and coherence will rise so that a greater fraction of the total system qubits will be usable for quantum volume tests. Trapped-ions systems seem poised to hit the 50-qubit limit much sooner than superconducting transmon qubit systems.

The advent of error-free logical qubits could completely change the picture, but it’s not clear how logical qubits would impact coherence time.

So, for now I presume that the 50-qubit limit for quantum volume applies to the limit of simulation and limited connectivity and coherence rather than the total available qubits for the entire system.

How might logical qubits impact the 50-qubit limit?

We’re still a long way from the advent and common use of error-corrected and error-free logical qubits, but they would definitely increase the size of quantum volume circuits which can be executed on real hardware.

But it’s not clear to me whether error-corrected and error-free logical qubits would have much impact on coherence time.

In fact, it’s not clear to me which is currently the more major factor in current quantum volume tests, gate errors or coherence time. I suspect it is gate errors, especially for swap networks, since with rather small values of quantum volume, even 128 using only seven qubits and a circuit depth of seven, circuits probably complete execution in less than the decoherence time. But I’m not completely sure about that.

But once we get up to dozens of qubits, the circuit depth may begin to bump into the limit of coherence time.

Trapped-ion systems tend to have longer coherence times, so they may bump into the 50-qubit limit much sooner than superconducting transmon qubits.

Possible need for an artificial limit on quantum volume

For future quantum computers which have dramatically lower error rates, and if the machine has more than 50 qubits — such as 65, 72, 127, or 413 qubits or greater — it would probably be appropriate to place some upper limit on quantum volume so that impossible simulations are not attempted.

What would be reasonable artificial limits?

2⁵⁰. A reasonable default — if sufficient resources are available.
2⁵⁵. Probably a general practical limit for current classical technology.
2⁶⁰. Probably an absolute limit for current classical technology.
2⁴⁰. May be a more reasonable default since 2⁵⁰ or higher would consume extreme resources.
2³⁸. If that’s what Google’s cloud-based simulator supports and if it does so efficiently.
2³⁵. May be a reasonable default unless the quantum hardware is exceptionally good.
2³². May be a good default for average, commodity hardware.

It might also be sensible to have a wall-clock time limit for simulation runs, and warn the user that quantum volume will be limited unless an extra amount of time is enabled.

What does the IBM paper say about lower error rates?

The IBM paper does have this footnote:

For error rates as low at 10E−4, we anticipate that model circuits U that can be successfully implemented will involve few enough qubits and/or low enough depth to compute HU classically. For lower error rates than this, the quantum volume can be superseded by new volume metrics or modified so classical simulations are not necessary.

The operant phrases:

quantum volume can be superseded by new volume metrics
or modified so classical simulations are not necessary

In other words, some other metric, not this current notion and method of measuring quantum volume, would be needed.

And as that footnote suggests, a method that does not require classical simulations will likely be needed — when error rates get that low.

In short, the days of quantum volume as currently designed are numbered. For now, it’s okay, but it won’t be okay once we get more robust qubits.

So what is the real limit?

There doesn’t appear to be any specific actual real or theoretical limit to how many qubits can be measured for IBM’s notion of quantum volume. Rather, there is only the practical limit of how large a quantum simulation the individual running the test can perform.

Rather than being a limit of 50 qubits per se, it might be 55 or 60 on the high end or 40 or 45 or even 35 or 32 on the low end. And a lot of people might have trouble running simulations for 28 or even 16 qubits.

The largest simulator capacity I know of is the AtoS Quantum Learning Machine, which is currently advertised as supporting up to 41 qubits.

Google recently announced that they would soon offer simulation of up to 38 qubits on their upcoming cloud-based Quantum Computing Service. They said they could push that higher, but that was the sweet spot for performance.

What is the practical limit?

The practical limit for measuring quantum volume is whatever simulator software and classical hardware is available to the person seeking to generate a measurement of quantum volume.

Some people may have access to an AtoS simulator capable of simulating 41 bits. Some may not. Others may have access to specialized systems or custom hardware that is capable of a few more or maybe a dozen more qubits (45 to 55, probably not 60 to 65, but who can say for sure.)

And some people may only be able to simulate 32 qubits, or less.

Some quantum state limits:

16 qubits = 2¹⁶ = 65,536 quantum states.
20 qubits = 2²⁰ = 1,048,576 (one million) quantum states.
24 qubits = 2²⁴ = 16,777,216 (16 million) quantum states.
28 qubits = 2²⁸ = 268,435,456 (268 million) quantum states.
32 qubits = 2³² = 4,294,967,296 (4 billion) quantum states.
36 qubits = 2³⁶ = 68,719,476,736 (68 billion) quantum states.
38 qubits = 2³⁸ = 274,877,906,943 (274 billion) quantum states.
40 qubits = 2⁴⁰ = 1,099,511,627,776 (one trillion) quantum states.
41 qubits = 2⁴¹ = 2,199,023,255,552 (two trillion) quantum states.
42 qubits = 2⁴² = 4,398,046,511,104 (four trillion) quantum states.
45 qubits = 2⁴⁵ = 35,184,372,088,832 (35 trillion) quantum states.
50 qubits = 2⁵⁰ = 1,125,899,906,842,624 (one quadrillion) quantum states.
53 qubits = 2⁵³ = 9,007,199,254,740,992 (nine quadrillion) quantum states.
54 qubits = 2⁵⁴ = 18,014,398,509,481,984 (18 quadrillion) quantum states.
55 qubits = 2⁵⁵ = 36,028,797,018,963,968 (36 quadrillion) quantum states.
60 qubits = 2⁶⁰ = 1,152,921,504,606,846,976 (one quintillion) quantum states.
64 qubits = 2⁶⁴ = 18,446,744,073,709,551,616 (18 quintillion) quantum states.
65 qubits = 2⁶⁵ = 36,893,488,147,419,103,232 (37 quintillion) quantum states.
72 qubits = 2⁷² = 4,722,366,500,000,000,000,000 (4 sextillion) quantum states.

How much storage is needed for quantum state?

We’ve discussed large numbers of quantum states, but it’s unclear how much storage would be required. The precise number of bits, bytes, and more needed for even a single quantum state is unclear.

Technically, a quantum wave function for a single unentangled qubit has two probability amplitudes, each a complex number consisting of two real numbers, so that would be two times two or four floating point numbers for each unentangled qubit. How much precision is needed for each floating point number is an open question.

Once qubit states become entangled, the number of quantum states rises dramatically, ultimately exponentially. Each quantum state being a single complex number, or a real and an imaginary part, two real numbers usually represented as two floating point numbers.

There are undoubtedly a variety of clever schemes for encoding quantum states for common cases, but the question is what might the average case be, including how many decimal or binary digits of precision are needed.

The ultimate answer may simply be that it all depends and it varies, based on the particular algorithm being simulated.

I suspect that the randomly-generated circuits used for quantum volume are somewhat worst case, designed to push the hardware as dramatically as possible, so that the quantum state storage required for such a quantum circuit may be more storage-intensive than hand-crafted circuits.

In any case, the amount of storage needed to represent the full quantum state of a quantum volume generated algorithm will be a factor in how large a quantum volume can be measured.

What are the prospects for a quantum performance metric that doesn’t require classical simulation?

A quantum leap for a performance metric is simply uncharted territory. We can speculate or fantasize about a comparable performance metric that is virtually limitless and doesn’t bump into such classical limits, but no solid answers are available at present.

One thorny problem is that unlike classical computing where operations are absolutely 100% perfect and absolutely deterministic — and hence predictable, all quantum operations are inherently probabilistic so that you cannot readily calculate the answer for a complex quantum circuit since the statistics quickly grow beyond our ability to compute them using a classical computer, which is why quantum volume relies on simulation in the first place.

The essence of quantum volume is that you are trying to count how many gates you can string together before the tiny errors of each gate add up to a large enough error to invalidate the results of the circuit — and before you exceed the coherence time of the hardware. And that the circuit is more of a tree or graph so that errors propagate in two dimensions. The point is that the mathematicians don’t have adequate math to readily and cheaply precompute the cumulative errors for the entire graph, and that is cheaper than doing an exponential simulation of the quantum circuit.

Even if someday we finally get to true quantum error correction (QEC), even a perfect logical qubit is still probabilistic, by nature, by definition from quantum mechanics, so classically computing the result of a complex circuit will always be as expensive as a classic circuit simulation.

Also, even with perfect qubits, there will always be some upper limit of coherence before the quantum states of the qubits begin to deteriorate. So any benchmark of such a real machine will need to at least model such decay. If the decay is reasonably consistent and predictable, maybe then a simple linear mathematical calculation could at least compute the maximum circuit depth before decoherence hits some designated limit.

But all of this requires perfect or near-perfect qubits, which will not be readily available any time soon (the next 2–5 years.)

Finally, maybe somebody will come up with a clever but simple circuit design that doesn’t even theoretically require a full simulation across a large number of qubits. Maybe. It’s possible, but it simply hasn’t been done yet.

Potential for application-specific benchmarks

Another possibility is that as we finally begin to get true production-scale algorithms and applications, application-specific benchmarks could be developed, so that even if complex theoretical quantum circuits cannot be readily modeled classically, much more practical circuits could — maybe.

And such application-specific benchmarks could have much more practical value than benchmarks such as quantum volume which are too abstract to provide useful insight to algorithm designers or application developers.

An example would be quantum computational chemistry, where molecules such as chains of hydrogen atoms or groups of hydrogen and carbon could be dynamically selected to match a desired number of qubits to be benchmarked.

Selecting a specific width might be a little more difficult, but could simply be achieved by padding with fixed gate sequences which have readily predictable results.

Or, financial applications such as portfolio optimization can be readily scaled to match portfolio size to number of qubits to be tested in a given benchmark run.

Benchmarks based on verifiable algorithms

There are two classes of theoretically-hard algorithms — those whose results can be quickly validated or verified and those which can’t be quickly validated or verified. If your quantum circuit factored a very large semiprime number, it’s easy to check (verify) that the factors multiply to produce the initial semiprime number. But for a complex optimization such as protein folding or solving a complex traveling salesman problem, you can only validate whether the result seems acceptable, not whether it is in fact actually the most optimal since you can’t readily classically calculate the optimal result since if you could readily calculate it, it wouldn’t require a quantum computer to calculate it in the first place.

The method for quantum volume is based on dynamically-generated circuits whose results cannot be directly verified without a full quantum simulation.

A collection of carefully-crafted algorithms could be designed which thoroughly exploit most of the features of a typical quantum computer, and produce results which can be quickly and efficiently verified using classical means.

The main issue with that approach is that it doesn’t seamlessly scale to any arbitrary number of qubits. The solution to that issue is to design scalable algorithms which do scale based on number of qubits as a parameter to the circuit generation process. The added issue there is that there must be a second parameter to scale the depth of the generated circuit as well. It could get complicated — so now we understand why randomly-generated circuits were chosen. Yes, a more complex scheme is needed — that’s the cost of benchmarking a more complex system.

How many qubits are likely to be used for production-scale applications?

At present, there are no production-scale algorithms or applications, but it is worth contemplating how many qubits might be needed for production-scale applications. In particular, the likelihood that far more than 50 qubits are likely to be needed, blowing right past the key limitation of the quantum volume benchmarking metric.

First, it is important to note that the whole rationale for using a quantum computer is that it can tackle complex problems that are too big or too difficult for classical computers. This implies a quantum advantage.

By definition, quantum advantage is exploiting quantum parallelism and in particular an exponential speedup compared to classical computers.

Conceptually, a quantum algorithm using 20 qubits for a parallel computation could have a raw quantum advantage of 2²⁰ or one million to one. But then circuit repetitions or shot count is likely needed to overcome both errors and the probabilistic nature of quantum computing to achieve a statistically meaningful result. A shot count of 1,000 (or 1024) would reduce those 20 qubits to having only a 1,000 to one advantage, which typically wouldn’t justify using a quantum computer rather than a network of classical computers.

Even if 30 qubits were used, a shot count of 64K would reduce its effective or net quantum advantage dramatically.

In short, it might only be when algorithms are using upwards of 40 to 60 qubits before a quantum computer could deliver a net quantum advantage which is notable and worth the effort to switch from classical computing to quantum computing.

I would say that 60 to 120 is probably the low end of the range of qubits where we will start seeing a significant net quantum advantage for quantum computers.

And just because a quantum processor has 65 or 127 qubits does not mean that it supports algorithms of 60 or 120 qubits.

In reality, it might not be until we get to algorithms and applications using hundreds and even thousands of qubits before we see the combination of a dramatic net quantum advantage as well as the capacity to tackle real-world problems in terms of data size.

The quantum volume metric will work up to roughly the 50-qubit limit, but once a production-scale algorithm is using upwards of 60, 80, 100, or more qubits, classical simulation is completely off the table, which then precludes the use of the quantum volume metric.

Quantum volume is a deadend

Sure, quantum volume is a plausible (if weak) metric for smaller quantum computers and smaller quantum circuits, but clearly it won’t work well and won’t work at all for larger quantum computers and larger quantum algorithms.

Google has a 53-qubit machine with a 72-qubit machine coming soon. IBM already has a 53-qubit machine — without a quantum volume rating — with a 65-qubit machine which just came out, and a 127-qubit machine coming in the next year. So, clearly a successor metric is already needed.

But since connectivity, error rates, and coherence are still significant issues, especially for superconducting transmon qubits, algorithms are severely limited in size regardless of qubits available, so maybe quantum volume still has some life left in it, at least until we get to the stage where circuit depth exceeds 50 (or 50 layers in terms of quantum volume.)

Either way, quantum volume is not the wave of the future — beyond the next few years.

Why doesn’t IBM refer to quantum volume in their quantum roadmap?

When IBM initially posted their roadmap for their future quantum computers I thought it very odd that they had no mention of quantum volume. It was only a couple of weeks later as I started to dig deeper into the technical details of quantum volume that I finally realized why, as exemplified by this informal paper — because all of their future machines have a lot more than 50 qubits and 50 qubits is roughly the limit for practical measurement of quantum volume.

The smallest quantum computer on the IBM roadmap is 65 qubits, well above the 50-qubit limit of the IBM protocol for quantum volume. Granted, IBM has separately published a quantum volume of 32 for their 65-qubit system, but honestly it seems silly to go through the trouble of building a system for 65 qubits when the best available benchmark for capacity uses only five of their 65 qubits (2⁵ = QV 32.) The bottom line is that this new 65-qubit machine doesn’t support 50-qubit algorithms, at least when measured using IBM’s own benchmark of quantum volume.

The next smallest quantum computer on the IBM roadmap is 127 qubits, which is much further from practical measurement of quantum volume — at least in terms of algorithms which use more than a tiny fraction of the available qubits.

And it just gets much worse after that.

Also, IBM didn’t publish any specs for connectivity and coherence of the machines on their roadmap, so even if full classical simulations could be run, they might produce quantum volume measurements which are only able to utilize a very small subset of the qubits, just as their recent tests of their 27-qubit Falcon processor were only able to utilize 6 of the 27 qubits to achieve a successful test at a quantum volume of 64.

Why doesn’t IBM publish the quantum volume of their 53-qubit machine?

In fact, IBM’s current largest quantum computer (before the 65-qubit machine was just announced), at 53 qubits, is already beyond the 50-qubit limit for measurement of quantum volume.

Sure, they could “cheat” and measure quantum volume using less than 50 of the total qubits — a lot less than 50, it turns out — but it would be impractical to measure quantum volume of a 53-qubit machine which had more perfect qubits and sufficiently long coherence time.

Also, as we saw when IBM published a quantum volume of 64 for their 27-qubit Falcon processor, using only 6 of the 27 qubits, it looks kind of silly to have so many qubits that can’t be used effectively. IBM probably could publish a quantum volume for their 53-qubit processor, but it might not be much better than the quantum volume for the 27-qubit Falcon, so… why bother?

It baffled me for quite some time why IBM was not publishing a quantum volume metric for their 53-qubit machine. Now I know exactly why.

But IBM does say that their 65-qubit system has a quantum volume of 32

Even though IBM hasn’t published a quantum volume for their 53-qubit system, they did publish (okay, they tweeted) that their new 65-qubit successor to the 53-qubit system has a quantum volume of 32.

Got that? More than twice as many qubits than a 27-qubit Falcon but only half the quantum volume. What’s the point of doing this?!

So, the overall system does have more qubits than the 50-qubit limit proposed in their original paper, but the connectivity, errors, and coherence are insufficient to perform a quantum volume circuit execution on a real machine for more than five qubits (2⁵ = QV 32).

IBM hasn’t published technical details for achieving QV 32 on this 65-qubit system. It is unclear whether the quantum volume software itself deduces that larger circuits do not need to be either tested or simulated, or maybe the investigator manually limited the tests, or even engaged in a little trial and error, rather than let the software attempt to simulate randomly-generated quantum circuits for a full 65 qubits.

They haven’t disclosed how they configured the simulator. It wouldn’t be practical to configure it for all 65 qubits. Only five, and maybe a few more, would be needed to run the quantum volume simulation to achieve a quantum volume of 32 (2⁵ = QV 32.)

How does one configure a simulator for a system with more than 50 qubits?

I haven’t seen any mention or publication of techniques for configuring simulations of quantum computers with over 50 qubits (or even 40 or even 32 qubits.) Clearly there are practical resource constraints. I don’t want to speculate, but I suspect that you could configure the simulator for just the subset of qubits used by your algorithm.

If IBM did indeed measure a quantum volume of 32 for a 65-qubit system, they had to perform simulation runs to compute that metric. Clearly they wouldn’t have configured the simulator for all 65 qubits.

There are lots of technical details to consider, such as the exact topology of the qubits and what portion of that topology is being simulated.

In a real quantum computer the exact performance characteristics will vary from qubit to qubit, so all of that would need to be configurable. Although for quantum volume tests qubit performance should be ideal and free of errors or decoherence anyway.

And then there is the question of what upper limit there is for the simulation. 50 qubits? 40? 32? 16? Who knows, until technical details are published.

But for now, with real systems having such small values of quantum volume, only small numbers of qubits need be simulated, well under the 50-qubit limit proposed by the original IBM paper.

Did IonQ achieve a quantum volume of 4 million for 32 qubits?

Although their press release seems to assert that they achieved a quantum volume of “greater than 4,000,000” for a 32-qubit system, something seems amiss in IonQ’s announcement.

First, quantum volume is always a power of 2, so “over 4,000,000” probably indicates 2²² or 4,194,304.

But, that would indicate that only 22 qubits were used in the circuit.

If they used their full 32 qubits, that would yield a quantum volume of 2³² or 4,294,967,296 or four billion, not four million.

Technically, it’s possible that they performed simulation runs with 22 qubits or even 32 qubits, but I’m skeptical. After all, they said “giving it an expected quantum volume…”, implying that they hadn’t actually tested and verified the quantum volume.

In any case, even at 32 qubits and QV of four billion, they’re still short of the 50-qubit limit indicated in the original IBM paper.

The future

Whatever technological advances the coming years and decades bring, we definitely need more helpful benchmarking metrics. And benchmark metrics that work both for more capable hardware and the needs of less-sophisticated application developers.

I expect that architectural advances in the next five to ten years will force a leap beyond the current notion of quantum volume for the algorithmic capacity of quantum computers.

At some point I do intend to write an informal paper which more fully discusses the details and issues of quantum volume, such as the topics listed in the preceding section.

For more of my writing: List of My Papers on Quantum Computing.