Shots and Circuit Repetitions: Developing the Expectation Value for Results from a Quantum Computer

  1. Motivation
  2. The problem
  3. Two forms of uncertainty in a quantum computer
  4. Statistics to the rescue
  5. Statistical aggregation to develop the expectation value
  6. How does a classical computer avoid the inherent probabilistic nature of quantum mechanics?
  7. Apparent determinism = probability + statistics
  8. Stochastic processes
  9. Quantum computing seeks to exploit the inherent uncertainty of quantum mechanics
  10. Everything is fine, until you measure qubits
  11. Applying statistics to measurement
  12. Don’t forget the noise and errors
  13. Environmental interference
  14. Noise vs. errors vs. environmental interference
  15. Measurement errors
  16. Each qubit has its own noise and error profile
  17. Shot count, shots, or circuit repetitions
  18. How many shots or circuit repetitions are needed?
  19. You need to know what results to expect
  20. Expectation value
  21. Simplified concept of expectation value
  22. Raw quantum results
  23. Bit string cases
  24. Quantum registers
  25. Cases of series of results
  26. Possibility of garbage results
  27. What to do about outliers
  28. Actual vs. requested shot count
  29. What to do when there is no clear dominant result
  30. Just take the average of the results — or the most common result
  31. Manually examine all results
  32. What fraction of shots should be dedicated to probabilistic nature vs. noise?
  33. Testing to characterize effects of noise, errors, and environmental interference on results
  34. Four types of dynamic shot count
  35. Dynamic shot count based on parameters
  36. Automatic dynamic shot count?
  37. Won’t quantum error correction solve the problem?
  38. Wall clock time
  39. Context of a quantum simulator
  40. Design for portability
  41. Is 10 ever a reasonable shot count?
  42. IBM Qiskit default shots is 1024
  43. Is 100 shots reasonable?
  44. What is the maximum shots for a single circuit?
  45. No clear rules, methodology, or guidance for shot count
  46. What’s a good design goal for shot count when designing new algorithms?
  47. Rules of thumb for shot count
  48. Formulas for shot count
  49. Intuitions for shot count
  50. Need to balance shot count and wall clock time
  51. How many digits of precision are reasonable?
  52. How does precision affect shot count?
  53. Is it still too soon to worry too much about shot counts?
  54. One exception: true random number generation — only a single shot required
  55. Don’t confuse circuit repetitions with the iterations or layers of variational methods
  56. Don’t confuse circuit repetitions with experiments
  57. Iterations of variational algorithms
  58. Application experiments
  59. Watch out for total circuit executions
  60. Don’t get too used to the nuances and glitches of the quantum machine you used yesterday
  61. How does quantum volume impact shot count?
  62. What is the impact of shots on quantum advantage?
  63. What about quantum supremacy?
  64. Scaling shot count for Shor’s algorithm
  65. Advice to authors of papers on quantum algorithms
  66. Recommendations
  67. Conclusions
  68. What’s next?

Motivation

  1. Formal — or even informal — treatment of the question of how to determine how many shots (circuit repetitions) are appropriate for a particular quantum circuit.
  2. Any consistency in focus on the importance of shot count.
  3. How shot count should scale as the input data scales.
  4. How exactly specific changes in error rates and coherence times should impact shot count.
  5. Limitations of increasing shot count — at what point can you simply not get a statistically valid result from a quantum circuit due either to the nature of the circuit itself (probability) or the quantum hardware being too noisy.
  6. How to analyze a circuit to determine an optimum shot count.
  7. Formal formulas for computing shot count.
  8. Even informal rules of thumb for estimating shot count.
  9. Robust guidance for when the vendor default for shot count — or 100 or 1,000 — fails to deliver satisfactory results, what should the algorithm or application designer do — in a more methodical manner rather than a haphazard trial and error manner.
  10. Could shot count be computed or estimated automatically based on circuit and some statement on required precision and any wall clock time limit? IOW, parameters which the application developer is likely to know and understand.
  11. How exactly an application developer who needs a deterministic value should treat the probabilistic results from a quantum calculation. What statistical techniques can or should be used to derive an expectation value?
  12. How to tune shot count based on desired precision of expectation value.
  13. How extreme shot count requirements might impact quantum advantage. For real-world scenarios.

The problem

Two forms of uncertainty in a quantum computer

  1. The inherent probabilistic nature of quantum computers and indeed of quantum mechanics itself.
  2. Noise and errors due to our inability to construct an ideal, perfect quantum computer at this time — noise and errors in the physical components as well as environmental interference from outside of the quantum computer that we are currently unable to filter out well enough to prevent the sensitive components of the quantum computer from being affected. This includes decoherence as well — the quantum state of a qubit will decay over time.

Statistics to the rescue

Statistical aggregation to develop the expectation value

How does a classical computer avoid the inherent probabilistic nature of quantum mechanics?

Apparent determinism = probability + statistics

Stochastic processes

Quantum computing seeks to exploit the inherent uncertainty of quantum mechanics

Everything is fine, until you measure qubits

Applying statistics to measurement

Don’t forget the noise and errors

Environmental interference

Noise vs. errors vs. environmental interference

  1. Noise differs from environmental interference in that the source of the noise is within the quantum computer itself — from electrical, electronic, and mechanical components, cabling, wiring, circuit connections, and intermittent interactions of any of the preceding, while environmental interference by definition has a source external to the quantum computer (see the preceding section.)
  2. Errors can be caused by noise and environmental interference, but errors are generally caused by the flawed execution of operations, such as quantum logic gates and measurement operations (so-called measurement errors), as well as decoherence. An operation on one qubit can also have an unintended effect on a nearby qubit — so-called spectator errors or crosstalk. Microwave signals and laser operations simply aren’t 100% foolproof, yet.
  3. Even if there were no environmental interference, no noise within the quantum computer, and all operations are executed flawlessly, there is still the possibility of errors as a result of decoherence, where an idle qubit gradually decays from a 0 to a 1 or from a 1 to a 0.

Measurement errors

Each qubit has its own noise and error profile

Shot count, shots, or circuit repetitions

How many shots or circuit repetitions are needed?

  1. Increase the number of shots or circuit repetitions until you get statistically accurate results that you are comfortable with.
  2. Reduce the number of shots or circuit repetitions until the total wall clock run time is acceptable for your needs while still producing statistically accurate results that you are comfortable with. It’s a balancing act and tradeoff between the two competing interests.

You need to know what results to expect

Expectation value

  1. https://en.wikipedia.org/wiki/Expectation_value_(quantum_mechanics)
  2. https://en.wikipedia.org/wiki/Expected_value
  3. https://mathworld.wolfram.com/ExpectationValue.html
  1. A single qubit whose value will be a binary 0 or 1. An average of 0.5 is not terribly useful when measuring qubits.
  2. A physical phenomenon such as energy levels of electrons, which are discrete levels rather than a continuous range of real numbers.
  3. A range of integers. A real, non-integer average is not very useful.
  4. A collection of bit strings — one bit per qubit, one bit string for each computational basis state. Each value is a pattern. A numerical average is not useful.

Simplified concept of expectation value

  1. What are the expectations about what a single value (single measurement) looks like. Such as its range of possible values and the theoretical distribution of those values, possibly even a probability for each value. That’s in line for the quantum mechanical notion of expectation value.
  2. How do we want to think about a series of measurements. What information do we want to extract from the series?

Raw quantum results

  1. A single qubit. A binary bit 0 or 1, or a boolean value false or true.
  2. A collection of single qubits considered as a group, a bit string, but each still a discrete binary value, typically potentially entangled, such as a so-called register, interpreted as a raw bit string but not interpreted as a single binary integer. For example, the two qubits of a Bell state, or three or more qubits in a GHZ or W state.
  3. A collection of qubits which represent the binary digits of an integer. Two bits for 0–3, three bits for 0–7, four bits for 0–15, 10 bits for 0–1023, and so on.
  4. A collection of qubits which represent the fractional binary digits of a binary point number, from 0 to 0.999… (up to by not including 1.0.) Two bits for 0, 0.25, 0.50, and 0.75. Three bits for 0 to 0.875. Four bits for 0.9375. And so on.
  5. Some combination of any number of the above — zero or more individual qubits, zero or more bit strings, zero or more k-bit integers, and zero or more k-bit binary-point fractions. Where each number may have its own value of k.

Bit string cases

  1. All bits independent, isolated, uncorrelated.
  2. 2-qubit Bell states — correlation or dependency between the quantum state of the two qubits.
  3. k-qubit GHZ state — all 0 or all 1, in theory.
  4. k-qubit W state — exactly one bit is 1, rest are 0, in theory.
  5. k-bit integer.
  6. k-bit binary fraction.

Quantum registers

Cases of series of results

  1. The raw values across all rows. Let the application sort them out.
  2. Exclude duplicates from raw values.
  3. The simple integer arithmetic average of a column across all rows. Simply add the column across all rows and divide by the number of rows.
  4. Exclude outliers from the simple integer arithmetic average. Specify count or threshold for outliers, smallest and largest values. Alternatively, outliers could be based on low-frequency integer values.
  5. Maximum value. What is the largest value which occurs across all rows for a given column.
  6. Most frequent value. Which binary value occurs most frequently across all rows for that column.
  7. Top k values. May be identical.
  8. Top k most common values. Ignoring duplicates.
  9. Cutoff threshold. How many values (rows) have a value in the column with a binary value above a specified binary value.
  10. Cutoff threshold average. Average of all of the values above the cutoff threshold.
  11. Bit string distribution. Frequency distribution for each unique bit string value from 0…0 to 1…1 for the number of qubits in the bit string. Generally only useful for relatively short bit strings since there will be 2^k entries, which is one million even for k of 20. Although, just because there are potentially 2^k entries, that doesn’t mean that every value is captured — the values may be relatively sparse. But even then, if your shot count is high and there is lots of noise and errors, many spurious values could be generated.
  12. Raw distribution. Counts for each unique value.
  13. Bucket distribution. Divide the k-bit values into some number of buckets and return a count or percentage of the row values which fall into each bucket.
  14. Maximum bucket. Simply the bounds of the bucket (or bucket number) which contains the most values. Which raw value occurred the most in that bucket. Should have the option of raw count of values in a bucket or a percentage, fraction, or probability based on the shot count.
  15. Bimodal distribution. Return the central value for each of the two peaks of a bimodal distribution, and possibly the number of values occurring within a range about the peaks.
  16. Multiple peaks. Return a list of all peaks, with a minimum distance between peaks. Optionally order the peaks by number of number of rows at the peak value or within some minimum distance from the peak value.
  17. Period of the peaks. Measure the distance between peaks. Or distances plural.
  18. Random. No apparent true, single value or pattern.
  19. Garbage. Not truly random, but no discernible value or pattern.
  1. Raw counts.
  2. Fractions.
  1. Were there more 1’s than 0’s — true or false.
  2. What percentage of the time was the result a 1. How close to 100%? Well above 50% or close to 50%?
  3. How random did the sequence appear? Longest run of 1’s or 0’s. Average percentage of time with runs of 1’s or 0’s longer than some threshold, say 3 or 4. Other measures of randomness.
  4. Raw binary values. Such as simply generating random bit values.

Possibility of garbage results

What to do about outliers

  1. Accept them as is. If the shot count is high enough, they may simply get washed away in the statistical aggregation.
  2. Ignore them. Based on some numeric threshold, values which are too far outside an expected range could simply be dropped.
  3. Ignore the fringe values — highs and lows. Regardless of any expected range, simply drop a percentage of the highest and lowest values.

Actual vs. requested shot count

What to do when there is no clear dominant result

Just take the average of the results — or the most common result

Manually examine all results

  1. There may be unexpected surprises.
  2. There may be some strange biases.
  3. There may be some anomalous results which should be discarded as outliers.
  4. There may be bimodal or multimodal results rather than a single peak — or vice versa.
  5. Your preconceptions and intuitions about what the results should look like may be a little bit off, skewed somewhat, or just plain completely wrong.

What fraction of shots should be dedicated to probabilistic nature vs. noise?

Testing to characterize effects of noise, errors, and environmental interference on results

  1. Are there any obvious patterns?
  2. Are there test cases which just give really bad results?
  3. Are there test cases which surprisingly give much better than expected results?
  4. Are all of your assumptions born out?
  5. Do you see things that you really didn’t expect?
  6. How do the test results affect how you conceptualize the expectation value for the quantum computation?

Four types of dynamic shot count

  1. Dependence on particular input data and parameter values. Particularly where the quantum circuit itself is dynamically generated based on the input data and parameters.
  2. Portability — to significantly or even somewhat different hardware.
  3. Calibration settings for the particular machine at the particular moment. Calibration can drift as well.
  4. Individual qubits or particular combinations of qubits can have distinct noise and error rates and coherence times.

Dynamic shot count based on parameters

  • 5,000, 25,000, 125,000, and 625,000
  • 10,000, 100,000, 1,000,000, and 10,000,000

Automatic dynamic shot count?

Won’t quantum error correction solve the problem?

Wall clock time

Context of a quantum simulator

  1. Shot count needed to achieve a desired precision of results based on a modeled degree of noise and errors.
  2. Degree of modeled noise and errors required to achieve the desired precision of results for a particular shot count.
  1. Predict what improvements would be needed in real hardware to achieve the desired precision and shot counts.
  2. Calculate or adjust shot count based on measured level of noise and errors on a particular machine. Real machines are periodically recalibrated and the calibration can vary over time.

Design for portability

Is 10 ever a reasonable shot count?

IBM Qiskit default shots is 1024

Is 100 shots reasonable?

What is the maximum shots for a single circuit?

No clear rules, methodology, or guidance for shot count

What’s a good design goal for shot count when designing new algorithms?

Rules of thumb for shot count

Formulas for shot count

Intuitions for shot count

Need to balance shot count and wall clock time

How many digits of precision are reasonable?

How does precision affect shot count?

Is it still too soon to worry too much about shot counts?

One exception: true random number generation — only a single shot required

Don’t confuse circuit repetitions with the iterations or layers of variational methods

Don’t confuse circuit repetitions with experiments

Iterations of variational algorithms

Application experiments

  • Number of application experiments.
  • Number of iterations within each experiment.
  • Shot count for each quantum circuit invocation within each iteration.

Watch out for total circuit executions

Don’t get too used to the nuances and glitches of the quantum machine you used yesterday

How does quantum volume impact shot count?

What is the impact of shots on quantum advantage?

What about quantum supremacy?

Scaling shot count for Shor’s algorithm

Advice to authors of papers on quantum algorithms

  1. Always be explicit about what shot count was needed or recommended.
  2. Be clear about the technical justification behind any shot count.
  3. Provide a solid mathematical formula whenever possible.
  4. Offer a crisp rule of thumb whenever possible.
  5. Preferably give a treatment for how shot count would scale as input grows.
  6. At least speculate what might need to happen for shot count as input grows.
  7. Consider and separate discussion of shot count for an ideal quantum simulator, an at least partially noisy quantum simulator, a current NISQ device, and speculation about better NISQ devices which may be available within 1–2 years.
  8. How shot count might impact quantum advantage. Especially when a very large shot count is needed.

Recommendations

  1. If all you are doing is very trivial demonstrations and relatively minor experiments, you can probably get by using the default setting for shot count. The only really important thing to bear in mind is that generally a quantum computation will be probabilistic and will not produce a single deterministic result, but a frequency distribution from which some inference must be made as to the likely result, or results plural, or range of likely results.
  2. Plan to run experiments to see how sensitive your results are to variations in circuit repetitions.
  3. Reflect on how shot count will, might, or could scale as input size or output precision grows.
  4. If you are authoring a formal or academic paper, please say something explicit about shot counts, how you determined them, how you validated them experimentally, and how they might scale with input size — see the preceding section.

Conclusions

What’s next?

  1. Development of formulas and methods for explicitly calculating shots and circuit repetitions for particular quantum circuits.
  2. Development of less flawed and less noisy qubits. Less noise and fewer errors — mean fewer shots are needed to compensate for them.
  3. Development of quantum computers better shielded from environmental interference.
  4. Improvements in coherence time of qubits. And how that impacts required shot count.
  5. Development of quantum algorithm techniques that can reduce circuit length and hence cumulative errors.
  6. Development of quantum algorithm techniques which can mitigate noise and errors within the quantum circuit, reducing the need for extra shots.

--

--

--

Freelance Consultant

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jack Krupansky

Jack Krupansky

Freelance Consultant

More from Medium

The Hype of Quantum Algorithms

Optimal Superdense Coding

Tracking The Quantum Computing Race — Building a Qubit

Qiskit 101 — How I began my quantum computing journey?

Qiskit banner