Abstract
We study a cardinality-constrained portfolio problem — hold exactly K of n assets —
whose selection landscape we make deliberately rugged with an explicit
anti-correlation term β. We race a D-Wave Advantage quantum annealer against
simulated annealing (SA), giving both solvers the identical selection QUBO and the
identical classical finish (greedy projection to exactly-K, then a convex QP for
the weights), so any objective difference between them measures selection-search
quality and nothing else. In controlled tests on real market data, D-Wave matches
the exact Gurobi MIQP optimum at every β we test, while SA lands 1.2–4.4% above it
at 28 assets and does not improve when given 100× more compute. In a live three-day
deployment (2,088 comparable races), SA never once produced a strictly better
portfolio than the QPU — 0 races, at any tolerance. The result is specific, and we
state its edges: the advantage appears only on rugged (β > 0) landscapes, it is
measured against simulated annealing (with tabu search matched equally trapped at
one configuration), and we have not yet tested parallel tempering.
1. The problem
Every optimization in our system solves the same problem: from a watchlist of n
assets, hold exactly K of them, and weight the holdings to trade expected return
against risk.
minimize (γ/2) · wᵀ Σ w − μᵀ w
w
subject to Σ wᵢ = 1 (fully invested)
exactly K of the wᵢ are > 0 (hold exactly K names)
w_min ≤ wᵢ ≤ w_max on held names
wᵢ = 0 on the rest
w portfolio weights (n-vector, the unknown)
μ expected returns (estimated from a trailing window)
Σ covariance of returns (estimated from a longer trailing window)
γ risk aversion (how much variance costs, in units of return)
K number of names to hold
w_min, w_max floor and cap on each held name's weight
The objective is the classic mean–variance form: −μᵀw pulls toward expected
return, (γ/2)·wᵀΣw charges for variance, and γ sets the exchange rate. Lower
objective = better portfolio.
The middle constraint is what makes this hard. “Exactly K of n” is a combinatorial
choice over C(n,K) subsets sitting on top of a continuous weighting problem —
a mixed-integer quadratic program (MIQP), and cardinality-constrained mean–variance
optimization is NP-hard in general. But the structure is very particular: an
outer combinatorial selection (which K names — the part with C(n,K) local
choices) wrapped around an inner convex QP (the weights on a fixed set of K
names — a single bowl with one global minimum, solvable in milliseconds). The hard
part and the easy part are different kinds of hard, and everything that follows
comes from taking that split seriously.
2. The objective, term by term
The two Markowitz terms are standard. The third term is the one this paper is
about. On selection indicators y_i (y_i = 1 if asset i is held, else 0), we add:
+ β · Σ_{i<j} ρ_ij · y_i · y_j ρ_ij = Σ_ij / √(Σ_ii · Σ_jj)
ρ is the correlation matrix, so the term reads pair by pair:
ρ_ij > 0 assets move together → co-holding them adds +β·ρ_ij (penalized)
ρ_ij < 0 assets move oppositely → co-holding them adds −β·|ρ_ij| (rewarded)
ρ_ij ≈ 0 unrelated → no effect
Mechanically it is an anti-correlation term; economically it is a diversification
preference — it pushes the selection toward a decorrelated basket. That is a
legitimate portfolio objective, not a device to flatter the quantum hardware, and
we are equally plain about the flip side: β changes the problem. A β > 0 optimum
trades some risk-adjusted return for decorrelation, so every comparison in this
paper is made against the exact optimum of the β-modified objective, computed by
an exact MIQP solver on the same instance.
Why add it at all, when the variance term (γ/2)·wᵀΣw already prices correlation?
Because the two terms act at different layers:
- The variance term prices correlation only as a weight-aggregated total. The
inner QP can hedge a correlated pair after the fact — shrink both weights — and
the selection never feels it. - The β term acts on the support itself, per pair, before weights exist. It is
a preference about which names you hold, independent of how much of each.
The β term therefore competes directly with return-seeking at the selection layer:
an asset that is excellent for the return term can be penalized for correlating
with another held name. Competing pulls that cannot all be satisfied at once is
the textbook recipe for a rugged optimization landscape — which is the point, and
the subject of the next section.
3. Frustration: why β makes the problem hard
Take three assets that are all pairwise positively correlated — in an equity
universe, the common case:
A
/ \
ρ>0 / \ ρ>0 β says: do not co-hold any of these pairs
/ \ K says: you must hold names
B ───── C pick any two → one edge is violated
ρ>0
Whichever two of the three you include, exactly one penalized pair rides along.
The triangle cannot be satisfied, only traded off. Physics has a name for this:
frustration — the same structure as three antiferromagnetic spins on a
triangle, where no assignment satisfies all three bonds. A real correlation matrix
is mostly positive, so the selection graph is laced with these triangles, and the
K-of-n constraint forces the solver into them.
Frustration is what shapes the landscape. With no β the selection objective is
dominated by the smooth mean–variance surrogate: basins are wide, local search
rolls downhill to the answer, and simulated annealing is exactly the right tool.
With β > 0, the frustrated couplings carve that surface into many near-degenerate
minima separated by barriers — subsets that are each locally excellent, mutually
exclusive, and hard to travel between by single-name swaps. A local searcher that
commits to the wrong basin has to climb out to fix its mistake.
The ruggedness is a dial, and it has a window:
- β too small — the smooth terms dominate; every solver we test finds the same
optimum and the race is a tie decided by speed. - β too large — pure anti-correlation dominates; the problem becomes easy
again (§5.2 shows this at 18 assets, where β at 1–2× the objective scale returns
to a tie). - β ≈ 0.25–0.5× the objective’s coefficient scale — return-seeking and
decorrelation genuinely compete. This is the frustrated regime, and it is where
the solvers separate.
In the live deployment (§6) we ramp β with basket size — off below n = 15, rising
linearly to its peak at the full 28-asset universe:
β(n) = 0.4 · [ 0.3 + 0.7 · (n − 15) / (28 − 15) ] 15 ≤ n ≤ 28
β(n) = 0 n < 15
so the largest baskets are also the most frustrated ones.
4. The hybrid pipeline
4.1 What didn’t work, and why (the encoding lesson)
Our first attempt put the whole problem on the annealer: encode weights as binary
units and fold every constraint into the QUBO as penalty terms, including
cardinality as A·(Σy − K)². This is the standard recipe, and it quietly rigs the
race against the QPU twice over:
(Σy − K)²expands into a coupling between every pair of selection bits — a
complete graph. Physical qubits on the Pegasus topology have limited
connectivity, so a clique must be embedded through long chains of qubits acting
as one logical variable. Long chains break; on hardware only ~2–10% of reads
decoded to a feasible portfolio. SA, running the logical QUBO in float64 with no
embedding, stayed ~100% feasible. Same math, one crippled competitor.- The penalty weight A must dominate the objective to enforce K — and then it
does dominate: the finance objective becomes a small perturbation on a
constraint-satisfaction problem. In our ablation the penalized encoding never
reached the true optimum at any penalty strength we tried (§5.1); the solver
returns feasible-but-mediocre subsets because feasibility is nearly all it can
see.
The QPU was losing to its own encoding, not to its hardware.
4.2 The fix: a selection-only QUBO and a shared classical finish
We stopped asking the annealer to do arithmetic and let it do only the
combinatorial part:
player triggers an optimization
│
▼
estimate μ, Σ from market data
│
▼
SELECTION QUBO over y ∈ {0,1}ⁿ (one bit per asset: hold it or not)
objective-only, penalty-free:
equal-weight mean–variance surrogate + β·Σ ρ_ij y_i y_j
│
├────────────────────────┐
▼ ▼
SA (CPU) D-Wave Advantage (QPU)
simulated annealing Pegasus clique sampler
│ bitstrings │ bitstrings
▼ ▼
┌────────── SHARED CLASSICAL FINISH (identical for every read) ──────────┐
│ greedy projector → repair to exactly K held │
│ convex QP → optimal weights w on those K (Σw=1, box) │
└──────────────────────────────────────────────────────────────────────────┘
│
▼
score every candidate by the TRUE objective(w) →
winner = best portfolio; speed reported alongside
No budget or cardinality penalties touch the QUBO — it is n sparse bits of pure
“which subset.” Exactly-K is enforced classically, by a greedy projector that
repairs every read (drop the weakest names / add the best, by surrogate value),
and the weights come from a convex QP on the chosen support. Three consequences:
- Feasibility is 100% by construction, for both solvers. The feasibility
confound of §4.1 is gone. - The finish is identical for both solvers, on every read. Given the same
chosen subset, SA and D-Wave produce the same weights and the same objective.
They can differ only by choosing different subsets — so an objective gap is a
pure measurement of selection-search quality. - The QUBO stays small and sparse — n binary variables rather than n × bits —
which is what lets the annealer embed far past the old wall (§5.4).
4.3 Hybrid does not mean approximate
Splitting a problem between machines usually smells like a compromise. Here the
split follows the structure from §1, and each machine gets the subproblem it is
provably suited to:
- The QPU gets the outer selection — the frustrated combinatorial search that
has no convex structure to exploit. - The classical convex QP gets the inner weights — a problem with a single
global minimum, which it finds in about 2 ms. On a fixed support our QP
reproduces Gurobi’s weights to within 2e-6 in any coordinate and 1e-9 in
objective (§5.4): the classical stage is exact, not heuristic.
So the pipeline’s answer is not “approximately the MIQP optimum.” Whenever the
sampler finds the optimal support, the convex finish returns the proven MIQP
optimum — identically what Gurobi certifies on the full problem. The only open
question in any race is whether a sampler found the right subset, and that is
exactly the quantity we score.
Scoring: the winner is the solver with the best portfolio objective; objectives
within 0.5% relative (|a−b| / max(|a|,|b|)) count as a tie and fall through to
speed. Reported solver time is compute time — for the QPU that is access time on
the annealer, deliberately excluding network and cloud-queue latency, which are
scheduling artifacts rather than properties of the computation. We state that
convention wherever we report speed.
5. Controlled results
All controlled runs use real market data, best-of-500 reads per solver, the shared
classical finish of §4.2 for every read, and an exact Gurobi MIQP solve of the same
instance as the optimality oracle. “Gap” is the relative distance of a solver’s
best portfolio objective from that proven optimum (lower is better; 0.00 means the
solver found the exact optimum).
5.1 The encoding ablation: penalty-free vs penalized
C2 is the penalty-free selection QUBO of §4.2; C3 is the same QUBO plus the
cardinality penalty A·(Σy − K)²:
size C2 (penalty-free) C3 (cardinality penalty)
12a/K5 ~0% (optimal) ~0% (tie — instance too small)
18a/K6 0% exact support 1.5–2.5% short
28a/K8 0% exact support 1.7–2.8% short
Sweeping C3’s penalty strength at 28 assets never closes the gap — 0.6–2.5% at
every strength tried; only penalty = 0 (which is C2) reaches the optimum. The
mechanism is not chain breaks (C3’s are ~0%): it is objective washout. A penalty
large enough to enforce K dominates the energy scale, the finance objective becomes
a small perturbation, and the sampler optimizes constraint satisfaction instead of
portfolios. Constraints belong in the classical finish, not in the QUBO.
5.2 The β sweep: ruggedness separates the solvers
C2 + β at 18 and 28 assets, three repeats per cell, gaps to the Gurobi optimum of
the same β-modified objective. uniq counts distinct subsets each solver’s 500
reads explored; DW win counts repeats where D-Wave’s portfolio was strictly
better than SA’s:
size β/scale SA gap% D-Wave gap% DW win SA uniq DW uniq
18a 0.00 0.00 0.00 0/3 1 4
18a 0.25 46.45 0.00 3/3 1 34
18a 0.50 113.73 0.00 3/3 1 78
18a 1.00 0.00 0.00 0/3 2 122
18a 2.00 0.00 0.00 0/3 4 163
28a 0.00 0.04 0.00 3/3 1 9
28a 0.25 3.51 0.00 3/3 1 113
28a 0.50 1.20 0.00 3/3 2 194
28a 1.00 4.41 0.00 3/3 1 277
28a 2.00 4.06 0.00 3/3 1 362
(A 12-asset sweep ties at every β — the instance is too small to separate the
solvers, so we omit its rows.) Read the two ends first. At β = 0 the landscape is smooth and both solvers find the
optimum — SA faster; a smooth problem gives the annealer nothing to exploit. At
β = 1–2× scale on 18 assets, pure anti-correlation dominates and the problem is
easy again. In between — the frustrated window of §3 — D-Wave finds the exact
optimum at every β while SA does not, in 3 of 3 repeats. Two honesty notes: the
18-asset 46–113% figures sit on a near-zero objective magnitude (β nearly cancels
risk − return there), so we treat the 28-asset band — 1.2 to 4.4% off optimum, on
a clean magnitude — as the solid evidence. And the uniq column already hints at
the mechanism: SA’s 500 reads collapse to 1–2 subsets; D-Wave’s explore 34–362.
5.3 SA is trapped, not under-powered
The obvious objection: give SA more compute. We did — up to 100× the sweeps and 4×
the reads at 18 assets (β = 0.5× scale), with D-Wave fixed at 500 reads / 20 µs:
SA config SA gap% SA time distinct subsets
500r × 1,000 sw 113.73 0.04 s 1
500r × 5,000 sw 113.73 0.20 s 1
500r × 20,000 sw 113.73 0.79 s 1
500r × 100,000 sw 113.73 3.87 s 1
2,000r × 20,000 sw 113.73 3.15 s 1
D-Wave (fixed) 0.00 0.056 s 78
The gap does not move. Every SA run converges to the same single subset and more
sweeps only descend deeper into the same basin. We repeated the stress test at 40
assets (K = 13, β = 0.25× scale) with an escalation to 50,000 reads and 20,000
sweeps, and added tabu search:
method gap% distinct subsets
SA (500r/1,000sw) 1.27 1
SA (50,000 reads) 1.27 1
SA (20,000 sweeps) 1.27 1
SA-heavy (10,000 sweeps) 1.27 1
Tabu search 1.27 –
D-Wave 0.00 (QPU 124 ms, 0.07% chain breaks)
Three different classical local searchers stall at the identical 1.27% gap — the
basin is a property of the landscape, not of a weak SA implementation. The
annealer’s sample diversity (many distinct subsets per 500 reads) is what escapes
it. Chain breaks stay low (0–4.6% across these runs) and our scoring pipeline
matches the oracle’s objective arithmetic to ~1e-19, so the gap is genuine
solution quality — not a feasibility, embedding, or scoring artifact.
5.4 Scaling: where the advantage lives, and what caps it
Scaling the select encoding on a 190-ticker daily-data universe (β = 0.4× scale,
Pegasus clique embedding, gaps to the best objective found by any solver):
n SA % Pegasus % chain-brk % Gurobi MIQP (30 s)
28 1.75 0.00 0.07 OPTIMAL
40 2.11 0.00 0.06 OPTIMAL
60 3.47 0.71† – TIME_LIMIT (37% MIP gap)
80 1.96 1.80 0.41 TIME_LIMIT (63% MIP gap)
100 3.57 2.40 0.48 TIME_LIMIT (104% MIP gap)
125 0.00 3.31 1.67 (past solver size cap)
150 0.00 7.08 11.36 —
175 0.00 8.96 16.91 —
† transient cloud failure on the recorded n=60 sweep; value from the first sweep.
An independent sweep on a 92-name universe replicates the pattern: D-Wave beats SA
at 40 assets (2.18% vs 6.24%) and 60 (1.38% vs 4.93%), and loses at 80 (6.53% vs
3.66%). Three boundaries emerge, and they are different machines’ boundaries:
- The exact oracle gives out first. Gurobi certifies optimality only to ~40
assets on the rugged objective; from 60 up it hits its time limit with MIP gaps
of 37–104%, and past ~100 the dense quadratic cannot even be built under our
solver license. Past that point, no tool in this study can certify optimality —
except the convex weight stage. - The weight QP is never the limit. On every winning support from n = 28 to
175, re-solving the weights classically matches an independent Gurobi convex
solve to ≤ 1.7e-6 in any weight and ≤ 2.3e-10 in objective, in 0.5–2.3 ms
(§4.3’s claim, measured). - The selection embedding is the binding ceiling. Chain breaks climb from
1.7% at n = 125 to 11.4% at 150 and 16.9% at 175, and the QPU’s gap grows with
them while plain SA overtakes it. Booth-grade quality holds to roughly 125
assets on Pegasus.
The regime where the annealer strictly beats simulated annealing on quality is
roughly 28–60 assets — which usefully overlaps the regime where the exact
classical oracle has already become intractable.
6. Field results: three days at the booth
We ran the pipeline as a public demo: each player picked a watchlist of up to 28
assets, set three sliders (risk aversion γ, position cap w_max, hold count K), and
received a virtual $10,000 bankroll; every press of “Optimize” raced SA against
the D-Wave QPU live, with the β ramp of §3 making larger baskets more frustrated.
We report the window 2026-06-24 00:00 → 2026-06-26 15:00 Eastern. This is
field validation under live, player-generated conditions — self-selected players,
one venue, three days — not a controlled benchmark; the controlled evidence is §5.
races in window 2,129
both solvers returned a result 2,129
comparable (both feasible, finite) 2,088
not comparable 41
SA side unusable (infeasible/non-finite) 36
D-Wave side unusable 5
Head-to-head portfolio quality on the 2,088 comparable races, using the live
system’s own materiality threshold (0.5% relative objective gap):
D-Wave strictly better portfolio 878 42.0%
objective tie 1,210 58.0%
SA strictly better portfolio 0 0.0%
Dropping the threshold to zero — any strictly lower objective counts:
D-Wave objective strictly lower 1,044 50.0%
exactly equal 1,044 50.0%
SA objective strictly lower 0 0.0%
SA never once produced a strictly better portfolio than the QPU in this window
— 0 of 2,088 races, at any tolerance. When the two solvers disagreed (1,044
races at zero tolerance), the annealer held the better portfolio in all 1,044.
This is the field echo of §5.3: the classical searcher never finds a subset the
annealer missed, while the reverse happens constantly. And when D-Wave’s win was
material, it was not marginal: the median quality win improved the portfolio
objective by 2.63% (mean 15.62%, maximum 199%) — the long tail is races where SA
sat in a basin far from the optimum.
By basket size (β ramps with n, so larger baskets are more rugged):
n races D-Wave win % tie % SA win %
15 1,233 39.4 60.6 0.0
16 223 58.3 41.7 0.0
17 272 33.8 66.2 0.0
28 150 47.3 52.7 0.0
(Basket sizes 18–27 each drew fewer than 100 races in this window — too noisy for
rates; every one of those cells also shows SA win % = 0.0.) The direction matches
§3 — the win rate is higher at the frustrated end (47.3% at n = 28) than at the
smooth end (39.4% at n = 15) — but we are explicit that in this window the ramp is
modest and the n = 28 cell holds 150 races; we rest the ramp claim on the
controlled sweep of §5.2, and the zero-SA-wins claim on all 2,088 races.
Speed, under the convention of §4.3 (QPU time = access time on the annealer;
network and queue latency excluded by design):
median mean p95
D-Wave 0.1145 s 0.1113 s 0.1194 s
SA 0.1190 s 0.1278 s 0.2120 s
D-Wave faster in 67.9% of comparable races
The QPU’s distribution is nearly flat — anneal-and-read is a fixed-cost operation
— while SA’s classical search carries a heavier tail. Speed only ever decided
ties: every race with a quality difference was scored on quality. Finally, the 41
non-comparable races lean the same way as everything else: SA was the unusable
side 36 times, D-Wave 5.
7. Limitations and next steps
The comparator is simulated annealing. Our claim is against SA (dwave-neal),
the classical solver the system races in production. Tabu search matched SA’s
trapped gap at the one 40-asset configuration we stress-tested (§5.3); parallel
tempering — a classical method built specifically for rugged landscapes — is
untested, and testing it is the next rigor step. We claim a measured advantage
over SA, not over all classical methods.
β changes the problem. The advantage exists only inside the rugged β window;
at β = 0 the solvers tie and SA is faster. We consider β a legitimate
diversification objective (§2), but a reader who rejects that premise should read
this paper as: on frustrated selection objectives of this family, the annealer
out-searches SA.
Speed is reported, not claimed. QPU time is annealer access time, excluding
network and cloud-queue latency by design; SA’s wall clock is often comparable or
better. Quality — which subset the solver finds — is the claim; speed only breaks
ties.
Estimation noise. μ and Σ come from finite trailing windows (168 h and 720 h
of hourly data in the live system; 63 d and 252 d daily in the scaling studies).
Objective magnitudes are small (~1e-3 to 1e-7), which is why every comparison in
this paper is relative, with the max(|a|,|b|) denominator stated in §4.3.
The field data is field data. Three days, one venue, self-selected players,
basket sizes concentrated where players actually played (59% of comparable races
at n = 15). It corroborates the controlled results; it does not replace them.
8. Reproduction
Everything needed to reproduce the controlled results of §5 with D-Wave Leap
access:
hardware D-Wave Advantage (Pegasus topology), clique embedding;
the ceiling study also used Advantage2 (Zephyr, clique limit 105)
sampler DWaveCliqueSampler; chain-strength multiplier 3.0; anneal 20 µs;
500 reads per solve (scaling runs: production reads-for-size curve)
classical racer simulated annealing via dwave-neal, 500 reads × 1,000 sweeps
(robustness sweeps up to 50,000 reads / 100,000 sweeps, §5.3)
encoding select: one binary variable per asset, penalty-free QUBO =
equal-weight (1/K) mean–variance surrogate + β·Σ_{i<j} ρ_ij y_i y_j
β fraction × max|QUBO coefficient|; fractions 0.25–2.0 swept in §5.2;
live ramp β(n) = 0.4·[0.3 + 0.7·(n−15)/13] for 15 ≤ n ≤ 28
finish greedy projector to exactly K (drop weakest / add best by
surrogate value), then box-constrained convex QP on the support
(Σw = 1, w_min ≤ w ≤ w_max) — verified against an independent
Gurobi convex solve to ≤1.7e-6 per weight (§5.4)
oracle Gurobi MIQP of the full β-modified problem, 30–60 s limit
data §5.2–5.3: live hourly prices, μ over 168 h, Σ over 720 h;
§5.4: 5 y of daily bars (92- and 190-ticker universes),
μ over 63 d, Σ over 252 d, winsorized ±50%/day
scoring true objective(w) on every repaired read; relative comparison
rel_gap = |a−b| / max(|a|,|b|); materiality threshold 0.005
Our experiment harnesses and the booth dataset are not published; the parameters
above are complete, and a third party with annealer access can rebuild the
pipeline from this section and §4 alone.