This notebook is a self-contained mathematical and computational companion to the Erdős minimum overlap section in the paper.
The task template, including the prompt given to the agents, is available here.
The outline is as follows:
- 1. Problem Definitions
- 2. S1. A new lower bound of .
- 3. Comparison with AlphaEvolve on the upper bound
- 4. Additional findings
The required packages are:
- Python 3.10 or later
NumPyIPython- A C compiler with the
MPFI,MPFR, andGMPlibraries
Run the notebook from its own directory, from top to bottom. The lower-bound replay uses directed MPFI arithmetic and stops if these libraries are unavailable; it has no floating-point substitute. Set MPFI_PREFIX only for a nonstandard installation. The four interval rows usually take 15–30 minutes.
1. Problem Definitions
Erdős’s minimum-overlap problem asks how evenly two complementary parts of an interval can avoid one another under translation. The constant below is its continuum form for balanced partitions of long integer intervals.
Let . An admissible profile is a measurable function , supported on , with . Put and
Both and are nonnegative and have mass one. Hence , , and .
The published bounds used for comparison in the paper are
from Kim and Pilanci (2026) and Ye et al. (2026), respectively.
The next cell loads the shared packages, authenticates both local inputs, and reads the certificate arrays without pickle.
Show code
Code cell 3 · In [1]
from __future__ import annotations
from decimal import Decimal, getcontext
from fractions import Fraction
from hashlib import sha256
from pathlib import Path
from tempfile import TemporaryDirectory
import os
import shutil
import subprocess
import sys
import numpy as np
from IPython.display import Markdown, display
ARTIFACT_DIR = Path.cwd()
DATA_PATH = ARTIFACT_DIR / "autocorr_6_5_certificate_data.npz"
VERIFIER_PATH = ARTIFACT_DIR / "mpfi_positive_budget.c"
EXPECTED_SHA256 = {
DATA_PATH.name: "c9fdb2881874c4797acca21be2ab4fcf7b7a43e75ed763ce1f17d17e269e7453",
VERIFIER_PATH.name: "a967d8dfd18456a80c79984c69eb28b88a7548bf001c505455f97808a704274d",
}
def file_sha256(path: Path) -> str:
digest = sha256()
with path.open("rb") as handle:
for block in iter(lambda: handle.read(1 << 20), b""):
digest.update(block)
return digest.hexdigest()
for path in (DATA_PATH, VERIFIER_PATH):
assert path.is_file(), f"missing required artifact: {path.name}"
observed = file_sha256(path)
assert observed == EXPECTED_SHA256[path.name], (path.name, observed)
with np.load(DATA_PATH, allow_pickle=False) as bundle:
assert str(bundle["schema"]) == "autocorr-6.5-public-certificate-v1"
artifact_data = {key: np.array(bundle[key], copy=True) for key in bundle.files}
print("artifact hashes verified")
print("python", sys.version.split()[0], "numpy", np.__version__)
Saved output 1
artifact hashes verified python 3.11.13 numpy 2.4.0
2. S1. A new lower bound of .
Theorem 2.1.
The minimum-overlap constant satisfies
We first prove why the finite certificate format implies a global theorem, then replay its four rows.
2.1 Phase-coupled Fourier support
Lemma 2.2.
For a real frequency , define
with . If , then every admissible profile satisfies
Proof. Use the Fourier convention and write . With the stated correlation orientation, the transform of is . Since , comparison of real and imaginary parts gives, up to the irrelevant sign convention for ,
Completing the square in yields
At a zero of , the same identities instead give and .
An equivalent form, used by the certificate, is the support inequality
Indeed, substitute the preceding inequality and maximize the resulting concave quadratic over . At a zero of , both sides of the support inequality are zero or the left side is negative, so the support form extends there as well. This formulation retains the sine information and permits arbitrary real, rather than only harmonic, frequencies.
2.2 The four-row certificate
Lemma 2.3 (one valid dual row).
Choose finitely many real frequencies , numbers , and a quadratic polynomial, and put
If
then every admissible with overlap first moment obeys
Proof. Let . Since and ,
The first two overlap moments are
To see the second identity, view as the density of , where independent variables and have densities and . Their means are opposite, while . Applying Lemma 2.2’s support form frequency by frequency now gives the stated .
The box constraint and the bathtub principle imply , hence . Reflection changes the sign of without changing , so it suffices to cover .
Certificate verification.
Each stored row is a decimal specification of and the triples . MPFI parses each decimal into an outward-rounded 128-bit interval. The accompanying C verifier then:
- covers all of by recursively bisected cells;
- encloses on each cell by a midpoint Taylor formula and the global bound for ;
- integrates sign-certified positive runs with an interval antiderivative;
- charges every unresolved terminal cell by , so a missed or tangential zero cannot invalidate the result; and
- evaluates the Fourier support charge with directed rounding, rejecting any row for which is not rigorously positive when , and subtracts a conservative guard from the constant coefficient of .
Thus budget_pass=true proves the continuous positive-part condition, not merely a grid sample. The final cover calculation uses the printed outward lower coefficients as exact decimal rationals.
Optional regeneration. The numerical witnesses can be regenerated with witness_generation/generate_witness.py. Regeneration is not required to verify the lower bound; the source is included for completeness.
The next cell authenticates and unpacks the four dual rows, then compiles the directed-interval verifier used by the certificate.
Show code
Code cell 5 · In [2]
ROW_KEYS = [f"lower_row_{j}_tsv" for j in range(4)]
ROW_SHA256 = [
"214a16dad78ed1903748634b1981b5a706c39a9cdfe2c761ba22ea3e421811f3",
"498bc70020abbcb314e2d33db3568e59c40bbe889f827f3e53f101b7b5ed56a8",
"d28cec137151a7ab16f91382c5e7c25cab4a3e3dc6cc5b4885e1d98e4f8462e8",
"9077fdba3821e09f667cd1ddf1389654b6a5aae477df775bcf0c899de2f282fe",
]
ROW_RUNS = [(20, 14), (20, 12), (20, 14), (20, 14)]
temporary_context = TemporaryDirectory(prefix="autocorr_certificate_")
temporary_dir = Path(temporary_context.name)
row_paths = []
for j, (key, expected_hash) in enumerate(zip(ROW_KEYS, ROW_SHA256)):
payload = artifact_data[key].tobytes()
assert sha256(payload).hexdigest() == expected_hash
row_path = temporary_dir / f"dual_row_{j}.tsv"
row_path.write_bytes(payload)
row_paths.append(row_path)
def compile_mpfi_verifier() -> Path:
compiler = shutil.which("gcc") or shutil.which("cc")
if compiler is None:
raise RuntimeError("A C compiler is required for the directed-interval replay.")
prefix_strings = [
os.environ.get("MPFI_PREFIX"),
os.environ.get("CONDA_PREFIX"),
sys.prefix,
]
prefixes = [Path(item) for item in prefix_strings if item]
selected = next(
(
prefix for prefix in prefixes
if (prefix / "include/mpfi.h").is_file()
and any((prefix / "lib").glob("libmpfi.*"))
),
None,
)
binary = temporary_dir / "mpfi_positive_budget"
command = [compiler, "-O3", "-std=c11"]
if selected is not None:
command += [
f"-I{selected / 'include'}",
str(VERIFIER_PATH),
f"-L{selected / 'lib'}",
f"-Wl,-rpath,{selected / 'lib'}",
]
else:
# Standard system installs expose mpfi.h and the libraries on the
# compiler's default search paths.
command += [str(VERIFIER_PATH)]
command += ["-lmpfi", "-lmpfr", "-lgmp", "-lm", "-o", str(binary)]
completed = subprocess.run(command, text=True, capture_output=True)
if completed.returncode != 0:
raise RuntimeError(
"MPFI verifier compilation failed. Install MPFI/MPFR/GMP or set "
"MPFI_PREFIX to their common prefix.\n" + completed.stderr
)
return binary
verifier_binary = compile_mpfi_verifier()
print("compiled", verifier_binary.name)
print("row atom counts:", [int(path.read_text().split()[3]) for path in row_paths])
Saved output 1
compiled mpfi_positive_budget row atom counts: [1090, 1098, 400, 400]
The next cell runs the verifier on all four rows and rejects any row whose continuous positive-part budget is not certified.
Show code
Code cell 7 · In [3]
def parse_verifier_output(output: str) -> dict[str, str]:
fields: dict[str, str] = {}
for line in output.splitlines():
if "=" in line and " " not in line.split("=", 1)[0]:
key, value = line.split("=", 1)
fields[key] = value
required = {
"total_positive_part_upper",
"support_charge_upper",
"quadratic_c0_lower",
"quadratic_a1_lower",
"quadratic_a2_lower",
"budget_pass",
}
assert required <= fields.keys(), required - fields.keys()
assert fields["budget_pass"] == "true"
return fields
verified_rows = []
for j, (row_path, (initial_cells, max_depth)) in enumerate(zip(row_paths, ROW_RUNS)):
print(f"verifying row {j + 1}/4 ...", flush=True)
result = subprocess.run(
[verifier_binary, row_path, str(initial_cells), str(max_depth)],
check=True,
text=True,
capture_output=True,
timeout=7200,
)
fields = parse_verifier_output(result.stdout)
verified_rows.append(fields)
print(
f"row {j}: budget <= {fields['total_positive_part_upper']}; "
f"c0 >= {fields['quadratic_c0_lower']}"
)
assert len(verified_rows) == 4
Saved output 1
verifying row 1/4 ...
Saved output 2
row 0: budget <= 9.99999999976307387212127448070e-1; c0 >= 3.80553385726178104971549433082e-1 verifying row 2/4 ...
Saved output 3
row 1: budget <= 9.99999690243467185352334005026e-1; c0 >= 3.80535903684482958019035618142e-1 verifying row 3/4 ...
Saved output 4
row 2: budget <= 9.99681961109477952102217976026e-1; c0 >= 3.80157336168305731734844629480e-1 verifying row 4/4 ...
Saved output 5
row 3: budget <= 9.99663207801089422162609863214e-1; c0 >= 3.69791719552889297350385845427e-1
The next cell checks the four mean intervals with exact rational arithmetic, prints their certified floors, and computes the gap-closing percentage used in the paper.
Show code
Code cell 9 · In [4]
MEAN_INTERVALS = [
(Fraction("0"), Fraction("0.00259038")),
(Fraction("0.00259038"), Fraction("0.035")),
(Fraction("0.035"), Fraction("0.065")),
(Fraction("0.065"), Fraction("1")),
]
CLAIMED_LOWER = Fraction(380552, 1_000_000)
def row_quadratic(fields: dict[str, str], mean: Fraction) -> Fraction:
c0 = Fraction(fields["quadratic_c0_lower"])
a1 = Fraction(fields["quadratic_a1_lower"])
a2 = Fraction(fields["quadratic_a2_lower"])
return c0 + a1 * mean + a2 * mean * mean / 2
cover_table = []
global_lower = None
previous_endpoint = Fraction(0)
for j, (fields, (lo, hi)) in enumerate(zip(verified_rows, MEAN_INTERVALS)):
assert lo == previous_endpoint
a2 = Fraction(fields["quadratic_a2_lower"])
assert a2 < 0 # concavity: the minimum on a closed interval is at an endpoint
q_lo, q_hi = row_quadratic(fields, lo), row_quadratic(fields, hi)
interval_lower = min(q_lo, q_hi)
assert interval_lower > CLAIMED_LOWER
global_lower = interval_lower if global_lower is None else min(global_lower, interval_lower)
cover_table.append((j, str(lo), str(hi), q_lo, q_hi, interval_lower))
previous_endpoint = hi
assert previous_endpoint == 1
assert global_lower is not None and global_lower > CLAIMED_LOWER
getcontext().prec = 50
def decimal(frac: Fraction, digits: int = 30) -> str:
return f"{Decimal(frac.numerator) / Decimal(frac.denominator):.{digits}f}"
table = [
"| row | certified mean interval | $q(\mathrm{left})$ | $q(\mathrm{right})$ | interval floor |",
"|---:|---:|---:|---:|---:|",
]
for j, lo, hi, q_lo, q_hi, floor in cover_table:
table.append(
f"| {j} | $[{lo},{hi}]$ | {decimal(q_lo, 15)} | "
f"{decimal(q_hi, 15)} | {decimal(floor, 15)} |"
)
display(Markdown("\n".join(table)))
print("certified global lower =", decimal(global_lower, 30))
print("margin above 0.380552 =", decimal(global_lower - CLAIMED_LOWER, 30))
PREVIOUS_LOWER = Fraction(37912, 100_000)
PUBLISHED_UPPER = Fraction(380868, 1_000_000)
fraction_closed = (global_lower - PREVIOUS_LOWER) / (PUBLISHED_UPPER - PREVIOUS_LOWER)
assert Fraction(81, 100) < fraction_closed < Fraction(83, 100)
print("published gap closed =", decimal(100 * fraction_closed, 6), "%")
Saved output 1
Saved output 2
certified global lower = 0.380552257389830222107376462494 margin above 0.380552 = 0.000000257389830222107376462494 published gap closed = 81.936922 %
Proof of Theorem 2.1. The four closed intervals meet exactly and cover ; reflection covers negative first moments. Lemma 2.3 therefore proves Theorem 2.1 for every admissible measurable profile.
Station Reference.
The certificate architecture is recorded in Archives #22, #31, and #49. The dense-frequency rows and completion work are recorded in Archive #53 and Research Center Evaluations #1038, #1042, and #1045. The run produced the phase-coupled relaxation, decisive witnesses, and a certificate above . The directed MPFI replay and one same-method bridge row were completed afterward; all four serialized rows are verified without trusting the optimizer.
Related Work. White (2023) retained complex Fourier phase data and used conic optimization. Kim and Pilanci (2026) added Toeplitz and Bochner moment constraints and proved the previous lower bound . The theorem above instead uses a generator-eliminated support inequality at arbitrary real frequencies, a dense nonharmonic frequency comb, and an outward-rounded certificate. It appears to give the first published lower bound above .
3. Comparison with AlphaEvolve on the upper bound
Theorem 3.1.
There is an admissible 2,048-cell step profile whose continuum overlap satisfies
Proof. Give the interval cells of width and let be the value on cell . Every stored binary64 value is interpreted as its exact dyadic rational. The stored sum misses by a tiny positive amount; the code adds that exact deficit to one interior coordinate and proves that all values remain in .
At an aligned shift ,
where only indices for which both cells exist are included. The code enumerates all aligned shifts using Python integers. For arbitrary real shift, the correlation of two equal-grid step functions is affine between consecutive aligned shifts, so its global maximum occurs at one of the enumerated breakpoints. The finite exact computation is therefore a proof for every real shift, not a sampled approximation.
The next cell interprets the stored profile as exact dyadic rationals, repairs its mass, and checks every aligned shift with integer arithmetic.
Show code
Code cell 12 · In [5]
values = artifact_data["upper_profile"]
repair_index = int(artifact_data["upper_repair_index"])
assert values.dtype == np.float64 and values.shape == (2048,)
assert np.isfinite(values).all()
n = values.size
ratios = [float(value).as_integer_ratio() for value in values]
exponent = max(denominator.bit_length() - 1 for _, denominator in ratios)
scale = 1 << exponent
weights = [
numerator * (scale // denominator)
for numerator, denominator in ratios
]
deficit = n * scale // 2 - sum(weights)
assert deficit > 0
assert 0 <= weights[repair_index] < scale
assert weights[repair_index] + deficit <= scale
weights[repair_index] += deficit
assert sum(weights) == n * scale // 2
assert 0 <= min(weights) <= max(weights) <= scale
maximum_numerator = -1
maximizing_shifts = []
for shift in range(-(n - 1), n):
start, stop = max(0, -shift), min(n, n - shift)
numerator = sum(
weights[j] * (scale - weights[j + shift])
for j in range(start, stop)
)
if numerator > maximum_numerator:
maximum_numerator = numerator
maximizing_shifts = [shift]
elif numerator == maximum_numerator:
maximizing_shifts.append(shift)
upper_score = Fraction(2 * maximum_numerator, n * scale * scale)
expected_upper_score = Fraction(
2174525160989298668356932628848891185116051721,
5708990770823839524233143877797980545530986496,
)
assert upper_score == expected_upper_score
assert upper_score < Fraction(380895, 1_000_000)
print(f"cells = {n}")
print(f"common dyadic denominator = 2^{exponent}")
print("exact mass repair =", decimal(Fraction(deficit, scale), 40))
print("maximizing aligned shifts =", maximizing_shifts)
print("maximum overlap =", decimal(upper_score, 40))
print("proved mu < 0.380895")
Saved output 1
cells = 2048 common dyadic denominator = 2^91 exact mass repair = 0.0000000000000480379612293450615331265127 maximizing aligned shifts = [697] maximum overlap = 0.3808948460912474797310388265227645656157 proved mu < 0.380895
Station Reference.
The profile comes from Research Center Evaluation #1681, initialized from the seed retained by Evaluation #1637; Archive #41 records the construction lineage. The exact mass correction and integer replay verify the stored profile without altering the construction method.
Related Work. Georgiev et al. (2025) reported , and Yüksekgönül et al. (2026) later reported an asymmetric construction near . Ye et al. (2026) subsequently gave the sharper published upper bound . Thus it independently improves the AlphaEvolve value while remaining above the current published record.
4. Additional findings
These are additional results that may be interesting but are not included in the spotlight.
4.1 Exact geometry under averaging and interpolation
The minimum-overlap problem is invariant under reflection, but this symmetry does not imply that averaging a profile with its reflection improves it. We derive an exact identity showing that reflection averaging adds a correction equal to one quarter of the autocorrelation of the asymmetric part; at zero shift, this is the positive penalty . We also prove that interpolation between two profiles acquires an analogous correction determined by the autocorrelation of their difference. These identities explain why asymmetric profiles can outperform their symmetrizations and why a straight path between two good constructions can pass through profiles with substantially larger overlap.
Write
Theorem 4.1 (symmetrization identity).
If and , then
Proof. Since , its first term is linear in and its second is quadratic. The evenness of gives . Expanding bilinearly and collecting terms leaves
At the correction is . Thus reflection averaging need not improve the objective: it can create a new central-overlap cost.
Theorem 4.2 (chord-curvature identity).
For admissible profiles and ,
Proof. The term is linear in . Expanding and subtracting it from that linear term gives
This exact correction explains why a straight interpolation between two good profiles can cross a substantial overlap barrier.
Station Reference.
The symmetrization identity is recorded in Archive #19 and Research Center Evaluations #351 and #373. The chord-curvature identity is recorded in Archive #60 and Evaluations #1124, #1129, and #1143.
Related Work. Correlation algebra, reflection symmetry, and polarization identities for quadratic functionals are classical. White (2023) analyzed an even-overlap restriction in a simplified linear program before passing to the unrestricted problem with a convex program. Applied here, polarization gives exact formulas for the change in overlap under reflection averaging and affine interpolation.
4.2 A rigorous local self-energy plateau
We use the self-energy to follow the minimum-overlap problem from the constant profile toward Boolean profiles. For the constrained optimum , the zero-shift overlap gives the universal lower bound , and we prove that this bound is attained throughout a nontrivial interval beginning at . The proof compares the linear loss of the constant profile under translation with the quadratic autocorrelation of a mean-zero perturbation. Along the constructed family , the zero shift remains globally maximizing for sufficiently small , providing a rigorous starting point for studying when off-center maximizers appear along controlled deformations.
For , define the constrained optimum
The identity immediately gives .
Theorem 4.3 (local plateau).
There exists such that
Proof. Choose a nonzero with and . Let its support have distance from the two endpoints, put , , and set
For , the function is admissible and . Write . If , the two linear boundary terms vanish because the translated interval still contains , and therefore
Translation in gives
so the chosen bound on makes the difference negative for . For , the unperturbed triangular overlap is at most ; Cauchy–Schwarz and give
Thus zero is the global maximizing shift and . Taking attains the universal lower bound throughout the stated interval.
Station Reference.
The self-energy parameter and the identity are recorded in Archive #5. The local plateau theorem and explicit- proof are recorded in Archive #71, answering the local part of Question Room Discussion #12. Archive #73 and Research Center Evaluations #1309, #1311, and #1318 record numerical exploration beyond the proved local interval.
Related Work. Scalar deformations of the minimum-overlap functional go back to Świerczkowski (1958) and Moser and Murdeshwar (1966), who parameterized the problem by mass and derived bounds near mass . Parameterizing instead by the self-energy , and the resulting exact local plateau, appear to be new.
4.3 Exact constructions for the periodic companion problem
We place the two-unit active interval on a circle of circumference and determine when the average lower bound is attained exactly. Equality is equivalent to assigning every nonzero Fourier mode of the active arc wholly to one side of the partition. Using this criterion, we construct constant-overlap optimizers for every and an additional rational optimizer at . The resulting Dirichlet-kernel ladder and ten-level rational construction provide explicit solvable models of how the boundary gap and Fourier spectrum interact.
This section concerns a companion problem, not a new bound on the original interval constant. Let be a circle of circumference , let be the indicator of an arc of length two, require and , put , and define the cyclic overlap
Write for the infimum of over these periodic profiles.
Preliminary observation. Fubini gives
so . Equality holds exactly when the overlap is constant. Since
this is equivalent to the Fourier-mask condition
Theorem 4.4 (an infinite exact Dirichlet-kernel ladder).
For every integer ,
Proof. Put , divide the circle into cells of width , use the first cells as the active arc and the last as the gap, and for every residue define
The last value is zero. The inequalities
show . Root-of-unity summation gives , hence the step function has mass one. At a nonzero finite Fourier mode, the arc indicator has coefficient . The profile has that same coefficient for and coefficient zero at every other nonzero mode. Thus it satisfies the Fourier-mask condition above. The cell-shape factor vanishes at every nonzero continuum mode divisible by , so the finite mask covers all continuum Fourier modes. The overlap is constant at its mean .
Theorem 4.5 (a rational tight point at ).
At , ten active cells and two gap cells, all of width , admit the exact active levels
The ten active levels give .
Proof. The levels lie in and sum to five, so both and its arc complement have mass one. The exact computation below proves that all twelve aligned cyclic correlations equal . A cyclic correlation of equal-grid step functions is affine between consecutive aligned shifts, so it is identically for every real shift. The averaging bound supplies the matching lower bound.
The next cell checks the mass, box constraints, and all twelve aligned cyclic overlaps of the rational construction exactly.
Show code
Code cell 18 · In [6]
periodic_f = [
Fraction(7, 12), Fraction(5, 6), Fraction(1, 2), Fraction(1, 4), Fraction(1, 3),
Fraction(1, 3), Fraction(1, 4), Fraction(1, 2), Fraction(5, 6), Fraction(7, 12),
Fraction(0), Fraction(0),
]
periodic_arc = [Fraction(1)] * 10 + [Fraction(0), Fraction(0)]
periodic_g = [arc_value - value for arc_value, value in zip(periodic_arc, periodic_f)]
cell_width = Fraction(1, 5)
assert all(Fraction(0) <= value <= arc_value for value, arc_value in zip(periodic_f, periodic_arc))
assert min(periodic_g) >= 0
assert cell_width * sum(periodic_f) == 1
assert cell_width * sum(periodic_g) == 1
cyclic_overlaps = [
cell_width * sum(
periodic_f[j] * periodic_g[(j + shift) % 12]
for j in range(12)
)
for shift in range(12)
]
assert cyclic_overlaps == [Fraction(5, 12)] * 12
print("all aligned overlaps =", cyclic_overlaps[0])
print("periodic optimum at L=12/5 verified exactly")
Saved output 1
all aligned overlaps = 5/12 periodic optimum at L=12/5 verified exactly
Station Reference.
The periodic averaging bound, Fourier-mask observation, and Dirichlet ladder are recorded in Archive #59 and its correction, with Research Center Evaluations #1084, #1089, #1096, #1127, and #1134; the companion problem arose in Question Room Discussion #10. The rational construction is recorded in Archive #79 and Evaluation #1430.
Related Work. Martos et al. (2023) study the minimum-overlap problem on finite abelian groups and obtain upper and lower bounds. Their setting provides finite-group background for the periodic companion problem. Here the Dirichlet-kernel construction gives an infinite ladder of exact optimizers, while the construction gives a further rational optimizer outside that ladder.