Manuscript |

On Completion Times under Memoryless Catastrophe

Tue, 15 Sep 2026 00:00:00 +0000

1 Introduction

1.1 Motivation

Stochastic processes subject to catastrophe undergo the total loss of accumulated progress at random intervals, necessitating a complete restart from the initial state. This abstraction of a base task disrupted by total resets arises across several disciplines:

•

Randomized algorithms restarted upon timeout, where [29] showed that a universal restart strategy is near-optimal up to a logarithmic factor;
•

Queueing systems flushed by disasters, initiated by the disaster models of [5] and developed through the negative-customer framework of [20];
•

Diffusive search under stochastic resetting, where [16] demonstrated that Poisson resetting of a Brownian particle produces a nonequilibrium steady state with finite mean first-passage time—launching a now-extensive literature surveyed in [15].

For such a process that must complete a task, its completion time, denoted by $T\text{,}$ depends on two ingredients: the base task (how long without catastrophe?) and the catastrophe mechanism (when does reset strike?). Much of the existing literature studies the first ingredient—optimizing restart timing for a given task [37, 33, 8]. We ask a complementary question about the second:

What algebraic structure does the catastrophe mechanism impose on the probability generating function of $T\text{,}$ and what are the consequences for moments, limit laws, and convergence rates?

The question has a clear theoretical lineage. The probability generating function (PGF) formula for completion under geometric restart was derived independently by [4] and [19]. Both observed the striking fact that $E[T]$ depends on the base process only through a single PGF evaluation—Flynn and Pilyugin called this “wonderful.” This is, in retrospect, the $k=1$ shadow of a deeper phenomenon. Meanwhile, in branching process theory, [1] and [38] exploited the fact that geometric compounding inherently produces linear fractional (Möbius) PGFs; in the catastrophe/restart context, this connection has not been made. Our work identifies the algebraic mechanism underlying these observations—affine PGF structure—and develops it into a systematic theory: a derivative reduction principle, a characterization theorem, an exponential limit with explicit Laplace expansion, sharp two-sided convergence rates, and a continuous-time counterpart that sharpens the characterization to full Poisson resetting.

1.2 Setup and the PGF Formula

Throughout, $T_{0}$ is a positive-integer-valued, almost surely finite random variable (the base completion time). Catastrophe occurs independently at each step with probability $q\in(0,1)\text{.}$ For an integer-valued random variable $X\text{,}$ write $G_{X}$ for its probability generating function. We set

$$s=1-q,\qquad g=G_{T_{0}},\qquad p=g(s),\qquad D_{k}=g^{(k)}(s).$$

We write $\operatorname{Exp}(\lambda)$ for the exponential distribution with rate $\lambda>0$ (density $\lambda e^{-\lambda x}\text{,}$ $x\geq 0\text{;}$ mean $1/\lambda$). In particular, $\operatorname{Exp}(1)$ denotes the standard exponential (mean $1$). The attempt decomposition

$$T=\sum_{i=1}^{N}A_{i}+B,$$

(1)

with $N\sim\mathrm{Geom}_{0}(p)$ counting failed attempts, $A_{i}$ i.i.d. failed-attempt lengths, and $B$ the successful-attempt length, is classical (see §2.1 for a self-contained derivation). The success probability $p=g(s)\text{,}$ a single PGF evaluation, is the simplest instance of what we call proportional single-point sufficiency.

The PGF of $T$ then takes the form (Theorem 2.2)

$$G_{T}(w)\;=\;\frac{g(u)(1-u)}{1-w+qw\,g(u)},\qquad u=ws.$$

(2)

This formula was obtained independently by [19] and [4]. Our contribution begins with the observation that (2) is affine in $g(u)$:

$$\underbrace{(1-u)}_{\alpha_{N}(w)}\cdot g(u)+\underbrace{0}_{\beta_{N}(w)}\quad\Big/\quad\underbrace{qw}_{\alpha_{D}(w)}\cdot g(u)+\underbrace{(1-w)}_{\beta_{D}(w)},$$

(3)

where the argument $u=ws$ is linear in $w\text{.}$ Since $\beta_{N}(1)=0=\beta_{D}(1)\text{,}$ the probability axiom $G_{T}(1)=1$ forces

$$\alpha_{N}(1)=\alpha_{D}(1)=q.$$

(4)

This coefficient equality is the organizing algebraic mechanism behind the derivative reduction principle and the subsequent approximation theory.

1.3 Main Results

We state our principal results in order: the coefficient equality (4) drives the derivative reduction principle (Theorem 1.1), which in turn underpins the Laplace expansion (Theorem 1.3) and the sharp rate (Theorem 1.4). The characterization (Theorem 1.2) closes the loop by showing that affine structure is not merely sufficient but also necessary within the class of age-based catastrophe mechanisms.

I. Derivative reduction principle. The coefficient equality (4) propagates through the Leibniz rule to produce an exact cancellation at every moment level.

Theorem 1.1 (Derivative reduction principle).

Define $\Delta_{k}=N^{(k)}(1)-D^{(k)}(1)\text{,}$ where $N$ and $D$ are the numerator and denominator of (2). Then

$$\Delta_{k}=\begin{cases}1-p,&k=1,\\ -k\,s^{k-1}\,D_{k-1},&k\geq 2.\end{cases}$$

In particular, $\Delta_{k}$ does not contain $D_{k}=g^{(k)}(s)\text{.}$ Consequently, the $k$-th factorial moment of $T$ depends on the $(k{-}1)$-jet $\{g(s),g^{\prime}(s),\ldots,g^{(k-1)}(s)\}\text{,}$ never on $g^{(k)}(s)\text{.}$

For details, see Theorem 2.6 and its proof. The theorem reveals a strict complexity staircase: each moment level requires exactly one new derivative of the base PGF. For base processes with unbounded support, the staircase is tight—no further reduction is possible (Proposition 2.10). In the coupon collector application (§6), this staircase is realized by the finite harmonic sums $H^{(k-1)}=\sum_{j=m+1}^{n+m}j^{-(k-1)}\text{,}$ so that the $k$-th moment introduces one new harmonic statistic that cannot be eliminated from the present recursion using lower-order data.

The cancellation rests on affine dependence on $g(u)\text{,}$ a linear shift $u=ws\text{,}$ and coefficient matching at the normalization point; see Lemma 2.7. This raises a natural question: is their conjunction characteristic of some identifiable class of catastrophe mechanisms?

II. Characterization of geometric-tail catastrophe. The answer is yes. Consider a general age-based catastrophe mechanism with hazard sequence $(h_{t})_{t\geq 1}\text{,}$ $h_{t}\in(0,1)\text{,}$ and survival function $S(t)=\prod_{i=1}^{t}(1-h_{i})\text{.}$ We say the mechanism has geometric tail if there exist $a\in(0,1)$ and $q\in(0,1)$ such that $h_{1}=1-a$ and $h_{t}=q$ for all $t\geq 2\text{;}$ equivalently, $S(t)=a\,s^{t-1}$ for $t\geq 1\text{.}$ The special case $a=s$ recovers full memorylessness.

Theorem 1.2 (Characterization).

Among age-based catastrophe mechanisms with hazard rates $h_{t}\in(0,1)\text{,}$ the following are equivalent:

(A)

Geometric-tail catastrophe: $S(t)=a\,s^{t-1}$ for $t\geq 1$.
(B)

Uniform affine PGF structure (Definition 2.4).
(C)

Derivative reduction: there exists $z_{0}\in(0,1)$ such that for every $k\geq 1$ and every finitely-supported $T_{0}$, the $k$-th factorial moment of $T$ depends on $T_{0}$ only through $\{g^{(j)}(z_{0})\}_{j=0}^{k-1}$.
(D)

Proportional single-point sufficiency: $p=\lambda\,g(z_{0})$ for some $z_{0}\in(0,1)$, $\lambda>0$, and all finitely-supported $T_{0}$.

Full memorylessness ($h_{t}\equiv q$) is further equivalent to $\lambda=1$ (Corollary 3.8).

For details, see Theorem 3.6. The hardest implication (C)$\Rightarrow$(A) uses only the $k=1$ case and proceeds by a Möbius rigidity argument (§3.3). The gap between geometric-tail and full memorylessness is exactly one degree of freedom—the first-step hazard $h_{1}\text{.}$ In continuous time, this gap closes entirely (Theorem 5.7); see VI below.

III. Exponential limit and the Laplace expansion. The attempt decomposition (1) expresses $T$ as a geometric sum plus a perturbation. Rényi’s classical theorem [36] asserts that normalized geometric sums converge to the exponential distribution. The coefficient equality (4) sharpens this into a quantitative expansion.

Theorem 1.3 (Laplace expansion).

Fix $q\in(0,1)\text{.}$ Let $W=T/E[T]$ and define

$$\alpha=\frac{s(p-qD_{1})}{1-p},\qquad\beta=\frac{1-qp-qsD_{1}}{1-p}.$$

Then $1+\alpha-\beta=0$ as an algebraic identity. For $p$ sufficiently small and $\theta>0$ satisfying $\theta pq/(1-p)\leq\epsilon_{0}\text{,}$

$$E[e^{-\theta W}]-\frac{1}{1+\theta}\;=\;\frac{\alpha\,\theta^{2}}{(1+\theta)^{2}}\;+\;O\!\left(\frac{|\alpha|^{2}\,\theta^{3}}{(1+\theta)^{3}}\right)\;+\;O\!\left(\frac{p\,\theta^{2}}{1+\theta}\right),$$

with implicit constants depending only on $q\text{.}$ In particular, $T/E[T]\xrightarrow{d}\mathrm{Exp}(1)$ as $p\to 0\text{.}$

See Theorem 4.4 for details. The identity $1+\alpha-\beta=0$ ensures that the first-order term in $\theta$ vanishes exactly—an algebraic consequence of the coefficient match (4), not an asymptotic cancellation. The parameter $\alpha$ measures the deviation of the successful attempt from a geometric target (§4.1).

IV. Sharp two-sided Kolmogorov bound. Converting the Laplace expansion to Kolmogorov distance via the smoothing inequality yields $d_{K}=O(|\alpha|\log(1/|\alpha|))$—a logarithmic artifact. The following theorem eliminates this loss.

Theorem 1.4 (Sharp Kolmogorov bound).

There exist constants $c_{A},C_{A},\delta_{A}>0$ depending only on $q$ such that, for $p+|\alpha|\leq\delta_{A}\text{,}$

$$c_{A}\,(p+|\alpha|)\;\leq\;d_{K}\!\left(\frac{T}{E[T]},\,\mathrm{Exp}(1)\right)\;\leq\;C_{A}\,(p+|\alpha|).$$

See Theorem 4.15 for details. The template $p+|\alpha|$ reflects two independent error sources: a lattice error $\Theta(p)$ from the integrality of the attempt count, and a perturbation error $\Theta(|\alpha|)$ from the successful attempt’s length mismatch. The $p$-term is irreducible, as shown by Proposition 4.16; the $|\alpha|$-term is forced by the Laplace-transform lower bound and is dominant in the CCP regime. The upper bound rests on a Bridge Theorem (Theorem 4.13) that decomposes the approximation via Brown’s quantitative Rényi theorem ([6]); the lower bound combines a lattice argument with evaluation of the Laplace expansion.

V. CCP specialization and phase transition. We instantiate the theory to the coupon collector problem with $m$ reset coupons among $n+m$ types (§6). The specialization yields:

•

Closed-form expressions for $E[T]$ and $\operatorname{Var}(T)\text{,}$ together with an explicit harmonic realization of the DRP staircase for higher moments (§6.3);
•

The CCP-specific bound $d_{K}\in[1-o(1),\,2+o(1)]\cdot|\alpha|$ with $|\alpha|\sim m!\,m\ln n/n^{m}$ (Theorem 6.4), closing the logarithmic gap left by the smoothing approach;
•

A discontinuous Gumbel-to-Exponential phase transition at $m=0\to m\geq 1\text{:}$ one reset coupon suffices to replace extreme-value waiting ($(T_{0}-n\ln n)/n\xrightarrow{d}\text{Gumbel}$) by geometric waiting ($T/E[T]\to$ Exponential).

VI. Continuous-time sharpening. Replacing PGFs by Laplace transforms and the multiplicative shift $u=ws$ by an additive shift $\sigma=\lambda+r\text{,}$ the entire discrete theory carries over to continuous time (§5), with one qualitative sharpening and one quantitative simplification.

The sharpening: the geometric-tail gap of Theorem 1.2 closes. Among continuous-time age-based catastrophe mechanisms, affine Laplace structure characterizes full Poisson resetting—constant hazard everywhere, with no residual first-step degree of freedom (Theorem 5.7). The mechanism is twofold: point masses $T_{0}=\delta_{x}$ with continuous $x$ determine the function $\psi$ on all of $(0,1)\text{,}$ and the relation $F=\rho\,S+c$ is differentiated rather than differenced, constraining the hazard rate at every point.

The simplification is conditional rather than unconditional: In continuous time, the lattice error disappears. In the regime $|\alpha|\gg p\text{,}$ the sharp template reduces to $|\alpha|\text{;}$ when $\alpha$ is small, a residual $O(p)$ term may remain. In the Brownian first-passage model of [16], this yields $d_{\mathrm{K}}\!\left(\frac{T}{E[T]},\,\operatorname{Exp}(1)\right)\in[1-o(1),\,2+o(1)]\cdot|\alpha|$ (§5.6).

1.4 Architecture of the Paper

The paper is organized as follows. §2 derives the PGF formula, identifies the affine structure, and establishes the coefficient equality. §2.2 proves the derivative reduction principle and the moment recursion. §3 establishes the characterization theorem via a Möbius rigidity argument. §4 develops the exponential approximation: the Laplace expansion, the Bridge Theorem, and the sharp two-sided Kolmogorov bound $d_{K}\asymp p+|\alpha|\text{,}$ including a counterexample showing that neither term is redundant. §5 presents the continuous-time counterpart—replacing PGFs by Laplace transforms and the multiplicative shift by an additive one—and shows that the geometric-tail gap closes: affine Laplace structure characterizes full Poisson resetting. §6 and §7 instantiate the theory to two structurally contrasting applications: the coupon collector with reset coupons, which produces a harmonic moment staircase and a Gumbel-to-Exponential phase transition, and a multi-phase task under catastrophe, which produces an algebraic staircase and a Gaussian-to-exponential transition with exponentially fast convergence. §8 discusses related work, and §9 outlines further directions.

2 Completion Time under Memoryless Catastrophe

2.1 Model and PGF Formula

Let $T_{0}$ be a positive-integer-valued, almost surely finite random variable, called the base completion time, with $\pi_{j}=P(T_{0}=j)$ for $j\geq 1$ and PGF $g(z)=G_{T_{0}}(z)=\sum_{k\geq 0}P(T_{0}=k)\,z^{k}$ for $|z|\leq 1\text{.}$

Definition 2.1 (Memoryless catastrophe).

Fix $q\in(0,1)$ and set $s=1-q\text{.}$ At each step, independently of everything else, a catastrophe occurs with probability $q\text{.}$ If the process survives (probability $s$), the base task may advance; if a catastrophe occurs and the base task has not yet completed, all accumulated progress is destroyed and the process restarts from scratch. The process terminates at the first step where it survives the catastrophe and the base task completes.

A maximal run of steps uninterrupted by catastrophe is called an attempt. Each attempt begins from the empty state and either succeeds (the base task completes) or fails (terminated by catastrophe). Since the base task completes at step $t$ with no prior catastrophe with probability $s^{t}\,\pi_{t}$ (independence), the success probability is

$$p\;=\;\sum_{t\geq 1}s^{t}\,\pi_{t}\;=\;g(s)\;\in\;(0,1).$$

(5)

Successive attempts are independent (each restarts from the empty state), so the number of failed attempts before the first success is $N\sim\mathrm{Geom}_{0}(p)\text{.}$¹¹ 1

 We write $\mathrm{Geom}_{0}(p)$ for the distribution on $\{0,1,2,\ldots\}$ with $P(N=k)=(1-p)^{k}p$ (failure count), and $\mathrm{Geom}_{1}(p)=\mathrm{Geom}_{0}(p)+1$ for the trial count.</span></span></span></p>

Denote by $A$ the length of a generic failed attempt, by $B$ the length of the successful attempt, and let $A_{1},A_{2},\ldots$ be i.i.d. copies of $A\text{.}$ The total completion time decomposes as

$$T\;=\;\sum_{i=1}^{N}A_{i}\;+\;B,$$

(6)

where $N\text{,}$ $(A_{i})_{i\geq 1}\text{,}$ and $B$ are mutually independent. Standard conditioning (Bayes’ rule on the success/failure event) yields the PGFs

$$\begin{aligned} \displaystyle G_{B}(w) & \displaystyle=\frac{g(ws)}{p}, \\[4pt] \displaystyle G_{A}(w) & \displaystyle=\frac{qw\bigl(1-g(ws)\bigr)}{(1-p)(1-ws)}, \end{aligned}$$

(7)(8)

where (8) uses the identity $\sum_{j\geq 0}z^{j}P(T_{0}>j)=(1-g(z))/(1-z)\text{.}$ Combining these with the geometric-sum PGF $E[G_{A}(w)^{N}]=p/(1-(1-p)G_{A}(w))$ gives the following.

Theorem 2.2 (PGF formula).

Under memoryless catastrophe with rate $q\text{,}$ the PGF of the completion time is

$$G_{T}(w)\;=\;\frac{g(u)(1-u)}{1-w+qw\,g(u)},\qquad u=ws.$$

(9)

Proof.

By (6) and mutual independence, $G_{T}=G_{B}\cdot E[G_{A}^{N}]\text{.}$ Substituting (7) and (8):

$$G_{T}(w)=\frac{g(u)}{p}\cdot\frac{p}{1-\dfrac{qw(1-g(u))}{1-u}}=\frac{g(u)(1-u)}{(1-u)-qw(1-g(u))}.$$

The denominator simplifies: $(1-ws)-qw+qw\,g(u)=1-w(s+q)+qw\,g(u)=1-w+qw\,g(u)\text{,}$ using $s+q=1\text{.}$ ∎

Corollary 2.3 (Expected completion time).

$\displaystyle E[T]=\frac{1-p}{qp}$.

Proof.

Differentiating (9): $G_{T}(w)-1=(1-w)(g(u)-1)/(1-w+qwg(u))$ (using $1-u-qw=1-w$), so $E[T]=\lim_{w\to 1}(G_{T}(w)-1)/(w-1)=(1-p)/(qp)\text{.}$ ∎

2.2 Affine Structure and the Derivative Reduction Principle

We now identify the structural property of (9) that drives all subsequent results.

Definition 2.4 (Uniform affine PGF structure).

A catastrophe mechanism has uniform affine PGF structure if there exist functions $a(w),b(w),c(w),d(w)\text{,}$ analytic in a neighborhood of $w=1\text{,}$ depending only on the mechanism (not on the base process $T_{0}$), and a linear function $u(w)$ with $u(1)\in(0,1)\text{,}$ such that for all base processes $T_{0}\text{,}$

$$G_{T}(w)=\frac{a(w)\,g(u(w))+b(w)}{c(w)\,g(u(w))+d(w)},$$

with $a$ and $c$ not identically zero, and the non-degeneracy condition $c(1)\,g(u(1))+d(1)\neq 0$ for all admissible $g\text{.}$²² 2

 Non-degeneracy excludes pathological representations obtained by multiplying numerator and denominator by a factor vanishing at $w=1\text{,}$ which would create a $0/0$ indeterminacy and invalidate the coefficient-matching argument.</span></span></span></p>

Proposition 2.5 (Affine structure and coefficient equality).

The PGF formula (9) exhibits uniform affine structure with

$$a(w)=1-u,\quad b(w)=0,\quad c(w)=qw,\quad d(w)=1-w,\quad u(w)=ws.$$

Writing $N(w)=a(w)\,g(u)+b(w)$ and $D(w)=c(w)\,g(u)+d(w)$ for the numerator and denominator, the normalization $G_{T}(1)=1$ forces

$$\alpha_{N}(1)\;:=\;a(1)\;=\;q\;=\;c(1)\;=:\;\alpha_{D}(1).$$

(10)

Proof.

Immediate from (9): at $w=1\text{,}$ $b(1)=0=d(1)\text{,}$ so $G_{T}(1)=1$ gives $a(1)\,g(s)=c(1)\,g(s)$ and $a(1)=c(1)$ (since $g(s)=p>0$); since $a(1)=1-s=q\text{,}$ the result follows. ∎

The coefficient equality (10) is a direct consequence of the probability axiom $G_{T}(1)=1$ applied to the affine form. Together with the linearity of $u(w)=ws$ and the affine dependence on $g(u)\text{,}$ it produces an exact derivative cancellation at every moment level.

The PGF formula expresses $G_{T}(w)=N(w)/D(w)\text{,}$ where

$$N(w)=(1-u)\,g(u),\qquad D(w)=(1-w)+qw\,g(u),\qquad u=ws.$$

(11)

The $k$-th factorial moment is $\eta_{k}=G_{T}^{(k)}(1)\text{.}$ Since $u=ws$ is linear, $\frac{d^{j}}{dw^{j}}g(ws)=s^{j}g^{(j)}(ws)$ with no Faà di Bruno corrections. Writing $D_{j}=g^{(j)}(s)\text{,}$ one might expect $\eta_{k}$ to depend on the full $k$-jet $\{D_{0},\ldots,D_{k}\}\text{.}$ The following theorem shows that $D_{k}$ cancels exactly.

Theorem 2.6 (Derivative reduction principle).

Define $\Delta_{k}=N^{(k)}(1)-D^{(k)}(1)$ for $k\geq 1\text{.}$ Then

$$\Delta_{k}=\begin{cases}1-p,&k=1,\\[3.0pt] -k\,s^{k-1}\,D_{k-1},&k\geq 2.\end{cases}$$

(12)

In particular, $\Delta_{k}$ does not contain $D_{k}=g^{(k)}(s)\text{.}$

Proof.

We compute $N^{(k)}(1)$ and $D^{(k)}(1)$ separately via the Leibniz rule, then subtract.

Case $k\geq 2\text{.}$ Write $N(w)=f_{1}(w)\cdot g(ws)$ with $f_{1}(w)=1-ws\text{.}$ Since $f_{1}(1)=q\text{,}$ $f_{1}^{\prime}(w)=-s\text{,}$ and $f_{1}^{(j)}=0$ for $j\geq 2\text{,}$ the Leibniz rule gives

$$N^{(k)}(1)=q\,s^{k}D_{k}-k\,s^{k}D_{k-1}.$$

(13)

Similarly, $D(w)=(1-w)+f_{2}(w)\cdot g(ws)$ with $f_{2}(w)=qw\text{.}$ For $k\geq 2\text{,}$ $\frac{d^{k}}{dw^{k}}(1-w)=0\text{.}$ Since $f_{2}(1)=q\text{,}$ $f_{2}^{\prime}(w)=q\text{,}$ and $f_{2}^{(j)}=0$ for $j\geq 2\text{:}$

$$D^{(k)}(1)=q\,s^{k}D_{k}+k\,q\,s^{k-1}D_{k-1}.$$

(14)

Subtracting (14) from (13):

$$\Delta_{k}=\underbrace{(q\,s^{k}-q\,s^{k})}_{=0}\,D_{k}+\bigl(-k\,s^{k}-k\,q\,s^{k-1}\bigr)\,D_{k-1}=-k\,s^{k-1}(s+q)\,D_{k-1}=-k\,s^{k-1}D_{k-1},$$

where the last step uses $s+q=1\text{.}$ The $D_{k}$-terms cancel because they enter $N^{(k)}(1)$ and $D^{(k)}(1)$ with the same coefficient $q\,s^{k}$—precisely the coefficient equality (10).

Case $k=1\text{.}$ Direct computation: $N^{\prime}(1)=-sp+qsD_{1}\text{,}$ $D^{\prime}(1)=-1+qp+qsD_{1}\text{.}$ Hence

$$\Delta_{1}=(-sp+qsD_{1})-(-1+qp+qsD_{1})=1-p(s+q)=1-p.$$

The next lemma extracts the cancellation mechanism behind Theorem 2.6.

Lemma 2.7 (Affine-ratio recursion).

Let

$$R(z)=\frac{a(z)h(u(z))+b(z)}{c(z)h(u(z))+d(z)},\qquad u(z)=u_{0}+\kappa(z-z_{*}),$$

where $a,b,c,d$ are $C^{\infty}$ near $z_{*}\text{,}$ and $a(z_{*})=c(z_{*}),b(z_{*})=d(z_{*}),D(z_{*}):=c(z_{*})h(u_{0})+d(z_{*})\neq 0\text{.}$ Set

$$N(z)=a(z)h(u(z))+b(z),\qquad D(z)=c(z)h(u(z))+d(z).$$

Then, for every $k\geq 1\text{,}$ $\Delta_{k}:=N^{(k)}(z_{*})-D^{(k)}(z_{*})$ depends only on $h(u_{0}),h^{\prime}(u_{0}),\dots,h^{(k-1)}(u_{0})\text{,}$ and

$$R^{(k)}(z_{*})=\frac{1}{D(z_{*})}\left(\Delta_{k}-\sum_{j=1}^{k-1}\binom{k}{j}D^{(j)}(z_{*})R^{(k-j)}(z_{*})\right).$$

Consequently, $R^{(k)}(z_{*})$ depends only on the $(k-1)$-jet of $h$ at $u_{0}\text{.}$

Proof.

Since $u$ is linear, $(h\circ u)^{(k)}(z_{*})=\kappa^{k}h^{(k)}(u_{0})$ and no Faà di Bruno terms appear. Hence the coefficient of $h^{(k)}(u_{0})$ in $N^{(k)}(z_{*})$ is $a(z_{*})\kappa^{k}\text{,}$ while in $D^{(k)}(z_{*})$ it is $c(z_{*})\kappa^{k}\text{;}$ these agree by $a(z_{*})=c(z_{*})\text{,}$ so $\Delta_{k}$ contains no $h^{(k)}(u_{0})\text{.}$ Differentiating the identity $D(z)R(z)=N(z)$ $k$ times at $z=z_{*}$ and isolating the $j=0$ and $j=k$ terms gives the displayed recursion. ∎

2.3 Moment Recursion and the Complexity Staircase

Corollary 2.8 (Moment recursion).

The factorial moments of $T$ satisfy $\eta_{0}=1$ and, for $k\geq 1\text{,}$

$$\eta_{k}=\frac{1}{pq}\left(\Delta_{k}-\sum_{j=1}^{k-1}\binom{k}{j}D^{(j)}(1)\,\eta_{k-j}\right),$$

(15)

where $D^{(j)}(1)$ is the $j$-th derivative of $D(w)=(1-w)+qw\,g(ws)$ at $w=1\text{.}$ Since $\Delta_{k}$ involves at most $D_{k-1}$ (Theorem 2.6) and $D^{(j)}(1)$ involves at most $D_{j}$ for $j\leq k-1$ (by (14)), induction on $k$ confirms: $\eta_{k}$ depends on the $(k{-}1)$-jet $\{D_{0},D_{1},\ldots,D_{k-1}\}\text{,}$ never on $D_{k}\text{.}$

Proof.

Apply the Leibniz rule to $D\cdot G_{T}=N$ at $w=1\text{,}$ separate the $j=0$ and $j=k$ terms, and use $\Delta_{k}=N^{(k)}(1)-D^{(k)}(1)$ to cancel $D^{(k)}(1)\text{.}$ ∎

The low-order derivatives of $D$ at $w=1$ are:

$$D(1)=qp,\qquad D^{\prime}(1)=-1+qp+qsD_{1},\qquad D^{\prime\prime}(1)=2qsD_{1}+qs^{2}D_{2}.$$

(16)

Example 2.9 (First two moments).

From (15) with $k=1\text{:}$ $\eta_{1}=(1-p)/(pq)\text{,}$ recovering Corollary 2.3. With $k=2\text{:}$ $\eta_{2}=\frac{1}{pq}(-2sD_{1}-2D^{\prime}(1)\,\eta_{1})\text{.}$ The variance $\operatorname{Var}(T)=\eta_{2}+\eta_{1}-\eta_{1}^{2}$ simplifies to

$$\operatorname{Var}(T)=\frac{(1-p)(1+ps)-2qsD_{1}}{p^{2}q^{2}},$$

(17)

which depends on $p=D_{0}$ and $D_{1}=g^{\prime}(s)$ but not on $D_{2}$—exactly as the DRP predicts.

The DRP asserts that each moment level requires at most $D_{k-1}\text{.}$ The following shows that, for base processes with sufficiently rich structure, it requires exactly $D_{k-1}\text{.}$

Proposition 2.10 (Tightness of the staircase).

For $k\geq 2\text{,}$ the total coefficient of $D_{k-1}$ in $\eta_{k}$ is $-ks^{k-1}/(p^{2}q)\neq 0\text{,}$ provided $D_{k-1}\neq 0\text{.}$ The latter holds whenever $P(T_{0}\geq k)>0\text{.}$ In particular, for base processes with unbounded support, every moment level introduces exactly one new derivative of $g\text{.}$

Proof.

By Theorem 2.6, $\Delta_{k}=-ks^{k-1}D_{k-1}$ for $k\geq 2\text{.}$ By induction, $\eta_{k-j}$ for $j\geq 1$ depends on $\{D_{0},\ldots,D_{k-j-1}\}\text{,}$ so $D_{k-1}$ enters the sum in (15) only through the $j=k-1$ term $D^{(k-1)}(1)\,\eta_{1}\text{,}$ where $D^{(k-1)}(1)$ contains $D_{k-1}$ with coefficient $qs^{k-1}\text{.}$ Combining: the total coefficient of $D_{k-1}$ in $\eta_{k}$ is $\frac{1}{pq}(-ks^{k-1}-k\,qs^{k-1}\,\eta_{1})=\frac{-ks^{k-1}}{pq}(1+q\eta_{1})=\frac{-ks^{k-1}}{p^{2}q}\text{,}$ using $\eta_{1}=(1-p)/(pq)\text{.}$

For the second claim: $g^{(k-1)}(s)=\sum_{t\geq k}\frac{t!}{(t-k+1)!}\pi_{t}s^{t-k+1}>0$ whenever some $\pi_{t}>0$ for $t\geq k\text{.}$ ∎

Together, Theorem 2.6 and Proposition 2.10 establish a sharp complexity staircase: for base processes with unbounded support, each moment of $T$ introduces exactly one new derivative of $g\text{,}$ and this staircase collapses only when $T_{0}$ has bounded support.

3 Characterization of Geometric-Tail Catastrophe

§2 established a chain of implications under memoryless catastrophe:

$$\text{memoryless catastrophe}\;\Longrightarrow\;\text{affine PGF structure}\;\Longrightarrow\;\text{derivative reduction (DRP)}.$$

A natural question arises: is memoryless catastrophe the only mechanism producing these properties, or do they hold more broadly? This section answers the question precisely. The answer is that affine PGF structure—and hence the DRP—characterizes a slightly larger class, which we call geometric-tail catastrophe: mechanisms whose hazard rate is constant from the second step onward, with the first-step hazard left free. Full memorylessness is the unique member of this class satisfying an additional algebraic condition ($p=g(s)$ rather than $p\propto g(s)$).

To state and prove this characterization, we first extend the framework of §2 to general age-based catastrophe mechanisms.

3.1 General Hazard Framework

Definition 3.1 (Age-based catastrophe mechanism).

An age-based catastrophe mechanism is specified by a hazard sequence $(h_{t})_{t\geq 1}$ with $h_{t}\in(0,1)$ for all $t\text{.}$ Here $h_{t}$ is the conditional probability of catastrophe at step $t\text{,}$ given that no catastrophe has occurred in steps $1,\ldots,t-1\text{.}$ The associated survival function is

$$S(0)=1,\qquad S(t)=\prod_{i=1}^{t}(1-h_{i}),\quad t\geq 1.$$

(18)

Thus $S(t)$ is the probability of surviving $t$ consecutive steps without catastrophe. The requirement $h_{t}\in(0,1)$ ensures $S(t)>0$ for all $t$ (every step has a positive probability of both survival and catastrophe).

Note that memoryless catastrophe (Definition 2.1) is the special case $h_{t}\equiv q\text{,}$ giving $S(t)=s^{t}\text{.}$ The attempt decomposition (6) remains valid under any age-based mechanism: each attempt starts from the empty state, so successive attempts are i.i.d. The success probability generalizes from $p=g(s)$ to

$$p=\sum_{t\geq 1}S(t)\,\pi_{t},$$

(19)

which is no longer a PGF evaluation unless $S(t)$ is exponential in $t\text{.}$ The expected completion time generalizes from Theorem 2.2 as follows.

Lemma 3.2 (Expected attempt length).

Under an age-based mechanism with survival function $S\text{,}$ the expected length of a single attempt (regardless of outcome) satisfies

$$E[\ell]=\sum_{j\geq 1}F(j)\,\pi_{j},\qquad F(j)=\sum_{t=0}^{j-1}S(t).$$

(20)

Proof.

Condition on $T_{0}=j\text{:}$ step $t\leq j$ is reached with probability $S(t-1)\text{.}$ ∎

Proposition 3.3 (General expected completion time).

Assume $E[\ell]=\sum_{j\geq 1}F(j)\pi_{j}<\infty\text{.}$ Then

$$E[T]=\frac{E[\ell]}{p}=\frac{\sum_{j\geq 1}F(j)\pi_{j}}{\sum_{j\geq 1}S(j)\pi_{j}}.$$

(21)

Proof.

The pairs $(\ell_{i},\mathrm{outcome}_{i})$ are i.i.d., and $M=N+1\sim\operatorname{Geom}_{1}(p)$ is a stopping time. Since $E[\ell]<\infty\text{,}$ Wald’s identity applies and gives $E[T]=E[\ell]\cdot E[M]=\frac{E[\ell]}{p}\text{.}$ ∎

Formula (21) expresses $E[T]$ as a ratio of two linear functionals of the base distribution $(\pi_{j})_{j\geq 1}\text{.}$ In the memoryless case, $S(t)=s^{t}$ makes the denominator $p=g(s)$—a PGF evaluation—and $E[T]$ depends on $T_{0}$ only through this single number. For a general mechanism, $E[T]$ depends on the full interaction between $(\pi_{j})$ and $S\text{.}$

3.2 Geometric-Tail Catastrophe

Definition 3.4 (Geometric-tail catastrophe).

A hazard sequence $(h_{t})_{t\geq 1}$ has geometric tail if there exist $a\in(0,1)$ and $q\in(0,1)$ such that

$$h_{1}=1-a,\qquad h_{t}=q\quad\text{for all }t\geq 2.$$

Equivalently, $S(t)=a\,s^{t-1}$ for $t\geq 1\text{,}$ where $s=1-q\text{.}$ The parameter $a=S(1)=1-h_{1}$ is the first-step survival probability. The special case $a=s$ (i.e., $h_{1}=q$) recovers memoryless catastrophe.

The geometric-tail class has one degree of freedom beyond full memorylessness: the first-step hazard $h_{1}$ may differ from $q\text{.}$ The next proposition shows that every member of this class produces a PGF with affine structure, establishing the forward direction of the characterization.

Proposition 3.5 (Geometric-tail PGF formula).

Under geometric-tail catastrophe with parameters $(a,q)\text{,}$ the success probability is $p=(a/s)\,g(s)$ and the completion-time PGF is

$$G_{T}(w)=\frac{a(1-u)\,g(u)}{s(1-w)(1-w(s-a))+aqw\,g(u)},\qquad u=ws.$$

(22)

This exhibits uniform affine PGF structure (Definition 2.4) with coefficient equality $\alpha_{N}(1)=\alpha_{D}(1)=aq\text{.}$ Setting $a=s$ recovers the memoryless formula (9).

Proof.

Since $S(t)=as^{t-1}\text{,}$ the success probability is $p=(a/s)\,g(s)$ and the successful-attempt PGF is $G_{B}(w)=a\,g(u)/(ps)\text{,}$ where $u=ws\text{.}$

For the failed-attempt PGF, note that the only departure from the memoryless calculation (§2) is the first-step hazard $h_{1}=1-a\neq q\text{.}$ Splitting accordingly:

$$(1-p)\,G_{A}(w)=w(1-a)+\frac{aqw}{s}\sum_{j\geq 1}u^{j}\,P(T_{0}>j).$$

The identity $\sum_{j\geq 1}u^{j}P(T_{0}>j)=(u-g(u))/(1-u)$ (used in §2) gives

$$(1-p)\,G_{A}(w)=w(1-a)+\frac{aqw(u-g(u))}{s(1-u)}.$$

Substituting into $G_{T}=p\,G_{B}/[1-(1-p)G_{A}]$ and clearing denominators, the denominator simplifies (using $s+q=1$ and $u=ws$) to $s(1-w)(1-w(s-a))+aqw\,g(u)\text{,}$ yielding (22). (Setting $a=s$ eliminates the $1-w(s-a)$ factor and recovers (9).)

The $g(u)$-coefficients are $\alpha_{N}(w)=a(1-u)$ and $\alpha_{D}(w)=aqw\text{,}$ giving $\alpha_{N}(1)=\alpha_{D}(1)=aq\text{.}$ Since the numerator has no $g$-free term ($\beta_{N}\equiv 0$) and $\beta_{D}(1)=0\text{,}$ we obtain $G_{T}(1)=1\text{.}$ Non-degeneracy: $D(1)=aqg(s)=pqs>0\text{.}$ ∎

Since the geometric-tail PGF has the same affine-ratio form, the same linear shift $u=ws\text{,}$ and coefficient matching at $w=1\text{,}$ Lemma 2.7 applies verbatim. Hence the DRP holds for geometric-tail mechanisms, with $\alpha_{N}(1)=\alpha_{D}(1)=aq$ replacing $q\text{.}$

3.3 The Characterization Theorem

We now prove the converse: geometric-tail catastrophe is the only age-based mechanism producing affine PGF structure. The result takes the form of a four-way equivalence linking the survival function, the PGF structure, the derivative reduction property, and the success probability formula.

Theorem 3.6 (Characterization).

Among age-based catastrophe mechanisms (Definition 3.1), the following are equivalent:

(A)

Geometric-tail catastrophe: $S(t)=a\,s^{t-1}$ for $t\geq 1$ (Definition 3.4).
(B)

Uniform affine PGF structure: Definition 2.4.
(C)

Derivative reduction: there exists $z_{0}\in(0,1)$ such that for every $k\geq 1$ and every finitely-supported $T_{0}$, the $k$-th factorial moment $\eta_{k}$ depends on $T_{0}$ only through $\{g^{(j)}(z_{0})\}_{j=0}^{k-1}$.
(D)

Proportional single-point sufficiency: there exist $z_{0}\in(0,1)$ and $\lambda>0$ such that $p=\lambda\,g(z_{0})$ for all finitely-supported $T_{0}$.

We prove the cycle (A)$\Rightarrow$(B)$\Rightarrow$(C)$\Rightarrow$(A), and independently prove (A)$\Leftrightarrow$(D). The implications (A)$\Rightarrow$(B) and (A)$\Rightarrow$(D) have been established; the substance lies in (C)$\Rightarrow$(A), which uses only the $k=1$ instance of (C) and proceeds by a Möbius rigidity argument.

Proof of (A)$\Rightarrow$(B).

This is Proposition 3.5. ∎

Proof of (B)$\Rightarrow$(C).

Suppose

$$G_{T}(w)=\frac{a(w)g(u(w))+b(w)}{c(w)g(u(w))+d(w)},\qquad u(1)=u_{1}\in(0,1),$$

with $u$ linear. From $G_{T}(1)=1$ for all finitely-supported $T_{0}\text{,}$ taking $T_{0}\equiv n$ and letting $n\to\infty$ gives $b(1)=d(1)\text{,}$ and then back-substitution gives $a(1)=c(1)\text{.}$ Since non-degeneracy yields $c(1)g(u_{1})+d(1)\neq 0\text{,}$ Lemma 2.7 applies with $R=G_{T}\text{,}$ $h=g\text{,}$ $z_{*}=1\text{,}$ and $u_{0}=u_{1}\text{.}$ Therefore the $k$-th factorial moment $\eta_{k}=G_{T}^{(k)}(1)$ depends only on $\{g^{(j)}(u_{1})\}_{j=0}^{k-1}\text{.}$ ∎

Proof of (C)$\Rightarrow$(A).

We use only the $k=1$ case: $E[T]$ depends on $T_{0}$ only through $g(z_{0})$ for some $z_{0}\in(0,1)\text{.}$ By Proposition 3.3, there exists $\psi:(0,1)\to\mathbb{R}$ such that

$$\frac{\sum_{j\geq 1}F(j)\,\pi_{j}}{\sum_{j\geq 1}S(j)\,\pi_{j}}=\psi\!\left(\sum_{j\geq 1}z_{0}^{j}\,\pi_{j}\right)\quad\text{for all finitely-supported }(\pi_{j}).$$

(23)

Step 1: Point masses. Taking $\pi=\delta_{j}$ gives $\psi(z_{0}^{j})=F(j)/S(j)=:f_{j}$ for each $j\geq 1\text{.}$

Step 2: Two-point distributions force $\psi$ to be Möbius. For $\pi_{a}=\alpha\text{,}$ $\pi_{b}=1-\alpha$ with $a<b\text{,}$ set $v=\alpha z_{0}^{a}+(1-\alpha)z_{0}^{b}\text{.}$ Substituting into (23) and solving for $\alpha$ in terms of $v$ gives

$$\psi(v)=\frac{(F(a)-F(b))\,v+F(b)z_{0}^{a}-F(a)z_{0}^{b}}{(S(a)-S(b))\,v+S(b)z_{0}^{a}-S(a)z_{0}^{b}}=:M_{ab}(v)$$

(24)

on $[z_{0}^{b},z_{0}^{a}]\text{.}$ This is a Möbius transformation of $v\text{.}$

Step 3: Global consistency. For $a<c<b\text{,}$ the Möbius transformations $M_{ab}$ and $M_{ac}$ agree on the interval $[z_{0}^{c},z_{0}^{a}]\text{,}$ which contains infinitely many points; since a non-degenerate Möbius transformation is determined by three values, $M_{ab}=M_{ac}$ as rational functions. For any two pairs $(a_{1},b_{1})\text{,}$ $(a_{2},b_{2})$ with $a_{i}<b_{i}\text{,}$ setting $a^{*}=\min(a_{1},a_{2})\text{,}$ $b^{*}=\max(b_{1},b_{2})$ gives $[z_{0}^{b_{i}},z_{0}^{a_{i}}]\subset[z_{0}^{b^{*}},z_{0}^{a^{*}}]$ for each $i$ (since $z_{0}\in(0,1)$ makes $j\mapsto z_{0}^{j}$ decreasing), so $M_{a_{1}b_{1}}=M_{a^{*}b^{*}}=M_{a_{2}b_{2}}\text{.}$ Hence all $M_{ab}$ coincide with a single

$$\psi(v)=M(v)=\frac{Av+B}{Cv+D}\qquad\text{on }(0,z_{0}].$$

(25)

The denominator slope of each $M_{ab}$ is proportional to $S(a)-S(b)\neq 0$ (since $h_{t}\in(0,1)$ makes $S$ strictly decreasing), so $C\neq 0\text{.}$

Step 4: Coefficient consistency forces $F=rS+c\text{.}$ Since $M_{ab}=M$ up to a common scalar, $A/C=(F(a)-F(b))/(S(a)-S(b))$ is constant across all $a\neq b\text{.}$ Denoting this constant by $r\text{,}$ we obtain $F(j)=rS(j)+c$ for all $j\geq 1\text{.}$

Step 5: Differencing yields constant hazard from step $2$ onward. From $F(j)=\sum_{t=0}^{j-1}S(t)\text{:}$ $F(j+1)-F(j)=S(j)\text{.}$ From $F=rS+c\text{:}$ $F(j+1)-F(j)=r(S(j+1)-S(j))=-rS(j)h_{j+1}\text{.}$ Equating and dividing by $S(j)>0\text{:}$

$$h_{j+1}=-1/r\quad\text{for all }j\geq 1.$$

Hence $h_{t}=q:=-1/r$ for all $t\geq 2\text{,}$ with $r<-1$ (since $q\in(0,1)$). Setting $a=S(1)\in(0,1)$ gives $S(t)=as^{t-1}$ for $t\geq 1\text{,}$ confirming geometric-tail catastrophe. ∎

Proof of (A)$\Rightarrow$(D).

If $S(t)=as^{t-1}\text{,}$ then $p=\sum_{t\geq 1}as^{t-1}\pi_{t}=(a/s)\,g(s)\text{,}$ with $z_{0}=s$ and $\lambda=a/s>0\text{.}$ ∎

Proof of (D)$\Rightarrow$(A).

The hypothesis $p=\lambda\,g(z_{0})$ for all finitely-supported $(\pi_{j})$ gives, upon taking $\pi=\delta_{t}\text{:}$ $S(t)=\lambda\,z_{0}^{t}$ for each $t\geq 1\text{.}$ Hence $h_{t}=1-S(t)/S(t-1)=1-z_{0}$ for all $t\geq 2\text{,}$ confirming geometric tail with $s=z_{0}\text{,}$ $q=1-z_{0}\text{,}$ and $a=\lambda z_{0}\text{.}$ ∎

Remark 3.7 (Emphasis of the (C)$\Rightarrow$(A) argument).

Only the $k=1$ instance of (C) is needed: the condition that $E[T]$ depends on $T_{0}$ through $g(z_{0})$ alone already forces geometric-tail catastrophe. The mechanism is Möbius rigidity: the ratio of two linear functionals of $(\pi_{j})$ equaling a function of a third forces that function to be linear fractional, and determination by three points propagates the local constraint globally. This is the same low-dimensionality phenomenon that drives the DRP—there, affine dependence on $g(u)$ forces the highest derivative to cancel; here, it forces the hazard sequence to stabilize.

3.4 From Geometric Tail to Full Memorylessness

The geometric-tail class has one free parameter ($a=S(1)$) beyond the catastrophe rate $q\text{.}$ The following corollary identifies the algebraic condition that pins down this parameter.

Corollary 3.8 (Full memorylessness).

Among geometric-tail mechanisms, full memorylessness ($h_{t}\equiv q$ for all $t\geq 1$) is equivalent to $\lambda=1$ in condition (D) of Theorem 3.6, i.e., the success probability is a literal PGF evaluation:

$$p=g(s).$$

Proof.

Geometric tail gives $p=(a/s)\,g(s)\text{,}$ so $\lambda=a/s\text{.}$ If $\lambda=1$ then $a=s\text{,}$ hence $h_{1}=1-a=1-s=q\text{,}$ giving full memorylessness. Conversely, memorylessness gives $a=s$ and $\lambda=1\text{.}$ ∎

Example 3.9 (Geometric-tail but not memoryless).

Set $h_{1}=1/2$ and $h_{t}=1/4$ for $t\geq 2\text{,}$ so $a=1/2\text{,}$ $q=1/4\text{,}$ $s=3/4\text{.}$ Then $S(t)=(1/2)(3/4)^{t-1}$ and $p=(2/3)\,g(3/4)\text{.}$ Since $h_{1}=1/2\neq 1/4=q\text{,}$ this is not memoryless, yet it has uniform affine PGF structure by Proposition 3.5, with $\alpha_{N}(1)=\alpha_{D}(1)=aq=1/8\text{.}$ This confirms that affine PGF structure characterizes the geometric-tail class, not full memorylessness.

Remark 3.10 (Proportionality is essential).

Condition (D) requires $p=\lambda\,g(z_{0})$—proportionality, not merely functional dependence. If one weakens (D) to “$p=\varphi(g(z_{0}))$ for some function $\varphi\text{,}$” then two-point distributions force $\varphi$ to be affine: $\varphi(x)=\lambda x+\mu\text{.}$ When $\mu>0\text{,}$ the resulting survival function $S(j)=\lambda z_{0}^{j}+\mu$ has $h_{j}\to 0$ as $j\to\infty\text{,}$ violating both (B) and (C). Thus proportionality ($\mu=0$) is necessary for the equivalence.

4 Exponential Approximation

The attempt decomposition (Theorem 2.2) writes $T=\sum_{i=1}^{N}A_{i}+B$ as a geometric sum plus a perturbation. A classical theorem of [36] establishes that normalized geometric sums converge to the exponential distribution; see [21, 25] for comprehensive treatments. §2 revealed a specific algebraic structure—the coefficient equality $\alpha_{N}(1)=\alpha_{D}(1)$—governing the PGF of $T\text{.}$ We now show that this structure sharpens Rényi’s result into:

(i)

a quantitative Laplace expansion whose leading error is second-order (the first-order term vanishes as an algebraic identity, not an asymptotic cancellation), and
(ii)

a sharp two-sided Kolmogorov bound $d_{K}(T/E[T],\operatorname{Exp}(1))\asymp p+|\alpha|$ that closes a logarithmic gap left by the smoothing inequality approach.

Both results hold for arbitrary base processes $T_{0}$ and are driven by the same coefficient equality.

4.1 The Perturbation Parameter

Recall the notation: $\mu=E[T]=(1-p)/(qp)\text{,}$ $b=E[B]\text{,}$ and write $W=T/\mu$ for the normalized completion time. From the decomposition (6), $W=U/\mu+B/\mu$ where $U=\sum_{i=1}^{N}A_{i}\text{.}$

Definition 4.1 (Perturbation parameters).

Define

$$\alpha=\frac{s(p-qD_{1})}{1-p},\qquad\beta=\frac{1-qp-qsD_{1}}{1-p},$$

(26)

where $D_{1}=g^{\prime}(s)\text{.}$

Proposition 4.2 (Properties of the perturbation parameters).

(a)

Fundamental identity: $1+\alpha-\beta=0$.
(b)

Successful-attempt identity: $b/\mu=sp/(1-p)-\alpha=qsD_{1}/(1-p)$.
(c)

Vanishing rate: for fixed $q\in(0,1)$, $|\alpha|=O_{q}(p|\ln p|)$ as $p\to 0$.

Proof.

(a) $1+\alpha-\beta=1+[sp-qsD_{1}-1+qp+qsD_{1}]/(1-p)=1+(p-1)/(1-p)=0\text{,}$ noting that $s+q=1\text{.}$

(b) Differentiating $G_{B}(w)=g(ws)/p$ at $w=1$ gives $b=sD_{1}/p\text{,}$ so $b/\mu=qsD_{1}/(1-p)\text{.}$ The first equality follows from the definition of $\alpha\text{.}$

(c) Set $Y=s^{T_{0}}\in(0,1]\text{,}$ so $p=E[Y]$ and $qsD_{1}=qE[-Y\ln Y]/|\ln s|\text{.}$ Since $\varphi(x)=-x\ln x$ is concave, Jensen gives $E[-Y\ln Y]\leq-p\ln p\text{,}$ whence $qsD_{1}=O_{q}(p|\ln p|)\text{.}$ Both terms in $\alpha=(sp-qsD_{1})/(1-p)$ are therefore $O_{q}(p|\ln p|)\text{.}$ ∎

Identity (a) is the Laplace-domain restatement of the coefficient equality (10): both arise from $s+q=1\text{.}$ Its consequence is that the first-order error in the exponential approximation vanishes exactly (Theorem 4.4).

Remark 4.3 (Interpretation of $\alpha$).

Identity (b) shows that $\alpha$ measures the deviation of the successful attempt from a “geometric target”: $\alpha=sp/(1-p)-b/\mu\text{.}$ If $\alpha=0\text{,}$ the rescaled successful attempt $B/\mu$ has exactly the expected length one would predict from a pure geometric model; if $\alpha\neq 0\text{,}$ $B$ is longer (when $\alpha<0$) or shorter (when $\alpha>0$) than this target. As we shall see, the Kolmogorov distance decomposes into two independent error sources: a lattice error $\Theta(p)$ from the integrality of the attempt count, and a perturbation error $\Theta(|\alpha|)$ from this mismatch.

4.2 The Laplace Expansion

Theorem 4.4 (Laplace expansion).

Fix $q\in(0,1)\text{.}$ There exist $p_{0}(q),\epsilon_{0}(q)>0$ such that for all $p\leq p_{0}$ and $\theta>0$ satisfying $\epsilon:=\theta pq/(1-p)\leq\epsilon_{0}\text{,}$

$$E[e^{-\theta W}]-\frac{1}{1+\theta}=\frac{\alpha\,\theta^{2}}{(1+\theta)^{2}}+O\!\left(\frac{|\alpha|^{2}\,\theta^{3}}{(1+\theta)^{3}}\right)+O\!\left(\frac{p\,\theta^{2}}{1+\theta}\right),$$

(27)

with implicit constants depending only on $q\text{.}$

Proof.

Set $\epsilon=\theta/\mu=\theta pq/(1-p)$ and $w=e^{-\epsilon}\text{.}$ Then $E[e^{-\theta W}]=G_{T}(w)\text{.}$ We expand the numerator $N(w)$ and denominator $D(w)$ of (9) around $w=1\text{.}$

Using $w=1-\epsilon+O(\epsilon^{2})\text{,}$ $u=ws=s-s\epsilon+O(\epsilon^{2})\text{,}$ and $g(u)=p-sD_{1}\epsilon+O(\epsilon^{2})\text{:}$

Numerator. $g(u)(1-u)=(p-sD_{1}\epsilon+O(\epsilon^{2}))(q+s\epsilon+O(\epsilon^{2}))=pq+(ps-qsD_{1})\epsilon+O(\epsilon^{2})\text{.}$

Denominator. $1-w+qwg(u)=\epsilon+O(\epsilon^{2})+q(1-\epsilon+O(\epsilon^{2}))(p-sD_{1}\epsilon+O(\epsilon^{2}))=qp+(1-qp-qsD_{1})\epsilon+O(\epsilon^{2})\text{.}$

Dividing both by $pq$ and substituting $\epsilon=\theta pq/(1-p)\text{:}$

$$G_{T}(w)=\frac{1+\alpha\theta+R_{N}(\theta)}{1+\beta\theta+R_{D}(\theta)},$$

(28)

where $|R_{N}|,|R_{D}|=O(\theta^{2}p)$ (with implicit constants bounded for fixed $q\text{,}$ since $s^{k}D_{k}\leq\max_{t\geq 1}t^{k}s^{t}=(k/(e|\ln s|))^{k}$).

By Proposition 4.2 (a), $\beta=1+\alpha\text{.}$ Computing:

$$\frac{1+\alpha\theta}{1+\beta\theta}-\frac{1}{1+\theta}=\frac{(1+\alpha-\beta)\theta+\alpha\theta^{2}}{(1+\beta\theta)(1+\theta)}=\frac{\alpha\theta^{2}}{(1+\beta\theta)(1+\theta)}.$$

Since $\beta=1+\alpha\text{:}$ $1/(1+\beta\theta)=1/((1+\theta)(1+\alpha\theta/(1+\theta)))\text{,}$ giving

$$\frac{\alpha\theta^{2}}{(1+\beta\theta)(1+\theta)}=\frac{\alpha\theta^{2}}{(1+\theta)^{2}}+O\!\left(\frac{|\alpha|^{2}\theta^{3}}{(1+\theta)^{3}}\right).$$

It remains to bound the contribution of the remainders $R_{N},R_{D}$ to the ratio (28). Writing $G_{T}(w)-1/(1+\theta)$ and using $1+\alpha-\beta=0\text{,}$ the numerator becomes $\alpha\theta^{2}+(1+\theta)R_{N}-R_{D}\text{.}$ For the denominator: $1+\beta\theta+R_{D}=(1+\theta)(1+\alpha\theta/(1+\theta)+R_{D}/(1+\theta))\text{.}$ Taking $p_{0}$ small enough that $|\alpha|\leq 1/4$ and $\epsilon_{0}\leq q/4\text{,}$ the parenthetical factor is at least $1/2\text{,}$ so $|1+\beta\theta+R_{D}|\geq(1+\theta)/2\text{.}$ The numerator remainder satisfies $|(1+\theta)R_{N}-R_{D}|=O((1+\theta)\theta^{2}p)\text{,}$ giving a contribution $O(p\theta^{2}/(1+\theta))\text{.}$ ∎

Remark 4.5 (Algebraic origin of the $\theta^{1}$ cancellation).

The identity $1+\alpha-\beta=0$ is equivalent to $E[W]=1$ (matching first moments). Its algebraic root is $s+q=1\text{,}$ the same identity underlying the coefficient equality $\alpha_{N}(1)=\alpha_{D}(1)$ that drives the DRP. This is an exact algebraic identity, not an asymptotic cancellation—a structural guarantee that the exponential approximation begins at second order for any base process.

Corollary 4.6 (Exponential limit).

Fix $q\in(0,1)\text{.}$ Under memoryless catastrophe with rate $q\text{,}$ $T/E[T]\xrightarrow{d}\operatorname{Exp}(1)$ as $p=g(s)\to 0\text{.}$

Proof.

By Proposition 4.2 (c), $|\alpha|\to 0\text{.}$ For each fixed $\theta>0\text{,}$ the condition $\epsilon=\theta pq/(1-p)\leq\epsilon_{0}$ is eventually satisfied, and both error terms in (27) vanish. Hence $E[e^{-\theta W}]\to 1/(1+\theta)$ pointwise. Pointwise convergence of Laplace transforms on $(0,\infty)$ implies convergence in distribution [21]. ∎

Remark 4.7 (Variable $q$ regime).

Corollary 4.6 assumes fixed $q\text{.}$ When $q\to 0$ simultaneously with $p\to 0$ (as in the CCP, where $q=m/(n+m)\to 0$), the implicit constants in (27) may deteriorate. The convergence $T/E[T]\xrightarrow{d}\operatorname{Exp}(1)$ in such regimes follows instead from the Bridge Theorem below (Theorem 4.13), which requires only $p\to 0$ and uniform control of certain moment ratios of $A\text{.}$

4.3 The Bridge Theorem

The Laplace expansion (Theorem 4.4) gives pointwise control on the Laplace transform error. Converting to Kolmogorov distance $d_{\mathrm{K}}$ via the smoothing inequality ([25]) yields

$$d_{\mathrm{K}}\lesssim\int_{0}^{R}\frac{|\alpha|\theta}{(1+\theta)^{2}}\,d\theta+\frac{1}{R},$$

which evaluates to $O(|\alpha|\ln(1/|\alpha|))$ upon optimizing $R\text{.}$ The logarithmic factor arises because the integrand decays only as $|\alpha|/\theta$ for large $\theta\text{,}$ accumulating a logarithmic contribution absent from the pointwise bound.

We now develop a direct probabilistic approach that avoids this loss entirely. The key idea is to decompose the approximation into three steps, each handling one component of $W=U/\mu+B/\mu\text{.}$

Lemma 4.8 (Failed-attempt tail bound).

Under memoryless catastrophe with rate $q\text{,}$ the failed-attempt length satisfies $P(A\geq t)\leq s^{t-1}$ for all $t\geq 1\text{.}$

Proof.

A failure at step $j$ requires survival through steps $1,\ldots,j-1$ (probability $s^{j-1}$), catastrophe at step $j$ (probability $q$), and $T_{0}\geq j\text{.}$ Hence

$$P(A\geq t)=\frac{\sum_{j\geq t}q\,s^{j-1}P(T_{0}\geq j)}{1-p}.$$

Factoring $s^{t-1}$ and using $P(T_{0}\geq t+k)\leq P(T_{0}\geq k+1)\text{:}$ the numerator is at most $s^{t-1}\sum_{k\geq 0}q\,s^{k}P(T_{0}\geq k+1)=s^{t-1}(1-p)\text{.}$ ∎

Lemma 4.9 (Uniform moment-ratio control).

Fix $q\in(0,1)\text{.}$ For any base process $T_{0}$ and any integer $k\geq 1\text{,}$

$$E[A^{k}]\leq M_{k}(q):=\sum_{t\geq 1}k\,t^{k-1}\,s^{t-1}<\infty,$$

and $E[A]\geq 1\text{.}$ Hence all moment ratios $E[A^{k}]/E[A]^{k}$ are bounded by constants depending only on $q\text{.}$

Proof.

The tail-sum formula $E[A^{k}]=\sum_{t\geq 1}[t^{k}-(t-1)^{k}]P(A\geq t)\text{,}$ combined with the mean value theorem bound $t^{k}-(t-1)^{k}\leq kt^{k-1}$ and Lemma 4.8, gives $E[A^{k}]\leq\sum_{t\geq 1}kt^{k-1}s^{t-1}<\infty\text{.}$ The bound $E[A]\geq 1$ is immediate since $A\geq 1\text{.}$ ∎

Lemma 4.10 (Scaling perturbation).

For any non-negative random variable $X$ and $r\in(0,1)\text{,}$

$$d_{\mathrm{K}}(rX,\operatorname{Exp}(1))\leq d_{\mathrm{K}}(X,\operatorname{Exp}(1))+(1-r).$$

Proof.

Set $\varepsilon=d_{\mathrm{K}}(X,\operatorname{Exp}(1))$ and $F_{E}(x)=1-e^{-x}\text{.}$ For all $x\geq 0\text{:}$

$$|P(rX\leq x)-F_{E}(x)|\leq\underbrace{|P(X\leq x/r)-F_{E}(x/r)|}_{\leq\,\varepsilon}+\underbrace{|F_{E}(x/r)-F_{E}(x)|}_{=\,e^{-x}-e^{-x/r}}.$$

The second term is non-negative (since $r<1$) and maximized at $x^{*}=r\ln(1/r)/(1-r)\text{,}$ where it equals $(1-r)r^{r/(1-r)}\leq 1-r\text{.}$ ∎

Lemma 4.11 (Additive perturbation).

Let $X\geq 0$ and $Y\geq 0$ be independent. Then

$$d_{\mathrm{K}}(X+Y,\operatorname{Exp}(1))\leq d_{\mathrm{K}}(X,\operatorname{Exp}(1))+E[Y].$$

Proof.

Set $\varepsilon=d_{\mathrm{K}}(X,\operatorname{Exp}(1))$ and $F_{E}(x)=1-e^{-x}\text{.}$

Upper bound. Since $Y\geq 0\text{,}$ $P(X+Y\leq x)\leq P(X\leq x)\leq F_{E}(x)+\varepsilon\text{.}$

Lower bound. We must show $F_{E}(x)-P(X+Y\leq x)\leq E[Y]+\varepsilon\text{.}$ Conditioning on $Y\text{:}$

$$F_{E}(x)-P(X+Y\leq x)=E\bigl[\bigl(F_{E}(x)-F_{X}(x-Y)\bigr)\mathbf{1}_{\{Y\leq x\}}\bigr]+F_{E}(x)\,P(Y>x).$$

For the first term, when $Y\leq x\text{,}$ $F_{E}(x)-F_{X}(x-Y)\leq\bigl[F_{E}(x)-F_{E}(x-Y)\bigr]+\varepsilon\leq Y+\varepsilon\text{,}$ using $|F_{E}^{\prime}|\leq 1\text{.}$ For the second term, $F_{E}(x)\,P(Y>x)\leq xP(Y>x)\leq E[Y\cdot\mathbf{1}_{\{Y>x\}}]\text{.}$ Combining them gives

$$F_{E}(x)-P(X+Y\leq x)\leq E[Y]+\varepsilon.$$

Rényi’s classical theorem ([36]) asserts that normalized geometric sums converge in distribution to $\operatorname{Exp}(1)\text{.}$ [6] established the first explicit Kolmogorov-distance bound, showing that the rate is linear in the geometric parameter $p\text{.}$

Theorem 4.12 (Theorem 2.1 (ii), [6]).

Let $U=\sum_{i=1}^{N}X_{i}\text{,}$ $N\sim\operatorname{Geom}_{0}(p)\text{,}$ where $0<p<1\text{,}$ the $X_{i}$ are i.i.d. nonnegative random variables, $P(X_{1}=0)<1\text{,}$ and $E[X_{1}^{2}]<\infty\text{.}$ Set $\bar{p}:=1-p\text{,}$ $\rho_{2}:=\frac{E[X_{1}^{2}]}{E[X_{1}]^{2}}\text{.}$ Then

$$d_{K}\!\left(\frac{U}{E[U]},\,\operatorname{Exp}(1)\right)\leq p\max\!\left(\rho_{2},\frac{\rho_{2}}{2\bar{p}}\right).$$

(29)

We can now state and prove the Bridge Theorem. The name reflects its role: it bridges the $O(p)$ exponential approximation of the pure geometric sum $U/\mu_{U}$ to the full completion time $W=T/\mu\text{.}$

Theorem 4.13 (Bridge Theorem).

For $0<p\leq 1/2\text{,}$

$$d_{\mathrm{K}}(W,\operatorname{Exp}(1))\leq C_{1}\,p+2\,\frac{b}{\mu},$$

(30)

where $C_{1}<\infty$ depends only on the distribution of $A$ (and is uniformly bounded over all base processes $T_{0}$ for fixed $q\text{,}$ by Lemma 4.9).

Proof.

Set $\mu_{U}=E[U]=(1-p)E[A]/p$ and $r=\mu_{U}/\mu=1-b/\mu\in(0,1)\text{.}$

Step 1: Geometric-convolution bound for $V=U/\mu_{U}\text{.}$ By Lemma 4.9, $E[A^{k}]\leq M_{k}(q)<\infty$ for all $k\geq 1\text{,}$ so Brown’s moment condition $E[A^{2}]<\infty$ is satisfied and the moment ratios $\rho_{j}=E[A^{j}]/E[A]^{j}\leq M_{j}(q)/1=M_{j}(q)$ are uniformly bounded. Theorem 4.12 applied with $X_{i}=A_{i}$ gives $d_{\mathrm{K}}(V,\operatorname{Exp}(1))\leq\rho_{2}(A)\,p\text{.}$ Since $\rho_{2}(A)=E[A^{2}]/E[A]^{2}\leq M_{2}(q)$ by Lemma 4.9, we may take $C_{1}=M_{2}(q)\text{,}$ and hence $d_{\mathrm{K}}(V,\operatorname{Exp}(1))\leq C_{1}p\text{.}$

Step 2: Scaling perturbation. Since $U/\mu=rV$ with $r=1-b/\mu\text{,}$ Lemma 4.10 gives $d_{\mathrm{K}}(U/\mu,\operatorname{Exp}(1))\leq C_{1}p+b/\mu\text{.}$

Step 3: Additive perturbation. Since $W=U/\mu+B/\mu$ and $U,B$ are independent (by mutual independence of $N\text{,}$ $(A_{i})\text{,}$ $B$ in (6)), Lemma 4.11 gives $d_{\mathrm{K}}(W,\operatorname{Exp}(1))\leq C_{1}p+b/\mu+E[B/\mu]=C_{1}p+2b/\mu\text{.}$ ∎

Remark 4.14 (Connection to the staircase).

The explicit correction terms in the Bridge Theorem—$p=g(s)$ and $b/\mu=qsD_{1}/(1-p)$—involve only the $1$-jet $\{g(s),g^{\prime}(s)\}\text{,}$ matching the staircase level of $\operatorname{Var}(T)$ (17). The Brown constant $C_{1}$ depends on the full distribution of $A$ (via moment ratios), so this staircase connection is more heuristic than exact.

4.4 The Sharp Two-Sided Bound

We now combine the Laplace expansion (Theorem 4.4) and the Bridge Theorem (Theorem 4.13) to obtain a sharp two-sided bound showing that $p+|\alpha|$ is the correct rate.

Theorem 4.15 (Two-sided bound).

There exist constants $c_{A},C_{A},\delta_{A}>0$ depending only on $q$ such that, for $p+|\alpha|\leq\delta_{A}\text{,}$

$$c_{A}\,(p+|\alpha|)\leq d_{\mathrm{K}}\!\left(\frac{T}{E[T]},\,\operatorname{Exp}(1)\right)\leq C_{A}\,(p+|\alpha|).$$

(31)

Proof of the upper bound.

From Theorem 4.13 and Proposition 4.2 (b): when $\alpha\leq 0\text{,}$ $b/\mu=|\alpha|+sp/(1-p)\text{;}$ when $\alpha>0\text{,}$ $b/\mu=sp/(1-p)-\alpha\leq sp/(1-p)\text{.}$ In both cases, $2b/\mu\leq 2|\alpha|+2sp/(1-p)\text{.}$ Hence

$$d_{\mathrm{K}}\leq C_{1}p+2|\alpha|+\frac{2sp}{1-p}\leq C_{A}(p+|\alpha|).$$

Proof of the lower bound.

We exhibit two independent error sources, each yielding a lower bound.

Lower bound from discreteness. Since $T\geq 1\text{,}$ $W=T/\mu\geq 1/\mu=pq/(1-p)\text{.}$ For $x=pq/(2(1-p))<1/\mu\text{:}$ $P(W\leq x)=0\text{,}$ while $F_{E}(x)=1-e^{-x}\geq x/2=pq/(4(1-p))$ (using $1-e^{-t}\geq t/2$ for $t\in[0,1]$). Hence

$$d_{\mathrm{K}}\geq\frac{pq}{4(1-p)}=\Omega(p).$$

(32)

Lower bound from the Laplace transform. For any non-negative random variables $W,Z$ and any $\theta>0\text{,}$ the integral representation $E[e^{-\theta X}]=\int_{0}^{\infty}\theta e^{-\theta x}(1-F_{X}(x))\,dx$ gives

$$d_{\mathrm{K}}(W,Z)\geq|E[e^{-\theta W}]-E[e^{-\theta Z}]|.$$

(33)

Applying (33) with $Z\sim\operatorname{Exp}(1)$ and $\theta=1\text{:}$ for $p$ sufficiently small, Theorem 4.4 gives $|E[e^{-W}]-1/2|=|\alpha|/4+O(|\alpha|^{2})+O(p)\text{,}$ so

$$d_{\mathrm{K}}\geq\frac{|\alpha|}{4}-O(|\alpha|^{2})-O(p)\geq c^{\prime}|\alpha|-c^{\prime\prime}p$$

(34)

for suitable $c^{\prime},c^{\prime\prime}>0\text{.}$

Combining. Set $K=2c^{\prime\prime}/c^{\prime}\text{.}$ Choose $\delta_{A}$ small enough that both (32) and (34) hold.

Case 1: $|\alpha|\leq Kp\text{.}$ Then $p+|\alpha|\leq(1+K)p\text{.}$ From (32), $d_{\mathrm{K}}\geq qp/4\geq q(p+|\alpha|)/(4(1+K))\text{.}$

Case 2: $|\alpha|>Kp\text{.}$ Then $p+|\alpha|\leq(1+1/K)|\alpha|\text{.}$ From (34), $d_{\mathrm{K}}\geq c^{\prime}|\alpha|/2\geq c^{\prime}K(p+|\alpha|)/(2(K+1))\text{.}$

In both cases, $d_{\mathrm{K}}\geq c_{A}(p+|\alpha|)$ with $c_{A}=\min\{q/(4(1+K)),\;c^{\prime}K/(2(K+1))\}>0\text{.}$ ∎

The following counterexample shows that the $p$-term in the template $p+|\alpha|$ cannot be dropped.

Proposition 4.16 (Counterexample: $p$ cannot be dropped).

There exists a family of base processes $(T_{0}^{(\varepsilon)})_{\varepsilon\to 0}$ with $q=1/2$ such that $|\alpha_{\varepsilon}|=o(p_{\varepsilon})$ but $d_{\mathrm{K}}(W_{\varepsilon},\operatorname{Exp}(1))=\Theta(p_{\varepsilon})\text{.}$

Proof.

Set $q=s=1/2$ and $P(T_{0}^{(\varepsilon)}=t)=\varepsilon\,\mathbf{1}_{t=1}+(1-\varepsilon)\,\mathbf{1}_{t=M_{\varepsilon}}$ where $M_{\varepsilon}=\lceil 3\log_{2}(1/\varepsilon)\rceil\text{.}$

Claim 1: $|\alpha_{\varepsilon}|=o(p_{\varepsilon})\text{.}$ Since $2^{-M_{\varepsilon}}\leq\varepsilon^{3}\text{:}$ $p_{\varepsilon}=\varepsilon/2+O(\varepsilon^{3})$ and $qD_{1,\varepsilon}=\varepsilon/2+o(\varepsilon)=p_{\varepsilon}+o(p_{\varepsilon})\text{,}$ so $\alpha_{\varepsilon}=o(p_{\varepsilon})\text{.}$

Claim 2: $d_{\mathrm{K}}=\Theta(p_{\varepsilon})\text{.}$ The event $\{T=1\}$ requires $T_{0}=1$ and no catastrophe: $P(T=1)=\varepsilon/2\text{.}$ At $x_{0}=1/E[T]\text{:}$ $P(W\leq x_{0})=P(T\leq 1)=p_{\varepsilon}+o(p_{\varepsilon})$ while $1-e^{-x_{0}}=p_{\varepsilon}/2+O(p_{\varepsilon}^{2})\text{.}$ Hence $d_{\mathrm{K}}\geq p_{\varepsilon}/2+o(p_{\varepsilon})\text{.}$ The upper bound $d_{\mathrm{K}}=O(p_{\varepsilon})$ follows from Theorem 4.13 (since $|\alpha_{\varepsilon}|=o(p_{\varepsilon})$). ∎

Remark 4.17 (Why $p+|\alpha|$ is the correct template).

The two-sided bound decomposes the approximation error into two sources with distinct physical origins:

•

The $p$-term reflects geometric discreteness: the attempt count is integer-valued, so $W$ takes values in $(1/\mu)\mathbb{Z}_{>0}\text{,}$ creating CDF jumps that no continuous distribution can match.
•

The $|\alpha|$-term reflects the successful-attempt perturbation: $T$ is not a pure geometric sum but includes the final attempt $B\text{,}$ whose expected length may deviate from the rescaling target.

The counterexample engineers near-cancellation of the perturbation ($\alpha\approx 0$) while preserving the irreducible lattice error, exposing the $\Theta(p)$ floor. In the CCP application (§6), the opposite regime prevails: $p\ll|\alpha|\text{,}$ so the perturbation term dominates.

5 Continuous-Time Theory

The discrete theory of §§2–4 rests on the affine dependence of $G_{T}$ on the base PGF $g$ at a linearly shifted argument. In continuous time, the same mechanism operates with PGFs replaced by Laplace transforms and the multiplicative shift $u=ws$ replaced by an additive shift $\sigma=\lambda+r\text{.}$ The entire theory carries over, with one notable sharpening: the geometric-tail gap of Theorem 3.6 closes, and affine Laplace structure characterizes full Poisson resetting with no residual degree of freedom (Theorem 5.7).

5.1 Setup

Let $T_{0}$ be a positive, almost surely finite, continuous random variable with Laplace transform $\hat{f}_{0}(\lambda)=E[e^{-\lambda T_{0}}]\text{.}$ Catastrophe occurs via a Poisson process of rate $r>0\text{,}$ independent of $T_{0}\text{.}$ The attempt decomposition (6) carries over with $N\sim\operatorname{Geom}_{0}(p)$ and

$$p=P(T_{0}<\tau)=E[e^{-rT_{0}}]=\hat{f}_{0}(r),$$

(35)

where $\tau\sim\operatorname{Exp}(r)$ is the first catastrophe time. Under any age-based mechanism, catastrophe and $T_{0}$ are independent, so $p=P(T_{0}<\tau)=E[S(T_{0})]\text{,}$ the continuous-time analog of (19). More generally, for any continuous-time catastrophe mechanism with hazard rate $h:(0,\infty)\to(0,\infty)\text{,}$ the survival function $S(x)=\exp\bigl(-\int_{0}^{x}h(t)\,dt\bigr)$ gives the probability of no catastrophe before time $x\text{;}$ Poisson resetting is the special case $h\equiv r\text{,}$ i.e., $S(x)=e^{-rx}\text{.}$ We write $\hat{f}_{T}(\lambda)=E[e^{-\lambda T}]$ for the Laplace transform of the completion time.

5.2 The Laplace Transform Formula and Affine Structure

The following is a standard consequence of the renewal structure under Poisson resetting (see, e.g., [15, 8]). We state it to exhibit its affine structure, which has not been identified previously.

Theorem 5.1 (Laplace transform under Poisson resetting).

For any base process $T_{0}$ under Poisson resetting at rate $r\text{,}$

$$\hat{f}_{T}(\lambda)=\frac{(\lambda+r)\,\hat{f}_{0}(\lambda+r)}{\lambda+r\,\hat{f}_{0}(\lambda+r)},\qquad\sigma:=\lambda+r.$$

(36)

Proof.

The successful-attempt Laplace transform is $\hat{f}_{B}(\lambda)=\hat{f}_{0}(\sigma)/p$ (tilting by $e^{-rT_{0}}/p$). For the failed attempt, $A=\tau\mid\tau<T_{0}$ with $\tau\sim\operatorname{Exp}(r)$ independent of $T_{0}\text{:}$

$$(1-p)\,\hat{f}_{A}(\lambda)=E[e^{-\lambda\tau}\mathbf{1}_{\tau<T_{0}}]=\frac{r(1-\hat{f}_{0}(\sigma))}{\sigma}.$$

Composing via $\hat{f}_{T}=\hat{f}_{B}\cdot p/[1-(1-p)\hat{f}_{A}]\text{,}$ the denominator becomes $1-r(1-\hat{f}_{0}(\sigma))/\sigma=(\lambda+r\hat{f}_{0}(\sigma))/\sigma\text{,}$ using $\sigma-r=\lambda\text{,}$ which gives (36).

∎

Corollary 5.2.

$E[T]=(1-p)/(rp)$, depending on $T_{0}$ only through $p=\hat{f}_{0}(r)\text{.}$

Proof.

By the attempt decomposition, $E[T]=E[\ell]/p$ with $E[\ell]=(1-p)/r$ (the expected attempt length under $\operatorname{Exp}(r)$ catastrophe), giving $E[T]=(1-p)/(rp)\text{.}$ ∎

As in the discrete case (§2.2), formula (36) is affine in $\hat{f}_{0}(\sigma)$ with linear shift $\sigma=\lambda+r\text{:}$

$$N(\lambda)=\underbrace{\sigma}_{\alpha_{N}(\lambda)}\cdot\hat{f}_{0}(\sigma)+\underbrace{0}_{\beta_{N}(\lambda)},\qquad D(\lambda)=\underbrace{r}_{\alpha_{D}(\lambda)}\cdot\hat{f}_{0}(\sigma)+\underbrace{\lambda}_{\beta_{D}(\lambda)}.$$

(37)

At the normalization point $\lambda=0\text{:}$ $\beta_{N}(0)=0=\beta_{D}(0)\text{,}$ so $\hat{f}_{T}(0)=1$ forces

$$\alpha_{N}(0)=\alpha_{D}(0)=r,$$

(38)

the continuous-time counterpart of (10). Since the transform has the same affine-ratio structure with linear shift and coefficient matching at $\lambda=0\text{,}$ Lemma 2.7 applies in continuous time as well.

5.3 Derivative Reduction

Theorem 5.3 (Continuous-time DRP).

Define $D_{k}=\hat{f}_{0}^{(k)}(r)$ and $\Delta_{k}=N^{(k)}(0)-D^{(k)}(0)\text{.}$ For all $k\geq 1\text{,}$

$$\Delta_{k}=\begin{cases}-(1-p),&k=1,\\ k\,D_{k-1},&k\geq 2.\end{cases}$$

(39)

In particular, $\Delta_{k}$ does not contain $D_{k}\text{.}$

Proof.

Since $N(\lambda)=(\lambda+r)\hat{f}_{0}(\lambda+r)$ and $\sigma=\lambda+r$ is linear ($\sigma^{\prime}=1$), the Leibniz rule gives $N^{(k)}(0)=rD_{k}+kD_{k-1}\text{.}$ For the denominator: $D^{(1)}(0)=1+rD_{1}$ and $D^{(k)}(0)=rD_{k}$ for $k\geq 2\text{.}$ Hence $\Delta_{k}=kD_{k-1}$ for $k\geq 2$ (the $D_{k}$ terms cancel because $\alpha_{N}(0)=\alpha_{D}(0)=r$), and $\Delta_{1}=(rD_{1}+p)-(1+rD_{1})=p-1\text{.}$ ∎

Remark 5.4 (Comparison with the discrete DRP).

The discrete formula $\Delta_{k}=-ks^{k-1}D_{k-1}$ (Theorem 2.6) contains the factor $s^{k-1}$ from the multiplicative shift $u=ws$ (chain rule coefficient $s$). The continuous formula $\Delta_{k}=kD_{k-1}$ has no such factor because the additive shift $\sigma=\lambda+r$ has chain rule coefficient $1\text{.}$ The sign difference ($-k$ vs. $+k$) reflects opposite monotonicity conventions: $G_{T}^{\prime}(1)=E[T]>0$ while $\hat{f}_{T}^{\prime}(0)=-E[T]<0\text{.}$

5.4 Characterization: Affine Laplace Structure Is Poisson

In discrete time, affine PGF structure characterizes geometric-tail catastrophe—constant hazard from the second step onward—leaving the first-step hazard $h_{1}$ as a free parameter (Theorem 3.6). In continuous time, this residual degree of freedom vanishes.

Definition 5.5 (Uniform affine Laplace structure).

A continuous-time catastrophe mechanism has uniform affine Laplace structure if there exist functions $a(\lambda),b(\lambda),c(\lambda),d(\lambda)\text{,}$ analytic in a neighborhood of $\lambda=0\text{,}$ depending only on the mechanism and a linear function $\sigma(\lambda)=\lambda+r$ ($r>0$), such that for all base processes $T_{0}\text{,}$

$$\hat{f}_{T}(\lambda)=\frac{a(\lambda)\,\hat{f}_{0}(\sigma)+b(\lambda)}{c(\lambda)\,\hat{f}_{0}(\sigma)+d(\lambda)},$$

with $a,c$ not identically zero and $c(0)\hat{f}_{0}(r)+d(0)\neq 0$ for all admissible $\hat{f}_{0}\text{.}$

Remark 5.6 (Why the shift has unit slope).

The restriction $\sigma(\lambda)=\lambda+r$ is not ad hoc. Starting from a general linear shift $\sigma(\lambda)=a\lambda+b$ with $a,b>0\text{,}$ the single-point sufficiency condition $p=\hat{f}_{0}(b)$ for all $T_{0}$ forces $S(x)=e^{-bx}$ (take $T_{0}=\delta_{x}$), identifying $b$ with the catastrophe rate $r\text{.}$ The Poisson catastrophe time then contributes a factor $e^{-rt}$ that combines with the Laplace kernel $e^{-\lambda t}$ into $e^{-(\lambda+r)t}\text{,}$ giving $a=1$ without further assumptions.

For the characterization result below, we temporarily enlarge the admissible class of base laws: the quantifiers over $T_{0}$ range over all positive, almost surely finite random variables, including point masses and two-point mixtures. Continuity of $T_{0}$ is imposed only later, in the lattice-free sharp-rate statement.

Theorem 5.7 (Continuous-time characterization).

Among continuous-time age-based catastrophe mechanisms specified by a measurable, locally integrable hazard rate $h:(0,\infty)\to(0,\infty)$ (so that $S(x)=\exp(-\int_{0}^{x}h(t)\,dt)$ is well-defined, strictly decreasing, and absolutely continuous), the following are equivalent:

(a)

$S(x)=e^{-rx}$ for all $x>0$ (Poisson resetting; equivalently, $h(t)=r$ for a.e. $t>0$).
(b)

Uniform affine Laplace structure (Definition 5.5).
(c)

Derivative reduction: there exists $r_{0}>0$ such that for every positive, almost surely finite $T_{0}$ and every $k\geq 1$, the $k$-th moment of $T$ is determined by $\{\hat{f}_{0}^{(j)}(r_{0})\}_{j=0}^{k-1}$.
(d)

Single-point sufficiency: there exists $r_{0}>0$ such that $p=\hat{f}_{0}(r_{0})$ for every positive, almost surely finite $T_{0}$.

Proof.

(a)$\Rightarrow$(b): Theorem 5.1.

(b)$\Rightarrow$(c): Suppose

$$\hat{f}_{T}(\lambda)=\frac{a(\lambda)\hat{f}_{0}(\sigma(\lambda))+b(\lambda)}{c(\lambda)\hat{f}_{0}(\sigma(\lambda))+d(\lambda)},\qquad\sigma(\lambda)=\lambda+r_{0}.$$

Applying $\hat{f}_{T}(0)=1$ to point masses $T_{0}=\delta_{x}$ and letting $x\to\infty$ gives $b(0)=d(0)\text{;}$ back-substitution yields $a(0)=c(0)\text{.}$ By non-degeneracy, Lemma 2.7 applies with $R=\hat{f}_{T}\text{,}$ $h=\hat{f}_{0}\text{,}$ $z_{*}=0\text{,}$ and $u_{0}=r_{0}\text{.}$ Hence $\hat{f}_{T}^{(k)}(0)$ depends only on $\{\hat{f}_{0}^{(j)}(r_{0})\}_{j=0}^{k-1}\text{,}$ which is exactly the derivative-reduction property.

(a)$\Rightarrow$(d): By definition (35),

$$p=P(T_{0}<\tau)=E[e^{-rT_{0}}]=\hat{f}_{0}(r).$$

(d)$\Rightarrow$(a): Under (d), for every positive, almost surely finite $T_{0}$ we have

$$E[S(T_{0})]=p=\hat{f}_{0}(r_{0})=E[e^{-r_{0}T_{0}}].$$

Taking $T_{0}=\delta_{x}$ gives $S(x)=e^{-r_{0}x}$ for every $x>0\text{.}$

(c)$\Rightarrow$(a): We use only the $k=1$ case. Thus there exist $r_{0}>0$ and a function $\psi:(0,1)\to\mathbb{R}$ such that

$$E[T]=\psi(\hat{f}_{0}(r_{0}))$$

for every positive, almost surely finite $T_{0}\text{.}$ For a general continuous-time age-based catastrophe mechanism,

$$p=E[S(T_{0})],\qquad E[\ell\,|\,T_{0}=x]=F(x):=\int_{0}^{x}S(t)\,dt,\qquad E[T]=\frac{E[\ell]}{p}=\frac{E[F(T_{0})]}{E[S(T_{0})]}.$$

Hence

$$\frac{E[F(T_{0})]}{E[S(T_{0})]}=\psi\!\bigl(E[e^{-r_{0}T_{0}}]\bigr)$$

(40)

for all positive, almost surely finite $T_{0}\text{.}$

Step 1: point masses determine $\psi$ on $(0,1)\text{.}$ Taking $T_{0}=\delta_{x}$ in (40) gives

$$\psi(e^{-r_{0}x})=\frac{F(x)}{S(x)},\qquad x>0.$$

Since $x\mapsto e^{-r_{0}x}$ is a bijection $(0,\infty)\to(0,1)\text{,}$ point masses determine $\psi$ on all of $(0,1)\text{.}$

Step 2: two-point laws force $\psi$ to be Möbius. Take $T_{0}=\alpha\delta_{a}+(1-\alpha)\delta_{b}$ with $0<a<b\text{.}$ Writing

$$v=\alpha e^{-r_{0}a}+(1-\alpha)e^{-r_{0}b}\in[e^{-r_{0}b},e^{-r_{0}a}],$$

equation (40) becomes

$$\psi(v)=\frac{\alpha F(a)+(1-\alpha)F(b)}{\alpha S(a)+(1-\alpha)S(b)}.$$

Solving for $\alpha$ in terms of $v$ shows that the right-hand side is a linear-fractional function of $v\text{,}$ namely

$$\psi(v)=\frac{(F(a)-F(b))\,v+F(b)e^{-r_{0}a}-F(a)e^{-r_{0}b}}{(S(a)-S(b))\,v+S(b)e^{-r_{0}a}-S(a)e^{-r_{0}b}}.$$

Thus $\psi$ is Möbius on every interval $[e^{-r_{0}b},e^{-r_{0}a}]\text{.}$

Step 3: overlapping intervals glue the local Möbius maps globally. Exactly as in Steps 2–4 of Theorem 3.6, the overlapping-interval consistency argument shows that all these local Möbius maps coincide with a single global one:

$$\psi(v)=\frac{Av+B}{Cv+D},\qquad v\in(0,1).$$

Because $S$ is strictly decreasing, the denominator slope in the two-point formula is nonzero, so the global denominator coefficient satisfies $C\neq 0\text{.}$

Step 4: coefficient consistency forces $F=\rho S+c\text{.}$ Comparing the coefficients of $v$ in the two-point representation with the global Möbius map shows that

$$\frac{F(a)-F(b)}{S(a)-S(b)}$$

is independent of the pair $(a,b)$ with $a\neq b\text{.}$ Denoting this constant by $\rho\text{,}$ we obtain

$$F(x)=\rho\,S(x)+c,\qquad x>0$$

for some constant $c\text{.}$

Step 5: differentiation yields constant hazard everywhere. Since $F(x)=\int_{0}^{x}S(t)\,dt$ is absolutely continuous and $F^{\prime}(x)=S(x)$ a.e., differentiating $F=\rho S+c$ gives

$$S(x)=\rho\,S^{\prime}(x)\qquad\text{for a.e.\ }x>0.$$

As $S^{\prime}(x)=-h(x)S(x)$ a.e. and $S(x)>0\text{,}$ we conclude

$$h(x)=-\frac{1}{\rho}=:r\qquad\text{for a.e.\ }x>0.$$

Hence $S(x)=e^{-rx}$ for all $x>0\text{,}$ which is exactly (a). ∎

Remark 5.8 (Why the geometric-tail gap closes).

The geometric-tail gap of Theorem 3.6 is a joint artifact of two discrete-time limitations: countable test points (requiring Möbius extension) and integer arithmetic (permitting only differencing). In continuous time, both limitations vanish simultaneously, forcing full Poisson resetting with no residual degree of freedom.

5.5 Exponential Approximation

The exponential approximation theory of §4 has a continuous-time counterpart. Define the continuous-time perturbation parameters:

$$\alpha=\frac{p+rD_{1}}{1-p},\qquad\beta=\frac{1+rD_{1}}{1-p},$$

(41)

where $D_{1}=\hat{f}_{0}^{\prime}(r)\text{.}$ Note that $D_{1}=-E[T_{0}e^{-rT_{0}}]<0\text{,}$ so the sign conventions in the discrete and continuous perturbation parameters are consistent: both $\alpha$ vanish when $b/\mu$ matches the geometric target (Remark 4.3). The fundamental identity

$$1+\alpha-\beta=0$$

(42)

holds by the same one-line computation as Proposition 4.2(a), replacing $s+q=1$ by direct cancellation of $rD_{1}\text{.}$ Differentiating $\hat{f}_{B}(\lambda)=\hat{f}_{0}(\lambda+r)/p$ at $\lambda=0$ gives $b=-D_{1}/p\text{,}$ so

$$\frac{b}{\mu}=\frac{-rD_{1}}{1-p}=\frac{p}{1-p}-\alpha,$$

(43)

the continuous-time analog of Proposition 4.2(b).

Theorem 5.9 (Continuous-time Laplace expansion).

Under Poisson resetting at rate $r\text{,}$ let $W=T/E[T]\text{.}$ For $\epsilon:=\theta rp/(1-p)\leq\epsilon_{0}\text{,}$

$$E[e^{-\theta W}]-\frac{1}{1+\theta}=\frac{\alpha\,\theta^{2}}{(1+\theta)^{2}}+O\!\left(\frac{|\alpha|^{2}\,\theta^{3}}{(1+\theta)^{3}}\right)+O\!\left(\frac{p\,\theta^{2}}{1+\theta}\right),$$

(44)

with implicit constants depending on $|D_{1}|$ and $|D_{2}|\text{,}$ which satisfy $|D_{k}|=|\hat{f}_{0}^{(k)}(r)|=E[T_{0}^{k}e^{-rT_{0}}]\leq(k/(er))^{k}\text{.}$

Proof.

Set $\mu=(1-p)/(rp)\text{,}$ $\epsilon=\theta/\mu\text{,}$ and expand $\hat{f}_{0}(r+\epsilon)=p+D_{1}\epsilon+O(\epsilon^{2})\text{.}$ Then $N(\epsilon)=(r+\epsilon)(p+D_{1}\epsilon+O(\epsilon^{2}))=rp+(p+rD_{1})\epsilon+O(\epsilon^{2})$ and $D(\epsilon)=\epsilon+r(p+D_{1}\epsilon+O(\epsilon^{2}))=rp+(1+rD_{1})\epsilon+O(\epsilon^{2})\text{.}$ Dividing by $rp$ and substituting $\epsilon=\theta rp/(1-p)$ yields the ratio $(1+\alpha\theta+R_{N})/(1+\beta\theta+R_{D})$ with $|R_{N}|,|R_{D}|=O(\theta^{2}p)\text{.}$ The identity $1+\alpha-\beta=0$ ensures the $\theta^{1}$-coefficient vanishes. The remainder analysis is identical to the proof of Theorem 4.4. ∎

Lemma 5.10 (Continuous-time $\alpha\to 0$).

For fixed $r>0\text{,}$ $|\alpha|=O(p|\ln p|)$ as $p=\hat{f}_{0}(r)\to 0\text{.}$

Proof.

Set $Y=e^{-rT_{0}}\in(0,1]\text{,}$ so $p=E[Y]$ and $rD_{1}=E[Y\ln Y]$ (since $Y\ln Y=-rT_{0}e^{-rT_{0}}$), giving $-rD_{1}=E[-Y\ln Y]\text{.}$ Since $\varphi(x)=-x\ln x$ is concave, Jensen’s inequality gives $E[-Y\ln Y]\leq-p\ln p\text{.}$ Hence $|p+rD_{1}|\leq p+E[-Y\ln Y]\leq p(1+|\ln p|)\text{,}$ so $|\alpha|=O(p|\ln p|)\text{.}$ ∎

In continuous time, the lattice error that contributes the $\Theta(p)$ floor in the discrete two-sided bound (Theorem 4.15) disappears: when $T_{0}$ has a continuous distribution, so does $T\text{,}$ and the CDF of $W$ has no jumps. The sharp rate therefore reduces to $|\alpha|$ alone. The following lemma provides the necessary moment convergence for the Bridge Theorem.

Lemma 5.11 (Escape to infinity).

Fix $r>0\text{.}$ Let $(T_{0,n})$ be a sequence of positive random variables with $p_{n}=E[e^{-rT_{0,n}}]\to 0\text{.}$ Then $T_{0,n}\to\infty$ in probability. Consequently, for each fixed $t\geq 0\text{,}$ $P(A_{n}\geq t)\to e^{-rt}\text{,}$ and for each fixed integer $k\geq 1\text{,}$ $E[A_{n}^{k}]\to k!/r^{k}\text{.}$

Proof.

For any fixed $x>0\text{,}$ $p_{n}\geq e^{-rx}P(T_{0,n}\leq x)\text{,}$ so $P(T_{0,n}\leq x)\leq e^{rx}p_{n}\to 0\text{.}$

For the failed-attempt tail: since $\tau\sim\operatorname{Exp}(r)$ is independent of $T_{0,n}\text{,}$

$$P(A_{n}\geq t)=\frac{P(\tau\geq t,\,\tau<T_{0,n})}{1-p_{n}}=\frac{E[(e^{-rt}-e^{-rT_{0,n}})\mathbf{1}_{\{T_{0,n}>t\}}]}{1-p_{n}}.$$

As $n\to\infty\text{:}$ $E[e^{-rT_{0,n}}\mathbf{1}_{\{T_{0,n}>t\}}]\leq p_{n}\to 0$ and $P(T_{0,n}>t)\to 1\text{,}$ so the numerator tends to $e^{-rt}$ and the denominator to $1\text{.}$

The uniform bound $P(A_{n}\geq t)\leq e^{-rt}/(1-p_{n})\leq 2e^{-rt}$ (for $p_{n}\leq 1/2$) provides a dominator for $t^{k-1}P(A_{n}\geq t)\text{,}$ so dominated convergence gives $E[A_{n}^{k}]=k\int_{0}^{\infty}t^{k-1}P(A_{n}\geq t)\,dt\to k!/r^{k}\text{.}$ ∎

Theorem 5.12 (Continuous-time sharp bound).

Under Poisson resetting at rate $r\text{,}$ for $T_{0}$ with continuous distribution, and $|\alpha|/p\to\infty$ as $p\to 0\text{:}$

$$d_{\mathrm{K}}\!\left(\frac{T}{E[T]},\,\operatorname{Exp}(1)\right)\in\bigl[1-o(1),\;2+o(1)\bigr]\cdot|\alpha|.$$

(45)

Proof.

Upper bound. The Bridge Theorem (Theorem 4.13) applies in continuous time: its three steps—Brown’s bound, scaling, and additive perturbation—require only that $N\sim\operatorname{Geom}_{0}(p)\text{,}$ the $A_{i}$ be i.i.d. positive with $E[A^{2}]<\infty\text{,}$ and $B$ be independent of $U\text{;}$ none assumes integrality. The tail bound $P(A\geq t)\leq e^{-rt}/(1-p)$ from the proof of Lemma 5.11 gives $E[A^{k}]<\infty$ for all $k\text{,}$ verifying Brown’s moment condition. By Lemma 5.11, $\rho_{j}=E[A^{j}]/E[A]^{j}\to j!$ as $p\to 0\text{,}$ so the Brown constant $C_{1}=O(1)\text{.}$ By (43), $2b/\mu\leq 2|\alpha|+2p/(1-p)\text{.}$ Hence $d_{\mathrm{K}}\leq C_{1}p+2|\alpha|+2p/(1-p)=(2+o(1))|\alpha|\text{,}$ using $p=o(|\alpha|)\text{.}$

Lower bound. Apply the Laplace expansion (Theorem 5.9) with $\theta=\theta_{n}=(|\alpha|/(2p))^{1/2}\to\infty\text{.}$ The validity condition $\epsilon=\theta rp/(1-p)\to 0$ holds since $\epsilon=O(r\sqrt{p|\alpha|})\to 0$ by Lemma 5.10. The error terms satisfy $p\theta^{2}/(1+\theta)\leq p\theta=\sqrt{p|\alpha|/2}=o(|\alpha|)$ and $|\alpha|^{2}\theta^{3}/(1+\theta)^{3}\leq|\alpha|^{2}=o(|\alpha|)\text{.}$ The leading term: $|\alpha|\theta^{2}/(1+\theta)^{2}=(1-o(1))|\alpha|$ since $\theta\to\infty\text{.}$ By (33), $d_{\mathrm{K}}\geq(1-o(1))|\alpha|\text{.}$ ∎

Remark 5.13 (Disappearance of the lattice error).

In the discrete two-sided bound (Theorem 4.15), the template is $p+|\alpha|$ because $W$ takes values in $(1/\mu)\mathbb{Z}_{>0}\text{,}$ creating an irreducible $\Omega(p)$ lattice error (Proposition 4.16). In continuous time, this error vanishes. When $|\alpha|/p\to\infty\text{,}$ the dominant error source is the successful-attempt perturbation encoded by $\alpha\text{;}$ when $\alpha\approx 0\text{,}$ a residual $O(p)$ error from higher-order terms persists.

5.6 Worked Example: Brownian First Passage

To illustrate the continuous-time theory concretely, consider a Brownian particle starting at $x_{0}>0$ with an absorbing barrier at the origin, subject to Poisson resetting at rate $r\text{.}$ This is the original model of [16]. The base first-passage time has Laplace transform $\hat{f}_{0}(\lambda)=e^{-x_{0}\sqrt{2\lambda}}\text{,}$ giving

$$p=e^{-x_{0}\sqrt{2r}},\qquad E[T]=\frac{e^{x_{0}\sqrt{2r}}-1}{r},\qquad D_{1}=-\frac{x_{0}}{\sqrt{2r}}\,e^{-x_{0}\sqrt{2r}}.$$

The perturbation parameter:

$$\alpha=\frac{(1-x_{0}\sqrt{r/2})\,e^{-x_{0}\sqrt{2r}}}{1-e^{-x_{0}\sqrt{2r}}}.$$

(46)

As $x_{0}\sqrt{r}\to\infty\text{:}$ $|\alpha|\sim x_{0}\sqrt{r/2}\cdot p\text{,}$ so $|\alpha|/p\to\infty$—the perturbation term dominates, and Theorem 5.12 gives $d_{\mathrm{K}}\in[1-o(1),2+o(1)]\cdot|\alpha|\text{.}$

At the critical point $x_{0}\sqrt{r/2}=1\text{,}$ the perturbation parameter $\alpha$ vanishes exactly: the leading $\alpha\theta^{2}/(1+\theta)^{2}$ term in (44) disappears, leaving only higher-order contributions. This does not imply distributional proximity to $\operatorname{Exp}(1)\text{:}$ at the critical point $p=e^{-2}$ is a fixed constant (not tending to zero), so the exponential limit theorem does not apply.

5.7 Discrete–Continuous Correspondence

Table 1 summarizes the parallel between the discrete and continuous theories. Every structural feature—affine form, coefficient matching, DRP, perturbation identity, Laplace expansion, sharp rate—carries over. The sole qualitative difference lies in the characterization’s scope and the sharp-rate template.

	Discrete time	Continuous time
Transform	PGF $G_{T}(w)=E[w^{T}]$	Laplace $\hat{f}_{T}(\lambda)=E[e^{-\lambda T}]$
Normalization point	$w=1$	$\lambda=0$
Shift	$u=ws$ (multiplicative)	$\sigma=\lambda+r$ (additive)
Catastrophe parameter	$q\in(0,1)$	$r>0$
Success probability	$p=g(s)$	$p=\hat{f}_{0}(r)$
$E[T]$	$(1-p)/(qp)$	$(1-p)/(rp)$
Coefficient match	$\alpha_{N}(1)=\alpha_{D}(1)=q$	$\alpha_{N}(0)=\alpha_{D}(0)=r$
DRP ($k\geq 2$)	$\Delta_{k}=-ks^{k-1}D_{k-1}$	$\Delta_{k}=kD_{k-1}$
Perturbation $\alpha$	$s(p-qD_{1})/(1-p)$	$(p+rD_{1})/(1-p)$
Fundamental identity	$1+\alpha-\beta=0$	$1+\alpha-\beta=0$
Laplace expansion	$\alpha\theta^{2}/(1+\theta)^{2}+\text{h.o.t.}$	$\alpha\theta^{2}/(1+\theta)^{2}+\text{h.o.t.}$
Bridge Theorem	$d_{\mathrm{K}}\leq C_{1}p+2b/\mu$	$d_{\mathrm{K}}\leq C_{1}p+2b/\mu$
Sharp-rate template	$p+\|\alpha\|$	$\|\alpha\|$ (no lattice error)
Characterizes	geometric tail	Poisson (full memoryless)
Gap	$h_{1}$ free	none

Table 1: Discrete–continuous correspondence. The two theories share every structural feature. The characterization sharpens in continuous time (the geometric-tail gap closes, Remark 5.8), and the sharp-rate template simplifies (the lattice error vanishes for continuous $T_{0}\text{,}$ Remark 5.13).

6 Application: Coupon Collector with Reset

The general theory of §§2–4 applies to any base process $T_{0}\text{.}$ We now instantiate it to the coupon collector problem with reset, obtaining closed-form moments, a sharp Kolmogorov bound that closes the logarithmic gap, and a discontinuous phase transition in the limit law. The pattern throughout is: compute CCP-specific quantities ($p\text{,}$ $D_{1}\text{,}$ $D_{2}\text{,}$ $\alpha$), then substitute into general formulas. The sole exception is the lower bound in the sharp Kolmogorov estimate (§6.4), which requires a product-structure argument specific to the CCP.

6.1 Model

A deck contains $n+m$ coupon types: types $1,\ldots,n$ are standard, and types $n+1,\ldots,n+m$ are reset. At each step, one type is drawn uniformly at random with replacement. Standard draws accumulate; any reset draw erases all collected coupons. The game ends when all $n$ standard types have been collected. This is the framework of §2.1 with $T_{0}$ the classical coupon collector completion time and

$$q=\frac{m}{n+m},\qquad s=\frac{n}{n+m}.$$

A closely related model was studied by [24], who derived expected values via Markov chain methods; our contribution is the distributional theory.

Notation. $C=\binom{n+m}{n}\text{;}$ $H=\sum_{j=m+1}^{n+m}1/j=H_{n+m}-H_{m}\text{;}$ $H^{(r)}=\sum_{j=m+1}^{n+m}1/j^{r}=H^{(r)}_{n+m}-H^{(r)}_{m}\text{.}$

6.2 Specialization of the PGF Data

The classical CCP has $T_{0}=X_{1}+\cdots+X_{n}$ with $X_{k}\sim\operatorname{Geom}_{1}((n{-}k{+}1)/n)$ independent, giving

$$g(z)=\prod_{k=1}^{n}\frac{(n{-}k{+}1)z/n}{1-(k{-}1)z/n}.$$

(47)

The general theory requires three inputs: $p=g(s)\text{,}$ $D_{1}=g^{\prime}(s)\text{,}$ and $D_{2}=g^{\prime\prime}(s)\text{.}$

Lemma 6.1 (Success probability).

$p=g(s)=1/C=1/\binom{n+m}{n}$.

Proof.

Algebraic. At $z=s=n/(n+m)\text{,}$ each factor of (47) evaluates to $(n{-}k{+}1)/(n{+}m{-}k{+}1)\text{.}$ The product telescopes: $\prod_{k=1}^{n}(n{-}k{+}1)/(n{+}m{-}k{+}1)=n!\,m!/(n{+}m)!=1/C\text{.}$

Combinatorial. An attempt succeeds if and only if all $n$ standard types appear before any reset type in the first-appearance permutation. By symmetry of i.i.d. uniform draws, this probability is $n!\,m!/(n+m)!=1/C\text{.}$ ∎

The derivatives are computed by logarithmic differentiation, exploiting the product structure $\log g=\sum_{k=1}^{n}\log G_{X_{k}}\text{.}$

Lemma 6.2 (PGF derivatives at $z=s$).

$$D_{1}=g^{\prime}(s)=\frac{(n+m)^{2}\,H}{n\,C},$$

(48)

$$D_{2}=g^{\prime\prime}(s)=\frac{(n+m)^{3}}{n^{2}\,C}\bigl((n+m)(H^{2}+H^{(2)})-2H\bigr).$$

(49)

More generally, the $r$-th logarithmic derivative of $g$ at $z=s$ is a polynomial in $H,H^{(2)},\ldots,H^{(r)}$ with rational coefficients in $(n,m)\text{.}$

Proof sketch.

Write $r_{k}(z)=G^{\prime}_{X_{k}}(z)/G_{X_{k}}(z)=1/(z\varphi_{k}(z))$ with $\varphi_{k}(z)=1-(k{-}1)z/n\text{.}$ At $z=s\text{:}$ $\varphi_{k}(s)=(n{+}m{-}k{+}1)/(n{+}m)\text{,}$ so $r_{k}(s)=(n{+}m)^{2}/(n(n{+}m{-}k{+}1))\text{.}$ Setting $j=n{+}m{-}k{+}1\text{:}$

$$(\log g)^{\prime}(s)=\sum_{k=1}^{n}r_{k}(s)=\frac{(n+m)^{2}}{n}\sum_{j=m+1}^{n+m}\frac{1}{j}=\frac{(n+m)^{2}\,H}{n}.$$

Then $D_{1}=g(s)\cdot(\log g)^{\prime}(s)\text{.}$ The second derivative follows from $g^{\prime\prime}=g[(\log g)^{\prime\prime}+((\log g)^{\prime})^{2}]$ and an analogous computation of $(\log g)^{\prime\prime}(s)\text{,}$ which introduces $H^{(2)}$ via the partial fractions of $r^{\prime}_{k}(s)\text{.}$ ∎

The pattern is transparent: the $r$-th logarithmic derivative introduces $H^{(r)}$ because each differentiation of $r_{k}(z)=1/(z\varphi_{k}(z))$ adds one power of $1/\varphi_{k}(s)=(n{+}m)/j\text{,}$ and summing over $k$ produces $\sum j^{-r}\text{.}$

6.3 Moments

Substituting the CCP data $(p,q,D_{1})$ into the general formulas of §2.2 gives closed-form first two moments:

$$E[T]=\frac{n+m}{m}(C-1),\qquad\operatorname{Var}(T)=\frac{n+m}{m^{2}}\bigl((C-1)((n{+}m)C+n)-2m(n{+}m)CH\bigr).$$

(50)

For $m=1\text{:}$ $E[T]=n(n+1)\text{.}$ For fixed $m$ as $n\to\infty\text{:}$ $E[T]\sim n^{m+1}/(m\cdot m!)\text{.}$ The variance involves both $p=1/C$ and $D_{1}$—but not $D_{2}$—exactly as the DRP guarantees. Higher moments follow analogously from the recursion (15). In particular, the third centered moment is the first level at which $D_{2}\text{,}$ and hence $H^{(2)}\text{,}$ enters; more generally, the $k$-th centered moment first involves the new harmonic statistic $H^{(k-1)}\text{.}$

The staircase of §2.3 takes a particularly illuminating form:

Moment of $T$	New input (DRP)	CCP realization	Origin
$E[T]$	$g(s)$	$C=\binom{n+m}{n}$	product telescope
$\operatorname{Var}(T)$	$+g^{\prime}(s)$	$+H=\sum 1/j$	1st log-derivative
$\mu_{3}(T)$	$+g^{\prime\prime}(s)$	$+H^{(2)}=\sum 1/j^{2}$	2nd log-derivative
$\mu_{k}(T)$	$+g^{(k-1)}(s)$	$+H^{(k-1)}=\sum 1/j^{k-1}$	$(k{-}1)$-th log-deriv.

Each row adds exactly one new finite harmonic sum: $H^{(k-1)}=\sum_{j=m+1}^{n+m}j^{-(k-1)}\text{.}$ The CCP thus provides the sharpest illustration of the harmonic complexity staircase—a feature absent in base processes with rational PGF structure (§7).

6.4 Phase Transition and Sharp Kolmogorov Bound

The CCP perturbation parameter, obtained by substituting $p=1/C$ and $qD_{1}=m(n+m)H/(nC)$ into Definition 4.1, is

$$\alpha=\frac{s-mH}{C-1},\qquad|\alpha|\sim\frac{m!\,m\ln n}{n^{m}}\quad\text{as }n\to\infty.$$

(51)

Since $p=1/C\sim m!/n^{m}$ and $|\alpha|\sim m!\,m\ln n/n^{m}\text{,}$ the ratio $p/|\alpha|\sim 1/(m\ln n)\to 0\text{:}$ the perturbation term dominates.

The Gumbel-to-Exponential phase transition. For the classical CCP ($m=0$), the limit law is Gumbel: $(T_{0}-n\ln n)/n\xrightarrow{d}\text{Gumbel}$ ([13, 3]). For $m\geq 1\text{,}$ the general theory (applied via the Bridge Theorem, Theorem 4.13) gives $T/E[T]\xrightarrow{d}\operatorname{Exp}(1)\text{.}$ The transition at $m=0\to m\geq 1$ is discontinuous:

	Classical ($m=0$)	Reset ($m\geq 1$)
Scale	$\Theta(n\log n)$	$\sim n^{m+1}/(m\cdot m!)$
Fluctuations	$\Theta(n)\ll E[T]$	$\Theta(E[T])$
Coefficient of variation	$\to 0$	$\to 1$
Limit law	Gumbel	Exponential
Mechanism	last coupon (extreme value)	lucky streak (geometric)

In the classical CCP, the bottleneck is the last missing coupon—an extreme value problem yielding the Gumbel law. A single reset coupon destroys this mechanism: completion now requires a lucky streak (an attempt succeeding without catastrophe), and the number of attempts is geometric. The geometric waiting time’s memorylessness propagates to $T/E[T]\to\operatorname{Exp}(1)\text{,}$ making the completion time fundamentally unpredictable.

Remark 6.3 (CCP convergence via the Bridge Theorem).

The CCP has $q=m/(n+m)\to 0$ as $n\to\infty\text{,}$ so Corollary 4.6 (which assumes fixed $q$) does not apply directly. Instead, CCP convergence follows from the Bridge Theorem (Theorem 4.13): the conditions $p\to 0$ and uniform boundedness of moment ratios of $A$ are verified below in the proof of Theorem 6.4.

Theorem 6.4 (CCP sharp bound).

For the coupon collector with $n$ standard and $m\geq 1$ reset coupons, $m$ fixed, $n\to\infty\text{:}$

$$d_{\mathrm{K}}\!\left(\frac{T}{E[T]},\,\operatorname{Exp}(1)\right)\in\bigl[1-o(1),\;2+o(1)\bigr]\cdot|\alpha|.$$

(52)

Proof of the upper bound.

To apply the Bridge Theorem (Theorem 4.13), it suffices to show that the moment ratios $\rho_{j}:=\frac{E[A^{j}]}{E[A]^{j}},j=2,3\text{,}$ are bounded uniformly in $n$ for each fixed $m\geq 1\text{.}$ We first bound the numerator. By Lemma 4.8, $P(A\geq t)\leq s^{t-1},t\geq 1\text{.}$ Hence, by the tail-sum formula,

$$E[A^{k}]=\sum_{t\geq 1}\bigl(t^{k}-(t-1)^{k}\bigr)P(A\geq t)\leq k\sum_{t\geq 1}t^{k-1}s^{t-1}.$$

Since $s=1-q\leq e^{-q}\text{,}$

$$\sum_{t\geq 1}t^{k-1}s^{t-1}\leq 2\sum_{t\geq 1}t^{k-1}e^{-qt}\leq C_{k}q^{-k},$$

for a constant $C_{k}$ depending only on $k\text{.}$ Therefore, $E[A^{k}]\leq C_{k}q^{-k}\text{.}$

For the lower bound on $E[A]\text{,}$ use the explicit failed-attempt law $P(A=j)=\frac{q\,s^{j-1}P(T_{0}>j)}{1-p},j\geq 1\text{.}$ In the CCP, $T_{0}\geq n$ almost surely, so $P(T_{0}>j)=1$ for $1\leq j\leq n-1\text{.}$ Hence

$$E[A]=\frac{1}{1-p}\sum_{j\geq 1}jqs^{j-1}P(T_{0}>j)\geq\sum_{j=1}^{n-1}jqs^{j-1}=\frac{1-ns^{\,n-1}+(n-1)s^{n}}{q}.$$

Since $q=m/(n+m)$ and $s=n/(n+m)\text{,}$ for each fixed $m\geq 1\text{,}$ $1-ns^{\,n-1}+(n-1)s^{n}\to 1-(m+1)e^{-m}>0\text{,}$ so there exists $c_{m}>0$ such that $E[A]\geq\frac{c_{m}}{q}$ for all sufficiently large $n\text{.}$

Combining the two estimates gives

$$\frac{E[A^{k}]}{E[A]^{k}}\leq\frac{C_{k}q^{-k}}{(c_{m}q^{-1})^{k}}=\frac{C_{k}}{c_{m}^{k}}=O_{m}(1).$$

Hence $\rho_{2},\rho_{3}=O_{m}(1)\text{,}$ and Brown’s constant in Theorem 4.13 satisfies $C_{1}=O_{m}(1)\text{.}$ Thus, from (51) and Proposition 4.2 (b): $b/\mu=\frac{mH}{C-1}=|\alpha|+\frac{s}{C-1}\text{,}$ and $\frac{s}{C-1}=O(p)\text{.}$ Finally, $d_{\mathrm{K}}\leq O_{m}(p)+2|\alpha|+O(p)=(2+o(1))|\alpha|\text{.}$ ∎

Proof of the lower bound.

This is the one point where CCP-specific structure goes beyond substitution into general formulas. Choose $\theta=\theta_{n}=(\ln n)^{1/2}$ and set $w=e^{-\theta/\mu}\text{,}$ $u=ws\text{,}$ $\epsilon=\theta/\mu\text{.}$ By (33), $d_{\mathrm{K}}\geq|G_{T}(w)-1/(1+\theta)|\text{.}$

Step 1: PGF reduction. From (9), setting $c=g(u)/p\text{:}$

$$G_{T}(w)=\frac{c+O_{m}(\theta p)}{c+\theta+O_{m}(\theta p)}.$$

Hence $G_{T}(w)-1/(1+\theta)=(c-1)\theta/((c+\theta)(1+\theta))+O_{m}(\theta p)\text{.}$

Step 2: Product-structure evaluation of $\log c\text{.}$ The product structure $g=\prod_{k=1}^{n}G_{X_{k}}$ gives

$$\log c=\sum_{k=1}^{n}\log(G_{X_{k}}(u)/G_{X_{k}}(s)).$$

Each ratio $G_{X_{k}}(u)/G_{X_{k}}(s)$ expands as

$$\frac{G_{X_{k}}(u)}{G_{X_{k}}(s)}=\frac{w(1-a_{k})}{1-a_{k}w}=\frac{1-\delta}{1+x_{k}},$$

where $x_{k}=a_{k}\delta/(1-a_{k})$ with $a_{k}=(k{-}1)/(n{+}m)$ and $\delta=1-w\text{.}$ Since $\sup_{k}x_{k}=O_{m}(\sqrt{\ln n}/n^{m})\to 0\text{,}$ the expansion $\log(1+x_{k})=x_{k}+O(x_{k}^{2})$ is uniformly valid. Summing and using the identity $\sum_{k=1}^{n}a_{k}/(1-a_{k})=(n+m)H-n\text{:}$

$$\log c=-\delta(n+m)H+O(\delta^{2}S_{2}),$$

where $S_{2}=\sum_{k=2}^{n}a_{k}^{2}/(1-a_{k})^{2}=O_{m}(n^{2})\text{.}$ Converting $\delta(n+m)H$ to $\theta|\alpha|(1+o(1))$ and verifying $\delta^{2}S_{2}=o(\theta|\alpha|)\text{:}$

$$\log c=-\theta|\alpha|(1+o(1)).$$

Since $\theta|\alpha|\to 0\text{,}$ this gives $c=1-\theta|\alpha|(1+o(1))\text{.}$

Step 3: Assembly. $(c-1)\theta/((c+\theta)(1+\theta))=-(1+o(1))\theta^{2}|\alpha|/(1+\theta)^{2}\text{,}$ and the correction $O_{m}(\theta p)=o(|\alpha|)\text{.}$ Hence $d_{\mathrm{K}}\geq(1+o(1))\theta^{2}|\alpha|/(1+\theta)^{2}=(1-o(1))|\alpha|\text{,}$ using $\theta^{2}/(1+\theta)^{2}=1-O(1/\sqrt{\ln n})\text{.}$ ∎

7 Application: Multi-Phase Task with Catastrophe

The CCP of §6 features a base PGF that is a product of $n$ distinct Möbius factors, producing a harmonic complexity staircase whose successive levels introduce new finite harmonic sums $H^{(r)}\text{.}$ We now present a structurally contrasting application: a base process whose PGF is a power of a single Möbius factor, yielding a purely algebraic staircase. The DRP applies identically in both cases; the difference lies entirely in what it saves.

7.1 Model and the Algebraic Staircase

A task consists of $r\geq 2$ sequential phases, each requiring $\operatorname{Geom}_{1}(\pi)$ steps independently. The base completion time is $T_{0}=X_{1}+\cdots+X_{r}\text{,}$ $X_{i}\stackrel{{\scriptstyle\text{i.i.d.}}}{{\sim}}\operatorname{Geom}_{1}(\pi)\text{,}$ with PGF

$$g(z)=\left[\frac{\pi z}{1-(1-\pi)z}\right]^{\!r}.$$

(53)

Catastrophe at each step (probability $q\text{,}$ independent) destroys all accumulated progress, resetting the phase counter to zero. Define $\sigma=1-(1-\pi)s=\pi+q(1-\pi)$ and $\rho=\pi s/\sigma\text{.}$ Then

$$p=g(s)=\rho^{r},\qquad E[T]=\frac{1-\rho^{r}}{q\rho^{r}},\qquad D_{1}=g^{\prime}(s)=\frac{r\rho^{r}}{s\sigma},$$

(54)

and the variance, obtained from (17), depends on $p$ and $D_{1}$ but not on $D_{2}$—exactly as the DRP guarantees.

The key structural feature is that $\log g=r\log f$ with $f(z)=\pi z/(1-(1-\pi)z)\text{,}$ so every logarithmic derivative

$$(\log g)^{(k)}(s)=r(k-1)!\left[\frac{(-1)^{k-1}}{s^{k}}+\frac{(1-\pi)^{k}}{\sigma^{k}}\right]$$

(55)

is a rational function of $(s,\pi,r)$—no new generalized harmonic statistic appears at any level. Displayed alongside the CCP realization:

Moment of $T$	New input (DRP)	CCP realization	Multi-phase realization
$E[T]$	$g(s)$	$C=\binom{n+m}{n}$	$\rho^{r}$
$\operatorname{Var}(T)$	$+g^{\prime}(s)$	$+H=\sum 1/j$	$+r/(s\sigma)$
$\mu_{3}(T)$	$+g^{\prime\prime}(s)$	$+H^{(2)}=\sum 1/j^{2}$	$+\,r\cdot[\text{rational in }s,\sigma,\pi]$
$\mu_{k}(T)$	$+g^{(k-1)}(s)$	$+H^{(k-1)}$	$+r\cdot[\text{rational}]$

The contrast traces to the PGF’s analytic structure: the CCP’s $n$ distinct poles produce new power sums $\sum j^{-k}$ at each level, while the multi-phase model’s single pole with multiplicity $r$ cycles through powers of the same two quantities $1/s$ and $(1-\pi)/\sigma\text{.}$

Remark 7.1 (What the DRP saves).

In the CCP, the DRP eliminates one new harmonic statistic at each level; here, it eliminates a routine rational computation. The mechanism—$\alpha_{N}(1)=\alpha_{D}(1)$ forcing cancellation of $g^{(k)}$ in $\Delta_{k}$—is identical. The DRP is a property of the catastrophe mechanism, not of the base process.

7.2 Phase Transition and Convergence Rate

Specializing Definition 4.1:

$$\alpha=\frac{\rho^{r}(s\sigma-qr)}{\sigma(1-\rho^{r})},\qquad|\alpha|\sim\frac{qr}{\sigma}\,\rho^{r},\quad p=\rho^{r}\qquad(r\to\infty).$$

(56)

Since $q$ is fixed, Corollary 4.6 applies directly: $T/E[T]\xrightarrow{d}\operatorname{Exp}(1)$ as $r\to\infty\text{.}$ Without catastrophe, $T_{0}\sim\mathrm{NegBin}(r,\pi)$ satisfies a CLT: $(T_{0}-r/\pi)/\sqrt{r(1-\pi)/\pi^{2}}\xrightarrow{d}N(0,1)\text{.}$ A single catastrophe mechanism thus induces a Gaussian-to-exponential phase transition—distinct from the CCP’s Gumbel-to-Exponential transition, but driven by the same algebraic cause: the coefficient equality $\alpha_{N}(1)=\alpha_{D}(1)$ forces the $\theta^{1}$-coefficient in the Laplace expansion to vanish exactly (Proposition 4.2 (a)).

The sharp two-sided bound (Theorem 4.15) gives $d_{\mathrm{K}}(T/E[T],\,\operatorname{Exp}(1))\asymp p+|\alpha|\asymp r\rho^{r}$ as $r\to\infty\text{:}$ convergence is exponentially fast in $r$ with a linear prefactor, contrasting with the CCP’s polynomial rate $\Theta(m!\,m\ln n/n^{m})\text{.}$

	CCP ($n\to\infty\text{,}$ $m$ fixed)	Multi-phase ($r\to\infty\text{,}$ $\pi,q$ fixed)
Without catastrophe	Gumbel	Gaussian
With catastrophe	Exponential	Exponential
Rate template	$p+\|\alpha\|\asymp m!\,m\ln n/n^{m}$	$p+\|\alpha\|\asymp r\rho^{r}$
Rate type	polynomial in $n$	exponential in $r$

Remark 7.2 (CV criterion and forced catastrophe).

The coefficient of variation of $T_{0}\sim\mathrm{NegBin}(r,\pi)$ satisfies $\operatorname{CV}^{2}=(1-\pi)/r<1$ for $r\geq 2\text{.}$ By the Pal–Reuveni criterion ([37]), adding voluntary restart to this base process is always detrimental. The present application demonstrates that the DRP framework remains fully operative—and produces non-trivial distributional information—even when catastrophe is imposed against the process’s interest. The algebraic structure does not consult the CV criterion.

Stochastic Resetting and the PGF Formula.

The renewal framework for stochastic resetting—developed in [16, 37, 33, 8] and surveyed in [15]—provides the attempt decomposition underlying Theorem 2.2. The PGF formula was derived independently by [19] and [4]. These works primarily use the resulting transform for computation or restart optimization, without identifying the affine structure or establishing convergence rates. Our framework places the CV-unity criterion of [37] in a broader structural context: the dependence on first and second moments appears as the $k=2$ shadow of the derivative reduction principle. Non-memoryless protocols [14, 9, 26] fall outside this affine transform class; our characterization theorems (Theorems 3.6, 5.7) identify the precise algebraic boundary.

Linear-Fractional Structure across Domains.

Geometric compounding inherently produces linear-fractional PGFs, exploited in branching process theory by [1, 38] (see also [23, 28]), in catastrophe queueing by [5, 20] (see [27, 12, 2] for extensions), and observed empirically in algorithmic restart, where [22] noted that restart converts heavy-tailed SAT runtimes into geometric-like distributions (see [29, 39, 40]). All these works treat linear-fractional structure as a convenient tractable family or an empirical regularity. Our characterization theorem reveals the complementary perspective: this structure is diagnostic—it identifies the geometric-tail class among all age-based catastrophe mechanisms—and the Möbius-rigidity argument applies it as a characterization tool rather than a model assumption.

Exponential Approximation of Geometric Sums.

[36] proved that normalized geometric sums converge to $\operatorname{Exp}(1)\text{.}$ [6] obtained optimal-order Kolmogorov bounds $d_{K}\leq C_{1}p\text{;}$ [35] introduced Stein’s method for this setting, achieving $O(p)$ Wasserstein and $O(\sqrt{p})$ Kolmogorov bounds (see [7, 34] for refinements, and [21, 25] for comprehensive treatments). Our Bridge Theorem matches Brown’s $O(p)$ rate for the geometric-sum component; the sharp two-sided bound $d_{\mathrm{K}}\asymp p+|\alpha|$ (Theorem 4.15) closes a logarithmic gap from the smoothing inequality by exploiting the specific two-parameter structure produced by the affine PGF analysis. Stein’s method does not directly yield this two-parameter template: existing Stein bounds apply to the pure geometric sum $U$ (achieving $O(\sqrt{p})$ in Kolmogorov distance [35], weaker than Brown’s $O(p)$), while the $|\alpha|$-term arises from the successful-attempt perturbation $B$—an additive component outside the geometric-sum framework that the Bridge Theorem handles via direct probabilistic arguments (Lemmas 4.10–4.11). The matching lower bound, obtained by evaluating the Laplace expansion at a fixed $\theta\text{,}$ is likewise orthogonal to the Stein approach.

[10] develop Stein’s method for the Gumbel approximation and obtain a $(\log n/n)$ rate for the classical coupon collector in a smooth Wasserstein-$1$ metric.

Coupon Collector Variants and Phase Transitions.

The classical Gumbel limit was established by [13] (see [3, 17, 18, 11, 31] for refinements and extensions). The closest prior phase-transition result is [30], who identified a Gumbel-to-Gaussian transition in the CCP with bonuses. [24] study a closely related reset-button CCP, deriving the waiting-time distribution in the unequal-probability setting and expected-value formulas with asymptotics in the equal-probability case. Our contribution is the structural theory (affine PGF framework, DRP, sharp exponential approximation), and the identification of the Gumbel-to-Exponential phase transition—which is qualitatively different from the Gumbel-to-Gaussian transition of [30] (it reflects memoryless catastrophe, not a CLT effect).

Our phase transitions differ from those of [32], who study transitions in the optimal restart rate. Ours are in the universality class of the limit distribution—Gumbel $\to$ Exponential, Gaussian $\to$ Exponential—driven by the coefficient equality $\alpha_{N}(1)=\alpha_{D}(1)$ killing the first-order Laplace error.

9 Further Directions

Theorem 3.6 characterizes affine PGF structure as equivalent to geometric-tail catastrophe, with full memorylessness singled out by $\lambda=1$ (Corollary 3.8). The gap—exactly one degree of freedom, the first-step hazard $h_{1}$—closes in continuous time (Theorem 5.7). In the discrete setting, supplementary algebraic conditions (e.g., requiring the successful-attempt PGF $G_{B}$ to also depend on $g$ at a single shifted point) may close this gap; formalizing this would strengthen the characterization.

A related stability question: for near-memoryless mechanisms with $|h_{t}-q|\leq\varepsilon$ for $t\geq 2\text{,}$ does the DRP cancellation error $|\Delta_{k}-(-ks^{k-1}D_{k-1})|$ admit an $O(\varepsilon)$ bound? More broadly, quantifying DRP degradation under Gamma-distributed inter-reset times (shape parameter near $1$) would connect the algebraic theory to the restart optimization literature. At the application level, the CCP sharp bound (Theorem 6.4) gives leading constants in $[1,2]\text{;}$ numerical evidence suggests the exact constant is closer to $1\text{,}$ likely requiring a finer analysis of the additive perturbation step.

Partial Reset.

When catastrophe clears each coupon independently with probability $r\in(0,1)\text{,}$ the i.i.d. attempt structure of (6) breaks down: partial reset creates state-dependent catastrophe that cannot be described by a hazard sequence $(h_{t})_{t\geq 1}\text{.}$ Theorem 3.6 predicts that affine PGF structure is lost, but a quantitative understanding—what replaces the DRP, what the correct rate template is—remains open. Partial reset interpolates between the memoryless regime ($r=1\text{,}$ exponential limit) and the classical CCP ($r=0\text{,}$ Gumbel limit); understanding this interpolation would reveal whether the Gumbel-to-Exponential phase transition is sharp or admits a crossover regime. A probabilistic proof of the DRP—perhaps via coupling—might extend to this setting where the PGF algebra breaks down.

Non-Uniform Coupon Probabilities.

For non-uniform CCP with probabilities $p_{1},\ldots,p_{n}\text{,}$ the PGF no longer telescopes, but the general theory (Theorem 2.2, DRP, exponential approximation) applies unchanged to the non-uniform base process. The challenge is evaluating the CCP-specific quantities ($D_{1}\text{,}$ $\alpha$) in terms of the non-uniform parameters and understanding how the phase transition depends on the coupon probability vector.

Universality Classes of Completion-Time Limits.

The present paper establishes that memoryless catastrophe (i.i.d. attempts, affine PGF structure) produces exponential limits. A structurally contrasting regime may arise in non-renewal recycling mechanisms with weakly dependent increments, where CLT-type arguments yield log-normal completion-time limits instead. This contrast raises a natural question: what properties of the catastrophe or recycling mechanism determine the universality class of the completion-time limit law?

References

[1] A. Agresti (1974) Bounds on the extinction time distribution of a branching process. Adv. in Appl. Probab. 6, pp. 322–335. Cited by: §1.1, §8.
[2] C. Banderier and M. Wallner (2017) Lattice paths with catastrophes. Discrete Math. Theor. Comput. Sci. 19 (1). Cited by: §8.
[3] L. E. Baum and P. Billingsley (1965) Asymptotic distributions for the coupon collector’s problem. Ann. Math. Statist. 36, pp. 1835–1839. Cited by: §6.4, §8.
[4] O. L. Bonomo and A. Pal (2021) First passage under restart for discrete space and time: Application to one-dimensional confined lattice random walks. Phys. Rev. E 103, pp. 052129. Cited by: §1.1, §1.2, §8.
[5] P. J. Brockwell, J. Gani, and S. I. Resnick (1982) Birth, immigration and catastrophe processes. Adv. in Appl. Probab. 14, pp. 709–731. Cited by: 2nd item, §8.
[6] M. Brown (1990) Error bounds for exponential approximations of geometric convolutions. Ann. Probab. 18 (3), pp. 1388–1402. Cited by: §1.3, §4.3, Theorem 4.12, §8.
[7] M. Brown (2015) Sharp bounds for exponential approximations under a hazard rate upper bound. J. Appl. Probab. 52, pp. 841–850. Cited by: §8.
[8] A. Chechkin and I. M. Sokolov (2018) Random search with resetting: A unified renewal approach. Phys. Rev. Lett. 121, pp. 050601. Cited by: §1.1, §5.2, §8.
[9] H. Chen and Y. Ye (2022) Random walks on complex networks under time-dependent stochastic resetting. Phys. Rev. E 106, pp. 044139. Cited by: §8.
[10] B. Costacèque and L. Decreusefond (2026) Convergence rate for the coupon collector’s problem with Stein’s method. Stochastic Process. Appl. 193, pp. 104835. Cited by: §8.
[11] A. V. Doumas and V. G. Papanicolaou (2012) The coupon collector’s problem revisited: Asymptotics of the variance. Adv. in Appl. Probab. 44, pp. 166–195. Cited by: §8.
[12] A. Economou and D. Fakinos (2003) A continuous-time Markov chain under the influence of a regulating point process and applications in stochastic models with catastrophes. European J. Oper. Res. 149, pp. 625–640. Cited by: §8.
[13] P. Erdős and A. Rényi (1961) On a classical problem of probability theory. Magyar Tud. Akad. Mat. Kutató Int. Közl. 6, pp. 215–220. Cited by: §6.4, §8.
[14] S. Eule and J. J. Metzger (2016) Non-equilibrium steady states of stochastic processes with intermittent resetting. New J. Phys. 18, pp. 033006. Cited by: §8.
[15] M. R. Evans, S. N. Majumdar, and G. Schehr (2020) Stochastic resetting and applications. J. Phys. A 53, pp. 193001. Cited by: 3rd item, §5.2, §8.
[16] M. R. Evans and S. N. Majumdar (2011) Diffusion with stochastic resetting. Phys. Rev. Lett. 106, pp. 160601. Cited by: 3rd item, §1.3, §5.6, §8.
[17] P. Flajolet, D. Gardy, and L. Thimonier (1992) Birthday paradox, coupon collectors, caching algorithms and self-organizing search. Discrete Appl. Math. 39, pp. 207–229. Cited by: §8.
[18] P. Flajolet and R. Sedgewick (2009) Analytic combinatorics. Cambridge University Press. Cited by: §8.
[19] J. M. Flynn and S. S. Pilyugin (2021) First passage with restart in discrete time: with applications to biased random walks on the half-line. Note: arXiv:2108.11508 Cited by: §1.1, §1.2, §8.
[20] E. Gelenbe (1991) Product-form queueing networks with negative and positive customers. J. Appl. Probab. 28, pp. 656–663. Cited by: 2nd item, §8.
[21] B. V. Gnedenko and V. Y. Korolev (1996) Random summation: limit theorems and applications. CRC Press. Cited by: §4.2, §4, §8.
[22] C. P. Gomes, B. Selman, N. Crato, and H. Kautz (2000) Heavy-tailed phenomena in satisfiability and constraint satisfaction problems. J. Automat. Reason. 24, pp. 67–100. Cited by: §8.
[23] N. Grosjean and T. Huillet (2017) Additional aspects of the generalized linear-fractional branching process. Ann. Inst. Statist. Math. 69, pp. 1075–1097. Cited by: §8.
[24] J. Jocković and B. Todić (2024) Coupon collector problem with reset button. Mathematics 12 (2), pp. 239. Cited by: §6.1, §8.
[25] V. V. Kalashnikov (1997) Geometric sums: bounds for rare events with applications. Springer. Cited by: §4.3, §4, §8.
[26] T. D. Keidar, O. Blumer, B. Hirshberg, and S. Reuveni (2025) Adaptive resetting for informed search strategies and the design of non-equilibrium steady-states. Nature Commun. 16, pp. 7259. Cited by: §8.
[27] B. Krishna Kumar and D. Arivudainambi (2000) Transient solution of an M/M/1 queue with catastrophes. Comput. Math. Appl. 40, pp. 1233–1240. Cited by: §8.
[28] A. Lindo and S. Sagitov (2018) General linear-fractional branching processes with discrete time. Stochastics: International Journal of Probability and Stochastic Processes 90, pp. 364–378. Cited by: §8.
[29] M. Luby, A. Sinclair, and D. Zuckerman (1993) Optimal speedup of Las Vegas algorithms. Inform. Process. Lett. 47, pp. 173–180. Cited by: 1st item, §8.
[30] T. Nakata and I. Kubo (2006) A coupon collector’s problem with bonuses. In DMTCS Proceedings, Vol. AG, pp. 215–224. Cited by: §8.
[31] P. Neal (2008) The generalised coupon collector problem. J. Appl. Probab. 45, pp. 621–629. Cited by: §8.
[32] A. Pal and V. V. Prasad (2019) Landau-like expansion for phase transitions in stochastic resetting. Phys. Rev. Research 1, pp. 032001. Cited by: §8.
[33] A. Pal and S. Reuveni (2017) First passage under restart. Phys. Rev. Lett. 118, pp. 030603. Cited by: §1.1, §8.
[34] E. A. Peköz, A. Röllin, and N. Ross (2013) Total variation error bounds for geometric approximation. Bernoulli 19, pp. 610–632. Cited by: §8.
[35] E. A. Peköz and A. Röllin (2011) New rates for exponential approximation and the theorems of Rényi and Yaglom. Ann. Probab. 39 (2), pp. 587–608. Cited by: §8.
[36] A. Rényi (1956) A Poisson-folyamat egy jellemzése. MTA Mat. Kutató Int. Közl. 1, pp. 519–527. Cited by: §1.3, §4.3, §4, §8.
[37] S. Reuveni (2016) Optimal stochastic restart renders fluctuations in first passage times universal. Phys. Rev. Lett. 116, pp. 170601. Cited by: §1.1, Remark 7.2, §8.
[38] S. Sagitov (2013) Linear-fractional branching processes with countably many types. Stochastic Process. Appl. 123, pp. 2940–2956. Cited by: §1.1, §8.
[39] A. P. A. van Moorsel and K. Wolter (2004) Analysis and algorithms for restart. In QEST 2004, pp. 195–204. Cited by: §8.
[40] K. Wolter (2010) Stochastic models for fault tolerance: restart, rejuvenation and checkpointing. Springer. Cited by: §8.

Retained-Set Descent for Diagonal Ramsey Numbers

Sun, 13 Sep 2026 00:00:00 +0000

1 Introduction

Let $R(k,\ell)$ be the least integer $N$ such that every red–blue coloring of $K_{N}$ contains a red $K_{k}$ or a blue $K_{\ell}\text{.}$ Ramsey’s theorem [Ram] guarantees that these numbers are finite. The Erdős–Szekeres recurrence [ES] gives $R(k,\ell)\leq\binom{k+\ell-2}{k-1}\text{,}$ and hence $R(k,k)\leq 4^{k}\text{.}$ In the other direction, Erdős’s random-coloring argument [Er] gives a lower bound of order $k2^{k/2}\text{;}$ Spencer [Spe] improved its leading constant using the Lovász local lemma.

Thomason [Tho] improved the classical upper bound by a polynomial factor using quasirandomness: a coloring close to the inductive bound must have degrees and small-subgraph counts close to those of a random coloring. Conlon [Con] developed this approach to obtain $R(k,k)\leq 4^{k}\exp(-c(\log k)^{2}/\log\log k)\text{.}$ Sah [Sah] strengthened the estimate to $R(k,k)\leq 4^{k}\exp(-c(\log k)^{2})\text{.}$ Each bound holds for a suitable absolute constant $c>0$ and all sufficiently large $k\text{.}$ These results gave successively larger subexponential improvements while retaining exponential base $4\text{.}$

Campos, Griffiths, Morris, and Sahasrabudhe [CGMS] obtained the first improvement in the exponential base, proving $R(k,k)\leq(4-\varepsilon)^{k}$ for an absolute $\varepsilon>0$ and all sufficiently large $k\text{.}$ Their book algorithm works with two vertex sets and controls the red density between them. Gupta, Ndiaye, Norin, and Wei [GNNW] replaced this algorithm by a candidate induction, obtained stronger off-diagonal estimates, and optimized the resulting parameters to give $R(k,k)\leq 3.8^{k+o(k)}\text{.}$ Their candidate induction and reuse of improved Ramsey bounds in this optimization are the starting points of our work.

We improve the diagonal bound to $3.69507^{k}$ for all sufficiently large $k\text{.}$ Our descent applies density bounds inside a vertex set $X$ while keeping a second set $Y$ fixed for a final inequality in $|X|^{w}|Y|\text{,}$ with $w>0\text{.}$ Each resulting bound can be used inside $X$ in a later descent. We also ask how small an exponent finitely many such repetitions can give from a fixed initial bound.

We call this initial bound the source. It has the form $\log R(a,b)\leq af(b/a)+o(a)$ for integers $1\leq b\leq a\text{,}$ where $f$ is a fixed concave function on $[0,1]$ and the error is uniform in $b\text{.}$ Section 2 specifies $f\text{,}$ and Appendix B proves the bound.

A conditional bound at density $p\in(0,1)$ and ratio $s\in(0,1]$ applies to colorings with at least a fraction $p$ of their edges red. An exponent $z$ means that, for some fixed $C\geq 1$ and all sufficiently large $k\text{,}$ order at least $C\mathrm{e}^{zk}$ guarantees a red $K_{k}$ or a blue $K_{\lfloor sk\rfloor}\text{.}$ We use families of such bounds on closed intervals of $s\text{,}$ with continuous exponents and common constants and thresholds.

Starting from the conditional bounds supplied by the source, we apply the descent rule using bounds proved at earlier steps. We may also restrict ratio intervals, increase a density threshold or an exponent, and combine bounds at the same density on a finite closed cover of a ratio interval. A finite proof uses only finitely many of these operations; Definition 4.1 gives the precise rules.

Let $U_{\rm fin}(p,s)$ be the infimum of the exponents obtained at density $p$ and ratio $s$ by such finite proofs. In the diagonal case $(p,s)=(1/2,1)\text{,}$ we construct a finite proof with rational exponent $z_{+}$ and a rational lower bound $z_{-}$ for all finite proofs. Their exact values are given in Section 6.

Theorem 1.

For this source and these finite proof rules,

$$1.305<z_{-}\leq U_{\rm fin}(1/2,1)\leq z_{+}<1.307.$$

(1)

The exponent $z_{+}$ has a finite proof. For all sufficiently large integers $k\text{,}$

$$R(k,k)\leq\left\lceil 150003\exp(z_{+}k)\right\rceil,$$

(2)

and hence $R(k,k)\leq 3.69507^{k}\text{.}$

For the Ramsey bound, choose the denser color as red and apply the conditional bound at $(p,s)=(1/2,1)\text{.}$

1.1 Proof Outline

We use disjoint sets $X,Y\text{,}$ keeping $Y$ fixed and requiring each vertex of $X$ to have at least a fixed positive fraction of $Y$ as red neighbors. The red target $k$ and the blue target $\ell$ in $Y$ stay fixed while the blue target $t$ in $X$ decreases. When $X$ is large enough for a conditional bound at density $q\text{,}$ we either finish at red density at least $q$ or descend to a blue neighborhood of size greater than $(1-q)(|X|-1)\text{.}$ Restricting $X$ to this neighborhood preserves its red degrees into $Y$ and reduces $t$ by one: a blue $K_{t-1}$ there extends through the chosen vertex.

Write the required size of $X$ at $s=t/k$ as $\exp(kg(s))\text{,}$ with $g$ piecewise affine. On a cell of slope $d\text{,}$ the threshold falls by a factor $\mathrm{e}^{-d}$ per step, so we take $d>-\log(1-q)$ to allow for the neighborhood loss. The profile $g$ must also exceed the exponents of the bounds used inside $X\text{.}$ With slopes and calls fixed, optimizing its vertical shift under these conditions and the initial and terminal inequalities gives the cost in Equation 12.

Reusing these bounds and iterating from the source gives a decreasing sequence of exponents whose limit is a greatest fixed point (Theorem 3). Strict inequalities persist on neighborhoods of ratios; compactness selects finitely many neighborhoods and hence a common finite depth for a whole descent. Hence every exponent strictly above the limit has a finite proof, uniformly near its target ratio.

For the lower bound, a concave polygon $F$ assigns exponent $F(s)$ at densities up to $1-\mathrm{e}^{-d}$ on a segment of slope $d\text{.}$ Where a descent using bounds that respect this estimate has $g<F\text{,}$ its call density must exceed $1-\mathrm{e}^{-d}\text{,}$ forcing the slope of $g$ above $d\text{.}$ Thus $g-F$ decreases as the descent moves to smaller ratios. We choose $F$ so that the terminal weighted inequality cannot hold below it. The supporting-line formula reduces this requirement to one-variable inequalities at its vertices (Theorem 4); the certificates in Section 6 complete the proof of Theorem 1.

The Lean 4 formalization and the Python code for exact certificate checks are available in the accompanying repository.¹¹ 1

 <a href="https://github.com/sichen-wang/diagonal-ramsey-numbers" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/sichen-wang/diagonal-ramsey-numbers</a></span></span></span></p>

2 Source Bounds

We now describe the source function and express its bound through pairs $(x,y)\text{.}$ These pairs supply the weighted inequality and the initial conditional bounds used in the descent.

All logarithms are natural. Let

$$h(s)=(1+s)\log(1+s)-s\log s,\qquad h(0)=0.$$

The classical Ramsey bound gives $\log R(k,\ell)\leq kh(\ell/k)$ for $1\leq\ell\leq k$ [ES].

Our fixed source is

$$f(s)=h(s)+s\mathrm{e}^{-s}P(s)\qquad(0\leq s\leq 1),$$

(3)

where the piecewise cubic $P$ is specified by the rational partition points, values, and derivatives in data/source.json. This source satisfies

$$\log R(a,b)\leq af(b/a)+o(a)\qquad(1\leq b\leq a),$$

(4)

with an error uniform in $b\text{.}$ Appendix B proves this bound and the properties of $f$ used below.

2.1 Source Pairs and Supporting Lines

The weighted inequality uses bounds of the form $R(a,b)\leq x^{-a}y^{-b}\text{.}$ We say that $(x,y)\in(0,1)^{2}$ is an admissible Ramsey pair if this bound holds for all sufficiently large $a+b\text{.}$ Color symmetry in Equation 4 leads to the extension

$$\widehat{f}(t)=\begin{cases}f(t),&0\leq t\leq 1,\\ tf(1/t),&t\geq 1.\end{cases}$$

The source exponent for arbitrary positive targets $a,b$ is then $b\widehat{f}(a/b)\text{.}$ We call $(x,y)\in(0,1)^{2}$ a source pair if $b\widehat{f}(a/b)\leq-a\log x-b\log y$ for all $a,b>0\text{,}$ and write $\mathcal{B}_{f}$ for their set. Every interior point of $\mathcal{B}_{f}$ is admissible: choose a slightly larger pair still in $\mathcal{B}_{f}$ and apply Equation 4 with color symmetry. The resulting error is $o(a+b)\text{,}$ which the strict coordinate gaps absorb.

Supporting lines give a convenient description of these pairs. The source verification shows that $f$ is continuous on $[0,1]$ and is $C^{1}$ on $(0,1]\text{,}$ where it is strictly concave and increasing. It also gives $f(0)=0$ and $2f^{\prime}(1)>f(1)\text{.}$ It follows that $\widehat{f}$ is increasing and concave. On each smooth piece with $t>1\text{,}$ $\widehat{f}^{\prime\prime}(t)=t^{-3}f^{\prime\prime}(1/t)<0\text{;}$ its first derivative is continuous at the reflected source partition points. At $t=1\text{,}$ the derivative drops from $f^{\prime}(1)$ to $f(1)-f^{\prime}(1)\text{.}$ Strict concavity and $f(0)=0$ give $f(s)-sf^{\prime}(s)>0\text{,}$ so the reflected branch is increasing. The expression in Equation 3 gives $\widehat{f}^{\prime}(0+)=\infty\text{,}$ $\widehat{f}^{\prime}(\infty)=0\text{,}$ and $\widehat{f}(t)/t\to 0$ as $t\to\infty\text{.}$

For $A>0\text{,}$ define the intercept of the supporting line with slope $A$ by

$$b(A)=\sup_{t\geq 0}\{\widehat{f}(t)-At\},\qquad\widehat{f}(t)=\inf_{A>0}\{At+b(A)\}\quad(t>0).$$

(5)

For fixed $A>0\text{,}$ $\widehat{f}(t)-At\to-\infty$ as $t\to\infty\text{,}$ while $f(t)/t\to\infty$ as $t\downarrow 0$ makes this expression positive for small positive $t\text{.}$ By continuity, the supremum is positive and attained at some $t>0\text{.}$ At each $t>0\text{,}$ concavity supplies a supporting slope $A>0\text{:}$ use the derivative, or any slope between the one-sided derivatives at $t=1\text{.}$ For this slope $b(A)=\widehat{f}(t)-At\text{,}$ proving the second identity.

Writing $x=\mathrm{e}^{-A}$ and $y=\mathrm{e}^{-B}\text{,}$ the source-pair condition becomes $B\geq b(A)\text{,}$ so

$$\mathcal{B}_{f}=\{(\mathrm{e}^{-A},\mathrm{e}^{-B}):A>0,\ B\geq b(A)\}.$$

The finite convex function $b$ is continuous on $A>0\text{,}$ so interior source pairs correspond to $B>b(A)\text{.}$ Taking $(a,b)=(1,s)$ in the support inequality gives

$$A+sb(A)\geq f(s)\qquad(0<s\leq 1).$$

(6)

2.2 The Weighted Candidate Bound

Write $N_{\chi}(v)$ for the neighborhood of $v$ in color $\chi\in\{R,B\}$ and $\deg_{\chi}(v,Z)=|N_{\chi}(v)\cap Z|\text{.}$ For disjoint nonempty vertex sets $X,Y\text{,}$ let $e_{R}(X,Y)$ be the number of red edges between them and write $d_{R}(X,Y)=e_{R}(X,Y)/(|X||Y|)\text{.}$ Following [GNNW], a candidate is an ordered pair $(X,Y)$ of such sets. We say that a candidate is $(k,\ell,t)$-good if $X\cup Y$ contains a red $K_{k}\text{,}$ or $X$ contains a blue $K_{t}\text{,}$ or $Y$ contains a blue $K_{\ell}\text{.}$ The target in the retained set $Y$ is $\ell\text{,}$ while the target $t$ in $X$ may decrease.

Lemma 2.1 (Weighted Candidate Bound).

Fix $w>0$ and $\mu,p,x,y\in(0,1)$ with $(x,y)\in\operatorname{int}\mathcal{B}_{f}$ and

$$x<(1-\mu)^{w}p^{1/(1-\mu)}.$$

For all sufficiently large $\ell\text{,}$ uniformly in positive integers $k,t\text{,}$

$$d_{R}(X,Y)\geq p,\qquad|X|^{w}|Y|\geq x^{-k}y^{-\ell}\mu^{-wt}$$

imply that $(X,Y)$ is $(k,\ell,t)$-good.

Proof.

Choose a slightly larger pair $(\widehat{x},\widehat{y})$ still in $\operatorname{int}\mathcal{B}_{f}\text{.}$ Once $\ell$ is large, admissibility guarantees a red $K_{a}$ or a blue $K_{\ell}$ in every vertex set of size at least $\widehat{x}^{-a}\widehat{y}^{-\ell}\text{,}$ uniformly for $a\geq 1\text{.}$ Applying Lemma A.1 with $C=1$ proves the claim. ∎

2.3 Conditional Bounds

To reuse a bound during a descent, we need it for an interval of blue-to-red target ratios, with a single size constant and a single threshold for $k\text{.}$

Definition 2.2 (Conditional Bound).

For $p\in(0,1)\text{,}$ a compact interval $I\subset(0,1]\text{,}$ and a continuous function $L:I\to(0,\infty)\text{,}$ we say that the conditional bound $\mathsf{D}(p,L,I)$ holds if some $C\geq 1,K$ have the following property uniformly for $r\in I$ and integers $k\geq K\text{:}$ a coloring of red density at least $p$ and order at least $C\mathrm{e}^{kL(r)}$ contains a red $K_{k}$ or a blue $K_{\lfloor rk\rfloor}\text{.}$ We choose $K$ so that all blue targets are positive. For a constant exponent $z\text{,}$ the notation $\mathsf{D}(p,z,I)$ means $L\equiv z\text{.}$

We call $(w,\mu,x,y)$ a strict control at density $p$ when it satisfies the parameter conditions of Lemma 2.1. Write $A=-\log x\text{,}$ $B=-\log y\text{,}$ and $\beta=-\log\mu\text{.}$ Apply the weighted candidate bound with $t=\ell$ to a balanced partition whose red cross-density is at least $p\text{;}$ such a partition exists by averaging. Both set sizes are a constant fraction of the graph’s order, so the corresponding exponent is

$$L(r)=\frac{A+rB+wr\beta}{1+w}.$$

(7)

Choose $y_{1}>y$ with $(x,y_{1})\in\operatorname{int}\mathcal{B}_{f}\text{.}$ For $\ell=\lfloor rk\rfloor$ and a host of order $N\geq\mathrm{e}^{kL(r)}\text{,}$ the partition satisfies

$$|X|^{w}|Y|\geq\frac{x^{-k}y^{-\ell}\mu^{-w\ell}}{3^{w+1}}\geq x^{-k}y_{1}^{-\ell}\mu^{-w\ell}.$$

The first inequality uses $kr\geq\ell$ and $B+w\beta>0\text{;}$ the second holds once $(y_{1}/y)^{\ell}\geq 3^{w+1}\text{.}$ Applying Lemma 2.1 at $(x,y_{1})$ proves $\mathsf{D}(p,L,I)$ with $C=1$ on every compact $I\subset(0,1]\text{.}$ The threshold is uniform in $r\in I\text{,}$ since $\ell\geq k\min I-1\text{.}$ These weighted source bounds are the starting points of our finite derivations, and we call them source leaves.

3 Retained-Set Descent and Its Cost

We now allow the conditional bound used in $X$ to change as its blue target decreases. Throughout the descent we keep $\deg_{R}(v,Y)\geq\pi|Y|$ for every $v\in X\text{,}$ for a fixed $\pi\in(0,1)\text{.}$ This gives cross-density at least $\pi$ after every restriction of $X\text{.}$ A positive function $g(s)$ specifies the required size $\exp(kg(s))$ of $X$ at target $t=sk\text{.}$ To use a bound $\mathsf{D}(q,L,I)\text{,}$ we require $g>L$ to absorb its size constant; the slope condition below absorbs the loss in a blue-neighborhood step.

Definition 3.1 (Route).

A route consists of a partition $0<\theta=u_{0}<\cdots<u_{m}=\lambda\leq 1$ and, on each closed cell $J_{j}=[u_{j-1},u_{j}]\text{,}$ a proved call $\mathsf{D}(q_{j},L_{j},I_{j})$ with $J_{j}\subseteq I_{j}\text{.}$ It also specifies slopes

$$d_{j}>c(q_{j}):=-\log(1-q_{j})$$

and a continuous positive function $g\text{,}$ constant on $[0,\theta]$ and affine with slope $d_{j}$ on $J_{j}\text{,}$ such that $g>L_{j}$ on every closed cell.

The descent starts at ratio $\lambda$ and stops at $\theta\text{.}$ Write $d(t)$ for the piecewise constant slope on $[\theta,\lambda]\text{,}$ taking the right-hand slope at $\theta$ and internal partition points and the left-hand slope at $\lambda\text{.}$

Theorem 2 (Retained-Set Rule).

Fix a route as above, an outer density $p\text{,}$ and a row density $0<\pi<p<1\text{.}$ Choose a strict control at density $\pi$ as the route’s terminal parameters. If

$$z>g(\lambda),\qquad z+wg(\theta)>A+\lambda B+w\theta\beta,$$

(8)

then $\mathsf{D}(p,z,\{\lambda\})$ holds with

$$C=\frac{3(1-\pi)}{p-\pi}.$$

(9)

The conclusion is uniform over truncations and vertical shifts of one fixed route when the new endpoint, output exponent, and shift depend continuously on a parameter in a compact set; the stopping ratio, slopes, densities, calls, and terminal controls stay fixed. At endpoint $r\text{,}$ retain the closed cells $[u_{j-1},\min\{u_{j},r\}]$ with $u_{j-1}<r$ and require positive uniform margins in $g>L_{j}$ and in Equation 8. Such a family may include $r=\theta\text{,}$ with no internal calls.

Proof.

If $\lambda=\theta\text{,}$ the size tests give

$$(1+w)z>z+wg(\theta)>A+\theta B+w\theta\beta.$$

The source leaf at density $\pi$ proves the conclusion at density $p>\pi\text{,}$ with constant $1\leq C\text{.}$

Suppose $\theta<\lambda\text{,}$ and take a graph of red density at least $p$ and order $N\geq C\mathrm{e}^{kz}\text{.}$ Averaging gives a balanced partition $X_{0},Y$ with cross-density at least $p$ and both parts of size at least $N/3\text{.}$ Keep in $X_{0}$ only the vertices with at least $\pi|Y|$ red neighbors in $Y\text{.}$ The surviving set $X$ satisfies

$$(p-\pi)|X_{0}||Y|\leq e_{R}(X,Y)-\pi|X||Y|\leq(1-\pi)|X||Y|.$$

The choice of $C$ gives $|X|,|Y|\geq\mathrm{e}^{kz}\text{.}$ Every nonempty subset of $X$ has red cross-density at least $\pi$ with this fixed $Y\text{.}$

Put $n_{0}=\lfloor\theta k\rfloor$ and $\ell=\lfloor\lambda k\rfloor\text{,}$ taking $k$ large enough that $n_{0}\geq 1\text{.}$ To track the loss, define for integers $0\leq t\leq\ell$

$$\Gamma_{k}(t)=kg(\theta)+\sum_{j=n_{0}+1}^{t}d(j/k)\quad(t>n_{0}),\qquad\Gamma_{k}(t)=kg(\theta)\quad(0\leq t\leq n_{0}).$$

For $V=d_{1}+\sum_{j<m}|d_{j+1}-d_{j}|\text{,}$

$$\Gamma_{k}(t)-\Gamma_{k}(t-1)=d(t/k)\quad(t>n_{0}),\qquad|\Gamma_{k}(t)-kg(t/k)|\leq V.$$

(10)

For the second bound, express $d$ as its initial step at $\theta$ and its subsequent jumps. For each unit step, the mesh count differs from $k$ times its integral by at most one, even when a jump lies on the mesh. Summing the absolute jump sizes gives $V\text{.}$

Let $C_{j}$ be the call constants and put

$$\delta=\min_{j}\min_{J_{j}}(g-L_{j})>0,\qquad\eta=\min_{j}(1-q_{j}-\mathrm{e}^{-d_{j}})>0.$$

Choose $k$ so that $k\delta>V+\max_{j}\log C_{j}$ and $\eta\mathrm{e}^{kg(\theta)}>1\text{,}$ and so that all call and candidate thresholds hold. We prove by induction on $t=1,\ldots,\ell$ that $(Z,Y)$ is $(k,\ell,t)$-good whenever $Z\subseteq X$ and $|Z|\geq\mathrm{e}^{\Gamma_{k}(t)}\text{.}$

For $t\leq n_{0}\text{,}$ the terminal test gives

$$\log(|Z|^{w}|Y|)\geq k[wg(\theta)+z]>k(A+\lambda B+w\theta\beta)\geq kA+\ell B+wt\beta.$$

Since $d_{R}(Z,Y)\geq\pi\text{,}$ Lemma 2.1 applies.

For $t>n_{0}\text{,}$ put $s=t/k$ and use the cell with assigned slope $d_{j}\text{.}$ If the red density in $Z$ is at least $q_{j}\text{,}$ then $\log|Z|\geq kg(s)-V>kL_{j}(s)+\log C_{j}\text{.}$ The call gives a red $K_{k}$ or a blue $K_{t}\text{.}$ Otherwise some vertex has a blue neighborhood $Z^{\prime}\subseteq Z$ with

$$|Z^{\prime}|>(\mathrm{e}^{-d_{j}}+\eta)(|Z|-1)>\mathrm{e}^{-d_{j}}|Z|\geq\mathrm{e}^{\Gamma_{k}(t-1)}.$$

The middle inequality follows from $\eta|Z|>1$ and $\mathrm{e}^{-d_{j}}+\eta<1\text{.}$ Induction applies to $(Z^{\prime},Y)\text{.}$ A blue $K_{t-1}$ extends through the chosen vertex; either of the other two outcomes already proves the claim.

Finally, Equation 10 and the endpoint test give $\Gamma_{k}(\ell)\leq kg(\lambda)+V<kz$ for large $k\text{.}$ The claim therefore applies to $Z=X$ at $t=\ell\text{.}$

For uniformity, take the minimum stopping height and the minima of the strict gaps over the compact parameter set, and the maxima of the finitely many call thresholds. The same $V$ bounds every truncated profile, since truncation only removes slope jumps. These choices give a common threshold for $k\text{;}$ at $\lambda=\theta$ use the fixed source control as above. ∎

3.1 The Cost of a Descent

Fix the partition, calls, slopes, and terminal controls, and vary the vertical position of $g\text{.}$ Put

$$S(s)=\int_{s}^{\lambda}d(t)\,dt,\qquad M=\max_{j}\max_{s\in J_{j}}\{L_{j}(s)+S(s)\}.$$

(11)

The quantity $S(s)$ is the cumulative logarithmic loss in descending from $\lambda$ to $s\text{.}$ Every $g$ with the specified slopes has the form $g(s)=H-S(s)$ on $[\theta,\lambda]\text{.}$ The internal conditions are exactly $H>M\text{;}$ they also make $g$ positive, since $M\geq L_{1}(\theta)+S(\theta)>S(\theta)\text{.}$ The conditions in Equation 8 now read

$$H<z,\qquad wH+z>A+\lambda B+w\theta\beta+wS(\theta).$$

There is such an $H>M$ exactly when

$$z>Z:=\max\left\{M,\frac{A+\lambda B+w\theta\beta+wS(\theta)}{1+w}\right\}.$$

(12)

Necessity follows from $z>H>M$ and $(1+w)z>z+wH\text{.}$ Conversely, if $z>Z\text{,}$ choose $H<z$ sufficiently close to $z$ to satisfy both inequalities. Thus $Z$ is the infimum output exponent over these vertical positions. The term $M$ accounts for every internal call together with the loss needed to reach it; the other term is the least exponent compatible with the weighted terminal inequality.

For one call $\mathsf{D}(q,L,[\theta,\lambda])$ with affine $L\text{,}$ choose a slope $d>c(q)\text{.}$ Here $S(s)=d(\lambda-s)\text{,}$ so $L+S$ is affine and its maximum occurs at an endpoint. Substitution in Equation 12 gives

$$Z=\max\left\{L(\lambda),\ L(\theta)+d(\lambda-\theta),\ \frac{A+\lambda B+w\theta\beta+wd(\lambda-\theta)}{1+w}\right\}.$$

The first two terms require enough vertices to use the call at both ends of the interval; the second includes all the loss incurred before reaching $\theta\text{.}$ The third requires enough vertices to apply the weighted bound at $\theta\text{.}$ Every $z>Z$ therefore gives a new conditional bound at $\lambda\text{.}$

3.2 Reuse on an Interval

To obtain bounds at smaller target ratios, keep the same calls and terminal controls. For $\theta\leq r\leq\lambda\text{,}$ restrict the profile $g$ to $[0,r]$ and allow a nonnegative vertical shift. Such a shift preserves the inequalities $g>L_{j}\text{.}$ The resulting infimum is

$$E(r)=\max\left\{g(r),\frac{A+rB+w\{\theta\beta+g(r)-g(\theta)\}}{1+w}\right\},\qquad\theta\leq r\leq\lambda.$$

(13)

For every $\varepsilon>0\text{,}$ the whole family $\mathsf{D}(p,E+\varepsilon,[\theta,\lambda])$ holds with the constant in Equation 9. To check strictness uniformly, use the shift $H_{r}=E(r)-g(r)+\varepsilon/2$ and output $z_{r}=E(r)+\varepsilon\text{.}$ The endpoint gap is $\varepsilon/2\text{,}$ and

$$z_{r}+w[g(\theta)+H_{r}]-A-rB-w\theta\beta\geq(1+w/2)\varepsilon.$$

Theorem 2 applies uniformly.

A source leaf is affine, and $E$ is the maximum of two affine functions on each affine piece of $g\text{.}$ When a new route uses one of these bounds as a call, refine its cell at the partition points of the called profile. On each resulting piece, adding the affine loss $S$ preserves convexity. The maximum in Equation 11 is therefore attained at an endpoint of a call cell or a partition point of the called profile inside that cell.

4 The Optimum over Finite Proofs

We keep the source $\mathcal{B}_{f}$ fixed and minimize over all finite repetitions of the retained-set rule.

Definition 4.1 (Finite Proof).

A finite proof is a finite derivation of conditional bounds using the following rules:

1.

Start with a weighted source bound from Equation 7.
2.

Apply Theorem 2 with previously derived conditional bounds as its calls, including its uniform families.
3.

Restrict to a closed ratio interval, increase the density, or increase the continuous exponent function pointwise.
4.

Combine finitely many bounds: on a finite closed cover of a compact ratio interval, use a continuous output that dominates a proved family at the same density on each cover interval.

Taking the maximum of the constants and thresholds validates the last rule. Let $U_{\rm fin}(p,s)$ be the infimum of the exponents obtained at $(p,s)$ by these finite proofs.

To compare these proofs across all finite depths, first let $U_{0}(p,s)$ be the infimum over source leaves alone. It is finite: take $w=1\text{,}$ $\mu=1/2\text{,}$ $A>\log 2-2\log p\text{,}$ and $B>b(A)\text{.}$ These strict controls give $0\leq U_{0}(p,s)<\infty\text{.}$

To treat all possible descents at once, let $u(p,s)$ specify the required exponent for a call at $(p,s)\text{.}$ Order these functions pointwise and consider the complete lattice $\mathcal{L}=\{u:(0,1)\times(0,1]\to\mathbb{R}:0\leq u\leq U_{0}\}\text{.}$

Define $T(u)(p,\lambda)$ as the infimum of $U_{0}(p,\lambda)$ and all $z>0$ admitted by the following data: a finite partition $0<\theta=u_{0}<\cdots<u_{m}=\lambda\text{,}$ densities $q_{j}\in(0,1)\text{,}$ slopes $d_{j}>c(q_{j})\text{,}$ and a continuous positive function $g\text{,}$ affine with slope $d_{j}$ on each $J_{j}$ and constant on $[0,\theta]\text{,}$ such that

$$g(s)>u(q_{j},s)\qquad(s\in J_{j}).$$

(14)

The terminal parameters form a strict control at some row density $0<\pi<p$ and satisfy Equation 8. All data are finite and fixed before $k$ is chosen.

The operator is monotone: decreasing the internal requirements can only add feasible routes. Its output is nonincreasing in the outer density $p\text{,}$ since any leaf or route valid at $p$ remains valid at a larger density. Set

$$U_{n+1}=T(U_{n}),\qquad U_{\infty}=\inf_{n\geq 0}U_{n}.$$

We say that $v\in\mathcal{L}$ is a lower bound preserved by the rules if $v\leq T(v)\text{.}$ In standard order terminology, such a function is a post-fixed point of $T\text{.}$ Equivalently, every leaf and every route using internal requirements $v$ has output at least $v\text{.}$ Here a proof on a relative neighborhood of $s$ means a conditional bound on a compact ratio interval containing that neighborhood.

Theorem 3 (Finite Proofs and Preserved Lower Bounds).

The sequence $U_{n}$ decreases to the greatest fixed point $U_{\infty}$ of $T$ in $\mathcal{L}\text{,}$ and

$$U_{\rm fin}(p,s)=U_{\infty}(p,s)=\sup_{\begin{subarray}{c}v\in\mathcal{L}\\ v\leq T(v)\end{subarray}}v(p,s).$$

(15)

Every $z>U_{\infty}(p,s)$ has a finite proof with constant output $z$ on a relative neighborhood of $s\text{,}$ with uniform constants.

The order comparison is the classical greatest-fixed-point principle [Tarski, Theorem 1]. Strict inequalities and compactness also identify the limit with finite conditional proofs, including the uniformity required for their reuse.

Proof.

Since $T(U_{0})\leq U_{0}\text{,}$ monotonicity gives $U_{n+1}\leq U_{n}\text{.}$ By induction on $n\text{,}$ we prove both that each ratio section $U_{n}(p,\cdot)$ is upper semicontinuous and that every $z>U_{n}(p,s)$ has a finite proof with constant output $z$ on a relative neighborhood of $s\text{.}$ Upper semicontinuity means that a strict inequality $U_{n}(p,s)<a$ persists near $s\text{.}$ At level zero, both conclusions follow by choosing a continuous source leaf strictly below $z\text{.}$

Suppose the claim holds at level $n\text{,}$ and let $z>U_{n+1}(p,\lambda)\text{.}$ Choose a leaf or a formal route against $U_{n}$ with output $z^{\prime}<z\text{.}$ A leaf again gives both conclusions by continuity. For a route, truncate or extend its last affine cell, keeping the density and slope. Upper semicontinuity preserves $g>U_{n}(q_{m},\cdot)$ near $\lambda\text{,}$ and the strict endpoint and terminal tests persist by continuity. Thus the same output $z^{\prime}$ is feasible for nearby endpoints, proving $U_{n+1}(p,r)<z$ there.

Choose a compact interval of endpoints within this range that contains a relative neighborhood of $\lambda\text{.}$ To replace the formal calls by finite proofs, consider the call cells up to the largest endpoint in this neighborhood. At each point $s$ of a cell with density $q_{j}\text{,}$ choose

$$U_{n}(q_{j},s)<z_{s}<g(s).$$

The induction hypothesis gives a finite proof with constant output $z_{s}$ near $s\text{.}$ Shrink this neighborhood until $z_{s}<g$ throughout. Compactness gives a finite subcover of each cell. Subdivide into closed subcells contained in these neighborhoods and use the corresponding proofs, with the original density and slope. The calls and their strict inequalities also hold at every shared endpoint. There are finitely many subcells, so their positive gaps have a common positive lower bound. The uniform part of Theorem 2 now gives a finite proof with constant output $z$ on the chosen endpoint neighborhood. This completes the induction.

We next show that $U_{\infty}$ is fixed. Monotonicity gives $T(U_{\infty})\leq U_{n+1}$ for every $n\text{,}$ hence $T(U_{\infty})\leq U_{\infty}\text{.}$ For the reverse inequality, take a route feasible against $U_{\infty}\text{.}$ On each closed cell $J_{j}\text{,}$ the sets

$$O_{n}=\{s\in J_{j}:U_{n}(q_{j},s)<g(s)\}$$

are relatively open, increase with $n\text{,}$ and cover $J_{j}\text{.}$ Compactness and nesting give a common $N$ with $O_{N}=J_{j}$ on every cell. The route is therefore feasible against $U_{N}\text{,}$ so its output is at least $U_{N+1}\geq U_{\infty}\text{.}$ The leaf alternative is also at least $U_{\infty}\text{.}$ Taking infima gives $T(U_{\infty})=U_{\infty}\text{.}$

If $v\in\mathcal{L}$ and $v\leq T(v)\text{,}$ then $v\leq U_{0}$ and

$$v\leq U_{n}\quad\Longrightarrow\quad v\leq T(v)\leq T(U_{n})=U_{n+1}.$$

Hence $v\leq U_{\infty}\text{.}$ As $U_{\infty}$ is itself fixed, it is the greatest post-fixed point.

Every finite proof has output at least $U_{\infty}\text{,}$ by induction through its rules. This holds for leaves since $U_{0}\geq U_{\infty}\text{.}$ A route whose calls are at least $U_{\infty}$ is feasible against $U_{\infty}\text{,}$ so its output is at least $T(U_{\infty})=U_{\infty}\text{.}$ The case $\lambda=\theta$ reduces to a leaf. Restriction, increasing the exponent, and finite patching preserve the comparison. Increasing the density does too, since $U_{\infty}$ is nonincreasing in that density. Thus $U_{\rm fin}\geq U_{\infty}\text{.}$

Conversely, every $z>U_{\infty}(p,s)$ exceeds $U_{n}(p,s)$ for some finite $n\text{.}$ The local proof constructed above gives $U_{\rm fin}\leq U_{\infty}$ and the claimed uniformity. ∎

5 Concave Lower Bounds

We construct a function $v\leq T(v)\text{,}$ giving the lower estimate by Theorem 3. Its exponent will be a concave polygon: a continuous piecewise affine concave function $F:[0,1]\to\mathbb{R}\text{.}$ Write its partition points and values as

$$0=s_{0}<s_{1}<\cdots<s_{N}=1,\qquad F_{i}=F(s_{i}),\qquad F_{0}=0,$$

and suppose the slopes $d_{i}=(F_{i}-F_{i-1})/(s_{i}-s_{i-1})$ satisfy $d_{1}\geq\cdots\geq d_{N}>0\text{.}$ A call at density $q>1-\mathrm{e}^{-d_{i}}$ requires a route slope greater than $c(q)>d_{i}\text{.}$ This relation suggests using $1-\mathrm{e}^{-d_{i}}$ as the density up to which $F$ gives a lower bound. Define

$$p_{F}(s)=1-\mathrm{e}^{-d_{i}}\quad(s_{i-1}<s\leq s_{i}),\qquad v_{F}(p,s)=\begin{cases}F(s),&p\leq p_{F}(s),\\ 0,&p>p_{F}(s).\end{cases}$$

At each $s_{i}>0\text{,}$ $p_{F}$ uses the slope of the segment ending there.

Proposition 5.1 (A Preserved Lower Bound).

Suppose every strict terminal control of row density at most $p_{F}(\lambda)$ satisfies

$$F(\lambda)+wF(\theta)\leq A+\lambda B+w\theta\beta\qquad(0\leq\theta\leq\lambda\leq 1,\ \lambda>0).$$

(16)

Then $v_{F}\leq U_{\infty}\text{.}$ If $d_{N}\geq\log 2\text{,}$ in particular $F(1)\leq U_{\infty}(1/2,1)\text{.}$

Proof.

Taking $\theta=\lambda$ in Equation 16 shows that every source leaf at density $p\leq p_{F}(\lambda)$ has exponent at least $F(\lambda)\text{.}$ Thus $0\leq v_{F}\leq U_{0}\text{.}$

Consider a route feasible against $v_{F}$ at density $p\leq p_{F}(\lambda)\text{,}$ with output $z<F(\lambda)\text{.}$ Its endpoint satisfies $g(\lambda)<F(\lambda)\text{.}$ Refine the call partition at the points $s_{i}\text{.}$ On a resulting cell $[a,b]\text{,}$ let $\ell_{j}$ and $d_{i}$ be the slopes of $g$ and $F\text{.}$ If $g(b)<F(b)\text{,}$ the closed call condition forces $q_{j}>p_{F}(b)=1-\mathrm{e}^{-d_{i}}\text{,}$ since $p_{F}$ uses the left slope at $b\text{.}$ Consequently $\ell_{j}>c(q_{j})>d_{i}\text{,}$ and

$$g(a)-F(a)=g(b)-F(b)-(\ell_{j}-d_{i})(b-a)<0.$$

Starting at $\lambda$ and applying this implication backward through the finite partition gives $g(\theta)<F(\theta)\text{.}$ The row density satisfies $\pi<p\leq p_{F}(\lambda)\text{,}$ so Equation 16 now contradicts $z+wg(\theta)>A+\lambda B+w\theta\beta\text{.}$ Every route and leaf therefore has output at least $v_{F}\text{,}$ giving $v_{F}\leq T(v_{F})\text{.}$ Apply Theorem 3; the final assertion follows from $p_{F}(1)\geq 1/2\text{.}$ ∎

5.1 Elimination of the Terminal Controls

At each positive partition point, we maximize over the stopping ratio and combine the terminal inequalities with the supporting-line formula. This gives a test in the single parameter $\mu\text{.}$ For $0<\mu<1\text{,}$ write $\tau=-\log(1-\mu)$ and $\beta=-\log\mu\text{,}$ and define

$$\Psi(\lambda,z,\kappa,\mu)=\frac{1-\mu}{\kappa}\left[z-\lambda\widehat{f}\!\left(\frac{1-\kappa}{\lambda}\right)\right],\qquad\lambda>0,\quad 0<\kappa<1.$$

(17)

Theorem 4 (Finite Polygon Criterion).

Suppose $F$ is as above and

$$0<F_{i}<\min\{f(s_{i}),h(s_{i})\}\qquad(1\leq i\leq N).$$

(18)

For each $\mu\in(0,1)$ define

$$D_{i}(\beta)=\max_{0\leq j\leq i}(F_{j}-s_{j}\beta),\qquad\kappa_{i}=D_{i}(\beta)/\tau,\qquad P_{i}=-\log(1-\mathrm{e}^{-d_{i}}).$$

(19)

If, for every $i$ and every $\mu$ with $D_{i}(\beta)>0\text{,}$

$$\Psi(s_{i},F_{i},\kappa_{i},\mu)\leq P_{i},$$

(20)

then $v_{F}\leq U_{\infty}\text{.}$

Proof.

At endpoint $s_{i}\text{,}$ the function $F(\theta)-\theta\beta$ is affine between consecutive partition points, so

$$\max_{0\leq\theta\leq s_{i}}\{F(\theta)-\theta\beta\}=D_{i}(\beta).$$

Concavity of $h$ and Equation 18 put $F(\theta)$ strictly below $h(\theta)$ for $\theta>0\text{.}$ The minimum of $\tau+\theta\beta$ over $0<\mu<1$ occurs at $\mu=\theta/(1+\theta)$ and equals $h(\theta)\text{.}$ Thus

$$F(\theta)<h(\theta)\leq\tau+\theta\beta\qquad(\theta>0).$$

The term at $\theta=0$ is zero, so $0\leq D_{i}(\beta)<\tau\text{.}$

Fix a strict control at row density $\pi\leq 1-\mathrm{e}^{-d_{i}}$ and put $P_{\pi}=-\log\pi\geq P_{i}\text{.}$ If the terminal inequality fails at $\lambda=s_{i}$ for some stopping ratio, then

$$wD_{i}>A+s_{i}B-F_{i}>0,\qquad A>w\tau+\frac{P_{\pi}}{1-\mu}.$$

Here $A+s_{i}B-F_{i}>0$ because $A+s_{i}B\geq f(s_{i})>F_{i}\text{.}$ In particular $0<\kappa_{i}=D_{i}/\tau<1\text{.}$ Multiplying the second inequality by $\kappa_{i}$ and using the first gives

$$(1-\kappa_{i})A+s_{i}B<F_{i}-\frac{\kappa_{i}P_{\pi}}{1-\mu}.$$

Since $B>b(A)\text{,}$ the supporting-line formula Equation 5 gives

$$s_{i}\widehat{f}\!\left(\frac{1-\kappa_{i}}{s_{i}}\right)\leq(1-\kappa_{i})A+s_{i}B.$$

Combining these inequalities yields $P_{\pi}<\Psi(s_{i},F_{i},\kappa_{i},\mu)\leq P_{i}\text{,}$ a contradiction. Thus Equation 16 holds for every stopping ratio at each positive partition point.

Now fix controls at row density at most $1-\mathrm{e}^{-d_{i}}$ and let $\lambda\in[s_{i-1},s_{i}]\text{.}$ Since $F$ is affine on this interval,

$$\max_{0\leq\theta\leq\lambda}\{F(\theta)-\theta\beta\}=\max\{D_{i-1}(\beta),\ F(\lambda)-\lambda\beta\},$$

where $D_{0}=0\text{.}$ The largest terminal difference over all stopping ratios is therefore

$$F(\lambda)-A-\lambda B+w\max\{D_{i-1}(\beta),\ F(\lambda)-\lambda\beta\}.$$

This is convex in $\lambda\text{,}$ being affine plus a positive multiple of a maximum of two affine functions. It is nonpositive at $s_{i}$ by the inequality already proved there. The same holds at $s_{i-1}>0$ because the left slope of $F$ there is at least $d_{i}\text{;}$ at zero the value is $-A<0\text{.}$ Convexity gives Equation 16 throughout the interval. Apply Proposition 5.1. ∎

5.2 The Full Control Interval

For the supplied polygon, the remaining variable is $\tau\in(0,\infty)\text{,}$ with $\mu=1-\mathrm{e}^{-\tau}$ and $\beta(\tau)=-\log(1-\mathrm{e}^{-\tau})\text{.}$ We verify the criterion analytically near zero and infinity, leaving a compact interval for the finite computation. Let $L=1/10$ and $H=10\text{.}$ The supplied data satisfy

$$\beta(L)>d_{1},\qquad\beta(H)<F_{N}/2,\qquad F_{N}<H,\qquad 2H\mathrm{e}^{-H}<P_{1}.$$

(21)

For $\tau\leq L\text{,}$ concavity gives $F_{j}\leq d_{1}s_{j}<\beta s_{j}$ for $j>0\text{,}$ so $D_{i}=0\text{.}$ For $\tau\geq H\text{,}$ concavity gives $F_{i}/s_{i}\geq F_{N}$ and hence $D_{i}\geq F_{i}-s_{i}\beta\geq F_{i}/2\text{.}$ Also $D_{i}\leq F_{N}<\tau\text{.}$ The source term in Equation 17 is nonnegative, so

$$\Psi(s_{i},F_{i},\kappa_{i},\mu)\leq\mathrm{e}^{-\tau}\frac{F_{i}\tau}{D_{i}}\leq 2\tau\mathrm{e}^{-\tau}\leq 2H\mathrm{e}^{-H}<P_{1}\leq P_{i}.$$

Thus only the closed interval $[L,H]$ requires subdivision.

On each control cell, outward interval arithmetic first encloses $\beta\text{.}$ The function $D_{i}$ is nonincreasing in $\beta\text{,}$ so its two endpoint evaluations enclose the finite maximum in Equation 19. The source $\widehat{f}$ increases, so its endpoint values enclose the source term in Equation 17. A cell is accepted if it proves $D_{i}=0$ throughout, or a nonpositive numerator in Equation 17 wherever $D_{i}>0\text{.}$ Points with $D_{i}=0$ already satisfy the terminal condition, since $F_{i}<A+s_{i}B\text{.}$ Otherwise the checker requires a positive lower bound for $\kappa_{i}$ before division, and accepts the cell only if it proves $P_{i}-\Psi>0\text{.}$ All other cells are subdivided. The checker verifies an exact closed cover of $[L,H]$ for every positive partition point. Theorem 4 then covers all weights, source supports, stopping ratios, endpoint ratios, and finite depths.

6 The Diagonal Bounds

We now evaluate the two sides of Equation 15. The repository specifies the source, a finite upper proof, and a rational lower polygon, with the following complete interval covers.

Construction	Size	Checked Cells
Source profile	2,279 cubic pieces	12,319
Upper proof	2,932 routes; depth 27	1,065,279
Lower polygon	2,048 affine pieces	853,082

The source inequalities are evaluated at 224 fractional binary bits, and the upper and lower inequalities at both 224 and 320 bits. An independent integer-prefix computation reproduces every route height and loss from the checked primitive enclosures. The repository contains the four rational inputs in data/, the Python certificate checker in proof/, and the Lean formalization in lean/. The Python checker uses only the standard library. Running python proof/replay.py --workers 16 from the repository root writes the results and input and code hashes to proof/build/replay/. The root README.md gives the build and reproduction instructions for both projects.

6.1 Exact Interval Arithmetic

Transcendental quantities are enclosed by intervals whose endpoints are integers divided by a fixed power of two. Rational inputs and arithmetic operations are rounded outward. For logarithms, reduce to $y\in[1,2]$ and put $u=(y-1)/(y+1)\in[0,1/3]\text{.}$ Then

$$\log y=2\sum_{j=0}^{m-1}\frac{u^{2j+1}}{2j+1}+E_{m},\qquad 0\leq E_{m}\leq\frac{3u^{2m+1}}{2m+1}.$$

(22)

The tail is at most $2u^{2m+1}/((2m+1)(1-u^{2}))\text{.}$ The same formula encloses $\log 2$ for reversing the scaling.

For the exponential, put $S_{m}(z)=\sum_{j=0}^{m}z^{j}/j!$ for integers $m\geq 0\text{.}$ Successive terms after degree $m$ have ratio at most $z/(m+2)\text{,}$ so a geometric sum gives

$$S_{m}(z)<\mathrm{e}^{z}\leq S_{m}(z)+\frac{z^{m+1}}{(m+1)!}\frac{1}{1-z/(m+2)}\qquad(0<z<m+2).$$

(23)

At zero the sum is exact. The interval backend reduces a nonnegative argument to $0\leq y\leq 1/8\text{,}$ where this tail is at most $2y^{m+1}/(m+1)!\text{.}$ Repeated squaring reverses the scaling; negative arguments use outward reciprocals. A logarithm or division is accepted only after its domain is checked. For the final exponential bound, a separate rational calculation applies Equation 23 directly with $m=140$ and compares the upper enclosure with $3.69507\text{.}$

6.2 A Finite Upper Proof

The file data/upper.json contains an ordered list of routes. A call names either a weighted source bound or an earlier route with a terminal control. All 2,932 routes are reachable from the root. The density, height, and output margins are all $10^{-12}\text{;}$ rounding is upward to multiples of $10^{-12}\text{.}$ The source pairs use the inward factor $1-10^{-9}\text{,}$ and we put $\rho=1-2\cdot 10^{-5}\text{.}$

For each control, let $\pi$ be its leaf density when it defines a source bound, and $\rho p$ when it ends a route at outer density $p\text{.}$ The checker computes

$$x_{\rm cap}=(1-\mu)^{w}\pi^{1/(1-\mu)}$$

and an explicit boundary pair $(\xi,\zeta)$ with $\xi\geq x_{\rm cap}\text{.}$ It uses $x=(1-10^{-9})x_{\rm cap}$ and $y=(1-10^{-9})\zeta\text{.}$ Both coordinates are strictly below a supporting pair, and $x<x_{\rm cap}\text{,}$ so the control is strict. Source supports are given explicitly in the data and checked by interval inequalities. For a source leaf, the coefficients $A/(1+w)$ and $(B+w\beta)/(1+w)$ are rounded upward, giving a rational affine upper bound for Equation 7.

For a reused child with stored profile $g$ and stopping ratio $\theta\text{,}$ upward enclosures of its terminal parameters give rational coefficients

$$a^{+}\geq\frac{A+w\theta\beta}{1+w},\qquad b^{+}\geq\frac{B}{1+w},\qquad t^{+}\geq\frac{w}{1+w}.$$

Its called family is

$$\max\{g(s),\ a^{+}+b^{+}s+t^{+}(g(s)-g(\theta))\}+10^{-12}.$$

This dominates Equation 13 with a strict allowance; rounding $t^{+}$ upward is valid because $g(s)-g(\theta)\geq 0\text{.}$

With these calls established, the parent route can be evaluated. Each slope is an upward-rounded enclosure of $c(q_{j})+10^{-12}\text{,}$ so the sums defining $S$ are rational. The endpoint rule in Equation 11 gives $M$ by checking the parent cell endpoints and the child profiles’ partition points inside each cell. With $Q=10^{12}\text{,}$ the parent stores

$$H=\frac{\lceil Q(M+10^{-12})\rceil}{Q},\qquad g=H-S.$$

The checker also verifies the full partitions, density implications, child-domain containment, and strict ordering of child indices. Induction through the ordered list and Theorem 2 prove all its conditional families.

The root has

$$p=\tfrac{1}{2},\qquad\pi=\tfrac{1}{2}-10^{-5},\qquad\theta=\tfrac{1}{4},\qquad\lambda=1,\qquad w=1,\qquad\mu=\tfrac{1}{4}.$$

It has 65 cells and uses the left source support in data/upper.json, at $t\approx 0.383\text{.}$ Exact reconstruction gives $g(1)>1.3\text{,}$ while the second term in Equation 13 is less than $1.2\text{.}$ Adding the output margin yields the constant conditional exponent

$$z_{+}=g(1)+10^{-12}=\frac{1306998938417}{10^{12}}.$$

Thus $U_{\infty}(1/2,1)\leq z_{+}\text{.}$

6.3 The Lower Polygon

The file data/lower.json specifies the 2,049 rational partition points and values of $F\text{.}$ Exact rational comparisons establish positive decreasing slopes and $F(0)=0\text{.}$ Interval comparisons establish Equation 18, the tail bounds in Equation 21, and $d_{N}>\log 2\text{.}$ The 853,082 closed control cells verify Equation 20. Theorem 4 and Proposition 5.1 therefore give

$$z_{-}:=F(1)=\frac{1305460275532}{10^{12}}\leq U_{\infty}(1/2,1).$$

The resulting interval for the finite-proof optimum has width less than $0.0016\text{.}$

Proof of Theorem 1.

The finite upper proof and the preserved polygon give Equation 1 by Theorem 3. In an arbitrary two-coloring, interchange the colors so that red has density at least $1/2$ and apply the root conditional bound. Its constant is

$$\frac{3(1-\pi)}{1/2-\pi}=150003,$$

which proves Equation 2. An independent rational Taylor enclosure gives $\mathrm{e}^{z_{+}}<3.69507\text{.}$ This strict gap absorbs the fixed factor and the ceiling for sufficiently large $k\text{.}$ ∎

References

[CGMS] M. Campos, S. Griffiths, R. Morris, and J. Sahasrabudhe, An exponential improvement for diagonal Ramsey, Annals of Mathematics 203 (2026), no. 3, 869–932. https://doi.org/10.4007/annals.2026.203.3.4.

[Con] D. Conlon, A new upper bound for diagonal Ramsey numbers, Annals of Mathematics 170 (2009), no. 2, 941–960. https://doi.org/10.4007/annals.2009.170.941.

[Er] P. Erdős, Some remarks on the theory of graphs, Bulletin of the American Mathematical Society 53 (1947), 292–294. https://doi.org/10.1090/S0002-9904-1947-08785-1.

[ES] P. Erdős and G. Szekeres, A combinatorial problem in geometry, Compositio Mathematica 2 (1935), 463–470. https://www.numdam.org/item/CM_1935__2__463_0/.

[GNNW] P. Gupta, Ndiamé Ndiaye, S. Norin, and L. Wei, Optimizing the CGMS upper bound on Ramsey numbers, arXiv:2407.19026v2, revised 29 August 2026. https://arxiv.org/abs/2407.19026v2.

[Ram] F. P. Ramsey, On a problem of formal logic, Proceedings of the London Mathematical Society (2) 30 (1930), 264–286. https://doi.org/10.1112/plms/s2-30.1.264.

[Sah] A. Sah, Diagonal Ramsey via effective quasirandomness, Duke Mathematical Journal 172 (2023), no. 3, 545–567. https://doi.org/10.1215/00127094-2022-0048.

[Spe] J. Spencer, Ramsey’s theorem—a new lower bound, Journal of Combinatorial Theory, Series A 18 (1975), 108–115. https://doi.org/10.1016/0097-3165(75)90071-0.

[Tarski] A. Tarski, A lattice-theoretical fixpoint theorem and its applications, Pacific Journal of Mathematics 5 (1955), 285–309. https://msp.org/pjm/1955/5-2/pjm-v5-n2-p11-s.pdf.

[Tho] A. Thomason, An upper bound for some Ramsey numbers, Journal of Graph Theory 12 (1988), no. 4, 509–517. https://doi.org/10.1002/jgt.3190120406.

Appendix A The Finite-Host Weighted Inequality

We prove the estimate used in Lemma 2.1. Its stopping assumption concerns only proper subsets of the host. This lets the same estimate prove the source bound by a smallest-counterexample argument in Appendix B.

Lemma A.1 (Finite-Host Weighted Candidate Bound).

Fix $w>0$ and parameters in $(0,1)$ satisfying

$$x_{0}<\widehat{x},\qquad y_{0}<\widehat{y},\qquad x_{0}<(1-\mu_{0})^{w}p^{1/(1-\mu_{0})}.$$

There is $L_{0}\text{,}$ depending only on these parameters, with the following property, uniformly in $C\geq 1$ and the finite host coloring $H\text{.}$ Fix $\ell\geq L_{0}\text{,}$ and suppose that for every integer $a\geq 1\text{,}$ every proper subset $Z\subsetneq V(H)$ of size

$$|Z|\geq C\widehat{x}^{-a}\widehat{y}^{-\ell}$$

(24)

contains a red $K_{a}$ or a blue $K_{\ell}\text{.}$ Then, for all $k,t\geq 1\text{,}$ every candidate in $H$ satisfying

$$d_{R}(X,Y)\geq p,\qquad|X|^{w}|Y|\geq C^{w+1}x_{0}^{-k}y_{0}^{-\ell}\mu_{0}^{-wt}$$

(25)

is $(k,\ell,t)$-good.

Proof.

We adapt the candidate induction of [GNNW, Lemma 12] to unequal weights. After removing rows with too few red neighbors, we either pass to a common blue neighborhood or choose a vertex whose red or blue child satisfies the induction hypothesis. A power of the excess red density controls the size loss in these steps.

Choose a fixed integer $r>\max\{1,w\}$ so large that $p^{1/r}>\mu_{0}$ and

$$x_{0}<(p^{1/r}-\mu_{0})^{r}(1-\mu_{0})^{w-r}.$$

The logarithm of the right side tends to $w\log(1-\mu_{0})+\log p/(1-\mu_{0})\text{.}$ The function

$$x_{0}^{1/r}(1-q)^{1-w/r}+\mu_{0}^{w/r}q^{1-w/r}$$

increases until $q=\mu_{0}/(\mu_{0}+x_{0}^{1/w})>\mu_{0}\text{,}$ and is less than $p^{1/r}$ at $q=\mu_{0}\text{.}$ By continuity choose

$$x_{0}<x<\widehat{x},\quad y_{0}<y<\widehat{y},\quad\mu_{0}<\mu<\beta<1,\quad 0<\delta<p/2,$$

and $\gamma>0$ so that, with $\bar{p}=p-\delta\text{,}$

$$\max_{0\leq q\leq\beta}\{x^{1/r}(1-q)^{1-w/r}+\mu^{w/r}q^{1-w/r}\}\leq(1-\gamma)\bar{p}^{1/r}.$$

(26)

All these choices precede $C,H,k,\ell,t\text{.}$

Put $n=k+t\text{,}$ $\delta_{n}=\delta/n\text{,}$ and induct on $n\text{,}$ for fixed sufficiently large $\ell\text{,}$ using the stronger hypothesis

$$d_{R}(X,Y)\geq p-\delta_{n},\qquad Q_{n}=(d_{R}(X,Y)-p+\delta_{n})^{r}|X|^{w}|Y|\geq C^{w+1}x^{-k}y^{-\ell}\mu^{-wt}.$$

(27)

After canceling $C^{w+1}\text{,}$ the conditions in Equation 25 imply this once $\ell$ is large, uniformly in $k,t\text{,}$ because

$$(\delta/n)^{r}(x/x_{0})^{k}(y/y_{0})^{\ell}(\mu/\mu_{0})^{wt}\geq 1.$$

The factors involving $k,t$ grow exponentially in $n\text{,}$ so their ratio to $n^{r}$ has a positive infimum; increasing $\ell$ makes the inequality hold. If $k=1$ or $t=1\text{,}$ any vertex of $X$ suffices. Henceforth $k,t\geq 2\text{,}$ and the density excess lies in $(0,1)\text{.}$

Regularize and stop if $Y$ is large. Delete rows of red degree less than $(p-\delta_{n})|Y|\text{.}$ The excess edge count $E=e_{R}(X,Y)-(p-\delta_{n})|X||Y|$ increases, and

$$Q_{n}=E^{r}|X|^{w-r}|Y|^{1-r}$$

does not decrease, since $r>w\text{.}$ Positive excess prevents deletion of the last row. Every nonempty $T\subseteq X$ now has $d_{R}(T,Y)\geq p-\delta_{n}\text{.}$ If $|Y|\geq C\widehat{x}^{-k}\widehat{y}^{-\ell}\text{,}$ apply the stopping assumption with $a=k\text{;}$ $Y$ is proper because $X\neq\varnothing\text{.}$ Otherwise $Q_{n}\leq|X|^{w}|Y|$ implies

$$|X|^{w}>C^{w}(\widehat{x}/x)^{k}(\widehat{y}/y)^{\ell}\mu^{-wt},\qquad|X|\geq C\exp(c(k+\ell+t))$$

(28)

for a fixed $c>0\text{.}$

Find a common blue neighborhood or a suitable vertex. Define

$$b=\left\lceil\frac{2r\log n+r\log(1/\delta)+w\log 2}{w\log(\beta/\mu)}\right\rceil,\qquad m=\lceil 10\beta^{-1}b^{2}\rceil,$$

and $W=\{v\in X:\deg_{B}(v,X)\geq\beta|X|\}\text{.}$ Put $\xi_{n}=\delta/(2n^{2})\text{.}$ We choose $L_{0}$ once so that

$$|X|\geq 5m^{2},\qquad R(k,m)<\bar{p}\xi_{n}|X|,\qquad\frac{2n^{2}}{\delta|X|}<\gamma.$$

To justify a uniform choice, note that $b=O(\log n)\text{,}$ $m=O((\log n)^{2})\text{,}$ and $R(k,m)\leq\binom{k+m-2}{m-1}\leq k^{m-1}\text{;}$ the last inequality counts nondecreasing tuples in $\{1,\ldots,k\}^{m-1}\text{.}$ The logarithm of each required lower bound on $|X|$ is therefore $O((\log n)^{3})\text{.}$ Subtracting $cn$ leaves a function bounded above in $n\text{,}$ so Equation 28 supplies all three bounds for one $L_{0}\text{,}$ independently of $C\geq 1,k,t\text{.}$ We use the common-neighborhood averaging argument of [GNNW, Lemma 9]. When $|W|\geq R(k,m)\text{,}$ either a red $K_{k}$ finishes the proof, or $W$ contains a blue $m$-clique $U\text{.}$ Put $N_{X}=|X|\text{.}$ The blue density $\sigma$ between $U$ and $X\setminus U$ satisfies

$$\sigma\geq\frac{\beta N_{X}-m}{N_{X}-m}=\beta-\frac{(1-\beta)m}{N_{X}-m}\geq\beta-\frac{1}{4m}.$$

Choose a uniform $b$-subset of $U\text{,}$ and let $T$ be its common blue neighborhood in $X\setminus U\text{.}$ Convexity of the piecewise-linear extension of $j\mapsto\binom{j}{b}$ on nonnegative integers gives

$$\mathbb{E}|T|\geq(N_{X}-m)\frac{\binom{\lfloor\sigma m\rfloor}{b}}{\binom{m}{b}}\geq(N_{X}-m)(\sigma-b/m)^{b}.$$

The second inequality follows factor by factor from the binomial product. Since $m\geq 10\beta^{-1}b^{2}\text{,}$

$$\sigma-b/m\geq\beta\left(1-\frac{1}{8b}\right)>0,\qquad N_{X}-m\geq\frac{4}{5}N_{X}.$$

Bernoulli’s inequality gives $\mathbb{E}|T|\geq(7/10)\beta^{b}N_{X}\text{.}$ Thus some blue $b$-clique $S\subseteq U$ has a disjoint common blue neighborhood $T$ with $|T|\geq\beta^{b}|X|/2\text{.}$

If $b\geq t\text{,}$ then $S$ already contains the required blue clique. For $b<t\text{,}$ we have

$$\delta_{n-b}-\delta_{n}\geq\delta/n^{2},\qquad Q_{n-b}(T,Y)\geq(\delta/n^{2})^{r}2^{-w}\beta^{wb}|X|^{w}|Y|\geq C^{w+1}x^{-k}y^{-\ell}\mu^{-w(t-b)}.$$

The last inequality uses $Q_{n}\leq|X|^{w}|Y|$ and $(\beta/\mu)^{wb}\geq 2^{w}(n^{2}/\delta)^{r}\text{,}$ which follows from the choice of $b\text{.}$ Induction applies; a blue $K_{t-b}$ in $T$ joins $S$ to form a blue $K_{t}\text{,}$ and either of the other two outcomes already suffices.

We may now assume $|W|<R(k,m)<\bar{p}\xi_{n}|X|\text{.}$ Put $Y_{v}=N_{R}(v)\cap Y\text{,}$ $d=d_{R}(X,Y)\text{,}$ and $E_{0}=e_{R}(X,Y)\text{.}$ Row regularization gives $|Y_{v}|\geq\bar{p}|Y|>0$ for every $v\in X\text{.}$ Counting by vertices in $Y$ and applying Cauchy–Schwarz gives

$$\sum_{v\in X}d_{R}(X,Y_{v})|Y_{v}|=\frac{1}{|X|}\sum_{y\in Y}\deg_{R}(y,X)^{2}\geq dE_{0}.$$

Removing the terms indexed by $W$ loses at most $|W||Y|\text{,}$ so

$$\sum_{v\notin W}d_{R}(X,Y_{v})|Y_{v}|\geq dE_{0}-|W||Y|>(d-\xi_{n})E_{0}\geq(d-\xi_{n})\sum_{v\notin W}|Y_{v}|.$$

Here $E_{0}\geq\bar{p}|X||Y|$ and $d-\xi_{n}>0\text{.}$ Hence some $v\notin W$ satisfies $d_{R}(X,Y_{v})\geq d-\xi_{n}\text{.}$

Pass to a red or blue child. Set $Y^{\prime}=Y_{v}\text{,}$ $X_{R}=N_{R}(v)\cap X\text{,}$ $X_{B}=N_{B}(v)\cap X\text{,}$ and

$$a=|X_{R}|/|X|,\quad q=|X_{B}|/|X|,\quad\alpha=d_{R}(X,Y^{\prime})-p+\delta_{n-1}.$$

Then $a+q\leq 1\text{,}$ $q\leq\beta\text{,}$ and $\alpha\geq d-p+\delta_{n}+\delta/(2n^{2})>0\text{.}$ For $\chi\in\{R,B\}\text{,}$ set $\alpha_{\chi}=d_{R}(X_{\chi},Y^{\prime})-p+\delta_{n-1}$ when $X_{\chi}\neq\varnothing\text{,}$ and set $\alpha_{\chi}=0$ otherwise. Write $u_{+}=\max\{u,0\}$ and

$$Q_{\chi}=(\alpha_{\chi})_{+}^{r}|X_{\chi}|^{w}|Y^{\prime}|.$$

Splitting $X$ into its two children and the root gives

$$1\leq a\frac{(\alpha_{R})_{+}}{\alpha}+q\frac{(\alpha_{B})_{+}}{\alpha}+\frac{1}{\alpha|X|}.$$

(29)

The root’s contribution is $(1-p+\delta_{n-1})/(\alpha|X|)\leq 1/(\alpha|X|)\text{.}$ We claim that

$$Q_{R}\geq xQ_{n}\qquad\text{or}\qquad Q_{B}\geq\mu^{w}Q_{n}.$$

If both inequalities failed, then $Q_{n}\leq\alpha^{r}|X|^{w}|Y|\text{,}$ $|Y^{\prime}|\geq\bar{p}|Y|\text{,}$ and Equation 29 would give

$$1\leq\bar{p}^{-1/r}\{x^{1/r}a^{1-w/r}+\mu^{w/r}q^{1-w/r}\}+\frac{1}{\alpha|X|}\leq 1-\gamma+\frac{2n^{2}}{\delta|X|}<1.$$

The second inequality uses $a\leq 1-q$ and Equation 26; the last is one of our three size bounds. Thus one child has a positive value of $Q_{n-1}$ meeting Equation 27, with $k$ reduced by one in the red child or $t$ reduced by one in the blue child. Induction applies. The root extends a red $K_{k-1}$ in $X_{R}\cup Y^{\prime}\text{,}$ or a blue $K_{t-1}$ in $X_{B}\text{,}$ respectively. Every other good outcome already gives a required clique. ∎

Corollary A.2 (Local Density Bound).

Under the stopping assumption of Lemma A.1, fix $w>0$ and $\mu,p,x,y\in(0,1)$ with $x<\widehat{x}\text{,}$ $y<\widehat{y}\text{,}$ and $x<(1-\mu)^{w}p^{1/(1-\mu)}\text{.}$ If a host has red density at least $p$ and order

$$N\geq Cx^{-k/(w+1)}(y\mu^{w})^{-\ell/(w+1)},$$

(30)

then it contains a red $K_{k}$ or a blue $K_{\ell}$ for all sufficiently large $\ell\text{,}$ with a threshold independent of $C\geq 1$ and $k\text{.}$

Proof.

Choose $y<y_{1}<\widehat{y}\text{.}$ A uniformly chosen balanced partition has expected cross-density equal to the host density, so some disjoint $X,Y\text{,}$ each of size at least $N/3\text{,}$ satisfy $d_{R}(X,Y)\geq p\text{.}$ For large $\ell\text{,}$ $(y_{1}/y)^{\ell}\geq 3^{w+1}\text{.}$ The order bound therefore supplies Equation 25 at $(x,y_{1})$ with $t=\ell\text{,}$ and Lemma A.1 applies. ∎

Appendix B Admissibility of the Source

We first give a criterion for a concave Ramsey profile, then verify it for the specified function $f\text{.}$ For a continuous $F$ on $[0,1]$ with $F(0)=0\text{,}$ define

$$G_{F}(a,b)=\max(a,b)F\!\left(\frac{\min(a,b)}{\max(a,b)}\right),$$

$$\mathcal{B}_{F}=\{(x,y)\in(0,1)^{2}:G_{F}(a,b)\leq-a\log x-b\log y\text{ for all }a,b>0\}.$$

Thus $\mathcal{B}_{F}$ consists of the exponential bounds that dominate the profile in both coordinates. For $F=f\text{,}$ this agrees with the source pairs in Section 2, since $G_{f}(a,b)=b\widehat{f}(a/b)\text{.}$ In the source criterion, $X,Y$ denote scalar support parameters.

Lemma B.1 (Source Criterion).

Suppose $F$ is continuous on $[0,1]\text{,}$ $F(0)=0\text{,}$ is $C^{1}$ and concave on $(0,1]\text{,}$ and has $F^{\prime}>0\text{.}$ If for every $s\in(0,1]$ there are $w>0\text{,}$ $\mu,Y\in(0,1)$ with

$$p_{0}(s)=1-\mathrm{e}^{-F^{\prime}(s)},\quad X=(1-\mu)^{w}p_{0}(s)^{1/(1-\mu)},\quad(X,Y)\in\mathcal{B}_{F},$$

and

$$(w+1)F(s)+\log X+s\log Y+ws\log\mu>0,$$

(31)

then, for every $\varepsilon>0\text{,}$ some $C_{\varepsilon}\geq 1$ satisfies

$$R(a,b)\leq\left\lceil C_{\varepsilon}\exp\bigl(G_{F}(a,b)+\varepsilon(a+b)\bigr)\right\rceil\qquad(a,b\in\mathbb{Z}_{\geq 1}).$$

(32)

Proof.

Fix $\varepsilon>0\text{.}$ We first choose finitely many local bounds, then apply one of them to a smallest counterexample. Since $F(0)=0$ and $F^{\prime}>0\text{,}$ we have $F\geq 0\text{.}$ Choose $0<\alpha<1$ with $h(\alpha)<\varepsilon/2\text{;}$ the classical bound will handle smaller ratios. At each $s_{0}\in[\alpha,1]\text{,}$ choose $w,\mu,X,Y$ as in the hypothesis and put $\bar{x}=\mathrm{e}^{-\varepsilon}X\text{,}$ $\bar{y}=\mathrm{e}^{-\varepsilon}Y\text{.}$ Scaling the controls this way and replacing $F(s_{0})$ by $F(s_{0})+\varepsilon(1+s_{0})$ increases the slack by $w\varepsilon(1+s_{0})\text{.}$ We may therefore choose

$$0<x<\widehat{x}<\bar{x},\qquad 0<y<\widehat{y}<\bar{y},\qquad 0<p<p_{0}(s_{0}),$$

close enough to $(\bar{x},\bar{y},p_{0}(s_{0}))$ that $x<(1-\mu)^{w}p^{1/(1-\mu)}$ and

$$H(t):=-\frac{\log x+t\log(y\mu^{w})}{w+1},\qquad H(s_{0})<F(s_{0})+\varepsilon(1+s_{0}).$$

Continuity gives a neighborhood on which this inequality holds and $p_{0}(s)>p+2\eta_{s_{0}}$ for some $\eta_{s_{0}}>0\text{.}$ A finite subcover gives $\eta>0$ and a common threshold $L\geq 2$ for Corollary A.2, both independent of $C\text{.}$ At every $s\in[\alpha,1]\text{,}$ one selected local bound satisfies $H(s)<F(s)+\varepsilon(1+s)$ and $p_{0}(s)>p+2\eta\text{.}$ Every selected pair also satisfies

$$-a\log\widehat{x}-b\log\widehat{y}\geq G_{F}(a,b)+\varepsilon(a+b)\qquad(a,b>0),$$

(33)

by the support condition and the scaling by $\mathrm{e}^{-\varepsilon}\text{.}$

Choose an integer $K$ with $\alpha K\geq L\text{,}$ and then $C\geq\max\{3,2/\eta,4^{K}\}\text{.}$ Suppose there is a coloring with no required clique at an integer order

$$N\geq C\exp\bigl(G_{F}(a,b)+\varepsilon(a+b)\bigr).$$

Choose one of smallest order among all parameter pairs and colorings. Interchange the colors if needed and write its targets as $k\geq\ell\text{.}$ The classical bound [ES] excludes $k<K\text{.}$ It also excludes $\ell/k\leq\alpha\text{,}$ since

$$\log R(k,\ell)\leq kh(\ell/k)\leq kh(\alpha)<\varepsilon k/2\leq kF(\ell/k)+\varepsilon(k+\ell)+\log C.$$

Thus $s=\ell/k\in[\alpha,1]$ and $\ell\geq L\text{.}$

For every integer $a\geq 1\text{,}$ minimality and Equation 33 give a red $K_{a}$ or a blue $K_{\ell}$ in each proper subset satisfying Equation 24. Thus the stopping assumption of Corollary A.2 holds for the local bound selected at $s\text{.}$ If the host’s red density is at least $p_{0}(s)-\eta>p\text{,}$ then $N>C\mathrm{e}^{kH(s)}\text{,}$ and Corollary A.2 gives a required clique.

Otherwise its blue density exceeds $q=\mathrm{e}^{-F^{\prime}(s)}+\eta<1\text{.}$ A blue neighborhood $Z$ has size at least $q(N-1)>N\mathrm{e}^{-F^{\prime}(s)}\text{,}$ since $N\eta>1\text{.}$ Concavity gives

$$k\bigl(F(s)-F(s-1/k)\bigr)\geq F^{\prime}(s).$$

Consequently

$$|Z|>N\mathrm{e}^{-F^{\prime}(s)}\geq\mathrm{e}^{\varepsilon}C\exp\bigl(kF(s-1/k)+\varepsilon(k+\ell-1)\bigr).$$

The proper subset $Z$ therefore has order above the proposed $(k,\ell-1)$ threshold. Minimality gives a red $K_{k}$ or a blue $K_{\ell-1}\text{;}$ in the latter case, add the chosen vertex. Either outcome contradicts the choice of the host.

Hence Equation 32 holds. For $1\leq b\leq a\text{,}$ its logarithm is at most $aF(b/a)+2\varepsilon a+\log(2C_{\varepsilon})\text{.}$ Since $\varepsilon$ is arbitrary, the error is uniform in $b\text{.}$ ∎

B.1 The Piecewise Cubic Profile

We now verify the criterion for $f$ in Equation 3. The function $P$ has 2,279 rational cubic pieces. The file data/source.json gives the rational partition points, values, and derivatives of $P\text{.}$ On a cell $[a,b]\text{,}$ set $u=t-a\text{,}$ $\delta=b-a\text{,}$ and $\Delta=P(b)-P(a)\text{.}$ Its exact polynomial is

$$P(a+u)=P(a)+P^{\prime}(a)u+\frac{3\Delta/\delta-2P^{\prime}(a)-P^{\prime}(b)}{\delta}u^{2}+\frac{P^{\prime}(a)+P^{\prime}(b)-2\Delta/\delta}{\delta^{2}}u^{3}.$$

Exact endpoint identities ensure that $P$ is globally $C^{1}\text{.}$ For $t>0$ in the interior of a piece, with $j(t)=t\mathrm{e}^{-t}P(t)\text{,}$

$$j^{\prime\prime}(t)=\mathrm{e}^{-t}\bigl((t-2)P+(2-2t)P^{\prime}+tP^{\prime\prime}\bigr),\qquad f^{\prime\prime}(t)=-\frac{1}{t(1+t)}+j^{\prime\prime}(t).$$

The cover in data/source_cells.json proves $1-t(1+t)j^{\prime\prime}(t)>0$ on each closed piece, using that piece’s endpoint derivatives, together with $f^{\prime}(1)>0$ and $2f^{\prime}(1)>f(1)\text{.}$ Consequently $f(0)=0\text{,}$ $f$ is continuous on $[0,1]\text{,}$ $C^{1}$ on $(0,1]\text{,}$ strictly concave and increasing.

To describe the boundary of $\mathcal{B}_{f}\text{,}$ put $x_{f}(t)=\mathrm{e}^{-f^{\prime}(t)}$ and $y_{f}(t)=\mathrm{e}^{tf^{\prime}(t)-f(t)}\text{.}$ Their endpoint limits are

$$x_{f}(t)=\frac{t}{1+t}\mathrm{e}^{-j^{\prime}(t)}\longrightarrow 0,\qquad y_{f}(t)=\frac{1}{1+t}\mathrm{e}^{tj^{\prime}(t)-j(t)}\longrightarrow 1.$$

Strict concavity makes $x_{f}$ increasing and $y_{f}$ decreasing. Also $x_{f}(t)<y_{f}(t)\text{:}$ the function $(1+t)f^{\prime}(t)-f(t)$ decreases to its positive value at $t=1\text{.}$ Writing $x_{1}=x_{f}(1)$ and $y_{1}=y_{f}(1)\text{,}$ the upper boundary is

$$Y_{f}(x)=\begin{cases}y_{f}(t),&0<x<x_{1},\quad x_{f}(t)=x,\\ \mathrm{e}^{-f(1)}/x,&x_{1}\leq x\leq y_{1},\\ x_{f}(t),&y_{1}<x<1,\quad y_{f}(t)=x.\end{cases}$$

(34)

These are the supporting lines of $\widehat{f}$ on its two smooth branches and at its corner. Thus $b(A)=-\log Y_{f}(\mathrm{e}^{-A})\text{.}$

B.2 The Source Inequalities

The file data/source_cells.json supplies a cover of 12,319 closed cells and rational controls $r,w>0$ on each cell. Set $\mu=tr\text{,}$ take $X$ as in Lemma B.1, and choose $Y=Y_{f}(X)\text{.}$ With $\Delta_{f}$ denoting the left side of Equation 31, the cell inequalities give

$$\Delta_{f}(t)/t>10^{-8}\qquad(0<t\leq 1).$$

The cell inequalities also establish $0<\mu,p_{0}<1$ for positive $t\text{.}$ Since Lemma B.1 requires controls at each ratio, they may be chosen separately on each cell.

Near zero, we cancel the logarithmic terms before evaluating the slack. Define

$$L_{+}(u)=\frac{\log(1+u)}{u},\qquad L_{-}(u)=\frac{-\log(1-u)}{u},\qquad L_{+}(0)=L_{-}(0)=1,$$

$$U=(1+t)L_{+}(t)+\mathrm{e}^{-t}P(t),\quad u=\frac{\mathrm{e}^{-j^{\prime}(t)}}{1+t},\quad V=wrL_{-}(tr)+\frac{uL_{-}(tu)}{1-tr},$$

$$J(s)=L_{+}(s)+s\mathrm{e}^{-s}(P(s)-P^{\prime}(s)).$$

Then $f(t)/t=-\log t+U$ and $\log X=-tV\text{.}$ On the right supporting branch, write $X=y_{f}(t\sigma)\text{,}$ $Y=x_{f}(t\sigma)\text{.}$ Here $\log Y=\log t+\log\sigma-\log(1+t\sigma)-j^{\prime}(t\sigma)$ and $\log\mu=\log t+\log r\text{.}$ The terms in $\log t$ cancel in the normalized slack, giving

$$\begin{aligned} \displaystyle\sigma J(t\sigma) & \displaystyle=V, \\ \displaystyle\Delta_{f}(t)/t & \displaystyle=(w+1)U-V+\log\sigma-\log(1+t\sigma)-j^{\prime}(t\sigma)+w\log r. \end{aligned}$$

As $t\downarrow 0\text{,}$ $X\to 1$ implies $t\sigma\to 0\text{,}$ since $y_{f}$ is strictly decreasing with limit 1 only at zero. Thus $\sigma J(t\sigma)=V$ and $J(0)=1$ give the continuous extension $\sigma(0)=V(0)=wr+\mathrm{e}^{-P(0)}>0\text{.}$ On each cell near zero, the checker verifies the branch, a positive bracket for $\sigma\text{,}$ and $t\sigma\leq 1\text{.}$ The root is unique: $\sigma\mapsto\sigma J(t\sigma)=-\log y_{f}(t\sigma)/t$ is strictly increasing for $t>0\text{,}$ and is the identity at zero.

The verifier evaluates complete finite polynomial expansions. Source inverse brackets are accepted only after outward-rounded endpoint inequalities prove them. When $t$ and $t\sigma(t)$ lie in the interiors of source pieces, possibly different ones,

$$\sigma^{\prime}=\frac{V^{\prime}-\sigma^{2}J^{\prime}(t\sigma)}{J(t\sigma)+t\sigma J^{\prime}(t\sigma)},$$

with a checked positive denominator. The integral representations

$$L_{+}(u)=\int_{0}^{1}\frac{da}{1+ua},\qquad L_{-}(u)=\int_{0}^{1}\frac{da}{1-ua}$$

give endpoint enclosures for these functions and their derivatives, including at zero. Away from zero, the function $u\mapsto\log Y_{f}(\mathrm{e}^{u})\text{,}$ with $u<0\text{,}$ has derivative $-s\text{,}$ $-1\text{,}$ or $-1/s$ on the three branches of Equation 34, where $s$ is the supporting parameter. These derivatives agree at the branch junctions.

Together with Equations 22 and 23, these identities enclose each slack and its derivative. An enclosure at the cell center plus the derivative interval times the displacement encloses the slack throughout the cell. When a cell crosses a source partition point, enclosures from every adjoining piece are combined. The continuous compositions are locally Lipschitz, and the stated derivative bounds hold almost everywhere; integrating those bounds justifies the same enclosure across a junction. The complete cover verifies the hypotheses of Lemma B.1, which gives Equation 4 with an error uniform in $b\text{.}$

A Bound Below 2.8 for Tuza's Conjecture

Sat, 12 Sep 2026 00:00:00 +0000

1 Introduction

A triangle packing in a graph $G$ is a set of pairwise edge-disjoint triangles, and a triangle transversal is a set of edges meeting every triangle. Let $\nu(G)$ and $\tau(G)$ denote their maximum and minimum sizes. A transversal contains an edge of every packed triangle, and the edges of a maximum packing form a transversal, so $\nu(G)\leq\tau(G)\leq 3\nu(G)\text{.}$ Tuza [Tuz90] conjectured that $\tau(G)\leq 2\nu(G)$ for every graph $G\text{.}$ The constant $2$ would be best possible, as $K_{4}$ and $K_{5}$ show.

Haxell [Hax99] proved $\tau(G)\leq(66/23)\,\nu(G)\text{,}$ where $66/23\approx 2.870\text{,}$ and noted without proof that the argument can be pushed to $(1+\sqrt{481})/8\approx 2.866\text{.}$ Very recently, Yi [Yi26] lowered the constant to $(162+4\sqrt{3})/59\approx 2.863$ by proving $\tau\leq(1+\sqrt{3})\nu$ for every family of triangles that admits a red-blue edge coloring with one red edge per triangle, and observed that this route alone cannot bring the constant below $54/19\approx 2.842\text{.}$ Independently of Yi’s work, we prove the following bound, whose constant $165/59\approx 2.797$ crosses this barrier and lies below $2.8\text{.}$

Theorem 1.

Every finite simple graph $G$ satisfies

$$\tau(G)\leq\frac{165}{59}\nu(G).$$

Our starting point is the following exchange property. Let the edges of a graph be colored red and blue, and consider the family of all triangles with exactly one red edge. Fix a maximum packing in this family and delete the blue edges of its triangles. By maximality, every triangle of the family that survives meets the packing only in its red edge. Suppose now that a surviving triangle $uvy$ contains the red edge $uv$ of a packed triangle $uvx\text{,}$ and that $xy$ is a red edge not used by the packing. Then $uvy$ is the only surviving triangle containing $uv\text{.}$ Indeed, $uxy$ has exactly one red edge, and the family contains every such triangle. So if $uvz$ were a second surviving triangle containing $uv\text{,}$ then replacing $uvx$ by $uxy$ and $uvz$ would enlarge the packing (Lemma 2.1). A red edge that lies in a single surviving triangle can be omitted from any transversal containing the other two edges of that triangle. Two transversals of the family, built in Section 2, allow this omission. Combining the two resulting bounds gives $\tau\leq(8/3)\nu$ for every family of this kind, and every packed triangle whose red edge lies in no other triangle of the family lowers the bound by a further $2/3$ (Lemma 2.3).

Families of this kind arise in Haxell’s construction. The construction fixes a maximum packing $P\text{,}$ calls the edges of its triangles old, and packs the remaining triangles in two further rounds: a maximum packing $A$ of triangles with one old edge, and, after the edges of $A$ are deleted, a packing $P^{\prime}$ with as many triangles with two old edges as possible. Color new edges red and old edges blue. After the deletion of the edges of $A\text{,}$ the triangles with two old edges form a family of the above kind, and so do the triangles that survive the further deletion of the old edges of $P^{\prime}\text{.}$ We build four transversals of $G$ from these packings, using books of triangles with a common edge, the bounds of Lemma 2.3, and a random bipartition of the vertex set. Each gives an inequality between $\tau(G)\text{,}$ $\nu(G)\text{,}$ and the sizes of the packings, and a linear combination of the four inequalities yields $165/59\text{.}$ Section 2 proves the exchange property and Lemma 2.3, and Section 3 carries out the construction. The complete proof has been formalized in Lean 4 with Mathlib.¹¹ 1

 <a href="https://github.com/sichen-wang/tuza-lean" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/sichen-wang/tuza-lean</a></span></span></span></p>

2 Colored Triangle Families

Throughout, graphs are finite and simple, a triangle is identified with its set of three edges, and two triangles meet if they share an edge. For a family of triangles, a packing is a set of pairwise edge-disjoint members, a transversal is a set of edges meeting every member, and a packing is maximal if no member of the family can be added to it. Maximum packings are maximal. For a set $P$ of triangles, $E(P)$ is the union of their edge sets, and $H-X$ is the graph $H$ with the edge set $X$ deleted.

This section proves the three lemmas used in Section 3: the exchange property (Lemma 2.1), a covering lemma for books (Lemma 2.2), and the two colored bounds (Lemma 2.3). Let $H$ be a graph whose edges are colored red and blue, and let $\mathcal{F}(H)$ be the family of all triangles of $H$ with exactly one red edge. We write $\mathcal{F}$ for $\mathcal{F}(H)$ when $H$ is clear. For a packing $P$ in $\mathcal{F}(H)\text{,}$ let $R(P)$ and $B(P)$ be the sets of red and blue edges of its members, and let

$$\mathcal{F}_{P}=\mathcal{F}(H-B(P))$$

be the family of triangles with one red edge that survive the deletion of the blue edges of $P\text{.}$ If $P$ is maximum, every member of $\mathcal{F}_{P}$ has its red edge in $R(P)\text{,}$ since otherwise it could be added to $P\text{.}$ Hence it meets $E(P)$ exactly in its red edge, and it meets exactly one member of $P\text{.}$ We say that an edge is private in a family of triangles if exactly one member of the family contains it.

Lemma 2.1.

Let $P$ be a maximum packing in $\mathcal{F}(H)\text{,}$ let $uvx\in P$ have red edge $uv\text{,}$ and let $uvy\in\mathcal{F}_{P}\text{.}$ If $xy$ is a red edge outside $R(P)\text{,}$ then $uv$ is private in $\mathcal{F}_{P}\text{.}$

Proof.

Suppose that $uvz\in\mathcal{F}_{P}$ for some $z\neq y\text{.}$ Since $uz\notin B(P)$ while $ux\in B(P)\text{,}$ also $z\neq x\text{.}$ The triangle $uxy$ has blue edges $ux,uy$ and red edge $xy\text{,}$ so it belongs to $\mathcal{F}(H)\text{,}$ and it meets $E(P)$ only in $ux\text{,}$ because $uy\notin B(P)$ and $xy\notin R(P)\text{.}$ The triangle $uvz$ meets $E(P)$ only in $uv\text{.}$ As $z\notin\{x,y\}\text{,}$ the triangles $uxy$ and $uvz$ are edge-disjoint, so replacing $uvx$ by both of them gives a packing in $\mathcal{F}(H)$ larger than $P\text{.}$ ∎

The transversals below are built from books. A book with spine $uv$ is a set of triangles containing the edge $uv\text{.}$ Its members are its pages, and the union of their edge sets is its support. If a book has exactly two pages $uvx$ and $uvy$ and $xy$ is an edge, then $xy$ is the opposite edge of the book.

Lemma 2.2.

Let $\mathcal{B}$ be a set of books with at most three pages each and pairwise edge-disjoint supports, and let $\mathcal{T}$ be a family of triangles such that every selection of one page from each book is a maximal packing in $\mathcal{T}\text{.}$ Let $C$ consist of all edges of the one-page books and the spines of the other books. Then every member of $\mathcal{T}$ avoiding $C$ is $uxy$ or $vxy$ for some two-page book $\{uvx,uvy\}$ in $\mathcal{B}\text{.}$ Consequently, $C$ together with the opposite edges of the two-page books is a transversal of $\mathcal{T}$ of size at most $\sum_{B\in\mathcal{B}}(4-|B|)\text{.}$

Proof.

Let $T\in\mathcal{T}$ avoid $C\text{.}$ If every book had a page edge-disjoint from $T\text{,}$ choosing such a page in each book would give a maximal packing in $\mathcal{T}$ to which $T$ could still be added, a contradiction. So $T$ meets every page of some book $B\text{.}$ Since all edges of a one-page book lie in $C\text{,}$ $B$ has two or three pages, and $T$ avoids its spine $uv\text{,}$ so $T$ contains $uz$ or $vz$ for every page $uvz$ of $B\text{.}$ If $B$ had three pages, these would be three edges of $T$ with distinct ends outside $\{u,v\}\text{,}$ but a triangle has only three vertices. So $B=\{uvx,uvy\}\text{,}$ and $T$ contains one of $ux,vx$ and one of $uy,vy\text{.}$ Two edges of a triangle share a vertex, so these are $ux,uy$ or $vx,vy\text{,}$ and $T$ is $uxy$ or $vxy\text{.}$ Finally, a book with $k$ pages contributes at most $4-k$ edges to the transversal. ∎

Lemma 2.3.

Let $P$ be a maximum packing in $\mathcal{F}(H)$ and $Q$ a maximum packing in $\mathcal{F}_{P}\text{,}$ with $p=|P|$ and $q=|Q|\text{,}$ and let $u$ be the number of members of $P$ whose red edge is private in $\mathcal{F}(H)\text{.}$ Then

$$\begin{aligned} \displaystyle\tau(\mathcal{F}(H)) & \displaystyle\leq\tfrac{8}{3}p-\tfrac{2}{3}u, \\ \displaystyle\tau(\mathcal{F}(H)) & \displaystyle\leq\tfrac{9}{4}p+\tfrac{3}{2}q-\tfrac{1}{4}u. \end{aligned}$$

(1)(2)

Proof.

Let $r$ be the number of members of $Q$ whose red edge is private in $\mathcal{F}_{P}\text{.}$ We build two transversals of $\mathcal{F}\text{,}$ which give

$$\tau(\mathcal{F})\leq 3p-2q+r-u\qquad\text{and}\qquad\tau(\mathcal{F})\leq 2p+3q-r,$$

and then combine these bounds, once directly and once after applying Equation 1 to $\mathcal{F}_{P}\text{.}$

The first transversal comes from books. Each member of $Q$ shares its red edge with exactly one member of $P\text{,}$ and the two form a book with that red edge as spine. Each unpaired member of $P$ forms a one-page book. No member of $P$ is paired twice, as $Q$ is a packing, and the supports are edge-disjoint, as the blue edges of $Q$ avoid $B(P)\text{.}$ Every selection of one page per book is a packing in $\mathcal{F}$ of size $p=\nu(\mathcal{F})\text{,}$ hence maximal. A member of $P$ whose red edge is private in $\mathcal{F}$ is unpaired, since a page of $Q$ on the same spine would be a second member of $\mathcal{F}$ containing that edge.

Let $C$ consist of the $3(p-q)$ edges of the one-page books and the $q$ spines. By Lemma 2.2, a member of $\mathcal{F}$ avoiding $C$ is $uxy$ or $vxy$ for a two-page book $\{uvx,uvy\}$ with $uvx\in P$ and $uvy\in Q\text{.}$ The two edges it shares with the pages are blue, so $xy$ is red, and $xy\notin R(P)$ since $R(P)\subseteq C\text{.}$ By Lemma 2.1, $uv$ is private in $\mathcal{F}_{P}\text{,}$ so $uvy$ is one of the $r$ members of $Q$ with a private red edge. Adding the opposite edges of these at most $r$ books to $C$ therefore gives a transversal of $\mathcal{F}\text{.}$ Finally, delete the red edges of the $u$ unpaired members of $P$ whose red edge is private in $\mathcal{F}\text{:}$ the only member of $\mathcal{F}$ containing such an edge is that member of $P\text{,}$ which is met by its blue edges in $C\text{.}$ The transversal now has at most $3(p-q)+q+r-u$ edges, which is the first bound.

The second transversal starts from $B(P)\cup E(Q)\text{.}$ A member of $\mathcal{F}$ avoiding $B(P)$ lies in $\mathcal{F}_{P}$ and, as $Q$ is maximal there, meets $E(Q)\text{.}$ So $B(P)\cup E(Q)$ is a transversal of $\mathcal{F}\text{.}$ Delete the red edges of the $r$ members of $Q$ whose red edge is private in $\mathcal{F}_{P}\text{:}$ a member of $\mathcal{F}$ avoiding the remaining edges lies in $\mathcal{F}_{P}$ and contains a deleted edge, so it is the corresponding member of $Q\text{,}$ which is met by its blue edges. The remaining $2p+3q-r$ edges give the second bound.

Twice the first bound plus the second gives $3\tau(\mathcal{F})\leq 8p-q+r-2u\leq 8p-2u\text{,}$ as $r\leq q\text{,}$ which is Equation 1. Next, $Q$ is a maximum packing in $\mathcal{F}(H-B(P))=\mathcal{F}_{P}\text{,}$ and $r$ of its members have a red edge that is private in $\mathcal{F}_{P}\text{,}$ so Equation 1 for the graph $H-B(P)$ gives $\tau(\mathcal{F}_{P})\leq\tfrac{8}{3}q-\tfrac{2}{3}r\text{.}$ Since $B(P)$ meets every member of $\mathcal{F}\setminus\mathcal{F}_{P}\text{,}$

$$\tau(\mathcal{F})\leq 2p+\tfrac{8}{3}q-\tfrac{2}{3}r.$$

Three times this bound plus the first bound gives $4\tau(\mathcal{F})\leq 9p+6q-u-r\leq 9p+6q-u\text{,}$ which is Equation 2. ∎

3 The General Bound

Proof of Theorem 1.

We follow Haxell’s construction [Hax99]: the packings below and the first transversal are taken from it, and the other three transversals rest on the lemmas of Section 2. Choose a maximum triangle packing $P$ in $G\text{,}$ and put $n=|P|=\nu(G)$ and $O=E(P)\text{.}$ Call edges in $O$ old and all other edges new. A triangle has type $j$ if it contains exactly $j$ old edges. By maximality of $P$ there are no type-0 triangles.

Choose a maximum packing $A$ of type-1 triangles and put $a=|A|\text{.}$ The first transversal comes from books formed by $P$ and $A\text{.}$ Pair each member of $A$ with the member of $P$ containing its old edge. If two members of $A$ met the same member of $P\text{,}$ replacing it by both would enlarge $P\text{,}$ since a type-1 triangle meets $E(P)$ only in its old edge. These pairs and the remaining members of $P$ form $n$ books with $n+a$ pages. Their supports are edge-disjoint, since the new edges of $A$ avoid $O\text{,}$ and every selection of one page per book is a packing of size $n=\nu(G)\text{,}$ hence maximal. Lemma 2.2 gives a transversal with at most $3(n-a)+2a$ edges, that is,

$$\tau(G)\leq 3n-a.$$

(3)

Put $G^{\prime}=G-E(A)$ and $O^{\prime}=O\setminus E(A)\text{.}$ Every type-1 triangle meets $E(A)$ by maximality of $A\text{,}$ so every triangle of $G^{\prime}$ has type $2$ or $3\text{,}$ and $|O^{\prime}|=3n-a\text{.}$ Each remaining transversal of $G$ consists of $E(A)$ and a transversal of $G^{\prime}\text{.}$ Color the new edges of $G^{\prime}$ red and its old edges blue. Then $\mathcal{F}(G^{\prime})$ is the family $\mathcal{F}_{2}$ of type-2 triangles of $G^{\prime}\text{.}$

Choose a packing $P^{\prime}$ in $G^{\prime}$ that maximizes first its number of type-2 members and then its size $m\text{.}$ Let $B^{*}$ be the set of type-2 members of $P^{\prime}$ and put $b=|B^{*}|\text{.}$ Any packing in $\mathcal{F}_{2}$ is a packing in $G^{\prime}\text{,}$ so $B^{*}$ is a maximum packing in $\mathcal{F}_{2}\text{.}$ The new edges of $P^{\prime}$ are the new edges of the members of $B^{*}\text{,}$ one for each. Let $N$ be their set, so that $R(B^{*})=N$ and $|N|=b\text{.}$ Since $A\cup P^{\prime}$ is a packing in $G\text{,}$ we have $a+m\leq n\text{.}$ The choice of $P^{\prime}$ makes every packing in $G^{\prime}$ with $m$ members, $b$ of them of type $2\text{,}$ maximal in $G^{\prime}\text{:}$ adding a triangle would give a packing with at least $b$ type-2 members and $m+1$ members.

Let $B(P^{\prime})$ be the set of old edges of $P^{\prime}\text{,}$ and put

$$\mathcal{S}=\mathcal{F}(G^{\prime}-B(P^{\prime})),$$

the family of type-2 triangles of $G^{\prime}$ whose old edges avoid $E(P^{\prime})\text{.}$ Since $B(B^{*})\subseteq B(P^{\prime})\text{,}$ we have $\mathcal{S}\subseteq(\mathcal{F}_{2})_{B^{*}}=\mathcal{F}(G^{\prime}-B(B^{*}))\text{,}$ and since $B^{*}$ is a maximum packing in $\mathcal{F}_{2}\text{,}$ the red edge of every member of $\mathcal{S}$ lies in $R(B^{*})=N\text{.}$ Choose a maximum packing $Q$ in $\mathcal{S}\text{,}$ put $c=|Q|\text{,}$ and let $h$ be the number of members of $Q$ whose red edge is private in $\mathcal{S}\text{.}$

The second transversal combines books formed by $B^{*}$ and $Q$ with a random bipartition. Pair each member of $Q$ with the member of $B^{*}$ containing its red edge. No member of $B^{*}$ is paired twice, as $Q$ is a packing. These pairs and the unpaired members of $B^{*}$ form $b$ books, whose supports are edge-disjoint since the old edges of $Q$ avoid $E(P^{\prime})\text{.}$ Every selection of one page per book consists of $b$ type-2 triangles and is therefore a maximum packing in $\mathcal{F}_{2}\text{.}$ Let $W$ be the set of old edges of the one-page books, so $|W|=2(b-c)\text{.}$ The set $C$ of Lemma 2.2 for these books lies in $W\cup N\text{,}$ since spines are edges of $N$ and a one-page book has its old edges in $W$ and its new edge in $N\text{.}$

Assign each vertex of $G^{\prime}$ a bit, independently and uniformly at random, and call an edge crossing if its ends receive different bits. Select $W$ and every non-crossing edge of $O^{\prime}\text{,}$ then add the new edge of each triangle of $G^{\prime}$ not yet met. A type-3 triangle has two vertices with the same bit and hence a non-crossing old edge, so the selected edges meet every triangle of $G^{\prime}\text{.}$ Their expected number is $|W|+\frac{1}{2}(|O^{\prime}|-|W|)$ plus the expected number of added new edges, which we bound in two parts.

A new edge is added only if both old edges of its triangle are crossing, and then it is non-crossing. Each edge of $N$ is non-crossing with probability $\frac{1}{2}\text{,}$ so the expected number of added edges in $N$ is at most $\frac{b}{2}\text{.}$

Now let an edge $xy\notin N$ be added as the new edge of a type-2 triangle $T$ not yet met. Then $T$ contains no edge of $N$ and avoids $W\text{,}$ hence avoids $C\text{,}$ and Lemma 2.2 for these books and $\mathcal{F}_{2}$ shows that $T$ is $uxy$ or $vxy$ for a two-page book $\{uvx,uvy\}$ with $uvx\in B^{*}$ and $uvy\in Q\text{.}$ The two edges that $T$ shares with the pages are old, so its new edge $xy$ is the opposite edge of the book. Since $uvy\in\mathcal{S}\subseteq(\mathcal{F}_{2})_{B^{*}}$ and $xy$ is a red edge outside $R(B^{*})=N\text{,}$ Lemma 2.1 shows that $uv$ is private in $(\mathcal{F}_{2})_{B^{*}}\text{,}$ hence in $\mathcal{S}\text{,}$ so $uvy$ is one of the $h$ members of $Q$ with a private red edge. Such a book contributes an added edge only if $uxy$ or $vxy$ is not yet met, which requires $x$ and $y$ to receive the same bit and $u$ or $v$ a different one. As $u,v,x,y$ are distinct, this has probability $\frac{1}{2}-\frac{1}{8}=\frac{3}{8}\text{.}$ Distinct added edges come from distinct books, as each book has one opposite edge, so the expected number of added edges outside $N$ is at most $\frac{3}{8}h\text{.}$ The expected total is at most

$$|W|+\frac{|O^{\prime}|-|W|}{2}+\frac{b}{2}+\frac{3}{8}h=\frac{|O^{\prime}|}{2}+\frac{3}{2}b-c+\frac{3}{8}h,$$

so some assignment of bits gives a transversal of $G^{\prime}$ of at most this size. Adding $E(A)$ and using $|O^{\prime}|=3n-a$ gives

$$\tau(G)\leq\tfrac{3}{2}n+\tfrac{5}{2}a+\tfrac{3}{2}b-c+\tfrac{3}{8}h.$$

(4)

For the third transversal, we add a third page to some of these books and include the remaining members of $P^{\prime}\text{.}$ Choose a maximum packing $Q_{1}$ in $\mathcal{S}_{Q}=\mathcal{F}(G^{\prime}-B(P^{\prime})-B(Q))$ and put $d=|Q_{1}|\text{.}$ The red edge of each member of $Q_{1}$ lies in $R(Q)\text{,}$ and distinct members of $Q_{1}$ have distinct red edges. Add each member of $Q_{1}$ as a third page to the book whose spine is its red edge, and add the members of $P^{\prime}\setminus B^{*}$ as one-page books. The old edges of $Q_{1}$ avoid $E(P^{\prime})\cup E(Q)\text{,}$ so the supports remain edge-disjoint. Every selection of one page per book is a packing in $G^{\prime}$ with $m$ members, $b$ of them of type $2\text{,}$ hence maximal in $G^{\prime}\text{.}$ These $m$ books have $m+c+d$ pages, so Lemma 2.2 gives a transversal of $G^{\prime}$ of size at most $3m-c-d\text{.}$ Adding $E(A)$ and using $a+m\leq n$ gives

$$\tau(G)\leq 3n-c-d.$$

(5)

The fourth transversal consists of $E(A)\text{,}$ the old edges of $P^{\prime}\text{,}$ and a transversal of $\mathcal{S}\text{.}$ The first two sets have at most $3a+3m-b\leq 3n-b$ edges. A triangle avoiding them lies in $G^{\prime}-B(P^{\prime})$ and is not of type $3\text{,}$ since a type-3 triangle avoiding $B(P^{\prime})$ would avoid $E(P^{\prime})$ and could be added to $P^{\prime}\text{.}$ So it belongs to $\mathcal{S}\text{.}$ Lemma 2.3 applied to $G^{\prime}-B(P^{\prime})\text{,}$ with $Q\text{,}$ $Q_{1}\text{,}$ $c\text{,}$ $d\text{,}$ $h$ in the roles of $P\text{,}$ $Q\text{,}$ $p\text{,}$ $q\text{,}$ $u\text{,}$ gives $\tau(\mathcal{S})\leq\frac{9}{4}c+\frac{3}{2}d-\frac{1}{4}h$ by Equation 2, so

$$\tau(G)\leq 3n-b+\tfrac{9}{4}c+\tfrac{3}{2}d-\tfrac{1}{4}h.$$

(6)

Multiply Equations 3, 4, 5 and 6 by $20,8,19,12\text{,}$ respectively, and add. The coefficients of $a,b,c,h$ cancel, giving

$$59\tau(G)\leq 165n-d\leq 165\nu(G).$$

Remark.

The weights are best possible for the system Equations 3, 4, 5 and 6. For $a=c=12n/59\text{,}$ $b=m=39n/59\text{,}$ and $d=h=0\text{,}$ which satisfy the relations $a+m\leq n\text{,}$ $d\leq c\leq b\leq m\text{,}$ and $h\leq c$ that hold in the proof, all four right-hand sides equal $165n/59\text{.}$ So no nonnegative combination of the four inequalities gives a smaller constant.

References

[Hax99] P. E. Haxell, Packing and covering triangles in graphs, Discrete Mathematics 195 (1999), 251–254. doi:10.1016/S0012-365X(98)00183-6.

[Tuz90] Z. Tuza, A conjecture on triangles of graphs, Graphs and Combinatorics 6 (1990), 373–380. doi:10.1007/BF01787705.

[Yi26] L. Yi, An improved upper bound for Tuza’s conjecture via 2-colorable triangle families, arXiv:2608.23010v1 (2026).

The Minimax Rate of Denoising Flows on Trees

Sat, 12 Sep 2026 00:00:00 +0000

1 Introduction

Two classical problems anchor Gaussian denoising under geometric constraints. Isotonic regression, the estimation of a monotone sequence, obeys the cube-root phase diagram of shape-constrained estimation (Zhang, 2002; Chatterjee et al., 2015). Mean estimation over a scaled simplex, the nonnegative part of an $\ell_{1}$ ball, obeys the $\sqrt{\log}$ diagram of sparse estimation (Donoho and Johnstone, 1994; Birgé and Massart, 2001). The two diagrams, each carrying decades of theory, are unlike in kind: the monotone problem pays a power of its depth, the sparse problem a logarithm of its width. Between the sequence and the simplex lies every geometry in which depth and branching interact, and there the picture has remained incomplete.

One model generalizes both. A tree flow (Chatterjee and Lafferty, 2018) assigns to the vertices of a rooted tree nonnegative values such that each vertex passes to its children at most what it receives: a mass enters at the root, travels downward, and may leak at any vertex. A path, depth without branching, is bounded isotonic regression; a star, branching without depth, is mean estimation over the scaled simplex. The same inequalities arise wherever hierarchical totals are measured with noise. In the consistency post-processing of differentially private counts they hold with equality (Hay et al., 2010; Abowd et al., 2022); in profiling a call tree the leak carries real mass, the self time of a procedure (Graham et al., 1982). Chatterjee and Lafferty studied the least squares estimator, the projection onto the flow cone and the natural tuning-free program for the model, and discovered that it behaves unlike its isotonic counterpart: its rate of convergence is not monotone in the depth of the tree, and it can miss the minimax rate by a power of $n\text{.}$ Where the projection fell short, the minimax rate was attained only by least squares over an exponentially large net, and their closing sentence poses the challenge of “closing the gap between the LSE and the minimax lower bound” with an efficient estimator. A decade later the challenge stood, and with it three gaps: the minimax risk was uncharacterized beyond two regimes of trees, the worst case of least squares at a fixed budget was unquantified, and no efficient estimator was known to attain the minimax rate where the projection did not.

This paper closes all three. We ask: for every finite rooted tree and every budget and noise level, what is the minimax risk of denoising a tree flow, and is it attained by a computationally efficient estimator? One combinatorial functional of the tree organizes the answers, stated in Section 1.1: a rate formula; an exact and efficient estimator; adaptation to the budget and, in an explicit regime, to the noise level; and the worst-case price of least squares. In the local entropy program for convex constraints (Neykov, 2023), the flow polytopes thereby form a nonsymmetric, combinatorially explicit family.

1.1 Our Results

Let $T$ be a finite rooted tree with vertex set $\mathsf{V}\text{,}$ root $o\text{,}$ and $n:=|\mathsf{V}|$ vertices. Write $u\preceq v\text{,}$ and call $u$ an ancestor of $v\text{,}$ when $u$ lies on the path from $o$ to $v\text{,}$ so that every vertex is an ancestor of itself; let $d_{T}$ denote graph distance, and let $H_{T}:=\max_{u,w\in\mathsf{V}}d_{T}(u,w)$ denote the diameter of $T\text{.}$ Each vertex $u$ carries its root-path indicator $p_{u}\in\{0,1\}^{\mathsf{V}}\text{,}$ defined by $p_{u}(v)=\mathbf{1}\{v\preceq u\}\text{.}$ The flow cone of $T$ is

$$\mathcal{F}(T):=\Bigl\{\textstyle\sum_{u\in\mathsf{V}}s_{u}p_{u}:\ s_{u}\geq 0\text{ for all }u\Bigr\},$$

(1.1)

whose elements are exactly the monotone flows on $T$ (Lemma 2.1). Fix a budget $V>0$ and a noise level $\sigma>0\text{.}$ The parameter set of this paper is the budget-$V$ slice of the cone, the flow polytope

$$\mathcal{F}_{V}(T):=\bigl\{\mu\in\mathcal{F}(T):\ \mu(o)=V\bigr\}\ =\ V\operatorname{conv}\{p_{u}:u\in\mathsf{V}\}.$$

(1.2)

The cone Equation 1.1 is the flow cone of Chatterjee and Lafferty (2018); they cap the root value, $\mu(o)\leq V\text{,}$ where we fix it. For $n\geq 2$ the two constraints carry the same minimax risk up to universal constants: the capped body contains the slice, raising the root value of a capped signal to $V$ changes no other coordinate, and the cap therefore adds only the bounded scalar $\mu(o)\text{,}$ at a cost $\min\{V^{2},\sigma^{2}\}$ that the two-point bound Equation 3.8 absorbs; the estimator of Theorem 2 transfers (Section S.1). We observe $Y=\mu+\sigma Z$ with $Z\sim N(0,I_{n})$ and $\mu\in\mathcal{F}_{V}(T)\text{,}$ and study the minimax risk

$$R^{*}_{T}(V,\sigma):=\inf_{\widehat{\mu}}\ \sup_{\mu\in\mathcal{F}_{V}(T)}\mathbb{E}_{\mu}\bigl\|\widehat{\mu}(Y)-\mu\bigr\|_{2}^{2},$$

(1.3)

the infimum running over all measurable maps $\widehat{\mu}:\mathbb{R}^{\mathsf{V}}\to\mathbb{R}^{\mathsf{V}}\text{;}$ estimators are not required to take values in $\mathcal{F}_{V}(T)\text{.}$

An ancestor $r$-net is a set $R\subseteq\mathsf{V}$ such that every vertex $u$ has an ancestor $a\in R$ with $d_{T}(a,u)\leq r\text{,}$ and $N^{\uparrow}_{T}(r)$ denotes the minimum cardinality of an ancestor $r$-net. The truncated ancestor profile of $T$ is

$$\alpha_{k}(T):=\sup_{r>0}\ r\,\min\Bigl\{k,\ \Bigl[\log\frac{N^{\uparrow}_{T}(r)}{k}\Bigr]_{+}\Bigr\},\qquad\delta_{k}(T):=\frac{\alpha_{k}(T)}{k}\qquad(k=1,2,\dots),$$

(1.4)

where $[x]_{+}:=\max\{x,0\}\text{,}$ and the crossing index is

$$k_{0}=k_{0}(T,V,\sigma):=\min\Bigl\{k\geq 1:\ V^{2}\delta_{k}(T)\leq A_{0}\sigma^{2}k\ \text{ or }\ k>n/C_{\star}\Bigr\},$$

(1.5)

where $A_{0}$ and $C_{\star}$ are fixed universal constants; one may take $A_{0}=144$ and $C_{\star}=e^{2}\text{.}$ The truncation is the one nonstandard feature: once describing the branching below a vertex costs more than the information budget $k\text{,}$ the cluster is blurred at its own diameter rather than charged per descendant (Remark 2.1).

The crossing index determines the minimax risk: Theorem 1, proved in Section 3, shows that

$$R^{*}_{T}(V,\sigma)\ \asymp\ \min\bigl\{V^{2}H_{T},\ \sigma^{2}k_{0}(T,V,\sigma)\bigr\}$$

for every finite rooted tree and all $V,\sigma>0\text{.}$ The minimum has three readings: a diameter branch $V^{2}H_{T}\text{,}$ where the body is small and the constant estimator $Vp_{o}$ is already optimal; an information branch $\sigma^{2}k_{0}\text{,}$ the cost of $k_{0}$ effective coordinates; and, when the crossing clause never fires, a dimension branch $\sigma^{2}k_{0}\asymp\sigma^{2}n\text{,}$ where the identity estimator is optimal. The formula is computable: a surrogate index evaluates it to within a factor of two in $O(n\log(n+1))$ operations (Lemma 2.3).

Behind the formula stands the entropy theory of Gaussian estimation, from global metric entropy to the local entropy fixed point of bounded convex constraints (Yang and Barron, 1999; Neykov, 2023). That theory identifies the governing quantity; computing it on a given body, and constructing the packings it promises when the body is an asymmetric cone, are the separate problems solved in Section 3. For $t>0$ let $H^{c}_{T}(t)$ denote the metric entropy of the flow polytope at squared radius $t\text{:}$ the logarithm of the smallest number of Euclidean balls of radius $\sqrt{t}\text{,}$ centered anywhere, whose union contains $\mathcal{F}_{V}(T)\text{.}$ Define the associated fixed point

$$\rho_{T}(V,\sigma)\ :=\ \inf_{t>0}\ \bigl[t+\sigma^{2}H^{c}_{T}(t)\bigr].$$

(1.6)

Free centers are the convenient convention, because the covers of Section 3 have centers outside the body; restricting centers to the body leaves Equation 1.6 unchanged (Section S.1). Proposition 1 gives $R^{*}_{T}(V,\sigma)\asymp\min\{\rho_{T}(V,\sigma),\ V^{2}H_{T},\ \sigma^{2}n\}\text{;}$ Theorem 1 thus evaluates this entropy expression in closed combinatorial form.

Table 1: The rate formula on three families (Corollaries 1, 2 and 3), with $n$ the number of vertices and universal constants in $\asymp\text{;}$ the path $P_{L}$ has $L$ edges, the star $S_{m}$ has $m$ leaves, and the complete binary tree $B_{h}$ has height $h\text{.}$ The first two rows recover the classical phase diagrams; the third carries a full logarithm that occurs at neither endpoint.

Tree	$R^{*}\asymp$	Classical counterpart	Phenomenon
path $P_{L}$	$\min\bigl\{V^{2}L,\ V^{2/3}\sigma^{4/3}L^{1/3},\ \sigma^{2}(L{+}1)\bigr\}$	bounded isotonic	cube root
star $S_{m}$	$\min\bigl\{V^{2},\ \sigma V\sqrt{1+[\log(m\sigma/V)]_{+}},\ \sigma^{2}m\bigr\}$	simplex mean	$\sqrt{\log}$
binary $B_{h}$	$\min\bigl\{V^{2}h,\ \sigma V\bigl(1+[\log(n\sigma/V)]_{+}\bigr),\ \sigma^{2}n\bigr\}$	no direct analogue	full logarithm

On three benchmark families the formula specializes in closed form; Table 1 collects the three rates, proved as Corollaries 1, 2 and 3 in Section 3.3. The first two are external checks: the path recovers the bounded isotonic diagram (Zhang, 2002; Chatterjee and Lafferty, 2018); the star recovers the simplex diagram of sparse estimation (Donoho and Johnstone, 1994; Birgé and Massart, 2001). The complete binary tree resembles neither: branching entropy repeats across $\Theta(\log n)$ depth scales, making the profile $\alpha_{k}$ quadratic in $\log(n/k)$ once $k$ exceeds that logarithm, and the crossing takes a square root, leaving the single full logarithm.

An exactly evaluable estimator attains the rate. Theorem 2, proved in Section 4, constructs a deterministic estimator with worst-case risk $O(R^{*}_{T}(V,\sigma))$ on every tree, computed exactly in $O(n^{2}\log(n+1))$ arithmetic operations: an exponentially weighted aggregate over a class of integer states. Together with Theorem 5, it answers the challenge of Chatterjee and Lafferty (2018): an efficient estimator closes the gap between least squares and the minimax risk.

The estimator of Theorem 2 takes $V$ and $\sigma$ as inputs; the next two theorems remove them. Theorem 3, proved in Section 4.3, removes the budget: a deterministic polynomial-time estimator free of $V$ attains $C(R^{*}_{T}(V,\sigma)+\sigma^{2})\text{,}$ and no budget-free estimator stays below an $\Omega(\sigma^{2})$ floor on adjacent budgets. On the cone, where the original model carries no budget parameter, this adaptation is the return to the model as Chatterjee and Lafferty (2018) posed it. Theorem 4 removes the noise level as well, in an explicit regime $Vw_{T}\leq c\sigma n\text{,}$ with $w_{T}$ a width functional that bounds how much of the tree’s difference structure the signal can contaminate, equal to $1$ on a path and $\Theta(\log n)$ on a balanced binary tree. Within the regime the noise dominates a majority of the tree’s difference statistics and a median estimates $\sigma\text{;}$ on families of bounded width, the regime sits within a $\sqrt{\log n}$ factor of a ceiling that no noise estimator can pass (Section S.6).

Every estimator above is built from the profile. The default estimator ignores it: the Euclidean projection onto the body, the tuning-free convex program of least squares at a known budget. Theorem 5, proved in Section 5, determines its worst case: on every tree and at all $V,\sigma>0$ the projection loses at most a factor $C(1+\log(en))^{2/5}$ over the minimax risk, and on an explicit one-parameter family of brooms, at a single budget and noise level, it loses at least $c(\log n)^{2/5}\text{.}$ The separation is an exact power of the logarithm, matched from both sides, where the separations of Chatterjee (2014); Kur et al. (2024) are polynomial in the sample size or the dimension. The equivalence of the fixed and capped budgets recorded above concerns minimax risks; Theorem 5 concerns the projection itself.

1.2 Technical Overview

The upper bound in Theorem 1 is soft: least squares over a covering net attains the entropy fixed point Equation 1.6, and two trivial estimators cap the risk at $V^{2}H_{T}$ and $\sigma^{2}n\text{.}$ Three tasks remain. The entropy must be realized, by hypotheses legal in a positive body (the lower bound); searched, exactly and quickly (the algorithm); and measured against the natural convex program (the least squares question). Two features of the flow polytope obstruct the classical routes to all three: positivity, the body being a simplex rather than a symmetric ball, so that a hypothesis moves only the leak mass it holds and sign-toggling lower bounds, whose one base point would have to fund every signed perturbation, fail; and overlap, root paths through a common vertex sharing coordinates, so that the contrast blocks realizing the entropy can overlap and are therefore priced (Section 3.2).

The lower bound realizes the entropy inside the body. Ancestor-covering numbers force a separated vertex set (Lemma 3.2); groups drawn from it, in subtrees with disjoint edge sets, share the budget exactly; a multiscale code fills each group; and a product form of Fano’s inequality (Lemma 3.3) prices the overlap between scales once, through a normalized Gram row sum, returning $R^{*}_{T}\geq c\min\{V^{2}r/q,\ \sigma^{2}q\log(eM/q)\}$ for $M$ vertices at pairwise distance $r$ in $q$ groups; matching $q$ to the profile recovers its truncated logarithm. Where Chatterjee and Lafferty (2018) allocate mass across vertex-disjoint paths, the same budget discipline runs on arbitrarily overlapping routes.

The upper bound builds one coded cover for all signals. Dyadic ancestor nets and their cells depend only on the tree and the information budget $k\text{;}$ for each signal, heavy cells split and light cells collapse to their roots, sparse sampling restoring the light exits; every subtree is an interval of the depth-first preorder, so one scalar, the running floor of cumulative masses along it, rounds all subtree sums at once. The resulting approximant is within squared error $O(V^{2}\delta_{k})$ of its signal and carries a fixed root sum, a code length $O(k)\text{,}$ and flow states confined to an $O(k)$ range; counting the class at $e^{O(k)}$ gives the entropy bound Equation 3.2. An integer-leak net already appears in the upper bound of Chatterjee and Lafferty (2018), at cost $\log n$ per unit of mass and at the diameter scale; the activation charges lower this to $O(1)$ per unit and the profile scale, and these reductions are what the estimator consumes.

The two halves meet at the crossing. In the interior case the nonincreasing profile scale $V^{2}\delta_{k}$ first falls below $A_{0}\sigma^{2}k$ at $k_{0}\text{;}$ the cover gives the rate $\sigma^{2}k_{0}$ there, the packing gives $\sigma^{2}(k_{0}-1)$ one step earlier, and the corner regimes are collected by a two-point bound and the dimension cap. This proves Theorem 1 and Proposition 1 as four cases of one argument.

The estimator replaces the search over the cover by an average, an exponentially weighted aggregate under a Kraft-type prior built from the code length. Its risk analysis composes two exact identities, Stein’s and the Gibbs variational identity, with no union bound and approximation constant one (Lemma 4.1). The confinement is what makes the computation possible: one integer state in $[0,3k]$ per vertex turns the partition function into a tree recursion of polynomial messages, evaluated exactly; one reverse-mode sweep then reads off the posterior mean at every coordinate. The recursion is dynamic programming over distributions, not values.

Adaptation reuses the cover. Removing the quantization from the approximant of Proposition 3.1 leaves amplitude-free profile subspaces; a Kraft-weighted penalized selection over them, joined with finite grids of candidate budgets, attains $C(R^{*}_{T}+\sigma^{2})$ without knowing $V\text{,}$ the additive $\sigma^{2}$ forced on adjacent budgets by a two-point test at the root. For unknown $\sigma\text{,}$ a median of tree difference statistics survives contamination when $Vw_{T}\leq c\sigma n\text{,}$ and the selection runs at the rounded estimate.

The broom of Theorem 5, a handle of $L$ edges feeding $e^{\Theta(L^{5/2})}$ terminal leaves, makes the blindness of the projection quantitative: the extreme leaf noise repays the quadratic cost of the full-budget route through the handle, forcing least squares to lose $L^{3/2}\text{,}$ while the profile sees one ancestor covering the terminal star at every radius $r\geq 1$ and the minimax risk stays at order $\sqrt{L}\text{,}$ that of the handle alone; with $\log n\asymp L^{5/2}\text{,}$ the ratio is the $2/5$ power of the logarithm. For the matching upper bound, the localized-width analysis of Chatterjee (2014) runs on the coded cover with centers chosen after the noise, and a width branch and a height branch meet at the same power.

Three devices are stated for reuse: the overlapping-block Fano inequality (Lemma 3.3), which refers to no tree and prices overlap inside an arbitrary convex constraint; the positive packing of Section 3.2, a template for bodies in which sign toggling is illegal; and the aggregate of Section 4, whose oracle inequality holds over any finite class and whose evaluation is exact whenever the class is dynamically programmable. Sections 2, 3, 4 and 5 prove the results in turn; the remaining details are in the appendices.

The closest work is Chatterjee and Lafferty (2018). They introduced tree flows and analyzed the projection onto the flow cone in two regimes: for trees of bounded depth they proved matching risk bounds in the principal range of budgets, and on trees consisting of many long disjoint paths they determined the exponents of both the projection and the minimax risk, which disagree. Their projection takes no budget input, and over part of that family its risk exceeds the minimax risk polynomially in $n$ at a fixed budget, by overfitting the zero signal where the cone has large statistical dimension; the projection of Theorem 5 receives the budget, which removes that failure and leaves exactly the $2/5$ power of the logarithm. The constraint family itself predates the statistics: Benabbas et al. (2011) smooth hierarchical data under the same inequalities in $\ell_{1}\text{.}$ Isotonic regression on richer orders (Han et al., 2019) shares the ambient structure but not the geometry: it acts on differences, while the flow constraint transports mass, thereby making the body a simplex. On the path the model meets the isotonic tradition (Zhang, 2002; Chatterjee et al., 2015; Bellec, 2018; Guntuboyina and Sen, 2018), in its bounded form (Luss and Rosset, 2017), and Corollary 1 returns its phase diagram. The obstacle on trees is therefore the budget slice, the simplex geometry, not monotonicity.

Metric entropy has determined minimax rates since Yang and Barron (1999) tied them to the global entropy of the function class; for bounded convex constraints in the Gaussian sequence model, Neykov (2023) gives the exact rate through a local entropy fixed point. On the flow polytopes the two halves of that program take concrete form: the profile computes the rate that the fixed point governs, in closed combinatorial form and in near-linear time, and the packing of Section 3.2 realizes the entropy inside a cone with no symmetry, one-sided perturbations replacing sign toggling; compare the cone geometry of testing in Wei et al. (2019). On the star, positivity leaves the sparse rate unchanged; what it changes is the lower bound, which must be built one-sided. The covering geometry has an operator-theoretic precedent: the body is $V$ times the image of the positive face of the $\ell_{1}$ ball under the tree’s summation operator, and Lifshits and Linde (2011a); Lifshits and Linde (2011b) bound the entropy numbers of such operators through ancestor covering numbers; the critical case stops a heavy–light partition at its light sets and counts the outcomes, the combinatorial core that Section 3.1 shares. Compactness of the symmetric image is their question; the positive slice, the minimax risk, and the certificates behind the estimator are this paper’s.

That the projection onto a convex body can miss the minimax rate by a large factor was shown by Zhang (2013) on an ellipsoid and by Chatterjee (2014) on a designed body, in both cases by a factor as large as $\sqrt{n}\text{;}$ the fixed-point analysis of least squares in the latter is the engine of our upper bound in Section 5. Separations of polynomial order in the sample size or the dimension are known for several natural programs (Kur et al., 2024; Vaškevičius and Zhivotovskiy, 2023), and on an $\ell_{p}$ ball Aolaritei et al. (2025) separate the projection from the minimax rate by a power of the logarithm. Prasadan and Neykov (2025) characterize the boundary exactly: least squares is minimax optimal if and only if the local Gaussian width map is Lipschitz at the relevant scales. By that characterization, Theorem 5 shows the Lipschitz bound cannot hold uniformly along the brooms, and it goes past the optimal-or-not dichotomy by bounding the worst-case ratio of least squares to the minimax rate from both sides. The upper bound speaks for the projection as much as against it: a loss beyond the $2/5$ power of the logarithm is impossible on any tree.

The estimator of Theorem 2 belongs to the aggregation tradition. Leung and Barron (2006) proved an exact Stein-based oracle inequality for exponentially weighted aggregates, and Lemma 4.1 descends from it; sharp PAC-Bayes bounds, optimal sparse aggregation, and unknown-variance mixing follow in Dalalyan and Tsybakov (2008); Rigollet and Tsybakov (2011); Giraud (2008). Exact mixtures over combinatorial families have a computing tradition of their own: context-tree weighting (Willems et al., 1995) and circuit differentiation for marginals (Darwiche, 2003), without squared-loss risk guarantees. The class of integer states sits in both traditions: its code length ties the Kraft prior, in the tradition of minimum description length (Barron and Cover, 1991), to the minimax rate, and its confined states make the posterior mean a tree recursion, evaluated exactly. Shape-constrained solvers compute hard projections (Robertson et al., 1988; Kyng et al., 2015), the object that Theorem 5 measures; the recursion here averages where those optimize. A recent line attains polynomial-time near-optimality on symmetric bodies under further regularity and oracle hypotheses (Neykov, 2026); the flow polytope is a simplex. The trade is structure for exactness: a combinatorial family of bodies, in exchange for the exact rate and the exact estimator.

2 The Flow Polytope and the Ancestor Profile

This section records the interfaces used by the rest of the paper: the two normal forms of the flow polytope, the Euclidean realization of the tree metric, and the working properties of the ancestor profile, including the surrogate index through which the profile is computed. Proofs are collected in Section S.1.

2.1 The Flow Polytope

We complete the tree notation of Section 1.1. For $v\neq o$ we write $\operatorname{pa}(v)$ for the parent of $v$ and identify $v$ with its parent edge $(\operatorname{pa}(v),v)\text{,}$ of which $v$ is the lower endpoint; $\operatorname{ch}(v)$ is the set of children of $v\text{,}$ and $T_{v}:=\{u\in\mathsf{V}:v\preceq u\}$ is the subtree rooted at $v\text{.}$ The depth of a vertex is its distance from the root, and the height of $T$ is $h_{T}:=\max_{v\in\mathsf{V}}\operatorname{depth}(v)\text{,}$ so that $h_{T}\leq H_{T}\leq 2h_{T}\text{.}$ Universal constants $c,C,c^{\prime},C^{\prime},\dots$ may change value from occurrence to occurrence, while subscripted constants such as $A_{0}$ and $C_{\star}$ keep their values fixed once introduced. Logarithms are natural. Complexity statements count arithmetic operations in the real-RAM model: arithmetic, comparisons, integer part, and evaluations of $\exp$ and $\log$ on real registers are exact unit-cost operations, and two polynomials of degree $m$ can be multiplied in $O(m\log m)$ operations by the fast Fourier transform (von zur Gathen and Gerhard, 2013); what exactness means for the estimator is stated in Remark 4.1.

The polytope has a barycentric normal form and a flow normal form: the first is the legality certificate behind every lower-bound construction of Section 3, the second identifies the body as a set of flows and explains its name.

Lemma 2.1 (Normal forms).

For $\mu\in\mathbb{R}^{\mathsf{V}}$ the following are equivalent:

1.

$\mu\in\mathcal{F}_{V}(T)$;
2.

$\mu=\sum_{u\in\mathsf{V}}s_{u}p_{u}$ with $s_{u}\geq 0$ for all $u$ and $\sum_{u\in\mathsf{V}}s_{u}=V$;
3.

$\mu(o)=V$ and $\mu(v)\geq\sum_{c\in\operatorname{ch}(v)}\mu(c)$ for every $v\in\mathsf{V}$.

The coefficients in (2) are unique, and they are tied to the coordinates by the subtree sums

$$\mu(v)\ =\ \sum_{u\succeq v}s_{u},\qquad s_{v}\ =\ \mu(v)-\sum_{c\in\operatorname{ch}(v)}\mu(c)\qquad(v\in\mathsf{V}).$$

Form (3) exhibits $\mathcal{F}_{V}(T)$ as the set of monotone flows at budget $V\text{,}$ with $s_{v}$ the leak at $v\text{,}$ the mass that stops there (Chatterjee and Lafferty, 2018). In a linear order of $\mathsf{V}$ that places ancestors before descendants, the matrix with columns $(p_{u})_{u\in\mathsf{V}}$ is upper triangular with unit diagonal (Section S.1), so the $p_{u}$ are linearly independent and the body is a nondegenerate $(n-1)$-dimensional simplex with vertex set $\{Vp_{u}:u\in\mathsf{V}\}\text{.}$ Finally, $\mu(o)=V$ identically on the body: the root coordinate carries no statistical content, and all estimation error lives on the nonroot coordinates.

The second interface converts tree combinatorics into Euclidean geometry: squared Euclidean distance between root-path indicators is tree distance, which is why the ancestor profile, defined by the tree metric, controls Euclidean covering and packing of the body.

Lemma 2.2 (Isometry and edge supports).

1.

$\|p_{u}-p_{w}\|_{2}^{2}=d_{T}(u,w)$ for all $u,w\in\mathsf{V}$; consequently $\operatorname{diam}^{2}\mathcal{F}_{V}(T)=V^{2}H_{T}$.
2.

For $a\preceq b$, $p_{b}-p_{a}=\mathbf{1}_{(a,b]}$, the indicator of $\{v:a\prec v\preceq b\}$; in general, the support of $p_{u}-p_{w}$ is exactly the set of lower endpoints of the edges on the path from $u$ to $w$. In particular, differences $p_{u}-p_{w}$ taken within edge-disjoint connected subtrees have disjoint supports.

Behind (1) is the ancestor count $\langle p_{u},p_{w}\rangle=\operatorname{depth}(u\wedge w)+1\text{,}$ where $u\wedge w$ denotes the deepest common ancestor of $u$ and $w\text{.}$

2.2 The Ancestor Profile

The profile $\alpha_{k},\delta_{k}$ and the crossing index $k_{0}$ were defined in Equations 1.4 and 1.5; this subsection collects the properties used below. A star shows why the truncation is there.

Remark 2.1 (Why the profile is truncated).

On the star with $m$ leaves rooted at the hub, a leaf is ancestor-covered only by itself or by the hub, so $N^{\uparrow}_{T}(r)=m+1$ for $0\leq r<1$ and $N^{\uparrow}_{T}(r)=1$ for $r\geq 1\text{;}$ the expression in Equation 1.4 evaluates exactly to

$$\alpha_{k}\ =\ \min\Bigl\{k,\ \Bigl[\log\frac{m+1}{k}\Bigr]_{+}\Bigr\}.$$

Without the truncation, a star with $\lceil e^{2k^{2}}\rceil$ leaves would be assigned $\alpha_{k}\geq k^{2}\text{,}$ hence $V^{2}\alpha_{k}/k\geq A_{0}\sigma^{2}k$ at $V^{2}=A_{0}\sigma^{2}\text{,}$ and the profile-to-risk half of Section 3 would assert a minimax risk of order at least $\sigma^{2}k\text{;}$ for large $k$ this contradicts the bound $R^{*}_{T}\leq V^{2}H_{T}=2A_{0}\sigma^{2}$ furnished by the constant estimator $Vp_{o}\text{,}$ since the star has $H_{T}=2\text{.}$

The root covers every vertex at radius $h_{T}\text{,}$ so $N^{\uparrow}_{T}(r)=1$ for $r\geq h_{T}$ and the supremum in Equation 1.4 is effectively over $r<h_{T}\text{.}$ Evaluating the profile exactly involves every integer radius below $h_{T}\text{;}$ a factor-of-two surrogate needs only geometrically spaced ones, and it is the surrogate that the estimator of Section 4 consumes. Define

$$\overline{\delta}_{k}(T):=\frac{2}{k}\,\max_{\begin{subarray}{c}q=2^{\ell}-1,\ \ell\in\{0,1,2,\dots\}\\ 2^{\ell}\leq h_{T}\end{subarray}}\ (q+1)\,\min\Bigl\{k,\ \Bigl[\log\frac{N^{\uparrow}_{T}(q)}{k}\Bigr]_{+}\Bigr\},$$

(2.1)

with $\overline{\delta}_{k}(T):=0$ if $h_{T}=0\text{,}$ and the algorithmic crossing index

$$k_{\mathrm{alg}}=k_{\mathrm{alg}}(T,V,\sigma):=\min\Bigl\{k\geq 1:\ V^{2}\overline{\delta}_{k}(T)\leq 2A_{0}\sigma^{2}k\ \text{ or }\ k>n/C_{\star}\Bigr\}.$$

(2.2)

Lemma 2.3 (Profile toolkit).

For every finite rooted tree $T\text{:}$

1.

$\delta_{k+1}(T)\leq\delta_{k}(T)\leq h_{T}$ for every $k\geq 1$;
2.

for $n\geq 2$ and every $k\geq 1$,

$$\alpha_{k}(T)=\max_{0\leq q<h_{T}}\ (q+1)\,\min\Bigl\{k,\ \Bigl[\log\frac{N^{\uparrow}_{T}(q)}{k}\Bigr]_{+}\Bigr\},$$ (2.3)

the maximum running over integer radii, and $\alpha_{k}(T)=0$ when $n=1$;
3.

$\delta_{k}\leq\overline{\delta}_{k}\leq 2\delta_{k}$ for every $k\geq 1$, the map $k\mapsto\overline{\delta}_{k}(T)$ is nonincreasing, and

$$k_{\mathrm{alg}}\ \leq\ k_{0}\ \leq\ 2\,k_{\mathrm{alg}}.$$

Moreover, a minimum ancestor $q$-net can be computed exactly in $O(n)$ operations for each integer $q\geq 0$, by a one-pass residual-depth greedy; consequently $k_{\mathrm{alg}}$ is computable in $O(n\log(n+1))$ operations, and $k_{0}$ in $O(n\,h_{T})$ operations.

The greedy selects a center $q$ levels above a deepest uncovered vertex, or the root when fewer remain; its exactness and the remaining proofs are in Section S.1.

Section 3 now converts the profile into the minimax rate: an upper bound through a coded cover, and a matching lower bound through a positive packing.

3 The Rate Formula

The main theorem of the paper is the rate formula.

Theorem 1 (Rate formula).

There are universal constants $c,C>0$ such that for every finite rooted tree $T$ and all $V,\sigma>0\text{,}$

$$c\,R^{*}_{T}(V,\sigma)\ \leq\ \min\bigl\{V^{2}H_{T},\ \sigma^{2}k_{0}(T,V,\sigma)\bigr\}\ \leq\ C\,R^{*}_{T}(V,\sigma).$$

The formula is insensitive to the choice of $(A_{0},C_{\star})$ in Equation 1.5: any pair for which Theorem 1 holds yields a formula $\asymp R^{*}_{T}\text{.}$ The entropy reading is the following.

Proposition 1 (Entropy fixed point).

For every finite rooted tree $T$ and all $V,\sigma>0\text{,}$

$$R^{*}_{T}(V,\sigma)\ \asymp\ \min\bigl\{\rho_{T}(V,\sigma),\ V^{2}H_{T},\ \sigma^{2}n\bigr\}$$

with universal constants.

Section 3.1 converts the profile into a coded cover of the flow polytope, yielding the upper bounds and the certificates that the estimators of Section 4 later consume; Section 3.2 converts the same profile into legal packings and the matching lower bounds; and Section 3.3 joins the two at the crossing index, proves both statements above, and derives the benchmark corollaries. The assembly is given here; the proofs of the two halves are in Sections S.2 and S.3.

3.1 The Coded Cover

Fix an integer $2\leq k\leq n/C_{\star}\text{.}$ The construction runs at the working scale

$$\bar{\alpha}\ :=\ k\,\overline{\delta}_{k}(T),$$

built from the surrogate Equation 2.1; the centers, and ultimately the estimator, owe their computability to this one choice. Everything rests on a fixed hierarchy of nets. Set $m_{j}:=2^{j}$ for $0\leq j\leq J\text{,}$ where $m_{J}$ is the largest power of two strictly below $k\text{,}$ let $S_{j}$ be the minimum ancestor $\lfloor\bar{\alpha}/m_{j}\rfloor$-net computed by the residual-depth greedy of Lemma 2.3(3), and put $R_{j}:=S_{0}\cup\dots\cup S_{j}\text{,}$ so that $R_{0}\subseteq\dots\subseteq R_{J}$ and each $R_{j}$ is an ancestor $(\bar{\alpha}/m_{j})$-net containing the root. The nets depend only on $(T,k)\text{,}$ not on any signal; every counting statement below rests on this. Their sizes are controlled by the profile itself: $N^{\uparrow}_{T}(\lfloor\bar{\alpha}/m_{j}\rfloor)\leq ke^{m_{j}}\text{,}$ as $m_{j}<k$ and a larger count would make the term of Equation 1.4 at that radius exceed $\bar{\alpha}\geq\alpha_{k}(T)\text{;}$ consequently $|R_{j}|\leq 2ke^{m_{j}}\text{.}$

For $v\in\mathsf{V}$ let $a_{j}(v)$ be the deepest ancestor of $v$ in $R_{j}\text{,}$ and call the fibers $B_{a,j}:=\{v:a_{j}(v)=a\}\text{,}$ $a\in R_{j}\text{,}$ the level-$j$ cells. At each level the cells partition $\mathsf{V}\text{;}$ each cell is rooted-connected, containing with any of its vertices the whole segment from its root $a$ down to that vertex; the cells of level $j+1$ refine those of level $j\text{;}$ and every vertex of a cell is within tree distance $\bar{\alpha}/m_{j}$ of its root. These are elementary consequences of the net property (Section S.2).

Define the active support, the first-appearance weight, and the activation charge

$$A_{k}:=R_{J},\qquad\omega(v):=\min\{m_{j}:\ v\in R_{j}\},\qquad\widetilde{\omega}(v):=\omega(v)\,\mathbf{1}\{v\notin R_{0}\},$$

and, for an integer vector $z$ supported on $A_{k}\text{,}$ the code length

$$\Gamma(z)\ :=\ \sum_{v\in A_{k}}\Bigl(|z_{v}|+\widetilde{\omega}(v)\,\mathbf{1}\{z_{v}\neq 0\}\Bigr):$$

(3.1)

coefficient mass, plus a charge on each support vertex equal to its dyadic discovery cost, the coarsest level being free. Section 4 converts exactly this additive functional into a prior.

The next proposition is the key technical result of the upper half. Beyond the covering statement it certifies three additive budgets simultaneously: a fixed root sum, a bounded code length, and a confined flow state. It is through these certificates that the cover later becomes an estimator.

Proposition 3.1 (Coded cover).

There are universal constants $C_{0},C_{1}$ such that for every finite rooted tree, every $V>0\text{,}$ and every integer $2\leq k\leq n/C_{\star}$ the following hold.

1.

For every $\mu\in\mathcal{F}_{V}(T)$ there is an integer vector $z$ supported on $A_{k}$ whose flow states $x_{v}:=\sum_{u\succeq v}z_{u}$ satisfy

$$\begin{gathered} \displaystyle\sum_{v}z_{v}=k,\qquad\Gamma(z)\leq 9k,\qquad 0\leq x_{v}\leq 3k\ \text{ for all }v\in\mathsf{V}, \\ \displaystyle\Bigl\|\mu-\frac{V}{k}\sum_{v}z_{v}p_{v}\Bigr\|_{2}^{2}\ \leq\ C_{0}V^{2}\delta_{k}(T). \end{gathered}$$
2.

The integer vectors supported on $A_{k}$ that satisfy the first three constraints displayed in (1) number at most $e^{C_{1}k}$.

Since $A_{k}$ and $\widetilde{\omega}$ depend only on $(T,k)\text{,}$ the centers built from these vectors form a class

$$\mathcal{D}_{k}\ :=\ \Bigl\{\tfrac{V}{k}\textstyle\sum_{v}z_{v}p_{v}:\ z\in\mathbb{Z}^{A_{k}},\ \sum_{v}z_{v}=k,\ \Gamma(z)\leq 9k,\ 0\leq x_{v}\leq 3k\ \forall v\Bigr\},$$

the coded cover, which does not depend on the signal and covers $\mathcal{F}_{V}(T)$ at squared radius $C_{0}V^{2}\delta_{k}(T)\text{;}$ in particular

$$H^{c}_{T}\bigl(C_{0}V^{2}\delta_{k}(T)\bigr)\ \leq\ C_{1}k\qquad\text{for every integer }2\leq k\leq n/C_{\star}.$$

(3.2)

Members of $\mathcal{D}_{k}$ may carry negative coefficients $z_{v}\text{,}$ so the cover is external to the body, which is why $H^{c}_{T}$ was taken with free centers; in flow coordinates, every member is nonnegative with root coordinate exactly $V$ and no coordinate above $3V\text{.}$

Proof.

By homogeneity take $V=1$ and write $f=\sum_{u}\lambda_{u}p_{u}$ in barycentric form (Lemma 2.1), with $\lambda(B):=\sum_{u\in B}\lambda_{u}\text{.}$ The construction stops a refinement of the cells at the signal’s own masses, collapses each stopped cell to its root, rounds the resulting skeleton, and samples the light exits; the budgets are then read off and the class is counted. The cell properties above, the collapse and exit estimates, the charge and skeleton accounting, and the counting computation are carried out in Section S.2.

Stopped refinement. Call a level-$j$ cell $B$ heavy when $\lambda(B)>m_{j}/k$ and light otherwise. Starting from level $0\text{,}$ stop at every light cell, refine every heavy cell, and stop everything at level $J\text{.}$ The terminal cells partition $\mathsf{V}\text{;}$ write $\mathcal{L}$ for their collection, and for $L\in\mathcal{L}$ let $a_{L}$ be its root, $m(L)$ its level weight $m_{j}\text{,}$ and $\lambda_{L}:=\lambda(L)\text{.}$ Heaviness is inherited upward through the refinement, so the maximal heavy cells are pairwise disjoint (Section S.2) and each has mass exceeding its own $m_{j}/k\text{;}$ since their masses total at most one,

$$\sum_{C\ \text{maximal heavy}}m(C)\ <\ k.$$

(3.3)

A heavy terminal cell sits at level $J$ and is maximal, so $m_{J}\geq k/2$ leaves room for at most one.

Collapse. Let $g:=\sum_{L\in\mathcal{L}}\lambda_{L}\,p_{a_{L}}\text{.}$ The error blocks $e_{L}:=\sum_{u\in L}\lambda_{u}(p_{u}-p_{a_{L}})$ are supported in $L\setminus\{a_{L}\}$ by rooted-connectedness and Lemma 2.2(2), so they have pairwise disjoint supports; each is controlled by the radius of its cell and, on a light cell, by its mass, and summing the disjoint blocks gives

$$\|f-g\|_{2}^{2}\ \leq\ \frac{3\bar{\alpha}}{k}$$

(3.4)

(Section S.2).

Skeleton and exits. Let $U$ consist of $R_{0}$ together with the roots of all heavy cells. Attach each terminal cell to $U\text{:}$ set $h(L):=a_{L}$ if $L$ is a level-$0$ cell or the heavy terminal cell; otherwise the parent cell of $L$ is heavy, since $L$ was reached by the refinement, and $h(L)$ is its root, so that $h(L)\in U$ and $h(L)\preceq a_{L}\text{.}$ With $q_{L}:=p_{a_{L}}-p_{h(L)}\text{,}$ Lemma 2.2(1) and the parent cell’s radius give $\|q_{L}\|_{2}^{2}=d_{T}(h(L),a_{L})\leq 2\bar{\alpha}/m(L)\text{,}$ and

$$g\ =\ g_{0}+\sum_{L}\lambda_{L}\,q_{L},\qquad g_{0}:=\sum_{h\in U}b_{h}\,p_{h},\quad b_{h}:=\sum_{L:\,h(L)=h}\lambda_{L}.$$

The nonzero $q_{L}$ are the light exits.

Preorder rounding. The skeleton masses are rounded through one scalar. List $U$ as $h_{1},\dots,h_{s}$ in a fixed depth-first preorder of $T\text{,}$ write $b_{i}:=b_{h_{i}}$ for the masses in this order, set $B_{0}:=0$ and $B_{i}:=\sum_{\ell\leq i}b_{\ell}\text{,}$ and put

$$Q_{i}\ :=\ \lfloor kB_{i}\rfloor-\lfloor kB_{i-1}\rfloor\ \in\ \mathbb{Z}_{\geq 0},\qquad\text{so that}\qquad\Bigl|\sum_{i\in I}Q_{i}-k\sum_{i\in I}b_{i}\Bigr|\ <\ 1$$

for every interval $I$ of the list, the left side being a difference of two floor errors. Write $Q$ for the vector with entry $Q_{i}$ at $h_{i}$ and zero elsewhere, a nonnegative integer leak vector of total mass $k$ supported on $U$ (Section S.2). The point of the preorder is that every subtree $T_{v}$ is a contiguous block of it, so $U\cap T_{v}$ is an interval of the list; coordinates being subtree sums (Lemma 2.1), the skeleton and its rounding $\bar{g}_{0}:=\frac{1}{k}\sum_{h\in U}Q_{h}\,p_{h}$ differ by less than $1/k$ at every vertex simultaneously. Both signals vanish outside the minimal rooted subtree spanning $U\text{,}$ which has $O(\bar{\alpha}k)$ edges (a chaining argument over the nets; Section S.2), whence

$$\|g_{0}-\bar{g}_{0}\|_{2}^{2}\ \leq\ C\,\frac{\bar{\alpha}}{k}.$$

(3.5)

Exit quantization. A light exit has $\lambda_{L}\leq m(L)/k\text{,}$ so the independent variables $Z_{L}:=\frac{m(L)}{k}\,\xi_{L}$ with $\xi_{L}\sim\mathrm{Bernoulli}(k\lambda_{L}/m(L))$ are well defined and unbiased. Independence makes the cross terms vanish in expectation no matter how the exits overlap, so $\mathbb{E}\|\sum_{L}(Z_{L}-\lambda_{L})q_{L}\|_{2}^{2}=\sum_{L}\operatorname{Var}(Z_{L})\|q_{L}\|_{2}^{2}\leq 2\bar{\alpha}/k\text{.}$ The selection weight $W:=\sum_{L}m(L)\mathbf{1}\{Z_{L}\neq 0\}$ has $\mathbb{E}W\leq k\text{;}$ by Markov’s inequality, some realization satisfies both $W\leq 2k$ and squared exit error at most $8\bar{\alpha}/k$ (Section S.2). Fix one and define, with $e_{v}$ the coordinate vector at $v\text{,}$ $z:=Q+\sum_{L:\,Z_{L}\neq 0}m(L)\bigl(e_{a_{L}}-e_{h(L)}\bigr)\text{.}$ Each transfer has coefficient sum zero, so $\sum_{v}z_{v}=k\text{;}$ and $z$ is supported on $A_{k}$ with $\|z\|_{1}\leq 5k\text{.}$

The flow states. Since $x_{v}=\sum_{u}z_{u}p_{u}(v)\text{,}$ the flow states of $z$ are the coordinates of $\sum_{u}z_{u}p_{u}\text{.}$ The skeleton $Q$ is a nonnegative leak vector of total mass $k\text{,}$ so its states lie in $[0,k]\text{;}$ and each selected transfer contributes $m(L)(p_{a_{L}}-p_{h(L)})=m(L)\mathbf{1}_{(h(L),a_{L}]}\geq 0$ by Lemma 2.2(2), a downward transport of mass that never creates a negative state and adds at most $m(L)$ to any state. Hence $0\leq x_{v}\leq k+W\leq 3k$ at every vertex.

Code length and error. The code length collects the activation charges: the selected exit roots contribute at most $W\leq 2k\text{,}$ and the heavy roots at most $2k\text{,}$ because the level weights along a chain of heavy cells total less than twice that of the maximal cell it descends to, which Equation 3.3 caps; so $\Gamma(z)\leq 9k$ (Section S.2). The error separates along the three stages of the construction,

$$f-\frac{1}{k}\sum_{v}z_{v}p_{v}\ =\ (f-g)+(g_{0}-\bar{g}_{0})+\sum_{L}(\lambda_{L}-Z_{L})q_{L},$$

so Equations 3.4 and 3.5 and the fixed realization give $\bigl\|f-\frac{1}{k}\sum_{v}z_{v}p_{v}\bigr\|_{2}^{2}\leq C\bar{\alpha}/k=C\overline{\delta}_{k}(T)\leq 2C\delta_{k}(T)$ by Lemma 2.3(3). Restoring the amplitude proves (1).

Counting. A vector obeying the three budgets has support split into a free part inside $R_{0}$ and a charged part of total activation charge at most $9k\text{;}$ the net sizes bound each charge group, so a generating-function estimate controls the number of supports, and on a fixed support, $\|z\|_{1}\leq 9k$ bounds the number of coefficient vectors. Multiplying the counts gives (2) (Section S.2). The covering statement and Equation 3.2 follow from (1) and (2). ∎

Two displays close the upper half. The first is generic. For any $t>0\text{,}$ least squares over a covering of the body at squared radius $t$ with $\log$-cardinality $H^{c}_{T}(t)\text{,}$ or the single center itself when one ball suffices, has risk at most $C(t+\sigma^{2}H^{c}_{T}(t))$ uniformly over the body, by the oracle inequality for least squares over a finite set of candidates (Section S.3). We optimize over $t$ and adjoin the constant estimator $Vp_{o}\text{,}$ whose risk is at most $\sup_{\mu}\|\mu-Vp_{o}\|_{2}^{2}\leq V^{2}h_{T}$ by convexity and Lemma 2.2, and the identity estimator $Y\text{,}$ whose risk is $\sigma^{2}n\text{.}$ This gives

$$R^{*}_{T}(V,\sigma)\ \leq\ C\bigl(\rho_{T}(V,\sigma)\wedge V^{2}H_{T}\wedge\sigma^{2}n\bigr),$$

(3.6)

the upper half of Proposition 1. The second display is where the profile enters: evaluating the infimum Equation 1.6 at $t=C_{0}V^{2}\delta_{k}(T)$ and inserting Equation 3.2,

$$\rho_{T}(V,\sigma)\ \leq\ C\bigl(V^{2}\delta_{k}(T)+\sigma^{2}k\bigr)\qquad\text{for every integer }2\leq k\leq n/C_{\star}.$$

(3.7)

Equation 3.6 knows nothing of the profile; Equation 3.7 is what the crossing argument of Section 3.3 consumes.

3.2 The Packing Lower Bound

The lower bound must realize the same entropy through hypotheses that live in the polytope, and here the two structural features of the body bind: a hypothesis may only move leak mass that it actually possesses, and root paths through a common vertex share coordinates, so the hypotheses cannot be taken orthogonal on an arbitrary tree. The construction of this subsection allocates disjoint mass budgets exactly, and measures the overlap it cannot remove.

Lemma 3.2 (Cover to packing).

For every $r>0$ there exist at least $N^{\uparrow}_{T}(r)$ vertices of $T$ whose pairwise tree distances are all at least $r/9\text{.}$

Proof sketch.

Under the isometry of Lemma 2.2(1), a maximal $(\sqrt{r}/3)$-separated subset of $\{p_{u}:u\in\mathsf{V}\}$ is a packing at pairwise tree distances at least $r/9$ and, by maximality, a cover of the same set at Euclidean radius $\sqrt{r}/3\text{.}$ Each covering ball yields one ancestor: the deepest common ancestor of the vertices it captures lies within tree distance $\tfrac{4}{9}\,r$ of each of them, because every captured vertex below it meets some other captured vertex exactly there. Collecting one ancestor per ball gives an ancestor $r$-net, so the balls, and with them the separated vertices, number at least $N^{\uparrow}_{T}(r)$ (Section S.3). ∎

A product form of Fano’s inequality extracts risk from overlapping hypotheses. The construction in this subsection supplies all three assumptions: blockwise scalability is legality under shrinking toward a reference point, the aspect bound rules out degenerate alphabets, and the Gram row sum is the one quantity through which arbitrary overlap enters.

Lemma 3.3 (Fano’s inequality for overlapping blocks).

Let $K\subseteq\mathbb{R}^{N}$ be convex, let $\mathcal{A}_{1},\dots,\mathcal{A}_{J}$ be finite alphabets with $|\mathcal{A}_{j}|\geq 2\text{,}$ and let

$$\mu_{\mathbf{a}}\ =\ \mu_{0}+\sum_{j=1}^{J}g_{j}(a_{j}),\qquad\mathbf{a}\in\textstyle\prod_{j}\mathcal{A}_{j},$$

be a family of points of $K\text{.}$ Write $d_{j}^{2}:=\min_{a\neq b}\|g_{j}(a)-g_{j}(b)\|_{2}^{2}\text{,}$ $D_{j}^{2}:=\max_{a,b}\|g_{j}(a)-g_{j}(b)\|_{2}^{2}\text{,}$ and $h_{j}:=\log|\mathcal{A}_{j}|\text{,}$ and assume, for constants $A\geq 1$ and $\gamma<1\text{:}$

1.

$\mu_{0}+\sum_{j}\tau_{j}g_{j}(a_{j})\in K$ for every $\tau\in[0,1]^{J}$ and every $\mathbf{a}$;
2.

$0<D_{j}^{2}\leq A\,d_{j}^{2}$ for every $j$;
3.

every family of nonzero contrasts $\Delta_{j}=g_{j}(a_{j})-g_{j}(b_{j})$ satisfies $\ \sup_{i}\sum_{j\neq i}\frac{|\langle\Delta_{i},\Delta_{j}\rangle|}{\|\Delta_{i}\|_{2}\,\|\Delta_{j}\|_{2}}\leq\gamma$.

Then the minimax risk over $K$ in the model $Y\sim N(\mu,\sigma^{2}I_{N})\text{,}$ $\mu\in K\text{,}$ satisfies

$$R^{*}(K,\sigma)\ \geq\ c_{A}(1-\gamma)\sum_{j=1}^{J}\min\bigl\{d_{j}^{2},\ \sigma^{2}h_{j}\bigr\},$$

where $c_{A}>0$ depends only on $A\text{.}$ The bound holds already for the Bayes risk of an explicit product prior on a blockwise-shrunken subfamily; and if every contrast $g_{j}(a)-g_{j}(b)$ is supported in a common coordinate set $S\text{,}$ it holds for the loss restricted to $S\text{.}$

Proof sketch.

For a small universal $\eta\text{,}$ shrink each block into its information budget by $\tau_{j}^{2}:=\min\{1,\,\eta\sigma^{2}h_{j}/D_{j}^{2}\}\text{,}$ legal by (1), and place the uniform product prior on $\mathbf{a}\text{.}$ The scale-invariant Gram bound (3) separates any two shrunken members by $(1-\gamma)\sum_{j}\tau_{j}^{2}\|\Delta_{j}\|_{2}^{2}\text{,}$ overlap paid for once through $\gamma\text{.}$ Revealing all other labels reduces block $j$ to a finite Gaussian test of Kullback–Leibler diameter $O(\eta h_{j})\text{,}$ on which Fano’s inequality (Tsybakov, 2009), or a direct binary test, makes every decoder err with probability at least $\tfrac{1}{4}\text{.}$ Decoding to the nearest shrunken member makes an error at block $j$ cost squared loss of order $(1-\gamma)\tau_{j}^{2}d_{j}^{2}\geq(\eta/A)(1-\gamma)\min\{d_{j}^{2},\sigma^{2}h_{j}\}$ by (2). Taking expectations assembles the bound; the per-block constants, the binary-alphabet test, and the restricted-loss variant are in Section S.3. ∎

The next proposition realizes these hypotheses inside the flow polytope.

Proposition 3.4 (Positive packing).

Let $U\subseteq\mathsf{V}$ consist of $M\geq 2$ vertices with pairwise tree distances at least $r>0\text{.}$ Then for every integer $1\leq q\leq M/2\text{,}$

$$R^{*}_{T}(V,\sigma)\ \geq\ c\,\min\Bigl\{\frac{V^{2}r}{q},\ \sigma^{2}q\log\frac{eM}{q}\Bigr\}.$$

Proof sketch.

The construction distributes the budget exactly and lets Lemma 3.3 price the overlap; the proof is completed in Section S.3.

Disjoint budgets. A bin-packing pass from the leaves of the minimal subtree spanning $U$ upward, sealing a connected component whenever it has collected on the order of $M/q$ marked vertices, extracts from $U$ marked sets $U_{1},\dots,U_{q^{\prime}}$ of sizes $M_{i}\asymp M/q$ lying in connected subtrees $\mathcal{C}_{1},\dots,\mathcal{C}_{q^{\prime}}$ with pairwise disjoint edge sets, where $q^{\prime}\asymp q$ (Section S.3). Give every group the budget $a:=V/q^{\prime}\text{;}$ the budgets sum to $V\text{.}$

A multiscale code in each group. Fix a group with marked set $U_{i}\text{.}$ Set $r_{\ell}:=16^{\ell}r$ and let $N_{\ell}\subseteq N_{\ell-1}$ be nested maximal $r_{\ell}$-separated subsets of $N_{0}:=U_{i}\text{;}$ assigning each point of $N_{\ell-1}$ to a point of $N_{\ell}$ within distance $r_{\ell}$ organizes $U_{i}$ into a hierarchy with a singleton top. Descend from the top along children of maximal weight, the weight of a node being the number of members of $U_{i}$ at or below it, so that the child counts along the chain multiply to at least $M_{i}\text{.}$ Discarding single-child levels and keeping one residue class $I$ of the rest modulo four produces alphabets $\mathcal{A}_{\ell}\text{,}$ $\ell\in I\text{,}$ the children of the chain node at level $\ell\text{,}$ with $h_{\ell}:=\log|\mathcal{A}_{\ell}|$ and total entropy $H:=\sum_{\ell\in I}h_{\ell}\geq\tfrac{1}{4}\log M_{i}\text{.}$ Two points of $\mathcal{A}_{\ell}$ are $r_{\ell-1}$-separated and within $2r_{\ell}$ of each other, so each block has aspect at most $32$ (Lemma 2.2(1)). The per-scale masses $a_{\ell}:=a\sqrt{h_{\ell}/r_{\ell-1}}/S\text{,}$ with normalizer $S:=\sum_{\ell^{\prime}\in I}\sqrt{h_{\ell^{\prime}}/r_{\ell^{\prime}-1}}\text{,}$ spend $\sum_{\ell\in I}a_{\ell}=a$ exactly, and Cauchy–Schwarz against the geometric radii gives

$$\sum_{\ell\in I}\min\{a_{\ell}^{2}r_{\ell-1},\,\sigma^{2}h_{\ell}\}\ \geq\ c\min\{a^{2}r,\,\sigma^{2}\log M_{i}\}$$

(Section S.3). Fix a reference letter $u^{0}_{\ell}\in\mathcal{A}_{\ell}$ at each retained scale; the group’s hypotheses are $\mu_{\mathbf{u}}=\mu_{0}+\sum_{\ell\in I}g_{\ell}(u_{\ell})$ with $\mu_{0}:=\sum_{\ell\in I}a_{\ell}\,p_{u^{0}_{\ell}}$ and $g_{\ell}(u):=a_{\ell}(p_{u}-p_{u^{0}_{\ell}})\text{,}$ so that $\mu_{\mathbf{u}}=\sum_{\ell\in I}a_{\ell}p_{u_{\ell}}$ places mass $a_{\ell}$ at the letter chosen from $\mathcal{A}_{\ell}\text{.}$

Overlap. A level-$\ell$ contrast has entries bounded by $a_{\ell}$ and support of size at most $32\,r_{\ell-1}$ (Lemma 2.2(2)), so its normalized inner product with a level-$\ell^{\prime}$ contrast, $\ell<\ell^{\prime}$ in $I\text{,}$ is at most $32\cdot 4^{-(\ell^{\prime}-\ell)}\text{;}$ retained levels differ by at least four, so every normalized Gram row sums to at most $\gamma=64/255<\tfrac{1}{2}\text{.}$

Assembly. Each blockwise contraction $\mu_{0}+\sum_{\ell}\tau_{\ell}g_{\ell}(u_{\ell})$ replaces $p_{u^{0}_{\ell}}$ by a convex combination of $p_{u^{0}_{\ell}}$ and $p_{u_{\ell}}\text{,}$ so it remains a nonnegative leak combination of total mass $a\text{;}$ Lemma 3.3 therefore applies with $A=32$ and $\gamma=64/255$ and bounds the group’s Bayes risk from below by $c\min\{a^{2}r,\sigma^{2}\log M_{i}\}$ in the group’s own coordinates. Sums of one member per group are legal points of $\mathcal{F}_{V}(T)$ (Lemma 2.1), and the groups’ contrasts occupy the pairwise disjoint edge supports of the $\mathcal{C}_{i}$ (Lemma 2.2(2)), so the joint Bayes risk dominates the sum of the isolated group bounds (Section S.3). With $q^{\prime}\asymp q$ and $M_{i}\asymp M/q\text{,}$ the group bounds sum to $R^{*}_{T}(V,\sigma)\geq c\,q\min\{V^{2}r/q^{2},\,\sigma^{2}\log(eM/q)\}\text{,}$ which is the claim. ∎

The lower half of the crossing now follows by matching the free parameter $q$ to the profile.

Proposition 3.5 (Profile to risk).

There is a universal constant $c_{0}>0$ such that for every finite rooted tree, every integer $k\geq 1\text{,}$ and all $V,\sigma>0\text{,}$

$$V^{2}\delta_{k}(T)\ \geq\ A_{0}\sigma^{2}k\qquad\Longrightarrow\qquad R^{*}_{T}(V,\sigma)\ \geq\ c_{0}\,\sigma^{2}k.$$

Proof.

The supremum in Equation 1.4 need not be attained, so choose $r>0$ whose term is at least half the supremum:

$$V^{2}r\,\min\Bigl\{1,\ \tfrac{1}{k}\Bigl[\log\tfrac{N^{\uparrow}_{T}(r)}{k}\Bigr]_{+}\Bigr\}\ \geq\ \tfrac{1}{2}\,V^{2}\delta_{k}(T)\ \geq\ \tfrac{A_{0}}{2}\,\sigma^{2}k\ >\ 0.$$

Positivity forces $N^{\uparrow}_{T}(r)>k\text{.}$ Lemma 3.2 supplies $M\geq N^{\uparrow}_{T}(r)>k$ vertices with pairwise tree distances at least $r^{\prime}:=r/9\text{,}$ and the displayed inequality survives with $(r^{\prime},M)$ in place of $(r,N^{\uparrow}_{T}(r))$ at the cost of a factor $18\text{,}$ absorbed into $A_{0}\text{.}$ It remains to choose $q$ in Proposition 3.4: the mass branch $V^{2}r^{\prime}/q$ shrinks and the entropy branch $\sigma^{2}q\log(eM/q)$ grows as $q$ increases, and at $q\asymp\max\bigl\{1,\ k/(1+\log(M/k))\bigr\}$ both sit above $\sigma^{2}k$ times a universal constant. This scalar matching, including the boundary cases, is proved in Section S.3. ∎

3.3 The Crossing and the Benchmarks

The two halves now meet. Throughout, $K:=\lfloor n/C_{\star}\rfloor\text{,}$ so that $k_{0}\leq K+1$ by Equation 1.5.

Proof of Theorem 1 and Proposition 1.

The upper half of Proposition 1 is Equation 3.6; it remains to prove

$$\text{(I)}\quad R^{*}_{T}\ \geq\ c\,\bigl(\rho_{T}\wedge V^{2}H_{T}\wedge\sigma^{2}n\bigr),\qquad\qquad\text{(II)}\quad R^{*}_{T}\ \asymp\ \min\bigl\{V^{2}H_{T},\ \sigma^{2}k_{0}\bigr\}.$$

For $n=1$ the body is the single point $Vp_{o}$ and $H_{T}=0\text{,}$ so $R^{*}_{T}=0$ and both sides of (I) and (II) vanish; assume $n\geq 2\text{.}$ Testing two points at distance $\min\{V\sqrt{H_{T}},\sigma\}$ along the diameter segment of the body, which lies in $\mathcal{F}_{V}(T)$ by Lemma 2.2(1), records a two-point bound

$$R^{*}_{T}(V,\sigma)\ \geq\ c\,\min\bigl\{V^{2}H_{T},\ \sigma^{2}\bigr\}\qquad(n\geq 2)$$

(3.8)

(Section S.3). The proof of (I) and (II) now splits into four exhaustive cases, according to how the crossing index fires.

Case $n<C_{\star}\text{.}$ Here $k_{0}=1$ through the dimension clause, and $\sigma^{2}\geq\sigma^{2}n/C_{\star}\text{,}$ so Equation 3.8 gives (I) and the lower half of (II), while Equation 3.6 gives the upper half (Section S.3).

Case $n\geq C_{\star}$ and $V^{2}\delta_{1}\leq A_{0}\sigma^{2}\text{.}$ Here $k_{0}=1$ through the crossing clause. Proposition 3.1 starts at $k=2\text{,}$ so we supply the missing entropy bound at $k=1\text{.}$ Every ancestor $r$-net with $r<h_{T}$ has at least two elements: the root lies in every net, being its own only ancestor, and it covers no deepest vertex. Hence $N^{\uparrow}_{T}(r)\geq 2$ for all $r<h_{T}\text{,}$ and letting $r\uparrow h_{T}$ in Equation 1.4 gives $\alpha_{1}\geq h_{T}\log 2\text{.}$ On the other hand the single center $Vp_{o}$ covers the body at squared radius $V^{2}h_{T}\text{,}$ since $\|\mu-Vp_{o}\|_{2}\leq\max_{u}\|Vp_{u}-Vp_{o}\|_{2}=V\sqrt{h_{T}}$ by convexity, so $H^{c}_{T}(V^{2}h_{T})=0$ and

$$\rho_{T}\ \leq\ V^{2}h_{T}\ \leq\ \frac{V^{2}\delta_{1}}{\log 2}\ \leq\ \frac{A_{0}}{\log 2}\,\sigma^{2}.$$

Both (I) and (II) follow, with Equation 3.8 for the lower bounds and Equation 3.6 for the upper ones (Section S.3).

Case $2\leq k_{0}\leq K\text{.}$ The crossing clause fired at $k_{0}\text{:}$ $V^{2}\delta_{k_{0}}\leq A_{0}\sigma^{2}k_{0}\text{,}$ and $k_{0}\leq n/C_{\star}\text{,}$ so Equation 3.7 at $k_{0}$ gives

$$\rho_{T}\ \leq\ C\bigl(V^{2}\delta_{k_{0}}+\sigma^{2}k_{0}\bigr)\ \leq\ C(A_{0}+1)\,\sigma^{2}k_{0}.$$

(3.9)

By minimality neither clause holds at $k_{0}-1\text{;}$ in particular $V^{2}\delta_{k_{0}-1}>A_{0}\sigma^{2}(k_{0}-1)\text{,}$ so Proposition 3.5 at $k_{0}-1$ gives

$$R^{*}_{T}\ \geq\ c_{0}\,\sigma^{2}(k_{0}-1)\ \geq\ \frac{c_{0}}{2}\,\sigma^{2}k_{0}.$$

(3.10)

Combining Equations 3.9 and 3.10 proves (I). For (II) the same two inequalities identify the information branch: $V^{2}H_{T}\geq V^{2}h_{T}\geq V^{2}\delta_{k_{0}-1}>A_{0}\sigma^{2}(k_{0}-1)\geq\sigma^{2}k_{0}$ by Lemma 2.3(1), so $\min\{V^{2}H_{T},\sigma^{2}k_{0}\}=\sigma^{2}k_{0}\text{,}$ which Equation 3.10 bounds below and Equations 3.6 and 3.9 bound above.

Case $k_{0}=K+1$ with $n\geq C_{\star}\text{.}$ No clause fired on $[1,K]\text{;}$ in particular $V^{2}\delta_{K}>A_{0}\sigma^{2}K\text{,}$ so Proposition 3.5 at $K$ gives $R^{*}_{T}\geq c_{0}\sigma^{2}K\geq c\,\sigma^{2}n\text{,}$ using $K=\lfloor n/C_{\star}\rfloor\geq n/(2C_{\star})\text{.}$ This is (I), the three-term minimum being at most $\sigma^{2}n\text{;}$ and (II) follows because $\sigma^{2}k_{0}\asymp\sigma^{2}n$ while $V^{2}H_{T}\geq V^{2}\delta_{K}>A_{0}\sigma^{2}K$ (Section S.3).

The four cases are exhaustive: for $n\geq C_{\star}\text{,}$ either the crossing clause fires at $k=1\text{,}$ or it first fires at some $2\leq k_{0}\leq K\text{,}$ or it never fires on $[1,K]$ and the dimension clause fires at $K+1\text{.}$ This proves (I) and (II). In the interior case the computation also pins the fixed point itself, $\rho_{T}\asymp\sigma^{2}k_{0}$ (Section S.3). ∎

The rate formula specializes in closed form on the three families of Table 1; in each corollary the equivalence holds for all $V,\sigma>0$ with universal constants.

Corollary 1 (Path).

Let $P_{L}$ be the path with $L\geq 1$ edges, rooted at one endpoint, so that $\mathcal{F}_{V}(P_{L})=\{\mu:V=\mu_{0}\geq\mu_{1}\geq\cdots\geq\mu_{L}\geq 0\}\text{.}$ Then

$$R^{*}_{P_{L}}(V,\sigma)\ \asymp\ \min\bigl\{V^{2}L,\ \ V^{2/3}\sigma^{4/3}L^{1/3},\ \ \sigma^{2}(L+1)\bigr\}.$$

Corollary 2 (Star).

Let $S_{m}$ be the star with $m\geq 2$ leaves rooted at the hub; the hub coordinate is pinned at $V\text{,}$ and in leaf coordinates $\mathcal{F}_{V}(S_{m})=\{\mu\in\mathbb{R}_{\geq 0}^{m}:\sum_{i\leq m}\mu_{i}\leq V\}\text{.}$ Then

$$R^{*}_{S_{m}}(V,\sigma)\ \asymp\ \min\Bigl\{V^{2},\ \ \sigma V\sqrt{1+\bigl[\log(m\sigma/V)\bigr]_{+}},\ \ \sigma^{2}m\Bigr\}.$$

Corollary 3 (Complete binary tree).

Let $B_{h}$ be the complete binary tree of height $h\geq 1\text{:}$ every internal vertex has two children and all $2^{h}$ leaves lie at depth $h\text{,}$ so that $n=2^{h+1}-1\text{.}$ Then

$$R^{*}_{B_{h}}(V,\sigma)\ \asymp\ \min\bigl\{V^{2}h,\ \ \sigma V\bigl(1+[\log(n\sigma/V)]_{+}\bigr),\ \ \sigma^{2}n\bigr\}.$$

The three corollaries follow one recipe: the ancestor-covering counts determine the profile through Equation 2.3, and the crossing selects the active branch. Interval covering gives the path counts $\lceil(L+1)/(q+1)\rceil\text{;}$ on the star only the radius $q=0$ contributes, and the profile is the exact expression of Remark 2.1; on the binary tree the counts decay geometrically in the radius, and, for $k\leq n/C_{\star}\text{,}$ the profile $\alpha_{k}$ is quadratic in $\log(n/k)$ wherever $k$ exceeds that logarithm. The counts, the profiles, and the scalar optimizations that locate $k_{0}$ and match the branches in every parameter regime are carried out in Section S.4.

The crossing consumed Proposition 3.1 only through Equation 3.2; the further certificates are spent in Section 4, the code length becoming a prior, the root sum an exact constraint, and the confined states a one-dimensional computation.

4 Efficient and Adaptive Estimation

This section proves the three estimation theorems; the first is stated here, the adaptation pair in Section 4.3.

Theorem 2 (Efficient estimation).

There is a deterministic estimator $\widehat{\mu}=\widehat{\mu}(T,V,\sigma,Y)\text{,}$ computed exactly in $O(n^{2}\log(n+1))$ arithmetic operations of the real-RAM model fixed in Section 2.1, such that for every finite rooted tree $T$ and all $V,\sigma>0\text{,}$

$$\sup_{\mu\in\mathcal{F}_{V}(T)}\mathbb{E}_{\mu}\bigl\|\widehat{\mu}-\mu\bigr\|_{2}^{2}\ \leq\ C\,R^{*}_{T}(V,\sigma)$$

with a universal constant $C\text{.}$

The estimator is an exponentially weighted aggregate over a class of integer states, evaluated exactly in the sense fixed by Remark 4.1. Section 4.1 constructs it and proves the risk bound, Section 4.2 gives the algorithm, and Section 4.3 removes first the budget and then the noise level.

4.1 Aggregation over Integer States

Least squares over the coded cover converts Equation 3.2 into the rate, but it is a search through as many as $e^{C_{1}k}$ centers (Proposition 3.1). The estimator of Theorem 2 replaces the search by an average under a prior built from the code length.

Fix an integer $2\leq k\leq n/C_{\star}$ and recall the active support $A_{k}\text{,}$ the charges $\widetilde{\omega}\text{,}$ and the code length Equation 3.1. The integer states at budget $k$ are the vectors

$$\mathcal{X}_{k}\ :=\ \Bigl\{x\in\{0,1,\dots,3k\}^{\mathsf{V}}:\ x_{o}=k\ \text{ and }\ z_{v}:=x_{v}-\textstyle\sum_{c\in\operatorname{ch}(v)}x_{c}=0\ \text{ for all }v\notin A_{k}\Bigr\},$$

(4.1)

each with its leak vector $z\text{,}$ its code length $\Gamma(x):=\Gamma(z)\text{,}$ and its center $\nu_{x}:=\frac{V}{k}\,x\text{;}$ the leaks telescope to the root state, $\sum_{v}z_{v}=x_{o}=k\text{.}$ The approximant of Proposition 3.1 is an integer state: its coordinates lie in $[0,3k]\text{,}$ its root coordinate is $k\text{,}$ and its leaks are supported on $A_{k}\text{.}$ The class relaxes the coded cover in exactly one respect: the budget $\Gamma\leq 9k$ is dropped from the constraints, so that $\mathcal{D}_{k}=\{\nu_{x}:\ x\in\mathcal{X}_{k},\ \Gamma(x)\leq 9k\}\text{.}$ The dropped budget resurfaces as the prior: as a constraint it would add a second dimension to the recursions of Section 4.2; as a soft penalty it costs only a normalizing sum. Their leaks may be negative, as for the coded cover.

The estimator at budget $k$ is the exponentially weighted aggregate

$$\widehat{\mu}_{k}\ :=\ \frac{\sum_{x\in\mathcal{X}_{k}}\nu_{x}\,e^{-2\Gamma(x)}\,e^{-\|Y-\nu_{x}\|_{2}^{2}/(4\sigma^{2})}}{\sum_{x\in\mathcal{X}_{k}}e^{-2\Gamma(x)}\,e^{-\|Y-\nu_{x}\|_{2}^{2}/(4\sigma^{2})}}\ ,$$

(4.2)

the posterior mean at temperature $4\sigma^{2}$ under the Kraft-type prior $\pi_{k}(x)\propto e^{-2\Gamma(x)}\text{.}$ The normalizing sum $\sum_{x\in\mathcal{X}_{k}}e^{-2\Gamma(x)}\text{,}$ which cancels from Equation 4.2 and is never computed, is at most $e^{k}\text{,}$ by a generating-function estimate from the net sizes of Section 3.1 (Section S.5). Every integer state has root coordinate $k$ and coordinates at most $3k\text{,}$ so $\widehat{\mu}_{k}(o)=V$ identically and every coordinate of $\widehat{\mu}_{k}$ lies in $[0,3V]\text{.}$

The risk of an exponentially weighted aggregate is controlled through the prior mass at a single candidate; no cardinality of the class enters (Leung and Barron, 2006, cf.). In the next lemma the true mean is arbitrary.

Lemma 4.1 (Aggregation oracle).

Let $F\subseteq\mathbb{R}^{N}$ be finite and nonempty, let $\pi$ be a probability vector on $F$ with $\pi_{f}>0$ for all $f\text{,}$ and let $Y=\mu+\sigma Z$ with $Z\sim N(0,I_{N})$ and $\mu\in\mathbb{R}^{N}\text{.}$ Define

$$\widehat{f}_{\pi}(Y)\ :=\ \sum_{f\in F}\widehat{\pi}_{f}(Y)\,f,\qquad\widehat{\pi}_{f}(Y)\ :=\ \frac{\pi_{f}\,e^{-\|Y-f\|_{2}^{2}/(4\sigma^{2})}}{\sum_{g\in F}\pi_{g}\,e^{-\|Y-g\|_{2}^{2}/(4\sigma^{2})}}\ .$$

Then

$$\mathbb{E}_{\mu}\bigl\|\widehat{f}_{\pi}-\mu\bigr\|_{2}^{2}\ \leq\ \min_{f\in F}\Bigl\{\|f-\mu\|_{2}^{2}+4\sigma^{2}\log\frac{1}{\pi_{f}}\Bigr\}.$$

Proof sketch.

Two exact identities. Differentiating the weights shows that the aggregate is smooth and bounded and identifies its Jacobian as $\frac{1}{2\sigma^{2}}$ times the posterior covariance of $f\text{,}$ so Stein’s identity (Stein, 1981) writes the risk through the divergence, a covariance trace; the bias–variance split of the posterior loss $\sum_{f}\widehat{\pi}_{f}(y)\,\|y-f\|_{2}^{2}$ produces the same trace with the opposite sign, and at this temperature the two cancel exactly, leaving $\mathbb{E}_{\mu}\|\widehat{f}_{\pi}-\mu\|_{2}^{2}=\mathbb{E}\sum_{f}\widehat{\pi}_{f}(Y)\,\|Y-f\|_{2}^{2}-N\sigma^{2}\text{.}$ The weights are the Gibbs minimizer of the posterior loss penalized by $4\sigma^{2}$ times the relative entropy to $\pi$ (see, e.g., Boucheron et al., 2013, Section 4.9), so the value of the penalized objective at the point mass on any $f_{0}\in F$ bounds the posterior loss pathwise by $\|Y-f_{0}\|_{2}^{2}+4\sigma^{2}\log(1/\pi_{f_{0}})\text{;}$ take expectations and insert $\mathbb{E}\|Y-f_{0}\|^{2}=\|f_{0}-\mu\|^{2}+N\sigma^{2}\text{.}$ The differentiation, the domain of Stein’s identity, and the two expansions are carried out in Section S.5. ∎

Set $K:=\lfloor n/C_{\star}\rfloor$ as in Section 3.3. From here through Section 4.2, set $k:=k_{\mathrm{alg}}(T,V,\sigma)\text{,}$ the algorithmic crossing index of Lemma 2.3(3), so that $k\leq K+1\text{.}$ The root coordinate carries no statistical content (Section 2.1), so define the root-corrected observation $\widetilde{Y}$ by $\widetilde{Y}(o):=V$ and $\widetilde{Y}(v):=Y(v)$ for $v\neq o\text{.}$ The estimator of Theorem 2 is

$$\widehat{\mu}\ :=\ \begin{cases}Vp_{o},&V^{2}H_{T}\leq\sigma^{2}k,\\ \widetilde{Y},&V^{2}H_{T}>\sigma^{2}k\ \text{ and }\ k>K,\\ Vp_{o},&V^{2}H_{T}>\sigma^{2}k\ \text{ and }\ k=1\leq K,\\ \widehat{\mu}_{k},&V^{2}H_{T}>\sigma^{2}k\ \text{ and }\ 2\leq k\leq K,\end{cases},$$

(4.3)

four exclusive and exhaustive branches: when the diameter is not the active scale, the index $k$ either exceeds the dimension range, or is too small for Proposition 3.1, or is interior.

The risk bound of Theorem 2 follows. In the interior branch Proposition 3.1 applies, and evaluating the oracle of Lemma 4.1 at the approximant $x^{\star}$ of a given $\mu\in\mathcal{F}_{V}(T)\text{,}$ with $\log(1/\pi_{k}(x^{\star}))\leq 2\Gamma(x^{\star})+k\leq 19k\text{,}$ gives

$$\sup_{\mu\in\mathcal{F}_{V}(T)}\mathbb{E}_{\mu}\bigl\|\widehat{\mu}_{k}-\mu\bigr\|_{2}^{2}\ \leq\ C_{0}V^{2}\delta_{k}(T)+C\sigma^{2}k\ \leq\ C\,\sigma^{2}k\ =\ C\min\{V^{2}H_{T},\ \sigma^{2}k\},$$

(4.4)

the second inequality because the surrogate crossing clause fired at $k=k_{\mathrm{alg}}\leq K\text{,}$ so that $V^{2}\delta_{k}\leq V^{2}\overline{\delta}_{k}\leq 2A_{0}\sigma^{2}k\text{,}$ and the equality by the branch condition. The three elementary branches obey the same bound by the diameter and first-budget facts of Section 3.3 (Section S.5). Since $k_{\mathrm{alg}}\leq k_{0}$ by Lemma 2.3(3), $\min\{V^{2}H_{T},\sigma^{2}k_{\mathrm{alg}}\}\leq\min\{V^{2}H_{T},\sigma^{2}k_{0}\}\text{,}$ which Theorem 1 bounds by $C\,R^{*}_{T}(V,\sigma)\text{.}$

4.2 Exact Evaluation

It remains to evaluate Equation 4.2, a ratio of sums over a class of exponential size. Computing $k_{\mathrm{alg}}$ and the nets behind $A_{k}$ costs $O(n\log(n+1))$ operations (Lemma 2.3), and the elementary branches are linear; the content is the interior branch.

The confined states are the mechanism. Every coordinate of an integer state lies in $\{0,\dots,3k\}\text{,}$ so subtree weights factorize over one integer state per vertex. Write $\varphi_{v}(x):=e^{-(Y_{v}-(V/k)x)^{2}/(4\sigma^{2})}$ for the local weight, so that the summand of Equation 4.2 at $x$ is $e^{-2\Gamma(x)}\prod_{v}\varphi_{v}(x_{v})\text{,}$ and define for each vertex the message

$$P_{v}(\zeta)\ :=\ \sum_{x=0}^{3k}P_{v}[x]\,\zeta^{x},$$

(4.5)

a polynomial of degree at most $3k$ in an indeterminate $\zeta\text{,}$ whose coefficient $P_{v}[x]$ is the total weight, local weights times prior kernel, of the admissible assignments on the subtree $T_{v}$ with state $x$ at $v\text{.}$ The message at $v$ arises from the product $\prod_{c\in\operatorname{ch}(v)}P_{c}(\zeta)$ of its children’s messages by pairwise polynomial multiplications and an $O(k)$ local pass, and correctness is an induction over subtrees; since every integer state has root coordinate $k\text{,}$ the denominator of Equation 4.2 is the single coefficient $P_{o}[k]\text{.}$

A child sum can exceed $3k$ where a state cannot. At an inactive vertex conservation kills such terms; at an active vertex the prior kernel is geometric in the leak, so they enter through one scalar, the child product evaluated at $e^{-2}\text{;}$ evaluations multiply where messages multiply, so the scalar travels with each message, and no coefficient beyond degree $3k$ outlives a merge (Section S.5).

The tree off the active support carries no state: a subtree disjoint from $A_{k}$ contributes a constant factor and is pruned, and a maximal chain of inactive vertices, each left with one child whose state it copies, compresses to a single edge whose diagonal kernel is generated in $O(k)$ from the chain’s length and observation sum.

The numerator of Equation 4.2 is recovered by differentiation. Treat the $3k+1$ local weights at each vertex as formal inputs of the arithmetic circuit that computes $P_{o}[k]\text{;}$ every admissible assignment carries the factor $\varphi_{v}(x_{v})$ exactly once, so

$$\widehat{\mu}_{k}(v)\ =\ \frac{V}{k}\,\sum_{x=0}^{3k}x\cdot\frac{\varphi_{v}(x)}{P_{o}[k]}\,\frac{\partial P_{o}[k]}{\partial\varphi_{v}(x)}\ ,$$

(4.6)

the ratio under the sum being the posterior probability of $\{x_{v}=x\}\text{.}$ One reverse-mode sweep of the circuit computes all these derivatives at once, at the same asymptotic cost as the forward pass (Section S.5).

Each vertex contributes $O(k)$ local work, each of the fewer than $n$ merges costs one multiplication of degree-$3k$ polynomials, $O(k\log(k+1))$ operations in the model of Section 2.1, and the reverse sweep matches the forward cost, so with the preprocessing the total is $O(n\log(n+1)+nk\log(k+1))\leq O(n^{2}\log(n+1))$ operations, the count of Theorem 2. On structured trees the account improves: sibling messages that agree up to a dilation of the variable multiply in batch through power sums, and on stars the entire evaluation takes $O(n\log^{2}n)$ operations. Every tie break is fixed, so the estimator is a deterministic function of $(T,V,\sigma,Y)\text{;}$ together with the risk bound of Section 4.1, this proves Theorem 2. The evaluation identities, the operation count, and the instance-sensitive refinements are proved in Section S.5.

Remark 4.1 (Computational model).

“Exact” refers to the estimator itself, not only to its risk order: in the arithmetic model fixed in Section 2.1, the messages, the overflow scalars, the compressed kernels, and the reverse sweep are algebraic identities, and the returned vector is Equation 4.2 with $k=k_{\mathrm{alg}}\text{,}$ evaluated without error.

4.3 Adaptation

The budget is not external to the model: on the cone Equation 1.1 it is the root value of the signal, observed directly as $Y_{o}=V+\sigma Z_{o}\text{,}$ so adaptation to it is cheap but not free.

Theorem 3 (Budget adaptation).

There are universal constants $c,C>0$ and a deterministic polynomial-time estimator $\widehat{\mu}=\widehat{\mu}(T,\sigma,Y)\text{,}$ not depending on $V\text{,}$ such that for every finite rooted tree $T\text{,}$ all $V,\sigma>0\text{,}$ and every $\mu\in\mathcal{F}_{V}(T)\text{,}$

$$\mathbb{E}_{\mu}\|\widehat{\mu}-\mu\|_{2}^{2}\ \leq\ C\bigl(R^{*}_{T}(V,\sigma)+\sigma^{2}\bigr).$$

Conversely, every estimator $\widehat{\mu}(T,\sigma,Y)$ satisfies, for every finite rooted tree $T$ and all $V,\sigma>0\text{,}$

$$\max_{V^{\prime}\in\{V,\,V+\sigma\}}\ \sup_{\mu\in\mathcal{F}_{V^{\prime}}(T)}\mathbb{E}_{\mu}\|\widehat{\mu}-\mu\|_{2}^{2}\ \geq\ c\,\sigma^{2}.$$

Proof of Theorem 3, necessity.

The points $Vp_{o}$ and $(V+\sigma)p_{o}$ lie in the slices at budgets $V$ and $V+\sigma$ and are at distance $\sigma\text{.}$ For any estimator $\widehat{\mu}(T,\sigma,Y)\text{,}$ decode the nearer of the two points: a decoding error forces squared loss at least $\sigma^{2}/4\text{,}$ and deciding between two Gaussian means at distance $\sigma$ is a one-dimensional shift of size one, on which every test errs with probability at least a universal constant under the uniform two-point prior. The larger of the two risks is therefore at least $c\,\sigma^{2}\text{.}$ ∎

The sufficiency halves of Theorems 3 and 4 rest on candidate families free of the unknown parameters. The construction behind Proposition 3.1 consumes the budget in exactly one place, the quantization of leaks in units of $V/k\text{;}$ dropping the quantization leaves subspaces. For an integer $2\leq k\leq n/C_{\star}\text{,}$ call $S\subseteq A_{k}\setminus R_{0}$ admissible when $\sum_{v\in S}\widetilde{\omega}(v)\leq 4k\text{,}$ and define the profile subspaces $\mathcal{V}_{k,S}:=\operatorname{span}\{p_{v}:\ v\in R_{0}\cup S\}\text{,}$ of dimension $O(k)\text{.}$ Everything here depends on $(T,k)$ alone, and the admissible supports number at most $e^{C_{1}k}\text{,}$ by the counting argument of Proposition 3.1. Every body point is close to one of these subspaces at the profile scale:

$$\min_{S\ \mathrm{admissible}}\ \operatorname{dist}^{2}\bigl(\mu,\ \mathcal{V}_{k,S}\bigr)\ \leq\ C\,V^{2}\overline{\delta}_{k}(T)\qquad\text{for every }\mu\in\mathcal{F}_{V}(T),\ 2\leq k\leq n/C_{\star}.$$

(4.7)

The unrounded skeleton and the sampled exits of the proof of Proposition 3.1 realize the bound, their support carrying total charge at most $4k$ (Section S.6).

Selection is by penalized least squares with Kraft weights. Section S.6 proves the following oracle, of Birgé–Massart type (Birgé and Massart, 2001), in the weighted and random-penalty form needed here: for Gaussian data on $\mathbb{R}^{N}$ with independent coordinates of variances at most $\sigma^{2}$ and mean $\theta\text{,}$ a countable collection $\mathcal{M}$ of affine models $m$ with weights $\Delta_{m}$ obeying $\sum_{m}e^{-\Delta_{m}}\leq 1\text{,}$ penalties $C_{2}\sigma^{2}(\dim m+\Delta_{m})$ for a sufficiently large universal $C_{2}\text{,}$ and a full model penalized at $C_{2}\sigma^{2}N\text{,}$ every minimizer $\widehat{\theta}$ of the criterion obeys

$$\mathbb{E}\bigl\|\widehat{\theta}-\theta\bigr\|_{2}^{2}\ \leq\ C\,\min\Bigl\{\ \inf_{m\in\mathcal{M}}\bigl(\operatorname{dist}^{2}(\theta,m)+\sigma^{2}(\dim m+\Delta_{m}+1)\bigr),\ \ \sigma^{2}N\Bigr\}\ +\ C\sigma^{2}.$$

(4.8)

If the penalties are random, nonnegative, and bracketed on an event $E$ between fixed multiples of the displayed ones, the same right side bounds $\mathbb{E}\bigl[\|\widehat{\theta}-\theta\|_{2}^{2}\mathbf{1}_{E}\bigr]\text{;}$ this is how the estimated noise scale enters in Theorem 4.

For $n$ below a universal threshold the estimator of Theorem 3 returns $Y\text{,}$ at risk $O(\sigma^{2})\text{;}$ above it, the estimator is one such selection, conditional on the root observation, over three groups of candidates: the amplitude-free subspaces $\mathcal{V}_{k,S}$ at dyadic budgets $2\leq k\leq k^{\star}:=\lfloor\log n/(2C_{1})\rfloor\text{;}$ the integer-state points $\tfrac{V^{\prime}}{k}\,x\text{,}$ $x\in\mathcal{X}_{k}\text{,}$ at the $O(\log n)$ amplitudes $V^{\prime}$ of a root-anchored geometric grid and a low-amplitude dyadic grid, with $\operatorname{span}(p_{o})$ and the root points $V^{\prime}p_{o}$ adjoined; and the full model, each candidate weighted by its dimension and code length. The system depends on $Y$ through $Y_{o}$ alone, so conditionally on $Y_{o}$ it is deterministic while the noise in the remaining coordinates is unchanged, and unconditioning costs $\mathbb{E}(Y_{o}-V)^{2}=\sigma^{2}\text{;}$ the precise grids, weights, and thresholds are fixed in Section S.6.

Proof sketch of Theorem 3, sufficiency.

By Theorem 1 and the sandwich of Lemma 2.3(3), $\min\{V^{2}H_{T},\sigma^{2}k_{\mathrm{alg}}\}\asymp R^{*}_{T}(V,\sigma)\text{,}$ and there are three regimes, according to the candidate at which the oracle Equation 4.8 is evaluated. In the diameter regime $V^{2}H_{T}\leq\sigma^{2}k_{\mathrm{alg}}\text{,}$ the model $\operatorname{span}(p_{o})$ is within squared distance $V^{2}h_{T}$ of $\mu\text{,}$ at constant dimension and weight. In the entropy regime with small crossing, $k_{\mathrm{alg}}\leq k^{\star}/2\text{,}$ the smallest dyadic $k\geq\max\{k_{\mathrm{alg}},2\}$ stays below $k^{\star}\text{,}$ and Equation 4.7 with the surrogate clause fired at $k_{\mathrm{alg}}$ gives an amplitude-free admissible subspace within squared distance $C\sigma^{2}k_{\mathrm{alg}}$ of $\mu\text{,}$ at dimension plus weight $O(k_{\mathrm{alg}})\text{,}$ so no quantization is paid. In the entropy regime with large crossing, $k_{\mathrm{alg}}>k^{\star}/2\text{,}$ the grids take over: for $V\geq\sigma/2\text{,}$ outside an event of loss contribution $O(\sigma^{2})\text{,}$ the anchored grid contains an amplitude $V^{\prime}\geq V$ with $\mathbb{E}(V^{\prime}/V)^{2}=O(1)$ and index price $O(\sigma^{2})\text{,}$ and, according to the regime that $V^{\prime}$ sees, the approximant of Proposition 3.1 at the dyadic budget nearest its crossing, the root candidate, or the full model is a candidate whose oracle value is $C(R^{*}_{T}(V,\sigma)+\sigma^{2})$ in expectation, by the scale bound $R^{*}_{T}(cV,\sigma)\leq Cc^{2}R^{*}_{T}(V,\sigma)$ for $c\geq 1\text{;}$ for $V<\sigma/2$ the low-amplitude grid plays the same role, its index price absorbed because $k_{\mathrm{alg}}>k^{\star}/2$ places the rate above $c\,\sigma^{2}\log n$ (Section S.6). The additive $\sigma^{2}$ collects the root replacement, the anchored-grid prices, and the constant floor in Equation 4.8; the two families are complementary, amplitude quantization being expensive only at small crossing, where the subspaces are few enough to enumerate. The selection runs in deterministic polynomial time, each grid candidate’s inner minimization over $\mathcal{X}_{k}$ being a min-plus analogue of Section 4.2; the operation counts, the grid moment bounds, and the conditioning argument are completed in Section S.6. ∎

It remains to remove the noise level, which enters the selection only through the penalties; these tolerate a constant-factor error, so a robust estimate suffices. The regime in which the estimate succeeds is described by a width functional of the tree. At each internal vertex of $T$ fix a child $c$ maximizing the number of vertices $u$ with $c\preceq u\text{,}$ breaking ties by a fixed rule; call the edge to that child heavy, every other child edge light, and a vertex branching when it has at least two children. Set $w_{T}:=w_{\mathrm{lt}}(T)+w_{\mathrm{br}}(T)\text{,}$ where $w_{\mathrm{lt}}(T)$ is one plus the maximum number of light edges on a root-to-leaf path, and $w_{\mathrm{br}}(T)$ is the maximum number of branching vertices on such a path.

Theorem 4 (Full adaptation).

There are universal constants $c,C>0$ and a deterministic polynomial-time estimator $\widehat{\mu}=\widehat{\mu}(T,Y)\text{,}$ depending on neither $V$ nor $\sigma\text{,}$ such that for every finite rooted tree $T$ and all $V,\sigma>0\text{,}$

$$\sup_{\mu\in\mathcal{F}_{V}(T)}\mathbb{E}_{\mu}\|\widehat{\mu}-\mu\|_{2}^{2}\ \leq\ C\bigl(R^{*}_{T}(V,\sigma)+\sigma^{2}\bigr)\qquad\text{whenever}\quad Vw_{T}\leq c\sigma n.$$

For $n$ below a universal threshold the estimator behind Theorem 4 returns $Y\text{,}$ at risk $O(\sigma^{2})\text{;}$ above it, the estimator rounds a median. The tree supplies at least $(n-1)/2$ difference statistics: heavy-edge and sibling differences, each Gaussian of variance $2\sigma^{2}$ around a signal difference. Monotonicity caps the contamination collectively: heavy differences telescope along vertex-disjoint heavy paths to their tops, sibling excesses charge their branching vertices, and a root path meets at most $w_{T}$ of these sites; for every $\varepsilon>0\text{,}$ at most $Vw_{T}/(\varepsilon\sigma)$ statistics therefore have mean magnitude at least $\varepsilon\sigma\text{.}$ Under $Vw_{T}\leq c\sigma n\text{,}$ the normalized median of the absolute values of an independent subfamily then lands in $[\sigma/2,\,2\sigma]$ except with probability $2e^{-cn}\text{;}$ rounded upward to a power of two, the estimate takes at most four values on that event, the penalties of Theorem 3 stay bracketed at the estimated scale, a constant weight shift pays for the union over the four values, and the off-event loss contributes $O(\sigma^{2})\text{.}$ The statistics, the contamination count, the concentration of the median, and the moment bookkeeping are carried out in Section S.6.

Uniform recovery of the noise level has a ceiling: once monotone flows with root value of order $\sigma n\sqrt{\log n}$ are admitted, every estimator of $\sigma$ fails, with constant probability, to land strictly within a factor $\sqrt{2}$ of the truth (Section S.6). On families of bounded width, stars and brooms among them, the scale-based route of Theorem 4 thus operates within a $\sqrt{\log n}$ factor of its ceiling.

5 The Suboptimality of Least Squares

The last theorem prices the default estimator. The body is compact and convex, so the data have a unique Euclidean projection onto it, the constrained least squares estimator

$$\widehat{\mu}_{\mathrm{LSE}}\ :=\ \Pi_{\mathcal{F}_{V}(T)}(Y)\ =\ \operatorname*{arg\,min}_{\theta\in\mathcal{F}_{V}(T)}\ \bigl\|Y-\theta\bigr\|_{2}^{2},$$

the solution of a tuning-free convex program; write

$$R^{\mathrm{worst}}_{\mathrm{LSE}}(T,V,\sigma)\ :=\ \sup_{\mu\in\mathcal{F}_{V}(T)}\mathbb{E}_{\mu}\bigl\|\widehat{\mu}_{\mathrm{LSE}}-\mu\bigr\|_{2}^{2}$$

for its worst-case risk.

Theorem 5 (Least squares).

There is a universal constant $C$ such that for every finite rooted tree with $n\geq 2$ and all $V,\sigma>0\text{,}$

$$R^{\mathrm{worst}}_{\mathrm{LSE}}(T,V,\sigma)\ \leq\ C\,\bigl(1+\log(en)\bigr)^{2/5}\,R^{*}_{T}(V,\sigma).$$

Conversely, there are universal constants $c>0$ and $n_{0}$ such that the following holds for every integer $n\geq n_{0}\text{.}$ Let $L$ be the largest integer with $e^{20L^{5/2}}\leq\sqrt{n}\text{,}$ and let $T_{L}$ be the broom on $n$ vertices whose handle $o=v_{0},v_{1},\dots,v_{L}$ is a path of $L$ edges and whose endpoint $v_{L}$ has $m:=n-L-1$ leaf children. Then, at budget $V=L^{1/4}$ and noise level $\sigma=1\text{,}$

$$R^{\mathrm{worst}}_{\mathrm{LSE}}\bigl(T_{L},L^{1/4},1\bigr)\ \geq\ c\,(\log n)^{2/5}\,R^{*}_{T_{L}}\bigl(L^{1/4},1\bigr).$$

We prove the broom lower bound first: the broom isolates the mechanism, extreme-value forcing through a terminal star, and every numerical threshold in the forcing step is explicit (Section S.4). The universal upper bound shows the resulting separation extremal over all trees and parameters; its proof, sketched at the end of the section, runs the coded cover of Section 3.1 through a localized-width analysis.

The broom couples the two classical geometries in series, a path feeding a star. For integers $L\geq 1$ and $m\geq 1\text{,}$ the broom $T_{L}$ consists of a handle, the path $o=v_{0},v_{1},\dots,v_{L}$ of $L$ edges, and of $m$ terminal leaves $w_{1},\dots,w_{m}\text{,}$ the children of $v_{L}\text{,}$ so that $n=L+1+m$ and $h_{T_{L}}=H_{T_{L}}=L+1\text{;}$ the budget and the noise level are fixed at $V:=L^{1/4}$ and $\sigma:=1\text{.}$

Proof of Theorem 5, lower bound.

Fix $n\geq n_{0}\text{,}$ with $n_{0}$ a universal constant large enough for the estimates below. The maximality of $L$ gives $e^{20L^{5/2}}\leq\sqrt{n}<e^{20(L+1)^{5/2}}\text{;}$ for $n\geq n_{0}$ it forces $L\geq 4$ and $L+1\leq\sqrt{n}\text{,}$ whence $m=n-L-1\geq n-\sqrt{n}\geq\sqrt{n}\geq e^{20L^{5/2}}\text{.}$ The true signal is $\mu=Vp_{o}\text{:}$ all mass rests at the root, and every nonroot coordinate vanishes. We show that one extreme leaf noise forces the projection to carry more than half the budget through the entire handle, at squared loss exceeding $L^{3/2}/4\text{,}$ on an event of probability at least $3/8\text{;}$ and that the minimax risk of the broom is of order $\sqrt{L}\text{.}$

Coordinates. Write $h_{j}$ for the coordinate of a body point at $v_{j}$ and $\ell_{i}$ for its coordinate at $w_{i}\text{.}$ By Lemma 2.1, nonnegativity of the leaks identifies the body, in these coordinates, with the chain

$$V\ \geq\ h_{1}\ \geq\ h_{2}\ \geq\ \cdots\ \geq\ h_{L}\ \geq\ \sum_{i=1}^{m}\ell_{i},\qquad\ell_{i}\ \geq\ 0\quad(1\leq i\leq m).$$

(5.1)

The root coordinate equals $V$ on the whole body, and at the true signal the nonroot data are pure noise: $Y_{v_{j}}=Z_{v_{j}}$ and $Y_{w_{i}}=Z_{w_{i}}\text{.}$ Expanding $\|Y-\theta\|_{2}^{2}$ and discarding the terms that do not depend on $\theta$ shows that the nonroot coordinates of the projection maximize over Equation 5.1 the objective

$$\Psi(h,\ell)\ :=\ 2\sum_{j=1}^{L}Z_{v_{j}}h_{j}-\sum_{j=1}^{L}h_{j}^{2}\ +\ 2\sum_{i=1}^{m}Z_{w_{i}}\ell_{i}-\sum_{i=1}^{m}\ell_{i}^{2}.$$

(5.2)

The forcing inequality. Aggregate the noise into three statistics, local to this proof:

$$W:=\sum_{j=1}^{L}Z_{v_{j}},\qquad Q:=\sum_{j=1}^{L}\bigl[Z_{v_{j}}\bigr]_{+}^{2},\qquad M:=\max_{1\leq i\leq m}Z_{w_{i}}.$$

Routing the full budget to a leaf attaining $M\text{,}$ that is, taking $h_{j}=V$ for every $j\text{,}$ $\ell_{i^{\star}}=V$ at one index $i^{\star}$ with $Z_{w_{i^{\star}}}=M\text{,}$ and $\ell_{i}=0$ elsewhere, is feasible in Equation 5.1. Its objective value is $2VW+2VM-(L+1)V^{2}\text{.}$ On the other side, suppose $M>0\text{;}$ then every feasible point with $h_{L}\leq V/2$ obeys

$$\Psi(h,\ell)\ \leq\ Q+MV,$$

(5.3)

since $2Z_{v_{j}}h_{j}-h_{j}^{2}\leq[Z_{v_{j}}]_{+}^{2}$ for every $h_{j}\geq 0\text{,}$ while discarding $-\sum_{i}\ell_{i}^{2}$ and replacing each $Z_{w_{i}}$ by $M$ bounds the leaf part by $2M\sum_{i}\ell_{i}\leq 2Mh_{L}\leq MV\text{,}$ the last two steps by Equation 5.1 and by the hypothesis $h_{L}\leq V/2$ (Section S.4). The routed candidate beats the cap Equation 5.3 exactly when

$$M\ >\ (L+1)V-2W+\frac{Q}{V}\ .$$

(5.4)

When $M>0$ and Equation 5.4 hold, no maximizer of $\Psi$ can have $h_{L}\leq V/2\text{,}$ so the coordinates $(\widehat{h},\widehat{\ell})$ of $\widehat{\mu}_{\mathrm{LSE}}$ satisfy $\widehat{h}_{L}>V/2\text{;}$ the chain Equation 5.1 propagates the bound up the handle, and since the true nonroot signal is zero,

$$\bigl\|\widehat{\mu}_{\mathrm{LSE}}-Vp_{o}\bigr\|_{2}^{2}\ \geq\ \sum_{j=1}^{L}\widehat{h}_{j}^{\,2}\ >\ L\,\frac{V^{2}}{4}\ =\ \frac{L^{3/2}}{4}\ .$$

(5.5)

The forcing event. Let

$$E\ :=\ \{W\geq-2\sqrt{L}\}\cap\{Q\leq 2L\}\cap\{M\geq 4L^{5/4}\}.$$

On $E\text{,}$ with $V=L^{1/4}\text{,}$ the right side of Equation 5.4 is less than $4L^{5/4}\leq M$ for $L\geq 4\text{,}$ so the forcing inequality holds; Markov’s inequality on the handle statistics and a Gaussian tail estimate on the leaf maximum, using $m\geq e^{20L^{5/2}}\text{,}$ give $\mathbb{P}(E)\geq\tfrac{3}{8}$ (Section S.4). Combining with Equation 5.5,

$$\mathbb{E}_{Vp_{o}}\bigl\|\widehat{\mu}_{\mathrm{LSE}}-Vp_{o}\bigr\|_{2}^{2}\ \geq\ \frac{L^{3/2}}{4}\,\mathbb{P}(E)\ \geq\ \frac{3}{32}\,L^{3/2}.$$

(5.6)

The minimax rate. The ancestor-covering numbers of the broom are explicit: $N^{\uparrow}_{T}(0)=n\text{,}$ and for integers $1\leq j\leq L\text{,}$

$$N^{\uparrow}_{T}(j)\ =\ \Bigl\lceil\frac{L+2}{j+1}\Bigr\rceil.$$

(5.7)

Both directions follow by counting along the root-to-leaf path $v_{0},\dots,v_{L},w_{1}\text{;}$ the single center $v_{L+1-j}$ covers every leaf at distance exactly $j$ (Section S.4). Thus every radius $r\geq 1$ sees a covering count independent of $m\text{:}$ the terminal star enters the profile Equation 1.4 only through radii $r<1\text{,}$ where the truncation caps its term at $k\text{.}$ What survives is path geometry: Section S.4 turns Equation 5.7 into the profile bounds

$$\delta_{k}(T_{L})\ \leq\ \max\Bigl\{1,\ \frac{2(L+2)}{ek^{2}}\Bigr\}\quad\text{for }k\geq 1,\qquad\quad\delta_{k}(T_{L})\ \geq\ \frac{L+2}{2ek^{2}}\quad\text{for }k\leq\frac{L+2}{2e}.$$

(5.8)

The crossing then solves at $k_{0}\asymp(V^{2}(L+2)/\sigma^{2})^{1/3}\asymp\sqrt{L}\text{,}$ with the dimension clause of Equation 1.5 silent and the diameter branch of Theorem 1 inactive, so

$$R^{*}_{T_{L}}\bigl(L^{1/4},1\bigr)\ \asymp\ k_{0}\ \asymp\ \sqrt{L}$$

(5.9)

(Section S.4).

Conclusion. The maximality of $L$ gives $\sqrt{n}<e^{20(L+1)^{5/2}}\text{,}$ so $\log n<40(L+1)^{5/2}$ and $L\geq c(\log n)^{2/5}\text{.}$ Dividing Equation 5.6 by Equation 5.9 gives a ratio at least $cL\geq c^{\prime}(\log n)^{2/5}$ at this signal, and the worst-case risk dominates this single-signal risk. ∎

The upper half is a statement about every tree and every parameter pair at once; its proof has three blocks, completed in Section S.7.

Proof sketch of Theorem 5, upper bound.

A localized projection principle. For $\mu\in\mathcal{F}_{V}(T)$ and $t>0\text{,}$ let

$$w_{\mu}(t)\ :=\ \mathbb{E}\,\sup\bigl\{\langle Z,\theta-\mu\rangle:\ \theta\in\mathcal{F}_{V}(T),\ \|\theta-\mu\|_{2}\leq t\bigr\}$$

be the localized Gaussian width of the body at the truth. If $t\geq\sigma$ and $\sigma\,w_{\mu}(s)\leq s^{2}/4$ for every $s\geq t\text{,}$ then $\mathbb{E}_{\mu}\|\widehat{\mu}_{\mathrm{LSE}}-\mu\|_{2}^{2}\leq 33\,t^{2}\text{.}$ On the event $\|\widehat{\mu}_{\mathrm{LSE}}-\mu\|_{2}\geq s\text{,}$ projection optimality forces the supremum defining $w_{\mu}(s)$ above twice its mean; the supremum is $s$-Lipschitz in $Z\text{,}$ so Gaussian concentration bounds its probability, and integrating the tail from $t$ gives the claim (Section S.7). Localizing least squares at a fixed point of the width is Chatterjee’s argument (Chatterjee, 2014).

The width of the body through the coded cover. Fix an integer $2\leq k\leq n/C_{\star}\text{,}$ write $\overline{\delta}:=\overline{\delta}_{k}(T)$ for the surrogate profile Equation 2.1, and let $\ell_{n}:=1+\log(en)\text{.}$ Section S.7 proves the truth-localized width bound

$$w_{\mu}(t)\ \leq\ C\,\bigl(t+V\sqrt{\overline{\delta}}\,\bigr)\sqrt{k}\ +\ C\,V\Bigl[\sqrt{k\overline{\delta}}\,\log(2+\ell_{n})+\sqrt{\overline{\delta}\,\ell_{n}}\,\Bigr]\qquad\text{for all }t>0.$$

(5.10)

The first term is the width of the coded cover $\mathcal{D}_{k}$ seen from the truth: recentering at a deterministic center $\nu_{\mu}\in\mathcal{D}_{k}$ (Proposition 3.1) confines every center met by the local ball to distance $t+2CV\sqrt{\overline{\delta}}\text{,}$ and the class has logarithmic cardinality $O(k)\text{,}$ so the maximum of the recentered pairings obeys a $\sqrt{k}$ bound. The second term bounds the residual $\theta-\nu_{\theta}\text{,}$ uniformly over the body, through the three components of the construction behind Proposition 3.1: the collapse and rounding residuals, bounded by a capped order-statistics sum and a union of norm concentrations over $e^{Ck}$ subspaces at $CV\sqrt{k\overline{\delta}}$ (Section S.7); and the exit quantization. Conditionally on $(\theta,Z)\text{,}$ the pairing of $Z$ with the quantization error is centered, so on the sampling event of that construction, of probability at least $\tfrac{1}{4}\text{,}$ its conditional expectation is at most four conditional standard deviations. Some realization in the event meets this bound, and the randomized rounding becomes, for each noise outcome, a deterministic choice of a nearby center in $\mathcal{D}_{k}\text{.}$

Closure of the exponent. Let $2\leq k_{0}\leq K\text{,}$ where $R^{*}_{T}\asymp\sigma^{2}k_{0}$ (Section 3.3). The crossing Equation 1.5 and the sandwich $\overline{\delta}_{k_{0}}\leq 2\delta_{k_{0}}$ give $V\sqrt{\overline{\delta}_{k_{0}}}\leq C\sigma\sqrt{k_{0}}\text{,}$ hence also $V\sqrt{k_{0}\overline{\delta}_{k_{0}}}\leq C\sigma k_{0}\text{.}$ Inserting these into Equation 5.10 at $k=k_{0}$ and feeding the result to the localization principle at $t^{2}\asymp\sigma^{2}[\,k_{0}\log(2+\ell_{n})+\sqrt{k_{0}\,\ell_{n}}\,]$ gives

$$\frac{R^{\mathrm{worst}}_{\mathrm{LSE}}(T,V,\sigma)}{R^{*}_{T}(V,\sigma)}\ \leq\ C\,\Bigl[\log(2+\ell_{n})+\sqrt{\ell_{n}/k_{0}}\,\Bigr].$$

(5.11)

This bound deteriorates as $k_{0}$ shrinks, but a small crossing index strengthens the projection: the profile remembers the height, $\delta_{k}\geq cH_{T}/k^{2}$ for $k\leq n/C_{\star}$ (Section S.7), so the crossing forces $V^{2}H_{T}\leq C\sigma^{2}k_{0}^{3}\text{,}$ while $R^{\mathrm{worst}}_{\mathrm{LSE}}\leq V^{2}H_{T}$ holds, the truth and its projection lying in the body. Hence

$$\frac{R^{\mathrm{worst}}_{\mathrm{LSE}}(T,V,\sigma)}{R^{*}_{T}(V,\sigma)}\ \leq\ C\,k_{0}^{2}.$$

(5.12)

The two bounds meet at $k_{0}=\ell_{n}^{1/5}\text{:}$ for $k_{0}\leq\ell_{n}^{1/5}\text{,}$ Equation 5.12 is at most $C\ell_{n}^{2/5}\text{;}$ for $k_{0}>\ell_{n}^{1/5}\text{,}$ Equation 5.11 is at most $C[\log(2+\ell_{n})+\ell_{n}^{2/5}]\leq C^{\prime}\ell_{n}^{2/5}\text{.}$ At the endpoints $k_{0}=K+1$ and $k_{0}=1$ the deterministic caps $R^{\mathrm{worst}}_{\mathrm{LSE}}\leq\min\{V^{2}H_{T},\ \sigma^{2}n\}$ close the argument against $R^{*}_{T}$ (Section S.7). ∎

The broom is where the two geometries of this paper part ways. Euclidean geometry sees $m$ orthogonal leaf directions, and the extreme noise among them pulls the projection through the handle; the ancestor geometry sees, at every radius $r\geq 1\text{,}$ one vertex that covers the entire star, and the truncation caps its cost at the information budget rather than per leaf. The truncated ancestor profile is therefore the right geometry for this body. On every finite tree it determines the minimax rate and yields an estimator evaluated exactly, one that survives the removal of the budget at an additive cost of order $\sigma^{2}\text{;}$ across all trees it prices the blindness of the default convex program at the power $2/5$ of the logarithm.

The appendices below contain the complete proofs. References of the form Lemma 2.3, (1.4), or Section 2 point to the main body above; statements and equations proved here carry the prefix S, and the notation of the main body remains in force.

S.1 The Profile Toolkit

This section proves the statements deferred from Section 2, Lemmas 2.1 to 2.3, and verifies two comparisons stated in Section 1.1: the center convention behind (1.6), and the capped constraint of Chatterjee and Lafferty (2018). The two normal forms and the isometry come first; the computational claims of Lemma 2.3 then rest on an exact linear-time algorithm for minimum ancestor nets.

Proof of Lemma 2.1.

Item (1) is equivalent to item (2). Definition (1.2) reads $\mu=V\sum_{u}\lambda_{u}p_{u}$ with $\lambda$ a probability vector on $\mathsf{V}\text{,}$ and the substitution $s_{u}=V\lambda_{u}$ turns this into $\mu=\sum_{u}s_{u}p_{u}$ with $s_{u}\geq 0$ and $\sum_{u}s_{u}=V\text{.}$ For uniqueness, order $\mathsf{V}$ so that every vertex precedes its descendants, for instance by nondecreasing depth, and let $P$ be the matrix with columns $(p_{u})_{u\in\mathsf{V}}$ in that order. Its entry at row $v$ and column $u$ is $\mathbf{1}\{v\preceq u\}\text{,}$ which vanishes when $v$ follows $u\text{,}$ because a strict ancestor of $u$ precedes $u\text{;}$ and $p_{u}(u)=1$ on the diagonal. So $P$ is upper triangular with unit diagonal, hence $\det P=1$ and $P$ is invertible, and the coefficient vector is determined by $\mu\text{.}$

Item (2) implies item (3) and both displayed identities. Evaluating $\mu=\sum_{u}s_{u}p_{u}$ at $v$ gives

$$\mu(v)\ =\ \sum_{u\in\mathsf{V}}s_{u}\,\mathbf{1}\{v\preceq u\}\ =\ \sum_{u\succeq v}s_{u},$$

(S.1.1)

the first identity; at $v=o$ it reads $\mu(o)=\sum_{u}s_{u}=V\text{,}$ every vertex being a descendant of the root. The strict descendants of $v$ are partitioned by the subtrees $T_{c}\text{,}$ $c\in\operatorname{ch}(v)\text{,}$ so subtracting Equation S.1.1 at the children from Equation S.1.1 at $v$ leaves the single term $u=v\text{:}$

$$\mu(v)-\sum_{c\in\operatorname{ch}(v)}\mu(c)\ =\ \sum_{u\succeq v}s_{u}-\sum_{c\in\operatorname{ch}(v)}\sum_{u\succeq c}s_{u}\ =\ s_{v}.$$

This is the second identity; since $s\geq 0$ it gives $\mu(v)\geq\sum_{c\in\operatorname{ch}(v)}\mu(c)\text{,}$ which with $\mu(o)=V$ is item (3).

Item (3) implies item (2). Set $s_{v}:=\mu(v)-\sum_{c\in\operatorname{ch}(v)}\mu(c)\text{,}$ nonnegative by hypothesis. We prove Equation S.1.1 by induction from the leaves upward. At a leaf the sum over children is empty, so $s_{v}=\mu(v)\text{,}$ which is the claim. At an internal vertex, the same partition of the strict descendants of $v$ and the inductive hypothesis at the children give

$$\sum_{u\succeq v}s_{u}\ =\ s_{v}+\sum_{c\in\operatorname{ch}(v)}\sum_{u\succeq c}s_{u}\ =\ s_{v}+\sum_{c\in\operatorname{ch}(v)}\mu(c)\ =\ \mu(v).$$

At $v=o$ this yields $\sum_{u}s_{u}=\mu(o)=V\text{,}$ and reading Equation S.1.1 at every coordinate says exactly $\mu=\sum_{u}s_{u}p_{u}\text{.}$ ∎

Proof of Lemma 2.2.

Part (1). A vertex is a common ancestor of $u$ and $w$ precisely when it is an ancestor of their deepest common ancestor $u\wedge w\text{,}$ and a vertex $x$ has exactly $\operatorname{depth}(x)+1$ ancestors, namely the vertices of the segment from $o$ to $x\text{.}$ Hence

$$\langle p_{u},p_{w}\rangle\ =\ \#\{v:\ v\preceq u\ \text{and}\ v\preceq w\}\ =\ \operatorname{depth}(u\wedge w)+1,$$

and in particular $\|p_{x}\|_{2}^{2}=\operatorname{depth}(x)+1\text{.}$ Expanding the square,

$$\|p_{u}-p_{w}\|_{2}^{2}\ =\ \operatorname{depth}(u)+\operatorname{depth}(w)-2\operatorname{depth}(u\wedge w)\ =\ d_{T}(u,w),$$

the last equality because the $u$-to-$w$ path is the concatenation of the segments from $u$ up to $u\wedge w$ and from $u\wedge w$ down to $w\text{.}$ The diameter of a convex hull equals that of its generating set, so $\operatorname{diam}^{2}\mathcal{F}_{V}(T)=\max_{u,w}\|Vp_{u}-Vp_{w}\|_{2}^{2}=V^{2}H_{T}$ by (1.2).

Part (2). For $a\preceq b$ every ancestor of $a$ is an ancestor of $b\text{,}$ so

$$(p_{b}-p_{a})(v)\ =\ \mathbf{1}\{v\preceq b\}-\mathbf{1}\{v\preceq a\}\ =\ \mathbf{1}\{a\prec v\preceq b\},$$

that is, $p_{b}-p_{a}=\mathbf{1}_{(a,b]}\text{.}$ For general $u,w$ put $m:=u\wedge w$ and subtract the two vertical cases,

$$p_{u}-p_{w}\ =\ (p_{u}-p_{m})-(p_{w}-p_{m})\ =\ \mathbf{1}_{(m,u]}-\mathbf{1}_{(m,w]}.$$

The sets $(m,u]$ and $(m,w]$ are disjoint, since a common element would be a common ancestor of $u$ and $w$ strictly below $m\text{;}$ their union is the set of lower endpoints of the edges on the $u$-to-$w$ path, by the description of that path just used.

For the last assertion, let $u$ and $w$ lie in a connected subtree $C\text{.}$ The $u$-to-$w$ path then runs inside $C\text{,}$ so the support of $p_{u}-p_{w}$ consists of lower endpoints of edges of $C\text{.}$ Distinct edges have distinct lower endpoints, an edge being determined by its lower endpoint, so differences taken within edge-disjoint connected subtrees occupy disjoint coordinate sets. ∎

Fix an integer radius $q\geq 0\text{.}$ The residual-depth greedy computes a minimum ancestor $q$-net in one postorder pass, maintaining one number per vertex. Process the vertices in postorder, with a fixed child order, and at each vertex $v$ compute, from the values $r_{c}$ already assigned at its children,

$$d_{v}:=\max\bigl(\{0\}\cup\{r_{c}+1:\ c\in\operatorname{ch}(v),\ r_{c}\neq-\infty\}\bigr);$$

(S.1.2)

if $d_{v}=q\text{,}$ select $v$ as a center and set $r_{v}:=-\infty\text{;}$ otherwise set $r_{v}:=d_{v}\text{.}$ After the root has been processed, select it as well if $r_{o}\neq-\infty\text{.}$ The number $r_{v}$ tracks the depth below $v$ of the deepest vertex of $T_{v}$ that no selected center covers, with $-\infty$ standing for none; the proof below verifies this reading.

Lemma S.1.1.

For every integer $q\geq 0\text{,}$ the residual-depth greedy outputs a minimum ancestor $q$-net of $T$ in $O(n)$ operations, and the output is a deterministic function of $(T,q)\text{.}$

Proof.

We first verify feasibility, then exhibit the selected centers, in a suitable order, as a run of a conceptually simpler procedure, the deepest-uncovered greedy, and finally prove by an exchange argument that every run of that procedure is minimum.

Feasibility. Call a vertex uncovered at a given moment when no center selected so far is an ancestor of it within distance $q\text{.}$ We claim that after $v$ is processed, $r_{v}$ is exactly the maximal depth below $v$ of an uncovered vertex of $T_{v}\text{,}$ with $r_{v}=-\infty$ when none exists, and that $r_{v}<q\text{.}$ When $v$ is examined, every selected center is an already processed vertex, and no ancestor of $v$ has been processed, so $v$ is uncovered. Moreover, an ancestor of a vertex of $T_{c}\text{,}$ for a child $c$ of $v\text{,}$ lies in $T_{c}$ or strictly above $c\text{,}$ and no center selected after $c$ was processed is of either kind: the subtree $T_{c}$ is finished, and the vertices above $c$ are not yet processed. The uncovered part of $T_{c}$ is therefore still the one described by $r_{c}\text{,}$ and the uncovered vertices of $T_{v}$ are $v$ itself together with those of the children’s subtrees, one level deeper. Hence $d_{v}$ in Equation S.1.2 is exactly the maximal depth below $v$ of an uncovered vertex of $T_{v}\text{,}$ and $d_{v}\leq q$ by the inductive bound $r_{c}<q\text{.}$ If $d_{v}=q\text{,}$ selecting $v$ covers every uncovered vertex of $T_{v}\text{,}$ all being descendants of $v$ within distance $q\text{,}$ and $r_{v}=-\infty$ is correct; otherwise no center is added and $r_{v}=d_{v}<q$ is correct. When the root has been processed, either $r_{o}=-\infty$ and nothing remains uncovered, or the final selection covers the remaining uncovered vertices, which lie within $r_{o}<q$ of the root. Each vertex is examined once at cost proportional to its number of children, so the pass costs $O(n)\text{;}$ and the child order being fixed, the algorithm makes no arbitrary choices, so the output is a deterministic function of $(T,q)\text{.}$

Reduction to the deepest-uncovered greedy. The comparison procedure repeats the following step until every vertex is covered: pick an uncovered vertex $u$ of maximum depth and select its ancestor at distance exactly $\min\{q,\operatorname{depth}(u)\}\text{.}$ To exhibit the algorithm’s output as a run of this procedure, attach to every center $g$ selected through the test $d_{g}=q$ a witness $u_{g}\in T_{g}$ realizing $d_{g}\text{,}$ so that $d_{T}(g,u_{g})=q\text{;}$ if the root is selected at the end, its witness is a deepest vertex left uncovered, at distance $r_{o}<q\text{.}$ Order the selected centers by nonincreasing depth, ties broken by processing order, with the final root selection last, and let $P$ denote the set of centers preceding a given center in this order.

Fix a center $g$ with $d_{T}(g,u_{g})=q\text{.}$ First, $u_{g}$ is uncovered when only $P$ is in place: a center covering $u_{g}$ is an ancestor of $u_{g}$ of depth at least $\operatorname{depth}(u_{g})-q=\operatorname{depth}(g)\text{,}$ so it lies on the vertical segment between $g$ and $u_{g}\text{,}$ inside $T_{g}\text{,}$ and was processed, and selected, before $g\text{;}$ had it covered $u_{g}\text{,}$ then $u_{g}$ would already be covered when $g$ was examined and could not realize $d_{g}\text{.}$ Second, every vertex $w$ strictly deeper than $u_{g}$ is covered by $P\text{:}$ the complete output is feasible, so some selected center $c$ satisfies $c\preceq w$ and $d_{T}(c,w)\leq q\text{,}$ whence

$$\operatorname{depth}(c)\ \geq\ \operatorname{depth}(w)-q\ >\ \operatorname{depth}(u_{g})-q\ =\ \operatorname{depth}(g),$$

and every selected center strictly deeper than $g$ precedes it. Thus at $g$’s turn the vertex $u_{g}$ is a deepest uncovered vertex, and $g$ is its ancestor at distance exactly $q=\min\{q,\operatorname{depth}(u_{g})\}\text{.}$

For the final root selection with witness $u\text{:}$ no nonroot center covers the witness, which is still uncovered after the whole tree has been processed; every $w$ strictly deeper than $u$ is covered by a nonroot center, since otherwise $w$ would be an uncovered vertex deeper than $u$ at that moment, contradicting the maximality defining $r_{o}\text{;}$ and the root is the ancestor of $u$ at distance $\operatorname{depth}(u)=\min\{q,\operatorname{depth}(u)\}\text{,}$ since $\operatorname{depth}(u)=r_{o}<q\text{.}$

Optimality by exchange. Suppose, inductively, that some minimum ancestor $q$-net $O$ contains the first $t-1$ centers of a run of the deepest-uncovered greedy; let $u$ be the uncovered vertex of maximum depth picked at step $t$ and $g$ its selected ancestor. Feasibility of $O$ provides $c\in O$ with $c\preceq u$ and $d_{T}(c,u)\leq q\text{;}$ being an ancestor of $u$ of depth at least $\operatorname{depth}(u)-q\text{,}$ the center $c$ lies on the segment between $g$ and $u\text{,}$ and $c$ is not among the first $t-1$ centers, which do not cover $u\text{.}$ Set $O^{\prime}:=(O\setminus\{c\})\cup\{g\}\text{.}$ A vertex $x$ covered by $c$ but not by $g$ is a descendant of $g\text{,}$ as $g\preceq c\preceq x\text{,}$ with $d_{T}(g,x)>q\text{,}$ so that

$$\operatorname{depth}(x)\ >\ \operatorname{depth}(g)+q\ \geq\ \operatorname{depth}(u),$$

and $x$ is covered by the first $t-1$ centers, which lie in $O^{\prime}\text{.}$ Thus $O^{\prime}$ is feasible, contains the first $t$ centers, and $|O^{\prime}|\leq|O|\text{,}$ so $O^{\prime}$ is again a minimum net. By induction the complete run is contained in a minimum net, and being itself feasible, it is minimum. ∎

Proof of Lemma 2.3.

Throughout, write $c_{q}:=\min\{k,[\log(N^{\uparrow}_{T}(q)/k)]_{+}\}$ for the truncated factor of the profile at integer radius $q\text{,}$ the dependence on the fixed index $k$ suppressed from the notation.

Part (1). Dividing the truncated factor in (1.4) by $k$ and using $\min\{k,x\}/k=\min\{1,x/k\}$ gives

$$\delta_{k}(T)=\sup_{r>0}\ r\,\min\Bigl\{1,\ \frac{1}{k}\Bigl[\log\frac{N^{\uparrow}_{T}(r)}{k}\Bigr]_{+}\Bigr\}.$$

(S.1.3)

Fix $r\text{.}$ As $k$ increases, $[\log(N^{\uparrow}_{T}(r)/k)]_{+}$ is nonnegative and nonincreasing, and so is $1/k\text{;}$ a product of two nonnegative nonincreasing functions is nonincreasing, and the cap by $1$ preserves this. Every term of the supremum in Equation S.1.3 is therefore nonincreasing in $k\text{,}$ and so is $\delta_{k}\text{.}$ For the upper bound, the root is an ancestor of every vertex within distance $h_{T}\text{,}$ so $N^{\uparrow}_{T}(r)=1$ and the term vanishes for $r\geq h_{T}\text{,}$ while for $r<h_{T}$ the term is at most $r<h_{T}\text{;}$ hence $\delta_{k}\leq h_{T}\text{.}$

Part (2). Tree distances take integer values, so the covering constraint $d_{T}(a,u)\leq r$ is equivalent to $d_{T}(a,u)\leq\lfloor r\rfloor\text{,}$ and $N^{\uparrow}_{T}(r)=N^{\uparrow}_{T}(\lfloor r\rfloor)$ for every $r\geq 0\text{.}$ The supremum in (1.4) therefore splits along integer radii:

$$\alpha_{k}(T)=\sup_{q\geq 0}\ \sup_{r\in[q,q+1)\cap(0,\infty)}\ r\,c_{q}=\sup_{q\geq 0}\ (q+1)\,c_{q},$$

the inner supremum being approached as $r\uparrow q+1\text{.}$ Radii $q\geq h_{T}$ have $N^{\uparrow}_{T}(q)=1$ and $c_{q}=0\text{,}$ so the outer supremum is a maximum over the integers $0\leq q<h_{T}\text{,}$ an empty maximum being zero; this also covers $n=1\text{,}$ where $h_{T}=0\text{.}$

Part (3): the two-sided comparison. Write $G:=\{2^{\ell}-1:\ \ell\in\{0,1,2,\dots\},\ 2^{\ell}\leq h_{T}\}$ for the grid, so that (2.1) reads $\overline{\delta}_{k}=\tfrac{2}{k}\max_{q\in G}(q+1)c_{q}\text{.}$ The grid consists of integer radii below $h_{T}\text{,}$ so part (2) gives $\max_{q\in G}(q+1)c_{q}\leq\alpha_{k}\text{,}$ that is, $\overline{\delta}_{k}\leq 2\delta_{k}\text{.}$ Conversely, fix an integer $0\leq q<h_{T}$ and set $s:=q+1\in[1,h_{T}]\text{;}$ the largest power of two $p\leq s$ satisfies $p>s/2\text{,}$ and $q^{\prime}:=p-1$ belongs to $G$ with $q^{\prime}\leq q\text{.}$ Every ancestor $q^{\prime}$-net is an ancestor $q$-net, so $N^{\uparrow}_{T}(q^{\prime})\geq N^{\uparrow}_{T}(q)$ and $c_{q^{\prime}}\geq c_{q}\text{,}$ whence

$$(q^{\prime}+1)\,c_{q^{\prime}}\ =\ p\,c_{q^{\prime}}\ \geq\ \frac{s}{2}\,c_{q}\ =\ \frac{q+1}{2}\,c_{q}.$$

Maximizing over $q$ gives $\max_{q\in G}(q+1)c_{q}\geq\alpha_{k}/2\text{,}$ that is, $\overline{\delta}_{k}\geq\delta_{k}\text{.}$ When $h_{T}=0$ both sides vanish by convention. Monotonicity of $k\mapsto\overline{\delta}_{k}$ follows as in part (1), applied to the finite maximum over $G\text{.}$

Part (3): the sandwich. Put $K:=\lfloor n/C_{\star}\rfloor\text{.}$ The dimension clause $k>n/C_{\star}$ first holds at $k=K+1\text{,}$ so the minima in (1.5) and (2.2) are over nonempty sets and $k_{0},k_{\mathrm{alg}}\leq K+1\text{.}$ For the lower half: if $k_{0}\leq K\text{,}$ the dimension clause fails at $k_{0}\text{,}$ so the profile clause holds there, $V^{2}\overline{\delta}_{k_{0}}\leq 2V^{2}\delta_{k_{0}}\leq 2A_{0}\sigma^{2}k_{0}\text{,}$ and $k_{\mathrm{alg}}\leq k_{0}\text{;}$ if $k_{0}=K+1\text{,}$ then $k_{\mathrm{alg}}\leq K+1=k_{0}$ directly. For the upper half: if $k_{\mathrm{alg}}=K+1\text{,}$ the lower half forces $k_{0}=K+1=k_{\mathrm{alg}}\text{.}$ Otherwise $k_{\mathrm{alg}}\leq K\text{,}$ so the dimension clause of (2.2) fails at $k_{\mathrm{alg}}\text{,}$ its profile clause holds, and $V^{2}\delta_{k_{\mathrm{alg}}}\leq V^{2}\overline{\delta}_{k_{\mathrm{alg}}}\leq 2A_{0}\sigma^{2}k_{\mathrm{alg}}\text{.}$ If $k_{0}=k_{\mathrm{alg}}$ there is nothing to prove. If $k_{0}>k_{\mathrm{alg}}\text{,}$ every integer $k$ with $k_{\mathrm{alg}}\leq k\leq k_{0}-1$ fails both clauses of (1.5) by the minimality of $k_{0}\text{;}$ in particular $V^{2}\delta_{k}>A_{0}\sigma^{2}k\text{,}$ and combining this with the monotonicity of part (1),

$$A_{0}\sigma^{2}k\ <\ V^{2}\delta_{k}\ \leq\ V^{2}\delta_{k_{\mathrm{alg}}}\ \leq\ 2A_{0}\sigma^{2}k_{\mathrm{alg}},$$

so $k<2k_{\mathrm{alg}}\text{.}$ At $k=k_{0}-1$ this reads $k_{0}-1<2k_{\mathrm{alg}}\text{,}$ that is, $k_{0}\leq 2k_{\mathrm{alg}}\text{.}$

Part (3): complexity. Lemma S.1.1 computes $N^{\uparrow}_{T}(q)$ exactly in $O(n)$ operations for each integer radius. The grid $G$ has $O(\log(h_{T}+1))$ elements, so storing $\{N^{\uparrow}_{T}(q)\}_{q\in G}$ costs $O(n\log(h_{T}+1))\text{,}$ after which each evaluation of $\overline{\delta}_{k}$ costs $O(\log(h_{T}+1))\text{.}$ The predicate in (2.2) is monotone in $k\text{:}$ in its profile clause the left side is nonincreasing by the monotonicity proved above and the right side is increasing, and its dimension clause is monotone; $k_{\mathrm{alg}}$ is therefore located by binary search over $\{1,\dots,K+1\}$ with $O(\log n)$ predicate evaluations. Since $h_{T}<n\text{,}$ the total is $O(n\log(n+1))\text{.}$ For $k_{0}\text{,}$ running Lemma S.1.1 at every integer radius $0\leq q<h_{T}$ costs $O(n\,h_{T})$ and stores all counts; each $\alpha_{k}$ then costs $O(h_{T})$ by (2.3), and scanning $k=1,2,\dots$ until the first index satisfying (1.5), at most $K+1\leq n$ values, costs $O(n\,h_{T})$ in total. ∎

Finally, we verify the center-convention comparison stated with (1.6). Write $H^{c,\mathrm{int}}_{T}(t)$ for the covering entropy of $\mathcal{F}_{V}(T)$ at squared radius $t$ with centers restricted to the body, and

$$\rho^{\mathrm{int}}_{T}(V,\sigma)\ :=\ \inf_{t>0}\ \bigl[t+\sigma^{2}H^{c,\mathrm{int}}_{T}(t)\bigr].$$

Since every cover with centers in the body is in particular a cover with free centers, $H^{c}_{T}(t)\leq H^{c,\mathrm{int}}_{T}(t)$ for every $t>0\text{.}$ Conversely, fix $t>0$ and a cover of the body by $\exp H^{c}_{T}(t)$ balls of radius $\sqrt{t}$ with arbitrary centers, and replace the center $y$ of each ball by its Euclidean projection $\pi_{y}:=\Pi_{\mathcal{F}_{V}(T)}(y)$ onto the body. The variational inequality of the projection, $\langle y-\pi_{y},\mu-\pi_{y}\rangle\leq 0$ for every $\mu\in\mathcal{F}_{V}(T)\text{,}$ gives

$$\|\mu-\pi_{y}\|_{2}^{2}\ \leq\ \|\mu-y\|_{2}^{2}-\|y-\pi_{y}\|_{2}^{2}\ \leq\ \|\mu-y\|_{2}^{2},$$

so every $\mu$ within $\sqrt{t}$ of $y$ is within $\sqrt{t}$ of $\pi_{y}\text{.}$ The projected balls, of the same radius and number, cover the body with internal centers, so $H^{c,\mathrm{int}}_{T}(t)\leq H^{c}_{T}(t)\text{.}$ The two entropies coincide at every radius, and $\rho^{\mathrm{int}}_{T}=\rho_{T}\text{:}$ the center convention is immaterial.

We also verify the comparison with the capped constraint of Chatterjee and Lafferty (2018), stated in Section 1.1. Let

$$\mathcal{K}_{V}(T)\ :=\ \bigl\{\mu\in\mathcal{F}(T):\ \mu(o)\leq V\bigr\},\qquad R^{\mathrm{cap}}_{T}(V,\sigma)\ :=\ \inf_{\widehat{\mu}}\ \sup_{\mu\in\mathcal{K}_{V}(T)}\mathbb{E}_{\mu}\bigl\|\widehat{\mu}(Y)-\mu\bigr\|_{2}^{2},$$

the infimum over the same maps as in (1.3), and assume $n\geq 2\text{.}$ Since $\mathcal{F}_{V}(T)\subseteq\mathcal{K}_{V}(T)\text{,}$ every estimator has at least as large a supremum risk on the larger set, so $R^{\mathrm{cap}}_{T}\geq R^{*}_{T}\text{.}$

For the reverse, split $\mu$ into its root value $\mu(o)$ and its remaining coordinates $\mu_{-}\text{,}$ and write $C_{a}$ for the set of $\mu_{-}$ arising from $\mu\in\mathcal{F}_{a}(T)\text{.}$ Adding $(V-a)p_{o}$ to such a $\mu$ raises its root value to $V$ and changes no other coordinate, so $C_{a}\subseteq C_{V}$ whenever $0<a\leq V\text{,}$ and $C_{0}:=\{0\}\subseteq C_{V}$ as well. On the slice the root value is known, so it is estimated at no loss, while $Y_{o}$ is independent of $Y_{-}$ with a law free of $\mu_{-}$ and therefore acts as pure randomization, which cannot lower a maximum risk under squared loss; hence $R^{*}_{T}(V,\sigma)$ is exactly the minimax risk of estimating $\mu_{-}\in C_{V}$ from $Y_{-}\text{.}$ Fix $\varepsilon>0\text{,}$ let $\widehat{\theta}$ attain that risk to within $\varepsilon\text{,}$ and on the capped body take $\widehat{\mu}_{-}:=\widehat{\theta}(Y_{-})$ together with

$$\widehat{\mu}_{o}\ :=\ \begin{cases}0,&V\leq\sigma,\\[2.0pt] \Pi_{[0,V]}(Y_{o}),&V>\sigma.\end{cases}$$

Every $\mu\in\mathcal{K}_{V}(T)$ has $\mu_{-}\in C_{\mu(o)}\subseteq C_{V}\text{,}$ so its nonroot risk is at most $R^{*}_{T}(V,\sigma)+\varepsilon\text{.}$ Its root risk is at most $\mu(o)^{2}\leq V^{2}$ in the first case, and at most $\mathbb{E}(Y_{o}-\mu(o))^{2}=\sigma^{2}$ in the second, projection onto an interval containing $\mu(o)$ being nonexpansive. Letting $\varepsilon\downarrow 0\text{,}$

$$R^{\mathrm{cap}}_{T}(V,\sigma)\ \leq\ R^{*}_{T}(V,\sigma)+\min\{V^{2},\sigma^{2}\}.$$

Finally $n\geq 2$ forces $H_{T}\geq 1\text{,}$ so the two-point bound (3.8) gives

$$\min\{V^{2},\sigma^{2}\}\ \leq\ \min\{V^{2}H_{T},\ \sigma^{2}\}\ \leq\ c^{-1}R^{*}_{T}(V,\sigma):$$

fixing the root value and capping it pose the same problem up to universal constants.

The comparison is constructive: the estimator of Theorem 2 transfers to the capped body. At budget $V$ its root coordinate is identically $V\text{,}$ and its nonroot coordinates depend on $Y$ only through $Y_{-}\text{:}$ every integer state has root coordinate $k\text{,}$ so the root factor cancels from the ratio (4.2), and the branch selection in (4.3) depends on $(T,V,\sigma)$ alone. Write $\widehat{\mu}^{V}_{-}(Y_{-})$ for this nonroot part. For $\mu\in\mathcal{K}_{V}(T)$ the raised signal $(V,\mu_{-})$ lies in $\mathcal{F}_{V}(T)\text{,}$ and $Y_{-}$ has the law of the nonroot data of the slice model there, so Theorem 2 bounds the risk of $\widehat{\mu}^{V}_{-}$ against $\mu_{-}$ by $C\,R^{*}_{T}(V,\sigma)\text{,}$ uniformly over $\mathcal{K}_{V}(T)\text{.}$ Paired with the root rule above, it yields a deterministic estimator, computed in $O(n^{2}\log(n+1))$ arithmetic operations, whose risk on the capped body is at most $C\bigl(R^{*}_{T}(V,\sigma)+\min\{V^{2},\sigma^{2}\}\bigr)\leq C^{\prime}\,R^{\mathrm{cap}}_{T}(V,\sigma)\text{.}$

S.2 The Coded Cover

This section completes the proof of Proposition 3.1. The main text gives the mechanism behind each stage. To make this section self-contained, we restate the objects below and supply the estimates and accounting: the floor on the working scale and the sizes of the nets, the structure of the cells, the disjointness and charge accounting of the heavy hierarchy, the collapse and exit estimates, the edge count of the skeleton together with the explicit constants of the error assembly, and the counting bound for the coded cover.

Throughout, $k$ is an integer with $2\leq k\leq n/C_{\star}$ and, as in Section 3.1, $\bar{\alpha}:=k\overline{\delta}_{k}(T)\text{;}$ the level weights are $m_{j}=2^{j}$ for $0\leq j\leq J\text{,}$ with $m_{J}$ the largest power of two strictly below $k\text{;}$ $S_{j}$ is the minimum ancestor $\lfloor\bar{\alpha}/m_{j}\rfloor$-net computed by the greedy of Lemma S.1.1; and $R_{j}:=S_{0}\cup\dots\cup S_{j}\text{.}$ The net hierarchy, the active support and the activation charges below therefore depend on $(T,k)$ alone, the signal entering only through the stopped refinement of the construction. The active support, the first-appearance weight, and the activation charge are

$$A_{k}:=R_{J},\qquad\omega(v):=\min\{m_{j}:\ v\in R_{j}\},\qquad\widetilde{\omega}(v):=\omega(v)\,\mathbf{1}\{v\notin R_{0}\},$$

and the code length of an integer vector $z$ supported on $A_{k}$ is

$$\Gamma(z)\ :=\ \sum_{v\in A_{k}}\Bigl(|z_{v}|+\widetilde{\omega}(v)\,\mathbf{1}\{z_{v}\neq 0\}\Bigr),$$

the display (3.1). Throughout, $e_{v}$ denotes the coordinate vector at $v\text{.}$

The first lemma contains the size facts quoted when the nets are introduced: the working scale is never degenerate, and the profile itself caps the size of every net.

Lemma S.2.1 (Scale floor and net sizes).

For every integer $2\leq k\leq n/C_{\star}\text{:}$

1.

$\bar{\alpha}\geq\alpha_{k}(T)\geq 2$;
2.

$N^{\uparrow}_{T}(\lfloor\bar{\alpha}/m_{j}\rfloor)\leq ke^{m_{j}}$ for every $0\leq j\leq J$;
3.

$|R_{j}|\leq 2ke^{m_{j}}$ for every $0\leq j\leq J$.

Proof.

Part (1). The surrogate dominates the profile, $\overline{\delta}_{k}\geq\delta_{k}$ by Lemma 2.3, so $\bar{\alpha}=k\overline{\delta}_{k}\geq k\delta_{k}=\alpha_{k}(T)\text{.}$ For the floor, tree distances are integers, so at radii $0<r<1$ every vertex is its own only ancestor within distance $r\text{;}$ every ancestor $r$-net then contains all of $\mathsf{V}\text{,}$ and $N^{\uparrow}_{T}(r)=n\text{.}$ Hence

$$\alpha_{k}(T)\ \geq\ \sup_{0<r<1}\ r\,\min\Bigl\{k,\ \Bigl[\log\frac{n}{k}\Bigr]_{+}\Bigr\}\ =\ \min\Bigl\{k,\ \Bigl[\log\frac{n}{k}\Bigr]_{+}\Bigr\}\ \geq\ \min\{2,\ \log C_{\star}\}\ =\ 2,$$

using $k\geq 2$ and $n/k\geq C_{\star}=e^{2}\text{.}$

Part (2). Fix $0\leq j\leq J$ and suppose $N^{\uparrow}_{T}(\lfloor\bar{\alpha}/m_{j}\rfloor)>ke^{m_{j}}\text{.}$ The radius $r:=\bar{\alpha}/m_{j}$ is positive by part (1), and the covering constraint depends on $r$ only through $\lfloor r\rfloor\text{,}$ so $N^{\uparrow}_{T}(r)=N^{\uparrow}_{T}(\lfloor r\rfloor)>ke^{m_{j}}\text{,}$ that is, $[\log(N^{\uparrow}_{T}(r)/k)]_{+}>m_{j}\text{.}$ Since also $m_{j}\leq m_{J}<k\text{,}$ the minimum of $k$ and this positive part exceeds $m_{j}\text{,}$ and the term of (1.4) at $r$ satisfies

$$r\,\min\Bigl\{k,\ \Bigl[\log\frac{N^{\uparrow}_{T}(r)}{k}\Bigr]_{+}\Bigr\}\ >\ \frac{\bar{\alpha}}{m_{j}}\cdot m_{j}\ =\ \bar{\alpha}\ \geq\ \alpha_{k}(T),$$

contradicting that $\alpha_{k}(T)$ is the supremum of these terms.

Part (3). The sets $S_{i}$ are minimum nets, so $|S_{i}|=N^{\uparrow}_{T}(\lfloor\bar{\alpha}/m_{i}\rfloor)\leq ke^{m_{i}}$ by part (2), and $|R_{j}|\leq k\sum_{i\leq j}e^{m_{i}}\text{.}$ For $j\geq 1$ the sum has $j$ terms before the last, each at most $e^{m_{j-1}}\text{,}$ and $j\leq e^{2^{j-1}}$ gives $je^{m_{j-1}}\leq e^{m_{j-1}+2^{j-1}}=e^{m_{j}}\text{,}$ so $\sum_{i\leq j}e^{m_{i}}\leq 2e^{m_{j}}\text{;}$ for $j=0$ this is immediate. ∎

The cells inherit their structure from the nesting of the nets alone. Parts (2)–(4) below are the rooted-connectedness, refinement, and radius properties quoted in the main body; part (3) also identifies the parent of each cell, the hierarchy used from here on.

Lemma S.2.2 (Cell structure).

Fix $0\leq j\leq J$ and let $a_{j}(v)$ be the deepest ancestor of $v$ in $R_{j}\text{.}$ Then:

1.

the cells $B_{a,j}=\{v:a_{j}(v)=a\}$, $a\in R_{j}$, partition $\mathsf{V}$, and each contains its root $a$;
2.

if $v\in B_{a,j}$ and $a\preceq z\preceq v$, then $z\in B_{a,j}$;
3.

if $j<J$ and $a_{j+1}(u)=b$, then $a_{j}(u)=a_{j}(b)$; consequently every level-$(j{+}1)$ cell is contained in a single level-$j$ cell, whose root is an ancestor of its root;
4.

$d_{T}(a,v)\leq\lfloor\bar{\alpha}/m_{j}\rfloor$ for every $v\in B_{a,j}$.

Proof.

All four parts are read off the chain structure of ancestors and the nesting $R_{j}\subseteq R_{j+1}\text{.}$

Part (1). The root lies in every ancestor net, being its own only ancestor, so every vertex has an ancestor in $R_{j}\text{;}$ the ancestors of $v$ form a chain, so among them the elements of $R_{j}$ have a unique deepest member, and $a_{j}(v)$ is well defined. The fibers of $v\mapsto a_{j}(v)$ partition $\mathsf{V}\text{,}$ and $a_{j}(a)=a$ for $a\in R_{j}\text{.}$

Part (2). Every ancestor of $z$ is an ancestor of $v\text{,}$ so every $R_{j}$-ancestor of $z$ has depth at most $\operatorname{depth}(a_{j}(v))=\operatorname{depth}(a)\text{;}$ and $a$ itself is an $R_{j}$-ancestor of $z\text{,}$ since $a\preceq z\text{.}$ Hence $a_{j}(z)=a\text{.}$

Part (3). Let $s$ be an $R_{j}$-ancestor of $u\text{.}$ Then $s\in R_{j+1}\text{,}$ so $\operatorname{depth}(s)\leq\operatorname{depth}(b)\text{;}$ and $s$ and $b$ are comparable, both being ancestors of $u\text{,}$ so $s\preceq b$ and $s$ is an $R_{j}$-ancestor of $b\text{.}$ Conversely, every $R_{j}$-ancestor of $b$ is an ancestor of $u\text{,}$ since $b\preceq u\text{.}$ The $R_{j}$-ancestors of $u$ and of $b$ therefore coincide, and $a_{j}(u)=a_{j}(b)\text{.}$ In particular $B_{b,j+1}\subseteq B_{a_{j}(b),j}\text{,}$ and $a_{j}(b)\preceq b\text{.}$

Part (4). The net property supplies $s\in R_{j}$ with $s\preceq v$ and $d_{T}(s,v)\leq\lfloor\bar{\alpha}/m_{j}\rfloor\text{.}$ The deepest $R_{j}$-ancestor $a=a_{j}(v)$ has $\operatorname{depth}(a)\geq\operatorname{depth}(s)\text{,}$ so $a$ lies on the segment from $s$ to $v$ and $d_{T}(a,v)\leq d_{T}(s,v)\leq\lfloor\bar{\alpha}/m_{j}\rfloor\text{.}$ ∎

The remaining facts concern the stopped refinement that the proof of Proposition 3.1 runs for a fixed signal. Fix $f=\sum_{u}\lambda_{u}p_{u}$ with $\lambda_{u}\geq 0$ and $\sum_{u}\lambda_{u}=1\text{,}$ write $\lambda(B):=\sum_{u\in B}\lambda_{u}\text{,}$ write $m(B):=m_{j}$ for the level weight of a level-$j$ cell $B\text{,}$ and call $B$ heavy when $\lambda(B)>m(B)/k$ and light otherwise. The refinement begins at level $0\text{,}$ refines every heavy cell into its level-$(j{+}1)$ subcells, stops at every light cell, and stops everything at level $J\text{.}$ The cells at which it stops are the terminal cells. They partition $\mathsf{V}$ because the cells of one level do and each refinement step replaces a cell by a partition of it (Lemma S.2.2(1),(3)), and $\mathcal{L}$ denotes their collection. For $L\in\mathcal{L}$ we write $a_{L}$ for its root and $\lambda_{L}:=\lambda(L)\text{.}$ As in the main proof, $U$ consists of $R_{0}$ together with the roots of all heavy cells; a heavy cell is maximal when no heavy cell lies strictly below it in the hierarchy of Lemma S.2.2(3).

Lemma S.2.3 (The heavy structure).

In this setting:

1.

distinct maximal heavy cells have disjoint underlying vertex sets;
2.

the minimal rooted subtree $S$ spanning $U$ has at most $5\bar{\alpha}k$ edges;
3.

$\sum_{v\in U\setminus R_{0}}\widetilde{\omega}(v)<2k$;
4.

$\sum_{C\ \text{maximal heavy}}m(C)<k$, which is the display (3.3) of the main paper.

Proof.

Part (1) is a chain argument in the hierarchy, and part (4) follows from it; part (2) charges the skeleton to vertical segments along the nets; part (3) charges activation weights to maximal heavy cells.

Parts (1) and (4). Suppose two distinct heavy cells $(B,i)$ and $(B^{\prime},j)$ with $i\leq j$ have intersecting underlying sets. Cells at one level partition $\mathsf{V}\text{,}$ so $i<j\text{.}$ Iterating Lemma S.2.2(3), $B^{\prime}$ is contained in a unique cell of each level below $j\text{;}$ its level-$i$ cell meets $B\text{,}$ hence equals $B\text{,}$ and the chain of cells containing $B^{\prime}$ at levels $i,\dots,j$ ascends from $(B^{\prime},j)$ to $(B,i)$ in the hierarchy. Each cell of this chain, at level $\ell$ say, has mass at least $\lambda(B^{\prime})>m_{j}/k\geq m_{\ell}/k$ and is therefore heavy. In particular the chain member at level $i{+}1$ is a heavy cell strictly below $(B,i)\text{,}$ so $(B,i)$ is not maximal. Distinct maximal heavy cells therefore cannot intersect. Each carries mass exceeding $m(C)/k\text{,}$ and their supports being disjoint, their masses total at most $\lambda(\mathsf{V})=1\text{,}$ so

$$\sum_{C\ \text{maximal heavy}}m(C)\ <\ k,$$

which is part (4); the empty family satisfies it trivially.

Part (2). The subtree $S$ lies in the union of two families of vertical segments, whose total length we bound.

First, connect $R_{0}$ to the root. For $a\in R_{0}\setminus\{o\}\text{,}$ the net property of $R_{0}$ at the parent of $a$ supplies $s\in R_{0}$ with $s\preceq\operatorname{pa}(a)$ and $d_{T}(s,\operatorname{pa}(a))\leq\lfloor\bar{\alpha}\rfloor\text{;}$ the deepest strict $R_{0}$-ancestor $\bar{a}$ of $a$ then lies on the segment from $s$ to $a\text{,}$ so

$$d_{T}(\bar{a},a)\ \leq\ d_{T}(s,a)\ \leq\ \bar{\alpha}+1\ \leq\ \tfrac{3}{2}\,\bar{\alpha},$$

by Lemma S.2.1(1). Iterating $a\mapsto\bar{a}$ from any element of $R_{0}$ reaches the root, which belongs to every ancestor net, so the segments $[\bar{a},a]\text{,}$ $a\in R_{0}\setminus\{o\}\text{,}$ connect $R_{0}$ to $o\text{;}$ by Lemma S.2.1(2) their total length is at most $\tfrac{3}{2}\bar{\alpha}\,|R_{0}|\leq\tfrac{3}{2}e\,\bar{\alpha}k\text{.}$

Second, connect the heavy roots to $R_{0}\text{.}$ The root $b$ of a heavy cell $B$ at level $j\geq 1$ lies in its parent cell, whose root $a_{j-1}(b)$ is an ancestor of $b$ within distance $\lfloor\bar{\alpha}/m_{j-1}\rfloor\leq 2\bar{\alpha}/m_{j}$ (Lemma S.2.2(3)–(4)); and the parent cell is heavy as well, its mass being at least $\lambda(B)>m_{j}/k>m_{j-1}/k\text{.}$ Iterating these segments joins every heavy root to the root of a level-$0$ cell, which lies in $R_{0}\text{.}$ At level $j$ the heavy cells are disjoint and each has mass exceeding $m_{j}/k\text{,}$ so there are fewer than $k/m_{j}$ of them, and the segments they contribute have total length less than

$$\sum_{j\geq 1}\frac{k}{m_{j}}\cdot\frac{2\bar{\alpha}}{m_{j}}\ =\ 2\bar{\alpha}k\sum_{j\geq 1}4^{-j}\ =\ \tfrac{2}{3}\,\bar{\alpha}k.$$

Every member of $U$ is thus connected to the root inside the union of the two families, so $S$ is contained in that union and $|E(S)|\leq\tfrac{3}{2}e\,\bar{\alpha}k+\tfrac{2}{3}\,\bar{\alpha}k\leq 5\bar{\alpha}k\text{.}$

Part (3). The set $U\setminus R_{0}$ consists of roots of heavy cells at levels $j\geq 1\text{.}$ Charge each $v\in U\setminus R_{0}$ to a heavy cell rooted at $v$ of minimal level $j(v)\geq 1\text{;}$ roots of level-$j$ cells lie in $R_{j}\text{,}$ so $\widetilde{\omega}(v)=\omega(v)\leq m_{j(v)}\text{,}$ and the charging is injective into the set $\mathcal{H}$ of heavy cells. Hence

$$\sum_{v\in U\setminus R_{0}}\widetilde{\omega}(v)\ \leq\ \sum_{(B,j)\in\mathcal{H}}m_{j}.$$

Assign every heavy cell to one maximal heavy cell weakly below it in the hierarchy, obtained by descending to heavy children while one exists. The cells assigned to a fixed maximal cell $(C,\ell)$ all lie on its ancestor chain, so their level weights total at most $m_{0}+m_{1}+\dots+m_{\ell}<2m_{\ell}\text{,}$ and

$$\sum_{(B,j)\in\mathcal{H}}m_{j}\ \leq\ 2\sum_{C\ \text{maximal heavy}}m(C)\ <\ 2k,$$

the last step by part (4), which also covers the case of an empty family. ∎

The construction of the main proof samples the light exits at random. The next lemma supplies the collapse bound quoted there, the exit estimates behind the fixed realization, and the budgets that realization carries. Its objects are the following.

Collapsing every terminal cell to its root gives $g:=\sum_{L\in\mathcal{L}}\lambda_{L}p_{a_{L}}\text{.}$ A heavy terminal cell is one the refinement did not split, so it sits at level $J$ and no heavy cell lies below it; being maximal it contributes $m_{J}\geq k/2$ to the sum in Lemma S.2.3(4), and there is therefore at most one. Each terminal cell $L$ is attached to $U$ at $h(L):=a_{L}$ when $L$ is a level-$0$ cell or the heavy terminal one, and otherwise at the root of the parent cell of $L\text{,}$ which is heavy because $L$ was reached by the refinement; either way $h(L)\in U\text{,}$ and $h(L)\preceq a_{L}$ by Lemma S.2.2(3). Put $q_{L}:=p_{a_{L}}-p_{h(L)}\text{.}$ In the first two cases $q_{L}=0\text{;}$ otherwise, $L$ sits at a level $j\geq 1\text{,}$ and its parent cell has level weight $m_{j}/2=m(L)/2$ and contains $a_{L}\text{,}$ so Lemma S.2.2(4) and Lemma 2.2(1) give

$$\|q_{L}\|_{2}^{2}\ =\ d_{T}\bigl(h(L),a_{L}\bigr)\ \leq\ \Bigl\lfloor\frac{2\bar{\alpha}}{m(L)}\Bigr\rfloor\ \leq\ \frac{2\bar{\alpha}}{m(L)}.$$

(S.2.1)

With $b_{h}:=\sum_{L:\,h(L)=h}\lambda_{L}$ this splits the collapsed signal as $g=g_{0}+\sum_{L}\lambda_{L}q_{L}\text{,}$ where $g_{0}:=\sum_{h\in U}b_{h}p_{h}$ is the skeleton and the nonzero $q_{L}$ are the light exits.

The skeleton is rounded through one scalar. List $U$ as $h_{1},\dots,h_{s}$ in a fixed depth-first preorder of $T\text{,}$ write $b_{i}:=b_{h_{i}}$ for the masses in that order, set $B_{i}:=\sum_{\ell\leq i}b_{\ell}$ with $B_{0}:=0\text{,}$ and put $Q_{i}:=\lfloor kB_{i}\rfloor-\lfloor kB_{i-1}\rfloor\in\mathbb{Z}_{\geq 0}\text{.}$ Let $Q$ be the integer vector with $Q_{h_{i}}:=Q_{i}$ and $Q_{v}:=0$ for $v\notin U\text{,}$ and set $\bar{g}_{0}:=\frac{1}{k}\sum_{h\in U}Q_{h}p_{h}\text{.}$ Over an interval of the list the $Q_{i}$ telescope, so their sum differs from $k$ times the corresponding mass by a difference of two floor errors, hence by less than $1\text{.}$ The terminal cells partition $\mathsf{V}\text{,}$ so $B_{s}=\sum_{L}\lambda_{L}=1$ and the whole list gives $\sum_{i}Q_{i}=\lfloor kB_{s}\rfloor=k\text{;}$ as the $B_{i}$ increase, $Q$ is a nonnegative integer leak vector of total mass $k$ supported on $U\text{.}$ Every subtree $T_{v}$ is a contiguous block of a depth-first preorder, so $U\cap T_{v}$ is such an interval, and coordinates are subtree sums by Lemma 2.1; therefore $g_{0}$ and $\bar{g}_{0}$ differ by less than $1/k$ at every vertex simultaneously.

Finally, write $\mathcal{L}_{\mathrm{ex}}:=\{L\in\mathcal{L}:q_{L}\neq 0\}$ for the light exits, every member of which is a light cell and therefore has $\lambda_{L}\leq m(L)/k\text{.}$ They are quantized by the independent variables $Z_{L}=\frac{m(L)}{k}\xi_{L}\text{,}$ $L\in\mathcal{L}_{\mathrm{ex}}\text{,}$ with $\xi_{L}\sim\mathrm{Bernoulli}(k\lambda_{L}/m(L))\text{,}$ which is a legitimate parameter by that bound. With the selection weight $W:=\sum_{L\in\mathcal{L}_{\mathrm{ex}}}m(L)\mathbf{1}\{Z_{L}\neq 0\}\text{,}$ the integer vector is

$$z\ =\ Q+\sum_{L\in\mathcal{L}_{\mathrm{ex}}:\,Z_{L}\neq 0}m(L)\bigl(e_{a_{L}}-e_{h(L)}\bigr),$$

and the flow states of an integer vector $z$ are the subtree sums $x_{v}:=\sum_{u\succeq v}z_{u}\text{,}$ that is, the coordinates of $\sum_{u}z_{u}p_{u}\text{.}$

Lemma S.2.4 (Collapse and exit estimates).

In this setting:

1.

$\|f-g\|_{2}^{2}\leq 3\bar{\alpha}/k$, the display (3.4);
2.

writing $\Sigma$ for the sum $\sum_{L\in\mathcal{L}_{\mathrm{ex}}}(Z_{L}-\lambda_{L})q_{L}$, one has $\mathbb{E}\|\Sigma\|_{2}^{2}\leq 2\bar{\alpha}/k$ and $\mathbb{E}W\leq k$, so with probability at least $\tfrac{1}{4}$ both $W\leq 2k$ and $\|\Sigma\|_{2}^{2}\leq 8\bar{\alpha}/k$ hold;
3.

on such a realization $z$ is supported on $A_{k}$ and satisfies $\|z\|_{1}\leq 5k$, and its selected exit roots carry total activation charge at most $W$.

Proof.

Part (1). The blocks $e_{L}=\sum_{u\in L}\lambda_{u}(p_{u}-p_{a_{L}})$ have pairwise disjoint supports, as recorded in the main proof, so $\|f-g\|_{2}^{2}=\sum_{L\in\mathcal{L}}\|e_{L}\|_{2}^{2}\text{.}$ Every $u\in L$ has $d_{T}(a_{L},u)\leq\bar{\alpha}/m(L)$ by Lemma S.2.2(4), so the triangle inequality and Lemma 2.2(1) give $\|e_{L}\|_{2}\leq\sum_{u\in L}\lambda_{u}\sqrt{d_{T}(a_{L},u)}\leq\lambda_{L}\sqrt{\bar{\alpha}/m(L)}\text{.}$ A light terminal cell has $\lambda_{L}\leq m(L)/k\text{,}$ whence $\|e_{L}\|_{2}^{2}\leq\lambda_{L}^{2}\bar{\alpha}/m(L)\leq\lambda_{L}\bar{\alpha}/k\text{,}$ and summing over the light cells with $\sum_{L}\lambda_{L}\leq 1$ bounds their total contribution by $\bar{\alpha}/k\text{.}$ At most one terminal cell is heavy; it sits at level $J\text{,}$ where $m_{J}\geq k/2\text{,}$ and contributes $\|e_{L}\|_{2}^{2}\leq\lambda_{L}^{2}\bar{\alpha}/m_{J}\leq 2\bar{\alpha}/k\text{.}$ Adding the two bounds proves (1).

Part (2). Each $Z_{L}$ has mean $\lambda_{L}$ and

$$\operatorname{Var}(Z_{L})\ \leq\ \Bigl(\frac{m(L)}{k}\Bigr)^{2}\frac{k\lambda_{L}}{m(L)}\ =\ \frac{\lambda_{L}m(L)}{k},$$

while $\|q_{L}\|_{2}^{2}\leq 2\bar{\alpha}/m(L)$ by Equation S.2.1. The $Z_{L}$ are independent, so the cross terms vanish in expectation. With every sum below running over $L\in\mathcal{L}_{\mathrm{ex}}\text{,}$

$$\mathbb{E}\Bigl\|\sum_{L}(Z_{L}-\lambda_{L})q_{L}\Bigr\|_{2}^{2}\ =\ \sum_{L}\operatorname{Var}(Z_{L})\,\|q_{L}\|_{2}^{2}\ \leq\ \frac{2\bar{\alpha}}{k}\sum_{L}\lambda_{L}\ \leq\ \frac{2\bar{\alpha}}{k}.$$

Also $\mathbb{E}W=\sum_{L}m(L)\,k\lambda_{L}/m(L)=k\sum_{L}\lambda_{L}\leq k\text{.}$ By Markov’s inequality $\mathbb{P}(W>2k)\leq\tfrac{1}{2}\text{,}$ and the squared norm exceeds $8\bar{\alpha}/k$ with probability at most $\tfrac{1}{4}\text{,}$ so both bounds hold with probability at least $\tfrac{1}{4}\text{.}$

Part (3). The skeleton $Q$ is supported on $U\subseteq A_{k}$ and each selected transfer on $\{a_{L},h(L)\}\text{;}$ the root of a level-$j$ cell lies in $R_{j}\subseteq A_{k}\text{,}$ so $\operatorname{supp}(z)\subseteq A_{k}\text{.}$ Since $Q$ is nonnegative with total mass $k$ and each selected transfer adds $2m(L)$ to the $\ell_{1}$ norm, $\|z\|_{1}\leq k+2W\leq 5k\text{.}$ The selected exit roots are distinct, being roots of distinct terminal cells, and $\widetilde{\omega}(a_{L})\leq\omega(a_{L})\leq m(L)$ because $a_{L}\in R_{j}$ with $m_{j}=m(L)\text{;}$ their charges therefore total at most $\sum_{L:\,Z_{L}\neq 0}m(L)=W\text{.}$ ∎

Two explicit constants follow. The subtree $S$ has at most $5\bar{\alpha}k+1\leq 6\bar{\alpha}k$ vertices, by Lemma S.2.3(2) and $\bar{\alpha}\geq 2\text{;}$ since $g_{0}$ and $\bar{g}_{0}$ differ by less than $1/k$ at every vertex and vanish outside $S\text{,}$ $\|g_{0}-\bar{g}_{0}\|_{2}^{2}<6\bar{\alpha}k/k^{2}=6\bar{\alpha}/k\text{:}$ the display (3.5) holds with $C=6\text{.}$ Together with the collapse and exit bounds of Lemma S.2.4, the error assembly of the main proof reads

$$\Bigl\|f-\frac{1}{k}\sum_{v}z_{v}p_{v}\Bigr\|_{2}^{2}\ \leq\ 3\Bigl(\frac{3\bar{\alpha}}{k}+\frac{6\bar{\alpha}}{k}+\frac{8\bar{\alpha}}{k}\Bigr)\ =\ \frac{51\,\bar{\alpha}}{k}\ =\ 51\,\overline{\delta}_{k}(T)\ \leq\ 102\,\delta_{k}(T),$$

the last step by Lemma 2.3, so part (1) of Proposition 3.1 holds with $C_{0}=102\text{.}$

It remains to prove the counting bound.

Lemma S.2.5 (Counting bound).

The integer vectors $z$ supported on $A_{k}$ with $\sum_{v}z_{v}=k\text{,}$ $\Gamma(z)\leq 9k\text{,}$ and all flow states in $[0,3k]$ number at most $e^{45k}\text{.}$ In particular part (2) of Proposition 3.1 holds with $C_{1}=45\text{.}$

Proof.

We bound the larger class defined by the support and code-length constraints alone. Each member $z$ determines its support, split into the free part $\operatorname{supp}(z)\cap R_{0}$ and the charged part $\operatorname{supp}(z)\setminus R_{0}\text{,}$ and its coefficient vector; we count the three in turn and multiply.

Free parts. Any subset of $R_{0}$ can occur, and $|R_{0}|\leq ke^{m_{0}}=ek$ by Lemma S.2.1(2), so the free parts number at most $2^{ek}\leq e^{ek}\text{.}$

Charged parts. Group the charged vertices by first appearance: $G_{j}:=\{v\in A_{k}\setminus R_{0}:\ \omega(v)=m_{j}\}$ satisfies $G_{j}\subseteq R_{j}\text{,}$ hence $|G_{j}|\leq 2ke^{m_{j}}$ by Lemma S.2.1(3), for $1\leq j\leq J\text{.}$ The groups partition $A_{k}\setminus R_{0}\text{,}$ so a charged part is the disjoint union of its intersections with them, and one with $q_{j}$ elements of $G_{j}$ carries activation charge $\sum_{j}m_{j}q_{j}\leq\Gamma(z)\leq 9k\text{.}$ Inserting the factor $e^{2(9k-\sum_{j}m_{j}q_{j})}\geq 1$ and then dropping the charge constraint,

$$\begin{aligned} \displaystyle\#\{\text{charged parts}\}\ \leq\ e^{18k}\prod_{j=1}^{J}\ \sum_{q\geq 0}\binom{|G_{j}|}{q}\,e^{-2m_{j}q}\space & \displaystyle=\ e^{18k}\prod_{j=1}^{J}\bigl(1+e^{-2m_{j}}\bigr)^{|G_{j}|} \\ & \displaystyle\leq\ e^{18k}\exp\Bigl(2k\sum_{j\geq 1}e^{-m_{j}}\Bigr)\ \leq\ e^{19k}, \end{aligned}$$

using $\log(1+x)\leq x\text{,}$ $|G_{j}|e^{-2m_{j}}\leq 2ke^{-m_{j}}\text{,}$ and $\sum_{j\geq 1}e^{-2^{j}}<\tfrac{1}{6}\text{.}$

Coefficients. Every charged support vertex carries charge at least $1\text{,}$ so a support has at most $ek+9k\leq 12k$ vertices. On a fixed support of size $d\text{,}$ the map $z\mapsto(z^{+},z^{-})$ is injective into the nonnegative integer vectors on $2d$ coordinates with total at most $9k\text{,}$ since $\|z\|_{1}\leq\Gamma(z)\leq 9k\text{;}$ these number $\binom{9k+2d}{2d}\leq 2^{9k+2d}\leq 2^{33k}\leq e^{23k}\text{.}$

Multiplying the three counts bounds the class by $e^{(e+19+23)k}\leq e^{45k}\text{.}$ ∎

The facts quoted in the main proof are now all in place, and Proposition 3.1 is proved.

S.3 The Packing Lower Bound

This section completes the lower-bound half of Section 3.2. Lemmas 3.2 and 3.3, whose proofs were sketched there, are proved in full first, with the constants of the latter fixed. Two self-contained lemmas follow: the additivity of Bayes risks across coordinate-disjoint blocks, and the edge-disjoint grouping of a separated vertex set. The section then completes the proofs of Propositions 3.4 and 3.5, and finally the deferred cases of Theorem 1 and Proposition 1.

Proof of Lemma 3.2.

Write $\mathcal{P}:=\{p_{u}:u\in\mathsf{V}\}$ and $\varepsilon:=\sqrt{r}/3\text{,}$ and fix a maximal $\varepsilon$-separated subset of $\mathcal{P}\text{;}$ by maximality, every point of $\mathcal{P}$ lies within Euclidean distance $\varepsilon$ of the subset. By Lemma 2.2(1), the vertices indexing the subset have pairwise tree distances at least $\varepsilon^{2}=r/9\text{,}$ so it suffices to bound their number below by $N^{\uparrow}_{T}(r)\text{.}$

Consider the ball of radius $\varepsilon$ around one point of the subset, let $C$ be the set of vertices whose indicators it contains, and let $a$ be the deepest common ancestor of $C\text{.}$ Fix $u\in C$ with $u\neq a$ and let $c$ be the child of $a$ on the path toward $u\text{.}$ Not all of $C$ lies in the subtree $T_{c}\text{,}$ since $c$ would then be a common ancestor of $C$ deeper than $a\text{;}$ fix $v\in C\setminus T_{c}\text{.}$ The meet $u\wedge v$ is at least as deep as $a\text{,}$ the latter being a common ancestor of $u$ and $v\text{;}$ and it is not deeper, since an ancestor of $u$ strictly below $a$ lies in $T_{c}\text{,}$ and $u\wedge v\in T_{c}$ would force $v\in T_{c}\text{.}$ Hence $u\wedge v=a\text{,}$ so the path from $u$ to $v$ passes through $a\text{,}$ and by Lemma 2.2(1),

$$d_{T}(a,u)\ \leq\ d_{T}(u,v)\ =\ \|p_{u}-p_{v}\|_{2}^{2}\ \leq\ (2\varepsilon)^{2}\ =\ \tfrac{4}{9}\,r\ <\ r,$$

the indicators $p_{u}$ and $p_{v}$ lying in one ball of radius $\varepsilon\text{.}$ The bound holds trivially at $u=a\text{,}$ so $a$ ancestor-covers every vertex of $C$ within distance $r\text{.}$ Every vertex of $T$ lies in the captured set of some ball, so collecting one such ancestor per ball yields an ancestor $r$-net, of size at most the number of balls; the number of balls is the size of the separated subset, and every ancestor $r$-net has at least $N^{\uparrow}_{T}(r)$ elements, which proves the bound. ∎

Two facts about Gaussian shifts are used without further comment (Tsybakov, 2009). The Kullback–Leibler divergence between $N(\theta,\sigma^{2}I_{N})$ and $N(\theta^{\prime},\sigma^{2}I_{N})$ is $\|\theta-\theta^{\prime}\|_{2}^{2}/(2\sigma^{2})\text{.}$ Fano’s inequality in its pairwise form states that if a label is uniform on $M\geq 2$ hypotheses whose data laws satisfy $\max_{i,j}\operatorname{KL}(P_{i},P_{j})\leq\kappa\text{,}$ then every decoder based on the data errs with probability at least $1-(\kappa+\log 2)/\log M\text{.}$

Proof of Lemma 3.3.

Fix $\eta:=\tfrac{1}{10}\text{,}$ the constant of the main-text sketch, and shrink each block into its information budget:

$$\tau_{j}^{2}\ :=\ \min\Bigl\{1,\ \frac{\eta\,\sigma^{2}h_{j}}{D_{j}^{2}}\Bigr\},\qquad\widetilde{\mu}_{\mathbf{a}}\ :=\ \mu_{0}+\sum_{j}\tau_{j}g_{j}(a_{j}),$$

so that $\tau_{j}\in[0,1]$ and, by hypothesis (1), the shrunken family lies in $K\text{.}$ Place the uniform product prior on $\mathbf{a}\text{.}$ Since the family lies in $K\text{,}$ the minimax risk over $K$ dominates the Bayes risk of this prior, and it suffices to bound the latter for an arbitrary estimator $\widehat{\mu}=\widehat{\mu}(Y)\text{.}$

Separation. Let $\widehat{\mathbf{a}}$ index a member of the shrunken family nearest to $\widehat{\mu}\text{,}$ ties broken by a fixed order, so that $\|\widehat{\mu}-\widetilde{\mu}_{\mathbf{a}}\|_{2}\geq\tfrac{1}{2}\|\widetilde{\mu}_{\widehat{\mathbf{a}}}-\widetilde{\mu}_{\mathbf{a}}\|_{2}$ by the triangle inequality, and write $\Delta_{j}:=g_{j}(\widehat{a}_{j})-g_{j}(a_{j})\text{,}$ blocks with $\Delta_{j}=0$ dropped. With $\rho_{ij}:=|\langle\Delta_{i},\Delta_{j}\rangle|/(\|\Delta_{i}\|_{2}\|\Delta_{j}\|_{2})\text{,}$ hypothesis (3) gives $\sum_{j\neq i}\rho_{ij}\leq\gamma$ for every $i\text{,}$ and

$$\Bigl\|\sum_{j}\tau_{j}\Delta_{j}\Bigr\|_{2}^{2}\ \geq\ \sum_{j}\tau_{j}^{2}\|\Delta_{j}\|_{2}^{2}-\sum_{i\neq j}\rho_{ij}\,\tau_{i}\|\Delta_{i}\|_{2}\,\tau_{j}\|\Delta_{j}\|_{2}\ \geq\ (1-\gamma)\sum_{j}\tau_{j}^{2}\|\Delta_{j}\|_{2}^{2},$$

the last step by $xy\leq\tfrac{1}{2}(x^{2}+y^{2})$ and the row sums. Since $\|\Delta_{j}\|_{2}\geq d_{j}$ whenever $\widehat{a}_{j}\neq a_{j}\text{,}$ pathwise

$$\|\widehat{\mu}-\widetilde{\mu}_{\mathbf{a}}\|_{2}^{2}\ \geq\ \frac{1-\gamma}{4}\sum_{j}\tau_{j}^{2}d_{j}^{2}\,\mathbf{1}\{\widehat{a}_{j}\neq a_{j}\}.$$

Per-block error. Fix $j$ and condition on the other labels $\mathbf{a}_{-j}\text{.}$ Under the product prior, $a_{j}$ remains uniform on $\mathcal{A}_{j}\text{,}$ and subtracting the known vector $\mu_{0}+\sum_{i\neq j}\tau_{i}g_{i}(a_{i})$ from $Y$ reduces the conditional problem to testing the finite Gaussian family $\{\tau_{j}g_{j}(a):a\in\mathcal{A}_{j}\}\text{,}$ whose pairwise Kullback–Leibler divergences are at most $\tau_{j}^{2}D_{j}^{2}/(2\sigma^{2})\leq\eta h_{j}/2\text{.}$ If $|\mathcal{A}_{j}|\geq 3\text{,}$ Fano’s inequality gives every decoder error probability at least $1-\eta/2-\log 2/\log|\mathcal{A}_{j}|\geq 1-\tfrac{1}{20}-\log 2/\log 3>\tfrac{1}{4}\text{.}$ If $|\mathcal{A}_{j}|=2\text{,}$ then $D_{j}=d_{j}$ and the two means are at distance $\tau_{j}D_{j}\leq\sigma\sqrt{\eta\log 2}\text{;}$ the likelihood-ratio test is optimal and projects the data onto the difference of the means; its two conditional errors are equal, and the Bayes error is

$$\Phi_{\mathcal{N}}\Bigl(-\frac{\tau_{j}D_{j}}{2\sigma}\Bigr)\ \geq\ \frac{1}{2}-\frac{\sqrt{\eta\log 2}}{2\sqrt{2\pi}}\ >\ \frac{1}{4},$$

where $\Phi_{\mathcal{N}}$ is the standard normal distribution function, whose density is at most $1/\sqrt{2\pi}\text{.}$ In either case, every decoder of $a_{j}$ that is measurable in $Y$ is, conditionally on $\mathbf{a}_{-j}\text{,}$ a decoder of the reduced problem, so $\mathbb{P}(\widehat{a}_{j}\neq a_{j})\geq\tfrac{1}{4}\text{.}$

Assembly. Taking expectations in the separation bound,

$$\mathbb{E}\|\widehat{\mu}-\widetilde{\mu}_{\mathbf{a}}\|_{2}^{2}\ \geq\ \frac{1-\gamma}{4}\sum_{j}\tau_{j}^{2}d_{j}^{2}\,\mathbb{P}(\widehat{a}_{j}\neq a_{j})\ \geq\ \frac{1-\gamma}{16}\sum_{j}\tau_{j}^{2}d_{j}^{2},$$

and by the definition of $\tau_{j}$ and hypothesis (2),

$$\tau_{j}^{2}d_{j}^{2}\ =\ \min\Bigl\{d_{j}^{2},\ \eta\,\sigma^{2}h_{j}\,\frac{d_{j}^{2}}{D_{j}^{2}}\Bigr\}\ \geq\ \min\Bigl\{d_{j}^{2},\ \frac{\eta}{A}\,\sigma^{2}h_{j}\Bigr\}\ \geq\ \frac{\eta}{A}\,\min\bigl\{d_{j}^{2},\ \sigma^{2}h_{j}\bigr\},$$

so the Bayes risk of the prior is at least $c_{A}(1-\gamma)\sum_{j}\min\{d_{j}^{2},\sigma^{2}h_{j}\}$ with $c_{A}:=\eta/(16A)=1/(160A)\text{.}$

The restricted loss. When every contrast is supported in the coordinate set $S\text{,}$ choose $\widehat{\mathbf{a}}$ nearest to $\widehat{\mu}$ in the norm restricted to $S\text{.}$ The differences $(\widetilde{\mu}_{\widehat{\mathbf{a}}}-\widetilde{\mu}_{\mathbf{a}})|_{S}=\sum_{j}\tau_{j}\Delta_{j}$ are unchanged, since the contrasts live in $S\text{,}$ so the separation bound holds for $\|(\widehat{\mu}-\widetilde{\mu}_{\mathbf{a}})|_{S}\|_{2}^{2}\text{;}$ and the per-block bound applies to any decoder measurable in the full data $Y\text{,}$ in particular to this one. The assembly is identical. ∎

The assembly across groups in Proposition 3.4 rests on the fact that coordinate-disjoint blocks contribute additively to the Bayes risk.

Lemma S.3.1 (Bayes risks add across disjoint blocks).

Let

$$\mu_{\boldsymbol{\theta}}\ =\ \mu_{0}+\sum_{i=1}^{m}h_{i}(\theta_{i}),\qquad\boldsymbol{\theta}=(\theta_{1},\dots,\theta_{m}),$$

where the $\theta_{i}$ are independent under a product prior and, for each $i\text{,}$ every contrast $h_{i}(\theta_{i})-h_{i}(\theta_{i}^{\prime})$ is supported in a coordinate set $S_{i}\text{,}$ the sets $S_{1},\dots,S_{m}$ pairwise disjoint. In the model $Y=\mu_{\boldsymbol{\theta}}+\sigma Z\text{,}$ $Z\sim N(0,I_{N})\text{,}$ the Bayes risk of estimating $\mu_{\boldsymbol{\theta}}$ in squared loss is at least the sum over $i$ of the Bayes risks of the isolated problems: observe $Y_{S_{i}}\text{,}$ which up to a fixed additive vector is $h_{i}(\theta_{i})|_{S_{i}}+\sigma Z_{S_{i}}\text{,}$ and estimate $\mu_{\boldsymbol{\theta}}|_{S_{i}}$ under the $i$-th marginal prior.

Proof.

The squared loss dominates the sum of its restrictions to the disjoint sets: $\|\widehat{\mu}-\mu_{\boldsymbol{\theta}}\|_{2}^{2}\geq\sum_{i}\|(\widehat{\mu}-\mu_{\boldsymbol{\theta}})|_{S_{i}}\|_{2}^{2}\text{.}$ Fix $i\text{.}$ For $\ell\neq i$ the restriction $h_{\ell}(\theta_{\ell})|_{S_{i}}$ does not depend on $\theta_{\ell}\text{:}$ two values of $\theta_{\ell}$ differ on $S_{i}$ by a contrast of block $\ell$ restricted to $S_{i}\text{,}$ which vanishes because those contrasts are supported in $S_{\ell}\text{,}$ disjoint from $S_{i}\text{.}$ Hence $\mu_{\boldsymbol{\theta}}|_{S_{i}}=c_{i}+h_{i}(\theta_{i})|_{S_{i}}$ for a fixed vector $c_{i}\text{,}$ and $Y_{S_{i}}$ is distributed as in the isolated problem. Moreover $Y_{S_{i}^{c}}$ is a function of $(\boldsymbol{\theta}_{-i},Z_{S_{i}^{c}})\text{,}$ which is independent of $(\theta_{i},Z_{S_{i}})$ under the product prior, so $Y_{S_{i}^{c}}$ is independent of the pair $(\theta_{i},Y_{S_{i}})$ and the conditional law of $\mu_{\boldsymbol{\theta}}|_{S_{i}}$ given $Y$ equals its conditional law given $Y_{S_{i}}\text{.}$ The Bayes-optimal estimate of $\mu_{\boldsymbol{\theta}}|_{S_{i}}\text{,}$ its posterior mean, is therefore a function of $Y_{S_{i}}$ alone, and $\mathbb{E}\|(\widehat{\mu}-\mu_{\boldsymbol{\theta}})|_{S_{i}}\|_{2}^{2}$ is at least the isolated Bayes risk for every estimator $\widehat{\mu}\text{.}$ Summing over $i$ completes the proof. ∎

The next lemma produces the edge-disjoint groups of the main-text sketch, with explicit comparison constants.

Lemma S.3.2 (Edge-disjoint grouping).

Let $U\subseteq\mathsf{V}$ consist of $M\geq 2$ vertices and let $q$ be an integer with $1\leq q\leq M/2\text{.}$ There are connected subtrees $\mathcal{C}_{1},\dots,\mathcal{C}_{q^{\prime}}$ of $T$ with pairwise disjoint edge sets, and pairwise disjoint sets $U_{i}\subseteq U\cap\mathcal{C}_{i}\text{,}$ such that

$$\frac{q}{4}\ \leq\ q^{\prime}\ \leq\ 6q,\qquad\frac{M}{6q}\ \leq\ |U_{i}|\ \leq\ \frac{M}{q}\quad\text{and}\quad|U_{i}|\ \geq\ 2\qquad\text{for every }i.$$

Proof.

Call the members of $U$ marks. Work in the minimal subtree of $T$ spanning $U\text{,}$ rooted by the orientation inherited from $T\text{,}$ and set $L:=\max\{2,\lfloor M/(4q)\rfloor\}\text{.}$

The procedure. Process the vertices in postorder, distinguishing each mark’s token, which will be assigned to at most one group, from the vertices and edges that only provide connectivity. Each vertex $v$ receives a collection of items, each a connected subgraph containing $v$ together with the unassigned tokens it carries: for every child $c$ of $v\text{,}$ the residual passed up from $c\text{,}$ extended by the edge $(v,c)\text{;}$ and, if $v$ is a mark, the one-point item $\{v\}$ carrying the token of $v\text{.}$ Insert the items into an open bin one at a time; whenever the bin first carries at least $L$ tokens, seal it: its subgraphs unite into a component, connected through $v\text{,}$ and its tokens, now permanently assigned, form the component’s mark set. After the last item, pass the union of the open bin and the bare vertex $\{v\}\text{,}$ carrying the bin’s tokens, upward as the residual of $v\text{.}$ At the root, discard the final residual.

The invariant. Sealed components and residuals are connected, being unions of items that share the vertex at which they formed. A residual carries fewer than $L$ tokens, its bin having never reached $L\text{;}$ consequently every item carries fewer than $L$ tokens, the one-point items because $L\geq 2\text{,}$ and a component seals with at least $L$ but fewer than $2L$ tokens. Each token is assigned at most once, so the mark sets $U_{i}$ are pairwise disjoint; and a token’s vertex belongs to every subgraph that carries it, so $U_{i}\subseteq U\cap\mathcal{C}_{i}\text{.}$ Finally, each edge of the spanning subtree enters the process in exactly one item and thereafter stays inside whichever bin, component, or residual that item joined; the components are therefore pairwise edge-disjoint.

Counting. Let $q^{\prime}$ be the number of sealed components; by the invariant, $|U_{i}|\in[L,2L)$ for each. Every token is either assigned at some sealing or sits in the final, discarded residual, which carries fewer than $L\text{;}$ hence

$$q^{\prime}L\ \leq\ M\ <\ 2Lq^{\prime}+L.$$

If $M\geq 12q\text{,}$ then $L=\lfloor M/(4q)\rfloor\geq 3$ and $M/(6q)\leq L\leq M/(4q)\text{,}$ since $\lfloor x\rfloor\geq x-1\geq\tfrac{2}{3}x$ for $x\geq 3\text{;}$ hence $q^{\prime}\leq M/L\leq 6q$ and $q^{\prime}>M/(2L)-\tfrac{1}{2}\geq 2q-\tfrac{1}{2}\text{,}$ so $q^{\prime}\geq 2q\geq q\text{,}$ while $|U_{i}|\in[L,2L)\subseteq[M/(6q),\,M/(2q)]\text{.}$ If $M<12q\text{,}$ then $L=2\text{:}$ every item then carries at most one token, so components seal with exactly two, and $|U_{i}|=2$ lies in $[M/(6q),M/q]$ because $2q\leq M<12q\text{.}$ For $M\geq 4$ the count gives $q^{\prime}\leq M/2\leq 6q$ and $q^{\prime}>(M-2)/4\geq M/8\geq q/4\text{;}$ for $M\in\{2,3\}\text{,}$ necessarily $q=1\text{,}$ and the procedure seals exactly one component: fewer than $4$ tokens admit at most one, and at least one seals because all unassigned tokens funnel into the bins at the root. ∎

Completion of the proof of Proposition 3.4.

Lemma S.3.2, applied with the parameter $q$ of the proposition, supplies the groups of the main-text sketch, with $q/4\leq q^{\prime}\leq 6q$ and $M/(6q)\leq M_{i}\leq M/q\text{;}$ every group receives the budget $a=V/q^{\prime}\text{.}$ Fix a group, with marked set $U_{i}$ of size $M_{i}\geq 2\text{,}$ and adopt the objects of the sketch: the radii $r_{\ell}=16^{\ell}r\text{,}$ the nested maximal separated subsets $N_{\ell}\text{,}$ the descent chain with its node weights and child counts $m_{\ell}\text{,}$ the retained levels $I\text{,}$ the alphabets $\mathcal{A}_{\ell}\text{,}$ and the masses $a_{\ell}$ with their normalizer $S\text{.}$ We verify the claims quoted there and assemble.

The hierarchy and its entropy. Maximality of $N_{\ell}$ inside $N_{\ell-1}$ means that every point of $N_{\ell-1}$ lies within $r_{\ell}$ of a point of $N_{\ell}\text{,}$ so parents can be assigned, each point of $N_{\ell}$ to itself; and once $r_{\ell}$ exceeds the diameter of $U_{i}$ an $r_{\ell}$-separated subset is a single point: the hierarchy has a top. The weight of a node is at most its number of children times the largest weight of a child, so descending along children of maximal weight from the top, whose weight is $M_{i}\text{,}$ to a leaf, whose weight is one, gives $M_{i}\leq\prod_{\ell}m_{\ell}\text{,}$ that is, $\sum_{\ell}\log m_{\ell}\geq\log M_{i}\text{.}$ Levels with $m_{\ell}=1$ contribute zero to this sum, and among the four residue classes modulo four of the remaining levels one carries at least a quarter of it; the retained class $I$ therefore has $H=\sum_{\ell\in I}h_{\ell}\geq\tfrac{1}{4}\log M_{i}\text{.}$

Separations and aspect. Distinct children $u\neq v$ of the chain node at level $\ell$ lie in $N_{\ell-1}\text{,}$ so $d_{T}(u,v)\geq r_{\ell-1}\text{;}$ both lie within $r_{\ell}$ of their parent, so $d_{T}(u,v)\leq 2r_{\ell}=32\,r_{\ell-1}\text{.}$ By Lemma 2.2(1), the contrasts $\Delta=a_{\ell}(p_{u}-p_{v})$ then satisfy $a_{\ell}^{2}r_{\ell-1}\leq\|\Delta\|_{2}^{2}\leq 32\,a_{\ell}^{2}r_{\ell-1}\text{,}$ so $d_{\ell}^{2}\geq a_{\ell}^{2}r_{\ell-1}$ and $D_{\ell}^{2}\leq 32\,a_{\ell}^{2}r_{\ell-1}\leq 32\,d_{\ell}^{2}\text{:}$ hypothesis (2) of Lemma 3.3 holds with $A=32\text{.}$

Overlap. For $\ell<\ell^{\prime}$ in $I\text{,}$ a contrast $\Delta_{\ell}$ has entries in $[-a_{\ell},a_{\ell}]$ and support of size $d_{T}(u,v)\leq 32\,r_{\ell-1}\text{,}$ by Lemma 2.2(2), so $|\langle\Delta_{\ell},\Delta_{\ell^{\prime}}\rangle|\leq 32\,r_{\ell-1}a_{\ell}a_{\ell^{\prime}}\text{,}$ and with the norm lower bounds just proved the normalized entry is at most $32\sqrt{r_{\ell-1}/r_{\ell^{\prime}-1}}=32\cdot 4^{-(\ell^{\prime}-\ell)}\text{.}$ Retained levels differ by at least four, so every row of the normalized Gram matrix sums to at most

$$2\sum_{t\geq 1}32\cdot 4^{-4t}\ =\ \frac{64}{255}\ =:\ \gamma\ <\ 1:$$

hypothesis (3) of Lemma 3.3 holds with this $\gamma\text{.}$

Legality and scalability. With the reference points $u_{\ell}^{0}\in\mathcal{A}_{\ell}$ and $\mu_{0}:=\sum_{\ell\in I}a_{\ell}p_{u_{\ell}^{0}}\text{,}$ a blockwise-scaled member of the group family is

$$\mu_{0}+\sum_{\ell\in I}\tau_{\ell}\,a_{\ell}\bigl(p_{u_{\ell}}-p_{u_{\ell}^{0}}\bigr)\ =\ \sum_{\ell\in I}a_{\ell}\bigl[(1-\tau_{\ell})\,p_{u_{\ell}^{0}}+\tau_{\ell}\,p_{u_{\ell}}\bigr],$$

a leak combination with nonnegative coefficients of total mass $\sum_{\ell\in I}a_{\ell}=a\text{;}$ hypothesis (1) of Lemma 3.3 holds with $K$ the convex set of all such mass-$a$ combinations.

The allocation. Since $a_{\ell}^{2}r_{\ell-1}=a^{2}h_{\ell}/S^{2}\text{,}$

$$\sum_{\ell\in I}\min\bigl\{a_{\ell}^{2}r_{\ell-1},\ \sigma^{2}h_{\ell}\bigr\}\ =\ \sum_{\ell\in I}h_{\ell}\,\min\Bigl\{\frac{a^{2}}{S^{2}},\ \sigma^{2}\Bigr\}\ =\ H\,\min\Bigl\{\frac{a^{2}}{S^{2}},\ \sigma^{2}\Bigr\}.$$

By the Cauchy–Schwarz inequality, $S^{2}\leq H\sum_{\ell\in I}r_{\ell-1}^{-1}\text{,}$ and the radii grow geometrically, so $\sum_{\ell\in I}r_{\ell-1}^{-1}\leq r^{-1}\sum_{i\geq 0}16^{-i}=16/(15r)\leq 2/r\text{;}$ hence $a^{2}/S^{2}\geq a^{2}r/(2H)$ and

$$H\,\min\Bigl\{\frac{a^{2}}{S^{2}},\ \sigma^{2}\Bigr\}\ \geq\ \min\Bigl\{\frac{a^{2}r}{2},\ \sigma^{2}H\Bigr\}\ \geq\ \frac{1}{4}\,\min\bigl\{a^{2}r,\ \sigma^{2}\log M_{i}\bigr\},$$

using $H\geq\tfrac{1}{4}\log M_{i}\text{.}$

Assembly. The contrasts of the group join members of $U_{i}\subseteq\mathcal{C}_{i}\text{,}$ and the connecting paths run inside the connected subtree $\mathcal{C}_{i}\text{,}$ so all group contrasts are supported in the set $S_{i}$ of lower endpoints of the edges of $\mathcal{C}_{i}$ (Lemma 2.2(2)). Lemma 3.3, in its restricted-loss form with $A=32$ and $\gamma=64/255\text{,}$ therefore bounds the Bayes risk of the shrunken group family, for every estimator measurable in the full data, below by

$$c_{A}(1-\gamma)\sum_{\ell\in I}\min\bigl\{d_{\ell}^{2},\sigma^{2}h_{\ell}\bigr\}\ \geq\ c^{\prime}\,\min\bigl\{a^{2}r,\ \sigma^{2}\log M_{i}\bigr\},\qquad c^{\prime}:=\frac{c_{A}(1-\gamma)}{4},$$

by the aspect and allocation bounds above. Sums of one shrunken member per group are legal points of $\mathcal{F}_{V}(T)\text{,}$ as verified in the main body; the sets $S_{i}$ are pairwise disjoint, the $E(\mathcal{C}_{i})$ being disjoint and distinct edges having distinct lower endpoints; and the prior is a product across groups. Lemma S.3.1 therefore bounds the joint Bayes risk below by the sum of the isolated group risks, and each isolated problem is, after subtracting its fixed vector, the group problem observed through $Y_{S_{i}}$ alone, so its Bayes risk also dominates the displayed bound. Hence

$$R^{*}_{T}(V,\sigma)\ \geq\ c^{\prime}\sum_{i=1}^{q^{\prime}}\min\bigl\{a^{2}r,\ \sigma^{2}\log M_{i}\bigr\}.$$

Finally, $a=V/q^{\prime}\geq V/(6q)\text{,}$ and $\log M_{i}\geq\tfrac{1}{7}\log(eM/q)\text{:}$ if $M/q\geq 36\text{,}$ then with $x:=\log(M/q)\text{,}$ $\log M_{i}\geq\log(M/(6q))=x-\log 6$ and $5(x-\log 6)-(1+x)=4x-5\log 6-1\geq 4\log 36-5\log 6-1=3\log 6-1>0\text{,}$ so the ratio to $\log(eM/q)=1+x$ is at least $\tfrac{1}{5}\text{;}$ if $M/q<36\text{,}$ then $\log M_{i}\geq\log 2$ and $\log(eM/q)<1+\log 36\text{,}$ and $7\log 2>4.8>1+\log 36\text{.}$ Therefore

$$R^{*}_{T}(V,\sigma)\ \geq\ \frac{q^{\prime}c^{\prime}}{36}\,\min\Bigl\{\frac{V^{2}r}{q^{2}},\ \sigma^{2}\log\frac{eM}{q}\Bigr\}\ \geq\ \frac{c^{\prime}}{144}\,\min\Bigl\{\frac{V^{2}r}{q},\ \sigma^{2}q\log\frac{eM}{q}\Bigr\},$$

which is the claim. ∎

The reduction of Proposition 3.5 rests on a scalar matching of the free parameter $q$ to the profile; the constant $8$ below is the one priced into $A_{0}=18\cdot 8=144\text{.}$

Lemma S.3.3 (Scalar matching).

Let $k\geq 1$ and $M>k$ be integers, let $r>0\text{,}$ and set $L:=\log(M/k)>0\text{.}$ If

$$V^{2}r\,\min\Bigl\{1,\ \frac{L}{k}\Bigr\}\ \geq\ 8\,\sigma^{2}k,$$

then some integer $1\leq q\leq M/2\text{,}$ comparable to $\max\{1,\ k/(1+L)\}$ within universal factors, satisfies

$$\min\Bigl\{\frac{V^{2}r}{q},\ \sigma^{2}q\log\frac{eM}{q}\Bigr\}\ \geq\ \frac{\sigma^{2}k}{7}.$$

Proof.

Write $B:=1+L$ and note $M\geq 2\text{.}$ There are three cases.

Case $L\geq k\text{.}$ Take $q:=1\text{,}$ which equals $\max\{1,k/B\}$ since $k/B<1\text{.}$ The hypothesis reads $V^{2}r\geq 8\sigma^{2}k\text{,}$ and $\sigma^{2}\log(eM)=\sigma^{2}(1+\log M)\geq\sigma^{2}(1+L)\geq\sigma^{2}k\text{,}$ so both branches are at least $\sigma^{2}k\text{.}$

Case $L<k$ and $M\geq 8k/B\text{.}$ Take $q:=\max\{1,\lfloor k/B\rfloor\}\text{.}$ Then $k/(2B)\leq q\leq 2k/B\text{:}$ if $k\geq B$ this is $\lfloor x\rfloor\in[x/2,x]$ at $x=k/B\geq 1\text{;}$ if $k<B\text{,}$ then $q=1\text{,}$ and $k/(2B)<\tfrac{1}{2}<1\leq 2k/B$ because $B=1+L<1+k\leq 2k\text{.}$ Also $q\leq 2k/B\leq M/4<M/2$ by the case hypothesis. The hypothesis of the lemma reads $V^{2}rL/k\geq 8\sigma^{2}k\text{,}$ so

$$\frac{V^{2}r}{q}\ \geq\ \frac{B}{2k}\,V^{2}r\ \geq\ \frac{L}{2k}\,V^{2}r\ \geq\ 4\,\sigma^{2}k.$$

For the entropy branch, $M=ke^{L}$ and $q\leq 2k/B$ give $eM/q\geq\tfrac{e}{2}\,Be^{L}\text{,}$ so

$$\log\frac{eM}{q}\ \geq\ 1+L+\log B-\log 2\ \geq\ 0.3\,B,$$

since $0.7(1+L)+\log(1+L)\geq 0.7>\log 2\text{;}$ hence $\sigma^{2}q\log(eM/q)\geq\sigma^{2}\,\tfrac{k}{2B}\cdot 0.3B=0.15\,\sigma^{2}k\text{.}$

Case $L<k$ and $M<8k/B\text{.}$ Take $q:=\lfloor M/2\rfloor\text{,}$ so $M/3\leq q\leq M/2\text{;}$ and $q$ is comparable to $\max\{1,k/B\}\text{,}$ since $k<M<8k/B$ forces $B<8\text{,}$ whence $q\geq(M-1)/2\geq k/2$ and $q<4k/B\leq 4k\text{.}$ The entropy branch has $\log(eM/q)\geq\log(2e)>1\text{,}$ so $\sigma^{2}q\log(eM/q)\geq\sigma^{2}M/3>\sigma^{2}k/3\text{;}$ and the mass branch, using $k/M>B/8$ and $B>L\text{,}$

$$\frac{V^{2}r}{q}\ \geq\ \frac{2V^{2}r}{M}\ =\ 2\,\frac{V^{2}rL}{k}\cdot\frac{k}{LM}\ \geq\ 16\,\sigma^{2}k\cdot\frac{B}{8L}\ \geq\ 2\,\sigma^{2}k.$$

In every case the minimum is at least $0.15\,\sigma^{2}k\geq\sigma^{2}k/7\text{.}$ ∎

Completion of the proof of Proposition 3.5.

The main text reduces the proposition to the following situation: an integer $k\geq 1\text{,}$ and $M\geq N^{\uparrow}_{T}(r)>k$ vertices with pairwise tree distances at least $r^{\prime}:=r/9\text{,}$ where

$$V^{2}r^{\prime}\,\min\Bigl\{1,\ \frac{1}{k}\Bigl[\log\frac{M}{k}\Bigr]_{+}\Bigr\}\ \geq\ \frac{1}{18}\,V^{2}\delta_{k}(T)\ \geq\ \frac{A_{0}}{18}\,\sigma^{2}k\ =\ 8\,\sigma^{2}k:$$

the term at $(r^{\prime},M)$ is at least one ninth of the term at $(r,N^{\uparrow}_{T}(r))\text{,}$ since $r^{\prime}=r/9$ and enlarging the count only increases the positive part. With $L:=\log(M/k)>0$ this is the hypothesis of Lemma S.3.3. Its conclusion returns an integer $1\leq q\leq M/2\text{,}$ comparable to $\max\{1,k/(1+\log(M/k))\}$ as asserted in the main body, with $\min\{V^{2}r^{\prime}/q,\ \sigma^{2}q\log(eM/q)\}\geq\sigma^{2}k/7\text{.}$ Proposition 3.4, applied to the $M$ separated vertices with this $q\text{,}$ gives $R^{*}_{T}(V,\sigma)\geq(c/7)\,\sigma^{2}k$ with $c$ the constant of that proposition, which proves Proposition 3.5. ∎

Section 3.3 assembles the two halves. We next record its two-point bound, the deferred comparisons from three of the four cases, and the fixed-point conclusion. First, the generic bound (3.6) rests on the oracle inequality for least squares over a finite set of candidates. The inequality is classical, and belongs to the Gaussian model selection theory of Birgé and Massart (2001) and Massart (2007); we give the short proof so that its constants are explicit.

Lemma S.3.4 (Finite-class least squares).

Let $Y=\mu+\sigma Z$ with $Z\sim N(0,I_{n})$ and $\mu\in\mathbb{R}^{n}\text{,}$ let $\mathcal{N}\subset\mathbb{R}^{n}$ be finite and nonempty, and let $\widehat{f}$ minimize $\|Y-f\|_{2}^{2}$ over $f\in\mathcal{N}\text{,}$ ties broken by a fixed rule. Then

$$\mathbb{E}_{\mu}\bigl\|\widehat{f}-\mu\bigr\|_{2}^{2}\ \leq\ 4\min_{f\in\mathcal{N}}\|f-\mu\|_{2}^{2}\ +\ 12\,\sigma^{2}\bigl(1+\log|\mathcal{N}|\bigr).$$

Proof.

Let $f^{\star}$ attain the minimum and $a:=\|f^{\star}-\mu\|_{2}\text{.}$ If $|\mathcal{N}|=1$ then $\widehat{f}=f^{\star}$ and the claim is immediate, so assume $N:=|\mathcal{N}|\geq 2\text{.}$ Optimality of $\widehat{f}$ and $Y-f=(\mu-f)+\sigma Z$ give

$$\|\mu-\widehat{f}\|_{2}^{2}+2\sigma\langle Z,\mu-\widehat{f}\rangle\ \leq\ a^{2}+2\sigma\langle Z,\mu-f^{\star}\rangle,$$

that is, $\|\mu-\widehat{f}\|_{2}^{2}\leq a^{2}+2\sigma\langle Z,\widehat{f}-f^{\star}\rangle\text{.}$ Writing $u_{f}:=(f-f^{\star})/\|f-f^{\star}\|_{2}$ for $f\neq f^{\star}$ and $G:=\bigl[\max_{f\neq f^{\star}}\langle Z,u_{f}\rangle\bigr]_{+}\text{,}$ we get $\langle Z,\widehat{f}-f^{\star}\rangle\leq\|\widehat{f}-f^{\star}\|_{2}\,G\text{,}$ whence, with $x:=\|\widehat{f}-\mu\|_{2}$ and the triangle inequality $\|\widehat{f}-f^{\star}\|_{2}\leq x+a\text{,}$

$$x^{2}\ \leq\ a^{2}+2\sigma Gx+2\sigma Ga\ \leq\ a^{2}+\Bigl(\tfrac{x^{2}}{2}+2\sigma^{2}G^{2}\Bigr)+\bigl(a^{2}+\sigma^{2}G^{2}\bigr),$$

so $x^{2}\leq 4a^{2}+6\sigma^{2}G^{2}\text{.}$ Each $\langle Z,u_{f}\rangle$ is standard normal, so $\mathbb{P}(G>s)\leq\min\{1,Ne^{-s^{2}/2}\}$ and, splitting the integral $\mathbb{E}G^{2}=\int_{0}^{\infty}2s\,\mathbb{P}(G>s)\,ds$ at $s_{0}:=\sqrt{2\log N}\text{,}$

$$\mathbb{E}G^{2}\ \leq\ s_{0}^{2}+N\int_{s_{0}}^{\infty}2s\,e^{-s^{2}/2}\,ds\ =\ 2\log N+2.$$

Combining the two displays proves the lemma. ∎

Completion of Theorem 1 and Proposition 1.

For the two-point bound, pick $u,w$ at tree distance $H_{T}\text{.}$ The segment from $Vp_{u}$ to $Vp_{w}$ lies in $\mathcal{F}_{V}(T)$ and has length $V\sqrt{H_{T}}$ by Lemma 2.2(1), so it contains two points at distance

$$\varrho\ :=\ \min\bigl\{V\sqrt{H_{T}},\ \sigma\bigr\}.$$

Testing one against the other is a one-dimensional Gaussian shift of size $\varrho/\sigma\leq 1\text{,}$ so under the uniform prior on the pair every test errs with probability at least a universal constant. For any estimator, decoding the nearer of the two points errs only when the squared loss is at least $\varrho^{2}/4\text{;}$ hence the maximum of the two risks is at least $c\varrho^{2}\text{,}$ which is the display (3.8).

Case $n<C_{\star}\text{.}$ By (3.8) and $\sigma^{2}\geq\sigma^{2}n/C_{\star}\text{,}$

$$R^{*}_{T}\ \geq\ c\min\{V^{2}H_{T},\sigma^{2}\}\ \geq\ \frac{c}{C_{\star}}\min\{V^{2}H_{T},\ \sigma^{2}n\}\ \geq\ \frac{c}{C_{\star}}\bigl(\rho_{T}\wedge V^{2}H_{T}\wedge\sigma^{2}n\bigr),$$

which is (I) and the lower half of (II); the upper half follows from (3.6), since $\rho_{T}\wedge V^{2}H_{T}\wedge\sigma^{2}n\leq C_{\star}\min\{V^{2}H_{T},\sigma^{2}\}\text{.}$

Case $k_{0}=1$ with $n\geq C_{\star}\text{.}$ The main text bounds $\rho_{T}\leq(A_{0}/\log 2)\,\sigma^{2}$ there, so

$$\rho_{T}\wedge V^{2}H_{T}\wedge\sigma^{2}n\ \leq\ \rho_{T}\ \leq\ C\min\{\sigma^{2},\ V^{2}H_{T}\},$$

using $\rho_{T}\leq V^{2}h_{T}\leq V^{2}H_{T}$ for the second branch; with (3.8) this gives (I). For (II), $\min\{V^{2}H_{T},\sigma^{2}k_{0}\}=\min\{V^{2}H_{T},\sigma^{2}\}\text{,}$ whose lower half is (3.8) and whose upper half is $R^{*}_{T}\leq C\rho_{T}\wedge CV^{2}H_{T}\leq C^{\prime}\min\{\sigma^{2},V^{2}H_{T}\}$ by (3.6).

Case $k_{0}=K+1$ with $n\geq C_{\star}\text{.}$ Here $K=\lfloor n/C_{\star}\rfloor\geq n/(2C_{\star})\text{,}$ so $\sigma^{2}k_{0}=\sigma^{2}(K+1)$ lies between $\sigma^{2}n/(2C_{\star})$ and $2\sigma^{2}n/C_{\star}\text{,}$ while $V^{2}H_{T}\geq V^{2}\delta_{K}>A_{0}\sigma^{2}K\geq\sigma^{2}k_{0}/2\text{.}$ Hence $\min\{V^{2}H_{T},\sigma^{2}k_{0}\}\asymp\sigma^{2}n\text{,}$ bounded below by the profile-to-risk lower bound proved in the main body and above by (3.6) through the branch $\sigma^{2}n\text{.}$

The fixed point. In the interior case $2\leq k_{0}\leq K\text{,}$ the bound (3.6) gives $\rho_{T}\geq\rho_{T}\wedge V^{2}H_{T}\wedge\sigma^{2}n\geq R^{*}_{T}/C\text{,}$ so (3.9) and (3.10) pin the fixed point itself: $c\,\sigma^{2}k_{0}\leq\rho_{T}\leq C\,\sigma^{2}k_{0}\text{.}$ ∎

S.4 Benchmark and Broom Profiles

This section proves Corollaries 1, 2, and 3, together with the broom profile bounds (5.8) and the remaining computations deferred from the broom lower bound of Section 5: the objective cap, the probability of the forcing event, the broom’s covering counts, and the crossing that pins its minimax rate. By Theorem 1 the minimax risk is $\min\{V^{2}H_{T},\sigma^{2}k_{0}\}$ up to universal constants, so each corollary reduces to pinning the covering counts and the profile with explicit constants, localizing $k_{0}\text{,}$ and identifying the active branch in every parameter regime. Throughout, $K:=\lfloor n/C_{\star}\rfloor$ with $C_{\star}=e^{2}\text{;}$ the dimension clause of (1.5) holds at $K+1\text{,}$ so $1\leq k_{0}\leq K+1$ on every tree. In the corollary proofs, $W:=V/\sigma\text{.}$

On the path, ancestor covering is interval covering, and the exact counts it yields reduce the profile to one scalar optimization.

Lemma S.4.1 (Path profile).

Let $P_{L}$ be the path with $L\geq 1$ edges, rooted at one endpoint, so that $n=L+1$ and $h_{P_{L}}=H_{P_{L}}=L\text{.}$ For integers $0\leq q<L\text{,}$

$$N^{\uparrow}_{T}(q)\ =\ \Bigl\lceil\frac{L+1}{q+1}\Bigr\rceil.$$

Moreover

$$\alpha_{k}(P_{L})\ \leq\ \frac{2(L+1)}{ek}\quad\text{for every }k\geq 1,\qquad\alpha_{k}(P_{L})\ \geq\ \frac{L+1}{2ek}\quad\text{for }1\leq k\leq\frac{L+1}{e^{2}}.$$

Proof.

Index the vertices by depth $0,\dots,L\text{.}$ The ancestors of $u$ are $0,\dots,u\text{,}$ so a center $a$ covers exactly the vertices $a,\dots,\min\{a+q,L\}\text{:}$ ancestor $q$-nets are coverings of $L+1$ points by blocks of $q+1$ consecutive integers anchored at their left endpoints. The centers $j(q+1)\text{,}$ $0\leq j\leq\lfloor L/(q+1)\rfloor\text{,}$ form such a covering, and there are $\lfloor L/(q+1)\rfloor+1=\lceil(L+1)/(q+1)\rceil$ of them; no net is smaller, because each center covers at most $q+1$ of the $L+1$ vertices and net sizes are integers.

Upper bound. Consider a radius $q$ whose term in the discrete formula (2.3) is nonzero, so that $N^{\uparrow}_{T}(q)>k\geq 1\text{;}$ then $\lceil(L+1)/(q+1)\rceil\geq 2$ forces $(L+1)/(q+1)>1\text{,}$ whence $N^{\uparrow}_{T}(q)<(L+1)/(q+1)+1<2(L+1)/(q+1)\text{.}$ With $y:=q+1$ and $A:=2(L+1)/k\text{,}$ the term is therefore at most

$$y\,\log\frac{A}{y}\ \leq\ \sup_{y>0}\ y\log\frac{A}{y}\ =\ \frac{A}{e}\ =\ \frac{2(L+1)}{ek},$$

the supremum being attained at $y=A/e\text{.}$

Lower bound. Set $x:=(L+1)/(ek)$ and $y:=\lfloor x\rfloor\text{.}$ The hypothesis $k\leq(L+1)/e^{2}$ gives $x\geq e>2\text{,}$ so $y\geq x-1\geq x/2\text{;}$ and $y\leq x\leq(L+1)/e\leq L\text{,}$ so $q:=y-1$ is an admissible radius. Then $N^{\uparrow}_{T}(q)=\lceil(L+1)/y\rceil\geq(L+1)/y\geq ek\text{,}$ so $[\log(N^{\uparrow}_{T}(q)/k)]_{+}\geq 1$ and the term at $q$ is at least $y\min\{k,1\}=y\geq(L+1)/(2ek)\text{.}$ ∎

Proof of Corollary 1.

By Theorem 1 and $H_{P_{L}}=L\text{,}$ $R^{*}_{P_{L}}(V,\sigma)\asymp\min\{V^{2}L,\sigma^{2}k_{0}\}\text{;}$ write

$$F\ :=\ \min\bigl\{V^{2}L,\ \ V^{2/3}\sigma^{4/3}L^{1/3},\ \ \sigma^{2}(L+1)\bigr\}$$

for the right side of Corollary 1. Since its middle branch is the geometric interpolation $(V^{2}L)^{1/3}(\sigma^{2})^{2/3}\text{,}$ every branch of $F\text{,}$ and hence $F$ itself, dominates $\min\{V^{2}L,\sigma^{2}\}\text{.}$ Here $K=\lfloor(L+1)/e^{2}\rfloor\text{,}$ and there are three cases.

Case $k_{0}=1\text{.}$ Here $\min\{V^{2}L,\sigma^{2}k_{0}\}=\min\{V^{2}L,\sigma^{2}\}\text{,}$ which $F$ dominates; and since $F\leq V^{2}L\text{,}$ the matching upper bound needs an argument only when $\sigma^{2}<V^{2}L\text{.}$ By (1.5), $k_{0}=1$ means $L+1<e^{2}$ or $V^{2}\delta_{1}\leq A_{0}\sigma^{2}\text{.}$ In the first case $F\leq\sigma^{2}(L+1)<e^{2}\sigma^{2}\text{.}$ Otherwise $L+1\geq e^{2}\text{,}$ so Lemma S.4.1 applies at $k=1$ and gives $\delta_{1}=\alpha_{1}\geq(L+1)/(2e)\text{,}$ whence $F\leq V^{2}L\leq 2e\,V^{2}\delta_{1}\leq 2eA_{0}\sigma^{2}\text{.}$

Case $2\leq k_{0}\leq K\text{.}$ By minimality no clause of (1.5) holds below $k_{0}\text{,}$ while at $k_{0}$ the crossing clause holds, the dimension clause being silent there. Lemma S.4.1 is available on all of $[1,K]\text{,}$ since $K\leq(L+1)/e^{2}\text{,}$ and with

$$\kappa\ :=\ \Bigl(\frac{V^{2}(L+1)}{eA_{0}\sigma^{2}}\Bigr)^{1/3}$$

the crossing inequalities at $k_{0}$ and $k_{0}-1$ read

$$\frac{V^{2}(L+1)}{2e\,k_{0}^{2}}\ \leq\ V^{2}\delta_{k_{0}}\ \leq\ A_{0}\sigma^{2}k_{0},\qquad A_{0}\sigma^{2}(k_{0}-1)\ <\ V^{2}\delta_{k_{0}-1}\ \leq\ \frac{2V^{2}(L+1)}{e\,(k_{0}-1)^{2}},$$

that is, $k_{0}^{3}\geq\kappa^{3}/2$ and $(k_{0}-1)^{3}<2\kappa^{3}\text{;}$ since $k_{0}\geq 2$ gives $k_{0}\leq 2(k_{0}-1)\text{,}$

$$2^{-1/3}\,\kappa\ \leq\ k_{0}\ \leq\ 2^{4/3}\,\kappa.$$

Hence $\sigma^{2}k_{0}\asymp\sigma^{2}\kappa\asymp V^{2/3}\sigma^{4/3}L^{1/3}\text{,}$ the middle branch of $F\text{,}$ using $L\leq L+1\leq 2L\text{.}$ That branch is the least of the three up to constants: failure of the crossing clause at $k=1$ gives $A_{0}\sigma^{2}<V^{2}\delta_{1}\leq 4V^{2}L/e\text{,}$ so $\sigma^{2}\leq CV^{2}L$ and the middle branch is at most $(V^{2}L)^{1/3}(CV^{2}L)^{2/3}=C^{2/3}V^{2}L\text{;}$ and it is $\asymp\sigma^{2}k_{0}\leq\sigma^{2}K\leq\sigma^{2}(L+1)\text{.}$ Therefore $F\asymp\sigma^{2}k_{0}\text{,}$ and the same middle-branch bound gives $\sigma^{2}k_{0}\leq CV^{2}L\text{,}$ so $\min\{V^{2}L,\sigma^{2}k_{0}\}\asymp\sigma^{2}k_{0}$ as well.

Case $k_{0}=K+1\geq 2\text{.}$ Then $K\geq 1\text{,}$ so $L+1\geq e^{2}\text{,}$ and no clause fired on $[1,K]\text{;}$ at $k=K\text{,}$

$$A_{0}\sigma^{2}K\ <\ V^{2}\delta_{K}\ \leq\ \frac{2V^{2}(L+1)}{eK^{2}},\quad\text{so}\quad V^{2}(L+1)\ >\ \frac{eA_{0}}{2}\,\sigma^{2}K^{3}\ \geq\ \frac{A_{0}}{16e^{5}}\,\sigma^{2}(L+1)^{3},$$

using $K\geq(L+1)/(2e^{2})\text{,}$ which holds because $\lfloor x\rfloor\geq x/2$ for $x\geq 1\text{.}$ Hence $V^{2}\geq c\,\sigma^{2}(L+1)^{2}$ with $c:=A_{0}/(16e^{5})\text{,}$ and both remaining branches of $F$ dominate the third: $V^{2}L\geq\tfrac{1}{2}V^{2}(L+1)\geq\tfrac{c}{2}\,\sigma^{2}(L+1)^{3}\geq\tfrac{c}{2}\,\sigma^{2}(L+1)\text{,}$ and $(V^{2}L)^{1/3}\sigma^{4/3}\geq(\tfrac{c}{2})^{1/3}\sigma^{2}(L+1)\text{.}$ Therefore $F\asymp\sigma^{2}(L+1)\text{.}$ On the other side, $(L+1)/e^{2}<K+1\leq 2(L+1)/e^{2}$ gives $\sigma^{2}k_{0}\asymp\sigma^{2}(L+1)\text{,}$ and $\min\{V^{2}L,\sigma^{2}k_{0}\}\asymp\sigma^{2}(L+1)$ as well, by $V^{2}L\geq\tfrac{c}{2}\,\sigma^{2}(L+1)\text{.}$ The three cases are exhaustive. ∎

On the star, $h_{S_{m}}=1\text{,}$ so only the count $N^{\uparrow}_{T}(0)=m+1$ remains; the profile is exact by Remark 2.1, splitting into a capped and a logarithmic phase, and the case analysis tracks which phase meets the crossing.

Proof of Corollary 2.

By Theorem 1 and $H_{S_{m}}=2\text{,}$ $R^{*}_{S_{m}}(V,\sigma)\asymp\min\{V^{2},\sigma^{2}k_{0}\}$ after absorbing the factor two; here $n=m+1\text{.}$ By Remark 2.1,

$$\alpha_{k}\ =\ \min\{k,\ \ell(k)\},\qquad\ell(x)\ :=\ \Bigl[\log\frac{m+1}{x}\Bigr]_{+}\quad(x>0),$$

and since $k$ grows while $\ell(k)$ does not, the set $\{k\geq 1:k\leq\ell(k)\}$ is an initial segment of the integers; it contains $1\text{,}$ because $\ell(1)=\log(m+1)>1\text{,}$ and is finite, because $\ell$ vanishes beyond $m+1\text{.}$ With $\bar{k}$ its largest element, the profile has two phases:

$$V^{2}\delta_{k}\ =\ \sigma^{2}W^{2}\ \ (k\leq\bar{k}),\qquad V^{2}\delta_{k}\ =\ \sigma^{2}W^{2}\,\frac{\ell(k)}{k}\ \ (k>\bar{k}).$$

Write

$$\Lambda\ :=\ 1+\bigl[\log(m/W)\bigr]_{+},\qquad F\ :=\ \min\bigl\{V^{2},\ \sigma^{2}W\sqrt{\Lambda},\ \sigma^{2}m\bigr\},$$

so that $F$ is the right side of Corollary 2.

Bounded stars. For any universal $m_{0}$ the corollary holds for $m\leq m_{0}$ with constants depending only on $m_{0}\text{:}$ from $1\leq k_{0}\leq K+1\leq m_{0}\text{,}$ $\min\{V^{2},\sigma^{2}k_{0}\}\asymp\min\{V^{2},\sigma^{2}\}\text{,}$ while $F\leq\min\{V^{2},\sigma^{2}m\}\leq m_{0}\min\{V^{2},\sigma^{2}\}$ and every branch of $F$ dominates $\min\{V^{2},\sigma^{2}\}\text{,}$ the middle one because $\sigma^{2}W\sqrt{\Lambda}\geq\sigma V\geq\min\{V^{2},\sigma^{2}\}\text{.}$ Now fix a universal $m_{0}$ with the following four properties for every $m\geq m_{0}$ (one may take $m_{0}=\lceil e^{10}\rceil$): (a) $\bar{k}\geq\tfrac{1}{2}\log(m+1)\text{;}$ (b) $\bar{k}+1\leq K\text{;}$ (c) $\log(m/w)\geq\tfrac{1}{2}(1+\log m)$ whenever $1\leq w\leq\sqrt{A_{0}\log(m+1)}\text{;}$ (d) $m\geq 4e^{4}(1+\log m)\text{.}$ Property (a) holds because $k^{\prime}:=\lceil\tfrac{1}{2}\log(m+1)\rceil$ satisfies $k^{\prime}+\log k^{\prime}\leq\log(m+1)$ once $m$ is large, so that $k^{\prime}\leq\ell(k^{\prime})\text{;}$ (b) and (d) compare linear with logarithmic growth; (c) is worst at $w=\sqrt{A_{0}\log(m+1)}\text{,}$ where it reads $\tfrac{1}{2}\log m\geq\tfrac{1}{2}+\tfrac{1}{2}\log(A_{0}\log(m+1))\text{.}$ We also use, unconditionally, $\bar{k}\leq\ell(\bar{k})\leq\log(m+1)\text{.}$ Assume $m\geq m_{0}\text{;}$ there are four cases in $W\text{.}$

Case $W^{2}\leq A_{0}\text{.}$ Since $\bar{k}\geq 1\text{,}$ the capped phase gives $V^{2}\delta_{1}=\sigma^{2}W^{2}\leq A_{0}\sigma^{2}\text{:}$ the crossing clause fires at once and $k_{0}=1\text{.}$ Then $\min\{V^{2},\sigma^{2}k_{0}\}\in[V^{2}/A_{0},\,V^{2}]\text{.}$ And $F\asymp V^{2}\text{:}$ it is at most $V^{2}\text{,}$ while $\sigma^{2}W\sqrt{\Lambda}\geq\sigma^{2}W\geq V^{2}/\sqrt{A_{0}}$ and $\sigma^{2}m\geq 2\sigma^{2}\geq 2V^{2}/A_{0}\text{.}$

Case $A_{0}<W^{2}\leq A_{0}\bar{k}\text{.}$ Every integer $k<W^{2}/A_{0}$ has $k<\bar{k}\text{,}$ so the capped phase gives $V^{2}\delta_{k}=\sigma^{2}W^{2}>A_{0}\sigma^{2}k\text{,}$ and $k<\bar{k}\leq K$ keeps the dimension clause silent: no clause fires below $W^{2}/A_{0}\text{.}$ At $k_{1}:=\lceil W^{2}/A_{0}\rceil\text{,}$ which is at most $\bar{k}\leq K$ because $W^{2}/A_{0}\leq\bar{k}$ and $\bar{k}$ is an integer, the bound $\delta_{k_{1}}\leq 1$ gives $V^{2}\delta_{k_{1}}\leq\sigma^{2}W^{2}\leq A_{0}\sigma^{2}k_{1}\text{.}$ Hence

$$k_{0}\ =\ \Bigl\lceil\frac{W^{2}}{A_{0}}\Bigr\rceil\ \in\ \Bigl[\frac{W^{2}}{A_{0}},\ \frac{2W^{2}}{A_{0}}\Bigr],\qquad\sigma^{2}k_{0}\ \asymp\ V^{2},$$

so $\min\{V^{2},\sigma^{2}k_{0}\}\asymp V^{2}\text{.}$ For $F\asymp V^{2}\text{:}$ the case hypothesis and $\bar{k}\leq\log(m+1)$ give $W\leq\sqrt{A_{0}\log(m+1)}\text{,}$ so property (c) yields $\Lambda\geq\tfrac{1}{2}(1+\log m)$ and

$$\sigma^{2}W\sqrt{\Lambda}\ \geq\ V^{2}\,\frac{\sqrt{\Lambda}}{W}\ \geq\ V^{2}\,\frac{\sqrt{(1+\log m)/2}}{\sqrt{A_{0}(1+\log m)}}\ =\ \frac{V^{2}}{\sqrt{2A_{0}}},$$

the denominator by $\log(m+1)\leq 1+\log m\text{.}$ Also $\sigma^{2}m\geq\sigma^{2}W^{2}=V^{2}\text{,}$ since property (d) gives $m\geq A_{0}\log(m+1)\geq W^{2}\text{.}$

Case $W^{2}>A_{0}\bar{k}$ and $W<m\text{.}$ Now $W>\sqrt{A_{0}}$ and $\log(m/W)>0\text{,}$ so $\Lambda=1+\log(m/W)\text{.}$ Set

$$\kappa\ :=\ W\sqrt{\Lambda},\qquad c_{1}\ :=\ \frac{1}{2\sqrt{A_{0}}}.$$

No crossing below $c_{1}\kappa\text{:}$ fix an integer $k\leq\min\{c_{1}\kappa,K\}\text{,}$ so that the dimension clause is silent at $k\text{.}$ If $k\leq\bar{k}\text{,}$ the capped phase and the case hypothesis give $V^{2}\delta_{k}=\sigma^{2}W^{2}>A_{0}\sigma^{2}\bar{k}\geq A_{0}\sigma^{2}k\text{.}$ If $\bar{k}<k\leq c_{1}\kappa\text{,}$ monotonicity of $\ell$ gives

$$\ell(k)\ \geq\ \ell(c_{1}\kappa)\ \geq\ \log\frac{m}{c_{1}W\sqrt{\Lambda}}\ =\ (\Lambda-1)+\log(2\sqrt{A_{0}})-\tfrac{1}{2}\log\Lambda\ \geq\ \frac{\Lambda}{4}+1,$$

the last step because $\log(2\sqrt{A_{0}})=\log 24>3$ and $\tfrac{3}{4}\Lambda+1\geq\tfrac{1}{2}\log\Lambda$ for every $\Lambda\geq 1\text{;}$ hence, using $A_{0}c_{1}^{2}=\tfrac{1}{4}\text{,}$

$$V^{2}\delta_{k}\ =\ \sigma^{2}\,\frac{W^{2}\ell(k)}{k}\ >\ \sigma^{2}\,\frac{W^{2}\Lambda}{4k}\ =\ A_{0}\sigma^{2}\,\frac{(c_{1}\kappa)^{2}}{k}\ \geq\ A_{0}\sigma^{2}k.$$

So no clause fires at $k\text{,}$ and $k_{0}>\min\{c_{1}\kappa,K\}\text{.}$ Crossing by $\lceil\kappa\rceil\text{:}$ if $\lceil\kappa\rceil\leq K\text{,}$ then $k_{1}:=\lceil\kappa\rceil$ has

$$\begin{gathered} \displaystyle\ell(k_{1})\ \leq\ \Bigl[\log\frac{2m}{\kappa}\Bigr]_{+}\ \leq\ \log 2+(\Lambda-1)-\tfrac{1}{2}\log\Lambda\ <\ \Lambda, \\ \displaystyle V^{2}\delta_{k_{1}}\ \leq\ \sigma^{2}\,\frac{W^{2}\Lambda}{k_{1}}\ \leq\ \sigma^{2}\kappa\ \leq\ A_{0}\sigma^{2}k_{1}, \end{gathered}$$

so the crossing clause fires by $k_{1}$ and $k_{0}\leq\lceil\kappa\rceil\text{.}$

If $\kappa\leq K\text{:}$ then $c_{1}\kappa<k_{0}\leq\lceil\kappa\rceil\leq 2\kappa\text{,}$ using $\kappa\geq W>1\text{,}$ so $\sigma^{2}k_{0}\asymp\sigma^{2}\kappa=\sigma^{2}W\sqrt{\Lambda}\text{,}$ the middle branch of $F\text{.}$ Moreover, by property (a) and the case hypothesis,

$$\Lambda\ \leq\ 1+\log m\ \leq\ 1+2\bar{k}\ <\ 1+\frac{2W^{2}}{A_{0}}\ \leq\ \frac{3W^{2}}{A_{0}},\qquad\text{so}\qquad\sigma^{2}\kappa\ \leq\ \sqrt{\tfrac{3}{A_{0}}}\,V^{2}\ \leq\ \tfrac{1}{6}\,V^{2}:$$

hence $\min\{V^{2},\sigma^{2}k_{0}\}\asymp\sigma^{2}\kappa\text{,}$ and $F\asymp\sigma^{2}\kappa$ as well: the first branch is at least $6\sigma^{2}\kappa\text{,}$ and the third is at least the second because $\kappa\leq K\leq m\text{.}$ If $\kappa>K\text{:}$ then $k_{0}>\min\{c_{1}\kappa,K\}\geq c_{1}K$ and $k_{0}\leq K+1\leq 2K\text{,}$ so $\sigma^{2}k_{0}\asymp\sigma^{2}K\asymp\sigma^{2}m\text{,}$ using $K\geq(m+1)/(2e^{2})\text{.}$ Also $V^{2}>\sigma^{2}m\text{:}$ squaring $W\sqrt{\Lambda}=\kappa>K\geq m/(2e^{2})$ gives

$$V^{2}\ =\ \sigma^{2}W^{2}\ >\ \frac{\sigma^{2}m^{2}}{4e^{4}\Lambda}\ \geq\ \sigma^{2}m,$$

by property (d) and $\Lambda\leq 1+\log m\text{.}$ Hence $\min\{V^{2},\sigma^{2}k_{0}\}\asymp\sigma^{2}m\text{,}$ and $F\asymp\sigma^{2}m\text{:}$ its first branch exceeds $\sigma^{2}m\text{,}$ and its second is $\sigma^{2}\kappa>\sigma^{2}K\geq\sigma^{2}m/(2e^{2})\text{.}$

Case $W\geq m\text{.}$ Then $\Lambda=1$ and $F=\sigma^{2}\min\{W^{2},W,m\}=\sigma^{2}m\text{.}$ For every integer $k\leq K\text{,}$

$$\ell(k)\ \geq\ \ell(K)\ =\ \log\frac{m+1}{K}\ \geq\ \log C_{\star}\ =\ 2,$$

so $\min\{k,\ell(k)\}\geq 1$ and $V^{2}\delta_{k}\geq\sigma^{2}W^{2}/k>A_{0}\sigma^{2}k$ whenever $k<W/\sqrt{A_{0}}\text{:}$ no clause fires below $\min\{W/\sqrt{A_{0}},\,K+1\}\geq m/12\text{,}$ the first entry because $W\geq m$ and $\sqrt{A_{0}}=12\text{,}$ the second because $K+1>(m+1)/e^{2}$ and $e^{2}<12\text{.}$ Hence $m/12\leq k_{0}\leq K+1\leq m$ and $\sigma^{2}k_{0}\asymp\sigma^{2}m\text{.}$ Since $V^{2}=\sigma^{2}W^{2}\geq\sigma^{2}m^{2}\geq\sigma^{2}m\text{,}$ also $\min\{V^{2},\sigma^{2}k_{0}\}\asymp\sigma^{2}m\asymp F\text{.}$

The four cases cover $m\geq m_{0}\text{:}$ if $W\geq m$ the fourth applies, and otherwise $W^{2}\leq A_{0}\text{,}$ $A_{0}<W^{2}\leq A_{0}\bar{k}\text{,}$ and $W^{2}>A_{0}\bar{k}$ partition the range. ∎

On the complete binary tree the covering counts decay exponentially in the radius. Radius therefore trades against entropy linearly, and the profile is governed by $\log(n/k)\text{:}$ quadratic in that logarithm when the budget exceeds it, and truncated at the budget otherwise.

Lemma S.4.2 (Binary profile).

Let $B_{h}$ be the complete binary tree of height $h\geq 1\text{,}$ with $n=2^{h+1}-1$ vertices, $h_{B_{h}}=h\text{,}$ and $H_{B_{h}}=2h\text{.}$ For integers $0\leq q<h\text{,}$

$$2^{h-q}\ \leq\ N^{\uparrow}_{T}(q)\ \leq\ 3\cdot 2^{h-q},$$

and for every integer $1\leq k\leq n/C_{\star}\text{,}$ with $\beta:=\log(n/k)\text{,}$ which is at least $2$ in this range,

$$c_{B}\,\beta\min\{k,\beta\}\ \leq\ \alpha_{k}(B_{h})\ \leq\ C_{B}\,\beta\min\{k,\beta\},\qquad c_{B}:=\frac{1}{8\log 2},\quad C_{B}:=3.$$

Proof.

Covering counts. An ancestor of a depth-$h$ leaf within distance $q$ has depth at least $h-q\text{,}$ and a vertex at depth $d\geq h-q$ has $2^{h-d}\leq 2^{q}$ leaf descendants; covering all $2^{h}$ leaves therefore needs at least $2^{h-q}$ centers. Conversely, let $R$ consist of the root together with every vertex whose depth is of the form $D_{j}:=h-q-j(q+1)\geq 0\text{,}$ $j\geq 0\text{.}$ A selected vertex covers exactly the depths $D_{j},\dots,D_{j}+q$ of its subtree; consecutive selected depths differ by $q+1\text{,}$ so the covered bands tile $\{D_{j^{*}},\dots,h\}\text{,}$ where $D_{j^{*}}$ is the smallest selected depth, and $D_{j^{*}}\leq q\text{,}$ since otherwise $D_{j^{*}}-(q+1)\geq 0$ would be selected as well; the root covers the depths below $D_{j^{*}}\text{.}$ Hence $R$ is an ancestor $q$-net, of size

$$1+\sum_{j:\,D_{j}\geq 0}2^{D_{j}}\ \leq\ 1+2^{h-q}\sum_{j\geq 0}2^{-j(q+1)}\ \leq\ 1+2\cdot 2^{h-q}\ \leq\ 3\cdot 2^{h-q}.$$

Profile, upper bound. Since $2^{h+1}=n+1$ and $\log(n+1)\leq\log n+\tfrac{1}{3}$ for $n\geq 3\text{,}$ every integer $0\leq q<h$ has, with $y:=q+1\text{,}$

$$\begin{aligned} \displaystyle\log\frac{N^{\uparrow}_{T}(q)}{k}\space & \displaystyle\leq\ (h-q)\log 2+\log 3-\log k \\ & \displaystyle=\ \log(n+1)+\log 3-\log k-y\log 2\ \leq\ \beta+\tfrac{3}{2}-y\log 2, \end{aligned}$$

and $\beta+\tfrac{3}{2}\leq\tfrac{7}{4}\beta$ because $\beta\geq 2\text{.}$ Substituting $u:=y\log 2$ in the discrete formula (2.3) and writing $B:=\tfrac{7}{4}\beta\text{,}$

$$\begin{aligned} \displaystyle\alpha_{k}(B_{h})\ \leq\ \frac{1}{\log 2}\,\sup_{u>0}\ u\,\min\bigl\{k,\ (B-u)_{+}\bigr\}\space & \displaystyle\leq\ \frac{1}{\log 2}\,B\,\min\Bigl\{\frac{B}{4},\ k\Bigr\} \\ & \displaystyle\leq\ \frac{7}{4\log 2}\,\beta\min\{k,\beta\}\ \leq\ C_{B}\,\beta\min\{k,\beta\}: \end{aligned}$$

if $k\geq B/4$ the supremum is at most $\sup_{u}u(B-u)_{+}=B^{2}/4\text{,}$ while if $k<B/4$ the product is at most $uk\leq(B-k)k$ for $u\leq B-k\text{,}$ and at most $u(B-u)_{+}\leq(B-k)k$ beyond, since $u(B-u)_{+}$ is nonincreasing past $B/2<B-k\text{.}$

Profile, lower bound. Set $y:=\lfloor\beta/(2\log 2)\rfloor\text{.}$ From $\beta\geq 2\text{:}$ $\beta/(2\log 2)\geq 1/\log 2>1\text{,}$ so $y\geq 1$ and, halving, $y\geq\beta/(4\log 2)\text{;}$ and $y\leq\beta/(2\log 2)\leq\log n/(2\log 2)\leq(h+1)/2\leq h\text{,}$ so $q:=y-1$ is an admissible radius. At this radius,

$$\log\frac{N^{\uparrow}_{T}(q)}{k}\ \geq\ (h-q)\log 2-\log k\ =\ \log(n+1)-\log k-y\log 2\ \geq\ \beta-\frac{\beta}{2}\ =\ \frac{\beta}{2},$$

so, using $\min\{k,\beta/2\}\geq\tfrac{1}{2}\min\{k,\beta\}\text{,}$ the term at $q$ is at least

$$y\,\min\Bigl\{k,\ \frac{\beta}{2}\Bigr\}\ \geq\ \frac{\beta}{4\log 2}\cdot\frac{\min\{k,\beta\}}{2}\ =\ c_{B}\,\beta\min\{k,\beta\}.$$

Proof of Corollary 3.

By Theorem 1 and $H_{B_{h}}=2h\text{,}$ $R^{*}_{B_{h}}(V,\sigma)\asymp\min\{V^{2}h,\sigma^{2}k_{0}\}$ after absorbing the factor two. Write $\beta(k):=\log(n/k)\text{,}$ as in Lemma S.4.2, and

$$\Lambda\ :=\ 1+\bigl[\log(n/W)\bigr]_{+},\qquad F\ :=\ \sigma^{2}\min\bigl\{W^{2}h,\ W\Lambda,\ n\bigr\},$$

the right side of Corollary 3.

Bounded trees. For any universal $n_{0}$ the corollary holds for $n\leq n_{0}$ with constants depending only on $n_{0}\text{:}$ from $1\leq k_{0}\leq K+1\leq n_{0}\text{,}$ $\min\{V^{2}h,\sigma^{2}k_{0}\}\asymp\min\{V^{2}h,\sigma^{2}\}\text{,}$ while $F\leq\min\{V^{2}h,\sigma^{2}n\}\leq n_{0}\min\{V^{2}h,\sigma^{2}\}$ and every branch of $F$ dominates $\min\{V^{2}h,\sigma^{2}\}\text{;}$ for the middle branch, when $W\leq 1$ this is $\sigma^{2}W\Lambda\geq\log 2\cdot\sigma^{2}W^{2}h\text{,}$ using $W\geq W^{2}$ and $\Lambda\geq 1+\log n\geq h\log 2\text{,}$ and when $W>1$ it is $\sigma^{2}W\Lambda\geq\sigma^{2}\text{.}$ Now fix a universal $n_{0}$ (one may take $n_{0}=2^{7}$) such that for every $n\geq n_{0}\text{:}$ $h\leq K\text{,}$ $n\leq 2e^{2}K\text{,}$ and $\log h\leq\tfrac{\log 2}{2}\,h\text{.}$ Assume $n\geq n_{0}\text{.}$

The diameter regime $W\leq 1\text{.}$ We claim $k_{0}\geq c\,W^{2}h$ with $c:=c_{B}\log 2/(4A_{0})\text{.}$ This is trivial when $cW^{2}h<1\text{;}$ otherwise fix an integer $k\leq cW^{2}h\leq ch\text{.}$ Then

$$\beta(k)\ \geq\ \log\frac{2^{h}}{h}\ =\ h\log 2-\log h\ \geq\ \frac{\log 2}{2}\,h\ \geq\ k,$$

the last step because $c\leq\log 2/2\text{;}$ so Lemma S.4.2 gives

$$V^{2}\delta_{k}\ \geq\ c_{B}\,\sigma^{2}W^{2}\beta(k)\ \geq\ \frac{c_{B}\log 2}{2}\,\sigma^{2}W^{2}h\ >\ A_{0}\sigma^{2}\,cW^{2}h\ \geq\ A_{0}\sigma^{2}k,$$

and the dimension clause is silent at $k\leq h\leq K\text{:}$ no clause fires at any $k\leq cW^{2}h\text{,}$ proving the claim. In either case $\sigma^{2}k_{0}\geq c\,V^{2}h\text{,}$ so

$$\min\{V^{2}h,\ \sigma^{2}k_{0}\}\ \asymp\ V^{2}h.$$

And $F\asymp V^{2}h\text{:}$ its first branch is $V^{2}h\text{;}$ its middle branch dominates $\log 2\cdot V^{2}h$ as above; and $\sigma^{2}n\geq\sigma^{2}h\geq V^{2}h\text{.}$ This proves the corollary for $W\leq 1\text{.}$

The entropy and dimension regimes $W>1\text{.}$ Put $\kappa:=W\Lambda\geq 1\text{.}$ First, if $\lceil\kappa\rceil\leq K\text{,}$ the crossing clause fires by $k_{1}:=\lceil\kappa\rceil\text{:}$ there $W\leq k_{1}\leq K\leq n\text{,}$ so $\beta(k_{1})\leq\log(n/W)\leq\Lambda\text{,}$ and Lemma S.4.2 gives

$$V^{2}\delta_{k_{1}}\ \leq\ C_{B}\,\sigma^{2}\,\frac{W^{2}\beta(k_{1})^{2}}{k_{1}}\ \leq\ C_{B}\,\sigma^{2}\,\frac{W^{2}\Lambda^{2}}{\kappa}\ =\ C_{B}\,\sigma^{2}\kappa\ \leq\ A_{0}\sigma^{2}k_{1};$$

hence $k_{0}\leq\lceil\kappa\rceil$ whenever $\lceil\kappa\rceil\leq K\text{.}$ Second, with $c:=c_{B}/(8A_{0})\text{,}$ no clause fires at any integer $k\leq\min\{c\kappa,K\}\text{.}$ The dimension clause is silent since $k\leq K\text{.}$ For the crossing clause, note first that $\beta(k)\geq\Lambda/4\text{:}$ if $W\geq n$ this reads $\beta(k)\geq 2>\tfrac{1}{4}=\Lambda/4\text{,}$ and if $W<n\text{,}$ then $k\leq cW\Lambda$ with $c\leq 1$ gives

$$\beta(k)\ \geq\ \log\frac{n}{W\Lambda}\ =\ (\Lambda-1)-\log\Lambda\ \geq\ \frac{\Lambda}{4}\quad(\Lambda\geq 3),\qquad\beta(k)\ \geq\ 2\ \geq\ \frac{3}{4}\ >\ \frac{\Lambda}{4}\quad(\Lambda<3),$$

the first branch because $\tfrac{3}{4}\Lambda\geq 1+\log\Lambda$ for $\Lambda\geq 3\text{.}$ Now split on $\min\{k,\beta(k)\}\text{.}$ If $k\leq\beta(k)\text{,}$ then, using $k\leq cW\Lambda\leq cW^{2}\Lambda$ and $A_{0}c=c_{B}/8\text{,}$

$$V^{2}\delta_{k}\ \geq\ c_{B}\,\sigma^{2}W^{2}\beta(k)\ \geq\ \frac{c_{B}}{4}\,\sigma^{2}W^{2}\Lambda\ >\ A_{0}\sigma^{2}\,cW^{2}\Lambda\ \geq\ A_{0}\sigma^{2}k.$$

If $k>\beta(k)\text{,}$ then

$$V^{2}\delta_{k}\ \geq\ c_{B}\,\sigma^{2}\,\frac{W^{2}\beta(k)^{2}}{k}\ \geq\ \frac{c_{B}}{16}\,\sigma^{2}\,\frac{\kappa^{2}}{k}\ >\ A_{0}\sigma^{2}k,$$

the last step because $k\leq c\kappa$ and $c=c_{B}/(8A_{0})<\sqrt{c_{B}}/(4\sqrt{A_{0}})\text{,}$ so that $k^{2}\leq c^{2}\kappa^{2}<\tfrac{c_{B}}{16A_{0}}\kappa^{2}\text{.}$ Hence $k_{0}>\min\{c\kappa,K\}\text{.}$

Combining the two tests: if $\kappa\leq K\text{,}$ then $c\kappa<k_{0}\leq\lceil\kappa\rceil\leq 2\kappa\text{,}$ so $\sigma^{2}k_{0}\asymp\sigma^{2}\kappa=\sigma^{2}\min\{W\Lambda,n\}\text{,}$ using $\kappa\leq K\leq n\text{;}$ if $\kappa>K\text{,}$ then $k_{0}>\min\{c\kappa,K\}\geq cK$ and $k_{0}\leq K+1\leq 2K\text{,}$ so $\sigma^{2}k_{0}\asymp\sigma^{2}K\asymp\sigma^{2}n\asymp\sigma^{2}\min\{W\Lambda,n\}\text{,}$ using $n\leq 2e^{2}K\text{.}$ In both regimes

$$\sigma^{2}k_{0}\ \asymp\ \sigma^{2}\min\{W\Lambda,\ n\},\quad\text{hence}\quad\min\{V^{2}h,\ \sigma^{2}k_{0}\}\ \asymp\ \sigma^{2}\min\{W^{2}h,\ W\Lambda,\ n\}\ =\ F,$$

which proves the corollary for $W>1\text{.}$ ∎

It remains to prove the profile bounds (5.8) for the broom $T_{L}$ of Section 5, the handle $o=v_{0},\dots,v_{L}$ with $m$ leaf children at $v_{L}\text{,}$ so that $h_{T_{L}}=L+1\text{.}$ Its ancestor-covering numbers were computed in (5.7) there: $N^{\uparrow}_{T}(0)=n\text{,}$ and $N^{\uparrow}_{T}(q)=\lceil(L+2)/(q+1)\rceil$ for integers $1\leq q\leq L\text{.}$

Lemma S.4.3 (Broom profile).

For the broom $T_{L}$ with $L\geq 1$ and every integer $k\geq 1\text{,}$

$$\delta_{k}(T_{L})\ \leq\ \max\Bigl\{1,\ \frac{2(L+2)}{ek^{2}}\Bigr\},$$

and whenever $k\leq(L+2)/(2e)\text{,}$

$$\delta_{k}(T_{L})\ \geq\ \frac{L+2}{2ek^{2}}\,.$$

Proof.

By the discrete formula (2.3), with $h_{T_{L}}=L+1\text{,}$

$$\alpha_{k}(T_{L})\ =\ \max_{0\leq q\leq L}\ (q+1)\,\min\Bigl\{k,\ \Bigl[\log\frac{N^{\uparrow}_{T}(q)}{k}\Bigr]_{+}\Bigr\}.$$

Upper bound. The term at $q=0$ is at most $k$ and contributes at most $1$ to $\delta_{k}=\alpha_{k}/k\text{.}$ Consider $1\leq q\leq L$ with a nonvanishing term and put $y:=q+1\text{.}$ Then $\lceil(L+2)/y\rceil>k\text{,}$ hence $(L+2)/y>k\text{,}$ since a real number at most the integer $k$ has ceiling at most $k\text{;}$ consequently

$$N^{\uparrow}_{T}(q)\ <\ \frac{L+2}{y}+1\ <\ \frac{2(L+2)}{y}\,,$$

and the term is at most

$$y\,\log\frac{2(L+2)}{ky}\ \leq\ \sup_{u>0}\ u\log\frac{A}{u}\ =\ \frac{A}{e}\,,\qquad A:=\frac{2(L+2)}{k}\,,$$

the supremum attained at $u=A/e\text{.}$ Hence $\alpha_{k}\leq\max\{k,\ 2(L+2)/(ek)\}\text{,}$ and dividing by $k$ proves the upper bound.

Lower bound. Set $x:=(L+2)/(ek)\text{,}$ at least $2$ in the stated range, and $y:=\lfloor x\rfloor\text{,}$ so that $y\geq 2$ and $y>x-1\geq x/2\text{.}$ The radius $q:=y-1$ lies in $[1,L]\text{:}$ $q\geq 1$ since $y\geq 2\text{,}$ and $q\leq L$ since $y\leq x\leq(L+2)/e<L+1$ for every $L\geq 1\text{.}$ Then

$$N^{\uparrow}_{T}(q)\ =\ \Bigl\lceil\frac{L+2}{y}\Bigr\rceil\ \geq\ \frac{L+2}{y}\ \geq\ \frac{L+2}{x}\ =\ ek,$$

so $[\log(N^{\uparrow}_{T}(q)/k)]_{+}\geq 1\text{,}$ and the term at $q$ is at least $y\cdot\min\{k,1\}=y\geq x/2\text{.}$ Hence $\alpha_{k}\geq(L+2)/(2ek)\text{,}$ and dividing by $k$ proves the lower bound. ∎

Four computations from the lower-bound proof of Theorem 5 remain, in the notation fixed there: $V=L^{1/4}\text{,}$ $\sigma=1\text{,}$ $m\geq e^{20L^{5/2}}\text{,}$ and the statistics $W=\sum_{j=1}^{L}Z_{v_{j}}\text{,}$ $Q=\sum_{j=1}^{L}[Z_{v_{j}}]_{+}^{2}\text{,}$ $M=\max_{1\leq i\leq m}Z_{w_{i}}\text{.}$

The covering counts (5.7). For the lower bound, the root-to-leaf path $v_{0},\dots,v_{L},w_{1}$ has $L+2$ vertices; an ancestor center on it covers at most $j+1$ consecutive ones, and a center at any other leaf covers none of them, so $N^{\uparrow}_{T}(j)\geq\lceil(L+2)/(j+1)\rceil\text{.}$ For the upper bound, the single center $v_{L+1-j}$ covers $v_{L+1-j},\dots,v_{L}$ together with every leaf, each at distance exactly $j\text{,}$ and $\lceil(L-j+1)/(j+1)\rceil$ further centers along the prefix $v_{0},\dots,v_{L-j}$ complete an ancestor $j$-net; the total is $1+\lceil(L-j+1)/(j+1)\rceil=\lceil(L+2)/(j+1)\rceil\text{.}$

The cap (5.3). For $h_{j}\geq 0$ the handle term obeys $2Z_{v_{j}}h_{j}-h_{j}^{2}\leq[Z_{v_{j}}]_{+}^{2}\text{:}$ the left side is nonpositive when $Z_{v_{j}}\leq 0\text{,}$ and equals $Z_{v_{j}}^{2}-(h_{j}-Z_{v_{j}})^{2}$ otherwise. The leaf part is at most $2M\sum_{i}\ell_{i}\leq 2Mh_{L}\leq MV$ by (5.1), the last step because the case under consideration has $h_{L}\leq V/2\text{.}$

The forcing event $E=\{W\geq-2\sqrt{L}\}\cap\{Q\leq 2L\}\cap\{M\geq 4L^{5/4}\}$ has probability at least $\tfrac{3}{8}\text{.}$ On $E\text{,}$

$$(L+1)V-2W+\frac{Q}{V}\ \leq\ L^{5/4}+L^{1/4}+4L^{1/2}+2L^{3/4}\ <\ 4L^{5/4}\ \leq\ M,$$

the middle inequality being equivalent to $L^{-1}+4L^{-3/4}+2L^{-1/2}<3\text{,}$ whose left side is decreasing in $L$ and below $3$ at $L=4\text{;}$ so the forcing inequality (5.4) holds on $E\text{.}$ Since $W\sim N(0,L)\text{,}$ Markov’s inequality applied to $W^{2}$ gives $\mathbb{P}(W<-2\sqrt{L})\leq\tfrac{1}{4}\text{;}$ since $\mathbb{E}Q=L/2\text{,}$ by $\mathbb{E}[Z_{v_{j}}]_{+}^{2}=\tfrac{1}{2}\text{,}$ the same inequality gives $\mathbb{P}(Q>2L)\leq\tfrac{1}{4}\text{;}$ hence $\mathbb{P}\{W\geq-2\sqrt{L},\ Q\leq 2L\}\geq\tfrac{1}{2}\text{.}$ For the leaves, set $t:=4L^{5/4}$ and integrate the standard normal density, decreasing on $[t,t+t^{-1}]\text{,}$ over that interval:

$$\mathbb{P}\bigl(Z_{w_{1}}\geq t\bigr)\ \geq\ \frac{1}{t\sqrt{2\pi}}\,e^{-(t+t^{-1})^{2}/2}\ \geq\ c_{G}\,L^{-5/4}\,e^{-8L^{5/2}},\qquad c_{G}:=\frac{e^{-33/32}}{4\sqrt{2\pi}}\ ,$$

the exponent being $\tfrac{1}{2}t^{2}+1+\tfrac{1}{2}t^{-2}\leq 8L^{5/2}+\tfrac{33}{32}$ by $t^{2}=16L^{5/2}\geq 16\text{.}$ Multiplying by $m\geq e^{20L^{5/2}}\text{,}$

$$m\,\mathbb{P}\bigl(Z_{w_{1}}\geq t\bigr)\ \geq\ c_{G}\,L^{-5/4}\,e^{12L^{5/2}}\ \geq\ \log 4,$$

the middle expression being increasing in $L$ and, at $L=1\text{,}$ equal to $c_{G}e^{12}\geq e^{12-33/32-3}\geq e^{7}\text{,}$ using $4\sqrt{2\pi}\leq e^{3}\text{.}$ The leaves are independent of one another, so $\mathbb{P}(M<t)=(1-\mathbb{P}(Z_{w_{1}}\geq t))^{m}\leq e^{-m\mathbb{P}(Z_{w_{1}}\geq t)}\leq\tfrac{1}{4}\text{,}$ and independent of the handle, whence $\mathbb{P}(E)\geq\tfrac{1}{2}\cdot\tfrac{3}{4}=\tfrac{3}{8}\text{.}$

The crossing index satisfies $k_{0}\asymp\sqrt{L}\text{.}$ Set $\kappa:=(V^{2}(L+2)/(2eA_{0}))^{1/3}\text{,}$ which is $\asymp\sqrt{L}$ since $V^{2}=\sqrt{L}\text{.}$ Every integer $k<\kappa$ lies in the lower-bound range of Lemma S.4.3: $(2e)^{2}\sqrt{L}\leq A_{0}(L+2)^{2}$ gives $\kappa\leq(L+2)/(2e)\text{,}$ and there $V^{2}\delta_{k}\geq V^{2}(L+2)/(2ek^{2})>A_{0}k\text{,}$ the strict inequality being $k^{3}<\kappa^{3}$ rearranged, while $n\geq e^{20L^{5/2}}$ keeps the dimension clause of (1.5) silent; hence $k_{0}\geq\kappa\text{.}$ At $k_{+}:=\lceil 4^{1/3}\kappa\rceil$ the upper bound of Lemma S.4.3 gives $V^{2}\delta_{k_{+}}\leq\max\{V^{2},2V^{2}(L+2)/(ek_{+}^{2})\}\leq A_{0}k_{+}\text{,}$ the first branch since $A_{0}k_{+}\geq A_{0}^{2/3}(2/e)^{1/3}\sqrt{L}\geq V^{2}$ and the second since $k_{+}^{3}\geq 4\kappa^{3}\text{;}$ so the crossing fires by $k_{+}$ and $\kappa\leq k_{0}\leq\lceil 4^{1/3}\kappa\rceil\text{,}$ whence $k_{0}\asymp\sqrt{L}\text{.}$ Finally $\kappa\leq(L+2)/(2e)$ gives $k_{0}\leq\lceil 4^{1/3}\kappa\rceil\leq L+3\leq\sqrt{L}\,(L+1)=V^{2}H_{T_{L}}\text{,}$ so the minimum in Theorem 1 is the information branch and $R^{*}_{T_{L}}(L^{1/4},1)\asymp k_{0}\asymp\sqrt{L}\text{.}$

S.5 Exact Evaluation of the Aggregate

This section proves the statements deferred from Sections 4.1 and 4.2: the bound on the normalizing sum of the prior, the completion of the proof of Lemma 4.1, the risk of the three elementary branches of (4.3), the correctness and cost of the exact evaluation of the aggregate (4.2), and the batched evaluation behind the instance sentence of Section 4.2, including the softly linear evaluation on stars.

Throughout, $K:=\lfloor n/C_{\star}\rfloor\text{;}$ except in the treatment of the elementary branches, $k$ is an integer with $2\leq k\leq K\text{.}$ The active support $A_{k}\text{,}$ the charges $\widetilde{\omega}\text{,}$ the code length $\Gamma\text{,}$ and the level weights $m_{j}=2^{j}$ for $0\leq j\leq J$ are those of Section 3.1; $\mathcal{X}_{k}$ is the class (4.1) of integer states of Section 4.1; and, as in Section 4.2 there, $\varphi_{v}(x)=e^{-(Y_{v}-bx)^{2}/(4\sigma^{2})}$ with the shorthand $b:=V/k\text{.}$

Lemma S.5.1 (Normalizing sum).

For every integer $2\leq k\leq K\text{,}$

$$\sum_{x\in\mathcal{X}_{k}}e^{-2\Gamma(x)}\ \leq\ e^{k}.$$

Proof.

The map $x\mapsto z$ is injective on state vectors, since the subtree sums $x_{v}=\sum_{u\succeq v}z_{u}$ recover $x\text{,}$ and $\Gamma(x)$ depends only on $z\text{.}$ Dropping the constraints $x_{o}=k$ and $0\leq x_{v}\leq 3k$ therefore bounds the sum by a product over the support:

$$\sum_{x\in\mathcal{X}_{k}}e^{-2\Gamma(x)}\ \leq\ \sum_{z\in\mathbb{Z}^{A_{k}}}\prod_{v\in A_{k}}e^{-2(|z_{v}|+\widetilde{\omega}(v)\mathbf{1}\{z_{v}\neq 0\})}\ =\ \prod_{v\in A_{k}}\Bigl(1+\tfrac{2}{e^{2}-1}\,e^{-2\widetilde{\omega}(v)}\Bigr),$$

the factor at $v$ being $1+e^{-2\widetilde{\omega}(v)}\cdot 2\sum_{t\geq 1}e^{-2t}\text{,}$ with $2\sum_{t\geq 1}e^{-2t}=\tfrac{2}{e^{2}-1}\text{.}$ The free level $R_{0}\text{,}$ where $\widetilde{\omega}=0\text{,}$ has $|R_{0}|\leq ke^{m_{0}}=ek$ vertices by Lemma S.2.1(2), and $\log(1+\tfrac{2}{e^{2}-1})\leq\tfrac{3}{10}\text{,}$ so it contributes at most $\tfrac{3}{10}ek<0.82\,k$ to the logarithm. The charged vertices split into the groups $G_{j}:=\{v\in A_{k}\setminus R_{0}:\ \omega(v)=m_{j}\}\subseteq R_{j}\text{,}$ $1\leq j\leq J\text{,}$ with $|G_{j}|\leq 2ke^{m_{j}}$ by Lemma S.2.1(3); since $\log(1+y)\leq y\text{,}$ their contribution is at most

$$\sum_{j\geq 1}2ke^{m_{j}}\cdot\frac{2}{e^{2}-1}\,e^{-2m_{j}}\ =\ \frac{4k}{e^{2}-1}\sum_{j\geq 1}e^{-2^{j}}\ \leq\ \frac{4k}{e^{2}-1}\cdot 0.16\ <\ 0.11\,k,$$

using $\sum_{j\geq 1}e^{-2^{j}}<e^{-2}+e^{-4}+2e^{-8}<0.16\text{.}$ The logarithm of the product is below $0.93\,k\text{.}$ ∎

In the setting of Lemma 4.1, write

$$\mathcal{Z}(y)\ :=\ \sum_{g\in F}\pi_{g}\,e^{-\|y-g\|_{2}^{2}/(4\sigma^{2})}$$

for the normalizing sum of the weights, positive and smooth on $\mathbb{R}^{N}\text{.}$

Completion of the proof of Lemma 4.1.

The Jacobian. The logarithm of a weight is $\log\pi_{f}-\|y-f\|_{2}^{2}/(4\sigma^{2})-\log\mathcal{Z}(y)\text{;}$ differentiating it in $y_{i}$ gives

$$\frac{\partial\log\widehat{\pi}_{f}}{\partial y_{i}}\ =\ -\frac{y_{i}-f_{i}}{2\sigma^{2}}+\sum_{g\in F}\widehat{\pi}_{g}(y)\,\frac{y_{i}-g_{i}}{2\sigma^{2}}\ =\ \frac{f_{i}-\widehat{f}_{i}(y)}{2\sigma^{2}}\ ,$$

that is, $\partial\widehat{\pi}_{f}/\partial y_{i}=\widehat{\pi}_{f}\,(f_{i}-\widehat{f}_{i})/(2\sigma^{2})\text{.}$ Multiplying by $f_{j}$ and summing over $f\text{,}$

$$\frac{\partial\widehat{f}_{j}}{\partial y_{i}}\ =\ \frac{1}{2\sigma^{2}}\Bigl(\sum_{f\in F}\widehat{\pi}_{f}\,f_{i}f_{j}-\widehat{f}_{i}\,\widehat{f}_{j}\Bigr)\ =\ \frac{1}{2\sigma^{2}}\operatorname{Cov}_{\widehat{\pi}(y)}(f_{i},f_{j}),$$

the Jacobian identity of the main-text sketch; taking $i=j$ and summing,

$$2\sigma^{2}\operatorname{div}\widehat{f}_{\pi}(y)\ =\ \operatorname{tr}\operatorname{Cov}_{\widehat{\pi}(y)}(f)\ =\ \sum_{f\in F}\widehat{\pi}_{f}(y)\,\bigl\|f-\widehat{f}_{\pi}(y)\bigr\|_{2}^{2}\ \geq\ 0.$$

(S.5.1)

The domain of Stein’s identity. The aggregate takes values in the convex hull of the finite set $F\text{,}$ so it is bounded; its partial derivatives are posterior covariances of coordinates of $F\text{,}$ bounded uniformly in $y\text{;}$ and both are smooth, since $\mathcal{Z}>0$ everywhere. For such a map, Stein’s identity (Stein, 1981) gives

$$\mathbb{E}_{\mu}\bigl\|\widehat{f}_{\pi}(Y)-\mu\bigr\|_{2}^{2}\ =\ \mathbb{E}\bigl\|\widehat{f}_{\pi}(Y)-Y\bigr\|_{2}^{2}+2\sigma^{2}\,\mathbb{E}\operatorname{div}\widehat{f}_{\pi}(Y)\ -\ N\sigma^{2}.$$

The cancellation. At fixed $y\text{,}$ expanding $\|y-f\|_{2}^{2}$ around $\widehat{f}_{\pi}(y)$ and averaging under $\widehat{\pi}(y)$ kills the cross term, since $\sum_{f}\widehat{\pi}_{f}\,(f-\widehat{f}_{\pi})=0\text{:}$

$$\sum_{f\in F}\widehat{\pi}_{f}(y)\,\|y-f\|_{2}^{2}\ =\ \bigl\|y-\widehat{f}_{\pi}(y)\bigr\|_{2}^{2}+\sum_{f\in F}\widehat{\pi}_{f}(y)\,\bigl\|f-\widehat{f}_{\pi}(y)\bigr\|_{2}^{2}.$$

The last sum is $2\sigma^{2}\operatorname{div}\widehat{f}_{\pi}(y)$ by Equation S.5.1, so substituting the expansion into Stein’s identity cancels the divergence terms and leaves the risk identity of the main-text sketch.

The variational bound. The relative entropy of probability vectors on $F$ is $\operatorname{KL}(\rho\,\|\,\pi):=\sum_{f}\rho_{f}\log(\rho_{f}/\pi_{f})\text{,}$ with $0\log 0:=0\text{.}$ For any such $\rho\text{,}$ inserting the definition of $\widehat{\pi}(y)$ gives the identity

$$\sum_{f\in F}\rho_{f}\,\|y-f\|_{2}^{2}+4\sigma^{2}\operatorname{KL}(\rho\,\|\,\pi)\ =\ 4\sigma^{2}\operatorname{KL}\bigl(\rho\,\|\,\widehat{\pi}(y)\bigr)\ -\ 4\sigma^{2}\log\mathcal{Z}(y),$$

so the left side is minimized exactly at $\rho=\widehat{\pi}(y)\text{.}$ Comparing the minimizer with the point mass at $f_{0}\text{,}$ and dropping the nonnegative divergence $\operatorname{KL}(\widehat{\pi}(y)\,\|\,\pi)$ from the minimum,

$$\sum_{f\in F}\widehat{\pi}_{f}(y)\,\|y-f\|_{2}^{2}\ \leq\ \|y-f_{0}\|_{2}^{2}+4\sigma^{2}\log\frac{1}{\pi_{f_{0}}}\ ,$$

the pathwise bound of the main body, whose expectation completes the proof there. ∎

The elementary branches of (4.3) are next. Their analysis rests on two facts: the root-corrected observation matches $\mu(o)=V$ exactly, and $Vp_{o}$ is within $V\sqrt{h_{T}}$ of every body point.

Lemma S.5.2 (Elementary branches).

Write $k:=k_{\mathrm{alg}}(T,V,\sigma)\text{.}$ In each of the three elementary branches of (4.3),

$$\sup_{\mu\in\mathcal{F}_{V}(T)}\mathbb{E}_{\mu}\|\widehat{\mu}-\mu\|_{2}^{2}\ \leq\ C\min\{V^{2}H_{T},\ \sigma^{2}k\}.$$

Proof.

For every $\mu=\sum_{u}\lambda_{u}\,Vp_{u}\in\mathcal{F}_{V}(T)\text{,}$ convexity of the norm and Lemma 2.2 give $\|\mu-Vp_{o}\|_{2}\leq\max_{u}\|Vp_{u}-Vp_{o}\|_{2}=V\sqrt{h_{T}}\text{.}$

Diameter branch ($V^{2}H_{T}\leq\sigma^{2}k$). The estimator is the constant $Vp_{o}\text{,}$ so its risk is at most $V^{2}h_{T}\leq V^{2}H_{T}\text{,}$ which is the minimum under the branch condition.

Dimension branch ($V^{2}H_{T}>\sigma^{2}k$ and $k>K$). Since $k\leq K+1$ always, $k=K+1\text{,}$ and $K=\lfloor n/C_{\star}\rfloor$ gives $n<C_{\star}(K+1)=C_{\star}k\text{.}$ The root coordinate of $\widetilde{Y}$ is exact, so

$$\mathbb{E}_{\mu}\|\widetilde{Y}-\mu\|_{2}^{2}\ =\ (n-1)\,\sigma^{2}\ <\ C_{\star}\,\sigma^{2}k,$$

and the minimum is $\sigma^{2}k$ under the branch condition.

First-budget branch ($V^{2}H_{T}>\sigma^{2}k$ and $k=1\leq K$). Here $n\geq C_{\star}\text{,}$ so $n\geq 2\text{,}$ and, as in Section 3.3, every ancestor $r$-net with $r<h_{T}$ has at least two elements: the root lies in every net and covers no deepest vertex. Hence $N^{\uparrow}_{T}(r)\geq 2$ for all $r<h_{T}\text{,}$ and letting $r\uparrow h_{T}$ in (1.4) gives $\alpha_{1}\geq h_{T}\log 2\text{.}$ The dimension clause of (2.2) is silent at $k=1\text{,}$ since $1\leq K\text{,}$ so the surrogate crossing clause fired there, $V^{2}\overline{\delta}_{1}\leq 2A_{0}\sigma^{2}\text{,}$ and with Lemma 2.3,

$$\sup_{\mu}\|\mu-Vp_{o}\|_{2}^{2}\ \leq\ V^{2}h_{T}\ \leq\ \frac{V^{2}\alpha_{1}}{\log 2}\ =\ \frac{V^{2}\delta_{1}}{\log 2}\ \leq\ \frac{V^{2}\overline{\delta}_{1}}{\log 2}\ \leq\ \frac{2A_{0}}{\log 2}\,\sigma^{2};$$

the minimum is $\sigma^{2}$ under the branch condition. ∎

The rest of the section evaluates the interior branch. Fix $2\leq k\leq K$ and collect the prior into local kernels: for $z\in\mathbb{Z}$ put

$$\kappa_{v}(z)\ :=\ \begin{cases}1,&z=0,\\ e^{-2\widetilde{\omega}(v)}\,e^{-2|z|},&z\neq 0\ \text{and}\ v\in A_{k},\\ 0,&z\neq 0\ \text{and}\ v\notin A_{k}.\end{cases}$$

(S.5.2)

For a state vector $x\in\{0,\dots,3k\}^{\mathsf{V}}$ with leaks $z_{v}=x_{v}-\sum_{c\in\operatorname{ch}(v)}x_{c}\text{,}$ the product $\prod_{v}\kappa_{v}(z_{v})$ equals $e^{-2\Gamma(x)}$ when the leaks vanish off $A_{k}$ and is zero otherwise. The denominator of (4.2) is therefore the total weight $\prod_{v}\kappa_{v}(z_{v})\,\varphi_{v}(x_{v})$ of all state vectors with $x_{o}=k\text{,}$ and it is positive: the vector with $x_{o}=k$ and all other states zero has its one leak at the root, where $\widetilde{\omega}(o)=0$ because $o\in R_{0}\text{,}$ so its weight is $e^{-2k}\varphi_{o}(k)\prod_{v\neq o}\varphi_{v}(0)>0\text{.}$ In the same terms, the coefficients of the message (4.5) are

$$P_{v}[x]\ =\ \sum_{\begin{subarray}{c}x^{\prime}\in\{0,\dots,3k\}^{T_{v}}\\ x^{\prime}_{v}=x\end{subarray}}\ \prod_{w\in T_{v}}\kappa_{w}(z^{\prime}_{w})\,\varphi_{w}(x^{\prime}_{w}),\qquad z^{\prime}_{w}:=x^{\prime}_{w}-\textstyle\sum_{c\in\operatorname{ch}(w)}x^{\prime}_{c},$$

(S.5.3)

the leaks computed within $T_{v}\text{.}$

Messages are polynomials of degree at most $3k\text{,}$ but the product of many children’s messages is not, and its degree must not be allowed to grow with their number. The device is a pair representation: a polynomial $F(\zeta)=\sum_{i\geq 0}F[i]\zeta^{i}$ with nonnegative coefficients is carried as its truncation $F_{0}(\zeta):=\sum_{i\leq 3k}F[i]\zeta^{i}$ together with its tail

$$\tau_{F}\ :=\ \sum_{i>3k}F[i]\,e^{-2(i-3k)},$$

the evaluation of the high part at $e^{-2}\text{,}$ shifted to the cap. Messages have zero tail; tails arise along partial products of messages and stay scalars.

Lemma S.5.3 (Product of pairs).

Let $F,G$ be polynomials with nonnegative coefficients, of arbitrary degrees, given by their pairs. Then $(FG)_{0}$ is the truncation of $F_{0}G_{0}\text{,}$ and

$$\tau_{FG}\ =\ \sum_{q=3k+1}^{6k}[\zeta^{q}](F_{0}G_{0})\,e^{-2(q-3k)}\ +\ F_{0}(e^{-2})\,\tau_{G}\ +\ \tau_{F}\,G_{0}(e^{-2})\ +\ e^{-6k}\,\tau_{F}\,\tau_{G}.$$

(S.5.4)

The pair of $FG$ is therefore computable from the pairs of $F$ and $G$ by one multiplication of two polynomials of degree at most $3k$ and $O(k)$ further operations.

Proof.

For $q\leq 3k\text{,}$ $[\zeta^{q}](FG)=\sum_{i+j=q}F[i]G[j]$ involves only indices $i,j\leq 3k\text{,}$ which gives the truncation claim. In $\tau_{FG}=\sum_{i+j>3k}F[i]G[j]e^{-2(i+j-3k)}\text{,}$ partition the index pairs according to whether $i\leq 3k$ and whether $j\leq 3k\text{.}$ The low–low pairs give the first term of Equation S.5.4. The low–high pairs factor as $\sum_{i\leq 3k}F[i]e^{-2i}\cdot\sum_{j>3k}G[j]e^{-2(j-3k)}=F_{0}(e^{-2})\tau_{G}\text{,}$ the high–low pairs symmetrically, and the high–high pairs as $e^{-6k}\tau_{F}\tau_{G}\text{,}$ since $e^{-2(i+j-3k)}=e^{-2(i-3k)}e^{-2(j-3k)}e^{-6k}\text{.}$ For the cost: the single multiplication $F_{0}G_{0}$ yields all coefficients through degree $6k\text{,}$ and every remaining ingredient of Equation S.5.4, including the evaluations $F_{0}(e^{-2})$ and $G_{0}(e^{-2})\text{,}$ is $O(k)$ arithmetic on them. ∎

The recursion of Section 4.2 now takes its precise form. At each vertex the children’s messages are merged pairwise in the pair representation, and one local pass applies the observation weight and the prior kernel.

Lemma S.5.4 (Message recursion).

For $v\in\mathsf{V}$ let $Q_{v}(\zeta):=\prod_{c\in\operatorname{ch}(v)}P_{c}(\zeta)\text{,}$ an empty product being $1\text{,}$ with coefficients $Q_{v}[s]$ and tail $\tau_{v}\text{,}$ and set

$$\psi^{-}_{x}\ :=\ \sum_{0\leq s<x}e^{-2(x-s)}\,Q_{v}[s],\qquad\psi^{+}_{x}\ :=\ \sum_{s>x}e^{-2(s-x)}\,Q_{v}[s]\qquad(0\leq x\leq 3k).$$

Then

$$P_{v}[x]\ =\ \begin{cases}\varphi_{v}(x)\,Q_{v}[x],&v\notin A_{k},\\[2.0pt] \varphi_{v}(x)\,\bigl(Q_{v}[x]+e^{-2\widetilde{\omega}(v)}(\psi^{-}_{x}+\psi^{+}_{x})\bigr),&v\in A_{k},\end{cases}$$

(S.5.5)

and the two sums obey the linear recurrences

$$\psi^{-}_{0}=0,\quad\psi^{-}_{x}=e^{-2}\bigl(Q_{v}[x-1]+\psi^{-}_{x-1}\bigr);\qquad\psi^{+}_{3k}=\tau_{v},\quad\psi^{+}_{x}=e^{-2}\bigl(Q_{v}[x+1]+\psi^{+}_{x+1}\bigr).$$

(S.5.6)

In particular the pair of $P_{v}$ is computed from the pairs of the children’s messages by $(|\operatorname{ch}(v)|-1)_{+}$ products of pairs and $O(k)$ further operations, and $P_{o}[k]$ is the denominator of (4.2).

Proof.

Assignments on the subtrees of distinct children are independent, and weights multiply, so $Q_{v}[s]$ is the total weight of the assignments on the child subtrees whose states at the children sum to $s\text{;}$ here $s$ ranges over all of $\{0,1,\dots\}\text{,}$ and the pair of $Q_{v}$ is assembled by Lemma S.5.3. A state $x$ at $v$ completes such a family to an assignment on $T_{v}$ with local weight $\varphi_{v}(x)\kappa_{v}(x-s)\text{,}$ so by Equation S.5.3

$$P_{v}[x]\ =\ \varphi_{v}(x)\sum_{s\geq 0}\kappa_{v}(x-s)\,Q_{v}[s].$$

If $v\notin A_{k}\text{,}$ the kernel forces $s=x\leq 3k\text{,}$ which is Equation S.5.5; child sums beyond $3k$ contribute nothing. If $v\in A_{k}\text{,}$ the kernel splits the sum at $s=x$ into $Q_{v}[x]+e^{-2\widetilde{\omega}(v)}(\psi^{-}_{x}+\psi^{+}_{x})$ with

$$\psi^{+}_{x}\ =\ \sum_{x<s\leq 3k}e^{-2(s-x)}Q_{v}[s]\ +\ e^{-2(3k-x)}\,\tau_{v},$$

the high child sums entering exactly through the tail. The recurrences Equation S.5.6 are immediate from the definitions; the seed $\psi^{+}_{3k}=\tau_{v}$ is the identity just displayed at $x=3k\text{.}$ Both passes cost $O(k)\text{,}$ as does applying Equation S.5.5, and the leaf base case ($Q_{v}\equiv 1$) is contained in the general one. At the root, Equation S.5.3 with $x=k$ sums the weights of all state vectors with $x_{o}=k\text{,}$ which is the denominator of (4.2). ∎

The pruning and the chain compression described in Section 4.2 modify this recursion; neither changes the posterior.

Lemma S.5.5 (Pruning and compression).

1.

If $T_{w}\cap A_{k}=\varnothing$, then every state vector of nonzero weight vanishes on $T_{w}$, and the subtree contributes the factor $\prod_{u\in T_{w}}\varphi_{u}(0)$ to every weight; dropping it, and every other factor common to all assignments, changes no posterior quantity. The estimator satisfies $\widehat{\mu}_{k}(u)=0$ for $u\in T_{w}$.
2.

Let $\mathcal{C}$ be a maximal chain of vertices not in $A_{k}$, each having exactly one child whose subtree meets $A_{k}$ and each, except the lowest, followed in $\mathcal{C}$ by that child; write $c_{\mathcal{C}}$ for that child of the lowest vertex of $\mathcal{C}$. Every state vector of nonzero weight is constant on $\mathcal{C}\cup\{c_{\mathcal{C}}\}$, and the chain contributes, after the state-independent factor $\exp(-\sum_{w\in\mathcal{C}}Y_{w}^{2}/(4\sigma^{2}))$ is dropped, the diagonal kernel

$$x\ \longmapsto\ \exp\Bigl(\frac{b\,x}{2\sigma^{2}}\sum_{w\in\mathcal{C}}Y_{w}\ -\ \frac{|\mathcal{C}|\,b^{2}x^{2}}{4\sigma^{2}}\Bigr),$$ (S.5.7)

generated in $O(k)$ operations from $|\mathcal{C}|$ and $\sum_{w\in\mathcal{C}}Y_{w}$. The estimator satisfies $\widehat{\mu}_{k}(w)=\widehat{\mu}_{k}(c_{\mathcal{C}})$ for $w\in\mathcal{C}$.

Proof.

For (1): on $T_{w}$ all kernels force $z\equiv 0\text{,}$ so leaf-up induction gives $x\equiv 0$ on $T_{w}$ for every assignment of nonzero weight; the surviving factor is $\prod_{u\in T_{w}}\kappa_{u}(0)\varphi_{u}(0)$ as claimed. A factor common to every assignment multiplies numerator and denominator of each posterior expectation alike. Since $x_{u}=0$ with posterior probability one, $\widehat{\mu}_{k}(u)=b\,\mathbb{E}_{Y}[x_{u}]=0\text{.}$

For (2): a vertex $w\in\mathcal{C}$ has $z_{w}=0$ and all of its child subtrees off the chain disjoint from $A_{k}\text{,}$ hence carrying state zero, so $x_{w}$ equals the state of its unique child toward $A_{k}\text{;}$ iterating down the chain, all these states equal $x_{c_{\mathcal{C}}}=:x\text{.}$ The chain’s likelihood factor is $\prod_{w\in\mathcal{C}}\varphi_{w}(x)\text{,}$ and expanding the exponent,

$$-\sum_{w\in\mathcal{C}}\frac{(Y_{w}-bx)^{2}}{4\sigma^{2}}\ =\ -\sum_{w\in\mathcal{C}}\frac{Y_{w}^{2}}{4\sigma^{2}}\ +\ \frac{bx}{2\sigma^{2}}\sum_{w\in\mathcal{C}}Y_{w}\ -\ \frac{|\mathcal{C}|\,b^{2}x^{2}}{4\sigma^{2}},$$

which is Equation S.5.7 after the constant is dropped. Pathwise equality of the states gives $\mathbb{E}_{Y}[x_{w}]=\mathbb{E}_{Y}[x_{c_{\mathcal{C}}}]\text{,}$ hence the coordinate claim. ∎

The forward computation of $P_{o}[k]$ is a straight-line program in the $n(3k+1)$ inputs $\varphi_{v}(x)$ whose gates are polynomial multiplications with their tail formulas Equation S.5.4, the linear recurrences Equation S.5.6, and the coordinatewise products Equation S.5.5. The reverse sweep differentiates it.

Lemma S.5.6 (Reverse sweep).

All $n(3k+1)$ partial derivatives $\partial P_{o}[k]/\partial\varphi_{v}(x)$ are computable in $O(1)$ times the cost of the forward computation, and

$$\widehat{\mu}_{k}(v)\ =\ b\sum_{x=0}^{3k}x\cdot\frac{\varphi_{v}(x)}{P_{o}[k]}\,\frac{\partial P_{o}[k]}{\partial\varphi_{v}(x)},$$

(S.5.8)

which is the display (4.6).

Proof.

By Equation S.5.3 at the root, $P_{o}[k]$ is a sum of monomials, one per state vector $x^{\prime}$ with $x^{\prime}_{o}=k\text{,}$ each containing the input $\varphi_{v}(x^{\prime}_{v})$ exactly once and otherwise only kernel constants and inputs at other vertices. Hence $\varphi_{v}(x)\,\partial P_{o}[k]/\partial\varphi_{v}(x)$ is the total weight of the state vectors with $x^{\prime}_{v}=x\text{,}$ so the ratio in Equation S.5.8 is the posterior probability of $\{x_{v}=x\}$ and the sum is $b\,\mathbb{E}_{Y}[x_{v}]=\widehat{\mu}_{k}(v)\text{.}$

For the cost, propagate the derivatives of $P_{o}[k]$ with respect to every intermediate quantity backward through the program, from the output to the inputs. By the chain rule, the derivative with respect to a quantity is assembled from the derivatives with respect to the quantities that consume it, so one backward pass over the acyclic program suffices; each gate is handled by the transpose of its linearization, at the cost of the gate itself. For a product of pairs, the derivative with respect to a coefficient of one factor is a correlation with the other,

$$\frac{\partial P_{o}[k]}{\partial F_{0}[i]}\ =\ \sum_{j=0}^{3k}\frac{\partial P_{o}[k]}{\partial\,[\zeta^{i+j}](F_{0}G_{0})}\ G_{0}[j]\qquad(0\leq i\leq 3k),$$

(S.5.9)

one multiplication of polynomials of degree at most $6k$ after reversing the coefficient order, plus the $O(k)$ coordinatewise contributions of Equation S.5.4 through its low coefficients, its evaluations, and its tails. The transposed recurrences Equation S.5.6 run in the opposite direction in $O(k)\text{,}$ and the coordinatewise products Equation S.5.5 are differentiated in $O(k)\text{.}$ Every gate’s transpose therefore costs the order of the gate, and the sweep visits each gate once. ∎

The operation count of Theorem 2. Computing $k_{\mathrm{alg}}$ costs $O(n\log(n+1))$ operations by Lemma 2.3; the nets $S_{j}$ at the $J+1=O(\log(k+1))$ dyadic radii cost $O(n)$ each by Lemma S.1.1, and the charges $\widetilde{\omega}$ one further pass. In the interior branch, the forward computation performs, over the whole tree, fewer than $n$ products of pairs, each $O(k\log(k+1))$ by Lemma S.5.3 and the multiplication model of Section 2.1, and $O(k)$ local work at each vertex (Lemma S.5.4); the reverse sweep matches this cost (Lemma S.5.6); and assembling Equation S.5.8 at every vertex costs $O(nk)\text{.}$ Pruning and compression (Lemma S.5.5) only remove work. The elementary branches cost $O(n)$ beyond the preprocessing. Every tie break (child orders, the greedy of Lemma S.1.1, the merge order) is fixed, so the whole map $(T,V,\sigma,Y)\mapsto\widehat{\mu}$ is deterministic, and the total is

$$O\bigl(n\log(n+1)+nk\log(k+1)\bigr)\ \leq\ O\bigl(n^{2}\log(n+1)\bigr)$$

operations, since $k\leq K+1\leq n\text{:}$ the count claimed in Theorem 2, whose proof is now complete.

The remaining two lemmas support the instance sentence of Section 4.2. Sibling messages that are dilations of a common polynomial multiply in batch: the mechanism is that logarithms turn the product into power sums of the dilation parameters. Two standard consequences of fast multiplication are used (von zur Gathen and Gerhard, 2013): Newton iteration computes truncated inverses and logarithms of power series with constant term $1\text{,}$ and exponentials of those with constant term $0\text{,}$ at the cost of $O(1)$ multiplications of the same degree; and a degree-$D$ polynomial is evaluated at $d$ points in $O((D+d)\log^{2}(D+d))$ operations by a product tree.

Lemma S.5.7 (Batched dilations).

Let $F$ be a polynomial of degree at most $3k$ with $F[0]=1\text{,}$ and let $\xi_{1},\dots,\xi_{d}>0\text{.}$ In $O((k+d)\log^{2}(k+d))$ operations one can compute

1.

the truncation of $\prod_{i=1}^{d}F(\xi_{i}\zeta)$ modulo $\zeta^{3k+1}$;
2.

the values $F(e^{-2}\xi_{i})$ and $F^{\prime}(e^{-2}\xi_{i})$ for all $i\leq d$;
3.

for any given $y_{0},\dots,y_{3k}$, the values $\ \xi_{i}\,\dfrac{\partial}{\partial\xi_{i}}\displaystyle\sum_{s=0}^{3k}y_{s}\,[\zeta^{s}]\prod_{j=1}^{d}F(\xi_{j}\zeta)\space$ for all $i\leq d$.

Proof.

Write $g(\zeta):=\log F(\zeta)=\sum_{m=1}^{3k}g_{m}\zeta^{m}$ modulo $\zeta^{3k+1}\text{,}$ computable by Newton iteration since $F[0]=1\text{,}$ and let $t_{m}:=\sum_{i\leq d}\xi_{i}^{m}$ be the power sums. Then, modulo $\zeta^{3k+1}\text{,}$

$$\prod_{i=1}^{d}F(\xi_{i}\zeta)\ =\ \exp\Bigl(\sum_{m=1}^{3k}g_{m}\,t_{m}\,\zeta^{m}\Bigr),$$

(S.5.10)

because $\log F(\xi_{i}\zeta)=\sum_{m}g_{m}\xi_{i}^{m}\zeta^{m}$ and the logarithms add. The power sums come from the auxiliary polynomial $N(\zeta):=\prod_{i\leq d}(1-\xi_{i}\zeta)\text{,}$ built by a product tree, through the logarithmic-derivative identity $-\zeta N^{\prime}(\zeta)/N(\zeta)=\sum_{m\geq 1}t_{m}\zeta^{m}\text{,}$ one derivative, one truncated inversion, and one multiplication; the exponential in Equation S.5.10 is one more Newton iteration. This proves (1), and (2) is multipoint evaluation of $F$ and $F^{\prime}$ at the points $e^{-2}\xi_{i}\text{.}$ For (3), write $P:=\exp(H)$ with $H:=\sum_{m}g_{m}t_{m}\zeta^{m}\text{;}$ then $dP=P\,dH\text{,}$ so the displayed quantity, as a function of the $t_{m}\text{,}$ has

$$\frac{\partial}{\partial t_{m}}\sum_{s\leq 3k}y_{s}\,[\zeta^{s}]P\ =\ g_{m}\sum_{s=m}^{3k}y_{s}\,[\zeta^{s-m}]P,$$

one correlation as in Equation S.5.9; and since $\xi_{i}\,\partial t_{m}/\partial\xi_{i}=m\,\xi_{i}^{m}\text{,}$ the value carried by the dilation parameter $\xi_{i}$ is the evaluation at $\xi_{i}$ of the polynomial $\sum_{m=1}^{3k}m\,g_{m}\bigl(\sum_{s\geq m}y_{s}[\zeta^{s-m}]P\bigr)\,\zeta^{m}\text{,}$ a second multipoint evaluation. Every step is a product tree, a truncated inversion, a logarithm or exponential, a correlation, or a multipoint evaluation at degree $O(k+d)\text{,}$ each within the stated bound. ∎

On the star, either the active support collapses to the root or every leaf message is a dilation of one template; in both cases the evaluation is softly linear.

Lemma S.5.8 (Stars).

On the star $S_{m}$ with $m\geq 2$ leaves, the estimator (4.3) is computed in $O(n\log^{2}n)$ operations, $n=m+1\text{.}$

Proof.

The preprocessing and the elementary branches cost $O(n\log(n+1))\text{,}$ so let the interior branch be selected, with budget $2\leq k\leq K$ and working scale $\bar{\alpha}=k\overline{\delta}_{k}\geq 2$ (Lemma S.2.1(1)). On the star, the minimum ancestor $r$-net with $r\geq 1$ is $\{o\}\text{,}$ a leaf being an ancestor only of itself, while at $r=0$ every vertex needs itself; so with $j_{0}:=\min\{j:\lfloor\bar{\alpha}/m_{j}\rfloor=0\}\text{,}$ understood as $j_{0}=J+1$ if no radius vanishes, the nets are $S_{j}=\{o\}$ for $j<j_{0}$ and $S_{j}=\mathsf{V}$ for $j\geq j_{0}\text{,}$ and $j_{0}\geq 1$ because $\lfloor\bar{\alpha}\rfloor\geq 2\text{.}$

If $j_{0}=J+1\text{,}$ then $A_{k}=\{o\}\text{:}$ the only state vector with nonzero weight has $x_{o}=k$ and all leaf states zero, the posterior is a point mass, and $\widehat{\mu}_{k}=\tfrac{V}{k}\,x=Vp_{o}$ is written in $O(n)\text{.}$

If $j_{0}\leq J\text{,}$ then $A_{k}=\mathsf{V}$ and every leaf carries the same charge $\widetilde{\omega}=m_{j_{0}}\text{.}$ Factor the local weight of leaf $v$ at state $x$ as $\varphi_{v}(x)=e^{-Y_{v}^{2}/(4\sigma^{2})}\,e^{-b^{2}x^{2}/(4\sigma^{2})}\,\xi_{v}^{x}$ with $\xi_{v}:=e^{bY_{v}/(2\sigma^{2})}\text{;}$ the first factor is common to every assignment and is dropped. By Lemma S.5.4, the message of leaf $v$ is then $F(\xi_{v}\zeta)$ for the common template

$$F(\zeta)\ :=\ 1\ +\ e^{-2m_{j_{0}}}\sum_{x=1}^{3k}e^{-2x}\,e^{-b^{2}x^{2}/(4\sigma^{2})}\,\zeta^{x},$$

and the child product of the root is $Q_{o}(\zeta)=\prod_{v}F(\xi_{v}\zeta)\text{.}$ Lemma S.5.7(1) gives its truncation, Lemma S.5.7(2) gives $Q_{o}(e^{-2})=\prod_{v}F(e^{-2}\xi_{v})\text{,}$ and the tail follows as

$$\tau_{o}\ =\ e^{6k}\Bigl(\,Q_{o}(e^{-2})-\sum_{s=0}^{3k}Q_{o}[s]\,e^{-2s}\Bigr),$$

exactly, from the definition of the tail. The root pass of Lemma S.5.4 then yields $P_{o}[k]$ in $O(k)$ operations.

For the marginals, $P_{o}[k]$ is a linear function of $(Q_{o}[0],\dots,Q_{o}[3k],\tau_{o})$ by Equation S.5.5 and Equation S.5.6, with coefficients computable in $O(k)\text{;}$ and every assignment’s weight carries the factor $\xi_{v}^{x_{v}}\text{,}$ so Euler differentiation gives

$$\mathbb{E}_{Y}[x_{v}]\ =\ \frac{\xi_{v}}{P_{o}[k]}\,\frac{\partial P_{o}[k]}{\partial\xi_{v}}.$$

(S.5.11)

The derivative splits along the linear form: the low coefficients contribute one instance of Lemma S.5.7(3), with weights that combine the direct coefficients of the linear form and the term $-e^{6k-2s}$ that $Q_{o}[s]$ inherits from $\tau_{o}\text{,}$ while $Q_{o}(e^{-2})$ contributes

$$\xi_{v}\,\frac{\partial Q_{o}(e^{-2})}{\partial\xi_{v}}\ =\ Q_{o}(e^{-2})\cdot\frac{e^{-2}\xi_{v}\,F^{\prime}(e^{-2}\xi_{v})}{F(e^{-2}\xi_{v})}\ ,$$

an $O(1)$ combination of the values from Lemma S.5.7(2), whose denominators are positive because $F$ has positive constant term and nonnegative coefficients. Then $\widehat{\mu}_{k}(v)=b\,\mathbb{E}_{Y}[x_{v}]$ at every leaf and $\widehat{\mu}_{k}(o)=V\text{.}$ All steps beyond the preprocessing cost $O((k+m)\log^{2}(k+m))\text{,}$ and $k\leq K<n$ makes this $O(n\log^{2}n)\text{.}$ ∎

Together, Lemmas S.5.7 and S.5.8 verify the instance sentence of Section 4.2: a group of sibling messages agreeing up to a dilation of the variable multiplies in batch, and on stars the entire evaluation is softly linear in $n\text{.}$

S.6 Adaptation

This section proves the statements deferred from Section 4.3: the selection oracle (4.8), in the weighted and random-penalty form stated there; the scale calculus for the minimax risk; the subspace approximation (4.7); the enumeration of admissible supports; the assembly of the sufficiency half of Theorem 3, including its running time; the difference statistics of Theorem 4 with their contamination bound, the concentration of the noise estimate, and the moment bookkeeping; and the noise-estimation ceiling stated at the end of Section 4.3. Throughout, $K:=\lfloor n/C_{\star}\rfloor$ and dyadic integers are powers of two; the selection constant of (4.8) is instantiated as $C_{2}:=128\text{.}$ The standard Gaussian tail bound $\mathbb{P}\bigl(N(0,\tau^{2})\geq t\bigr)\leq e^{-t^{2}/(2\tau^{2})}\text{,}$ valid for $t\geq 0\text{,}$ is used without further comment.

The selection engine comes first. The lemma below is the oracle promised with (4.8): the noise may have unequal, even zero, coordinate variances, which is how conditioning on the root observation will enter, and the penalties may be random as long as they are bracketed on an event, which is how the estimated noise scale will enter.

Lemma S.6.1 (Weighted affine selection).

Let $Y=\theta+Z\in\mathbb{R}^{N}\text{,}$ where $\theta\in\mathbb{R}^{N}$ and $Z$ is a centered Gaussian vector with independent coordinates whose variances are at most $\sigma^{2}\text{,}$ some possibly zero. Let $\mathcal{M}$ be a countable collection of affine models $m=c_{m}+W_{m}\subseteq\mathbb{R}^{N}\text{,}$ each $W_{m}$ a linear subspace of finite dimension $D_{m}\text{,}$ with weights $\Delta_{m}\geq 0$ such that $\sum_{m\in\mathcal{M}}e^{-\Delta_{m}}\leq 1\text{.}$ Let $\widehat{\theta}$ minimize $\|Y-\nu\|_{2}^{2}+\operatorname{pen}(m)$ over $m\in\mathcal{M}$ and $\nu\in m\text{,}$ and over a full model whose fit is $\nu=Y$ itself. The penalties are nonnegative, possibly random, and such that the minimum defining $\widehat{\theta}$ is attained, as it is in particular for finite $\mathcal{M}\text{.}$ Suppose that on an event $E\text{,}$ for a constant $\bar{\kappa}\geq 32\text{,}$

$$\begin{gathered} \displaystyle 32\,\sigma^{2}(D_{m}+\Delta_{m})\ \leq\ \operatorname{pen}(m)\ \leq\ \bar{\kappa}\,\sigma^{2}(D_{m}+\Delta_{m})\qquad\text{for every }m\in\mathcal{M}, \\ \displaystyle 32\,\sigma^{2}N\ \leq\ \operatorname{pen}(\mathrm{full})\ \leq\ \bar{\kappa}\,\sigma^{2}N. \end{gathered}$$

Then

$$\mathbb{E}\bigl[\|\widehat{\theta}-\theta\|_{2}^{2}\,\mathbf{1}_{E}\bigr]\ \leq\ C_{\bar{\kappa}}\Bigl(\min\Bigl\{\ \inf_{m\in\mathcal{M}}\bigl(\operatorname{dist}^{2}(\theta,m)+\sigma^{2}(D_{m}+\Delta_{m}+1)\bigr),\ \ \sigma^{2}N\ \Bigr\}\ +\ \sigma^{2}\Bigr),$$

where $C_{\bar{\kappa}}$ depends only on $\bar{\kappa}\text{.}$

Proof.

The selected model is compared with one fixed model through the basic inequality of penalized least squares; a single Kraft-weighted deviation variable controls the Gaussian fluctuations over the whole collection, and the full model is handled by its own comparison.

A projection bound. Let $U\subseteq\mathbb{R}^{N}$ be a fixed subspace of dimension $d$ and $P_{U}$ the orthogonal projection onto it. For an orthonormal basis $u_{1},\dots,u_{d}$ of $U\text{,}$ the vector $(\langle Z,u_{i}\rangle)_{i\leq d}$ is centered Gaussian with covariance $\Sigma\preceq\sigma^{2}I_{d}\text{:}$ its quadratic form at $a\in\mathbb{R}^{d}$ is $\operatorname{Var}\langle Z,\sum_{i}a_{i}u_{i}\rangle\leq\sigma^{2}\|\sum_{i}a_{i}u_{i}\|_{2}^{2}=\sigma^{2}\|a\|_{2}^{2}\text{.}$ For a centered Gaussian $g$ of variance $\tau^{2}\leq\sigma^{2}\text{,}$ direct integration gives $\mathbb{E}e^{g^{2}/(4\sigma^{2})}=(1-\tau^{2}/(2\sigma^{2}))^{-1/2}\leq\sqrt{2}\text{,}$ the case $\tau^{2}=0$ included; diagonalizing $\Sigma$ and multiplying over the independent coordinates,

$$\mathbb{E}\,e^{\|P_{U}Z\|_{2}^{2}/(4\sigma^{2})}\ \leq\ 2^{d/2};$$

hence, by Markov’s inequality and $2^{1/2}\leq e\text{,}$ for every $t\geq 0\text{,}$

$$\mathbb{P}\bigl(\|P_{U}Z\|_{2}^{2}\geq 4\sigma^{2}(d+t)\bigr)\ \leq\ 2^{d/2}\,e^{-d-t}\ \leq\ e^{-t}.$$

The deviation variable. Fix a comparison model $m_{0}\in\mathcal{M}$ and let $f_{0}\in m_{0}$ attain $a:=\operatorname{dist}(\theta,m_{0})\text{,}$ the metric projection of $\theta$ onto the closed affine set $m_{0}\text{.}$ For $m\in\mathcal{M}$ put

$$U_{m}\ :=\ W_{m}+W_{m_{0}}+\operatorname{span}(c_{m}-f_{0}),\qquad d_{m}\ :=\ \dim U_{m}\ \leq\ D_{m}+D_{m_{0}}+1,$$

so that every $\nu=c_{m}+w\in m$ has $\nu-f_{0}=(c_{m}-f_{0})+w\in U_{m}\text{,}$ and define

$$Z^{*}\ :=\ \sup_{m\in\mathcal{M}}\ \Bigl[\frac{\|P_{U_{m}}Z\|_{2}^{2}}{4\sigma^{2}}-(d_{m}+\Delta_{m})\Bigr]_{+}.$$

By the projection bound and a union bound, $\mathbb{P}(Z^{*}>t)\leq\sum_{m}e^{-\Delta_{m}-t}\leq e^{-t}\text{,}$ so $\mathbb{E}Z^{*}=\int_{0}^{\infty}\mathbb{P}(Z^{*}>t)\,dt\leq 1\text{;}$ and by its definition, $\|P_{U_{m}}Z\|_{2}^{2}\leq 4\sigma^{2}(d_{m}+\Delta_{m}+Z^{*})$ for every $m$ simultaneously.

A model is selected. Suppose the minimum is attained at some $\widehat{m}\in\mathcal{M}$ with fit $\widehat{\theta}\in\widehat{m}\text{,}$ and write $d:=\widehat{\theta}-\theta\text{.}$ Expanding the basic inequality against $(m_{0},f_{0})\text{,}$ $\|Y-\widehat{\theta}\|_{2}^{2}+\operatorname{pen}(\widehat{m})\leq\|Y-f_{0}\|_{2}^{2}+\operatorname{pen}(m_{0})\text{,}$ at $Y=\theta+Z$ and combining the two cross terms into $2\langle Z,\widehat{\theta}-f_{0}\rangle$ gives

$$\|d\|_{2}^{2}\ \leq\ a^{2}+2\langle Z,\widehat{\theta}-f_{0}\rangle-\operatorname{pen}(\widehat{m})+\operatorname{pen}(m_{0}).$$

Since $\widehat{\theta}-f_{0}\in U_{\widehat{m}}\text{,}$ the inequality $2xy\leq\tfrac{1}{4}x^{2}+4y^{2}$ and the deviation variable give

$$\begin{aligned} \displaystyle 2\langle Z,\widehat{\theta}-f_{0}\rangle\ =\ 2\bigl\langle P_{U_{\widehat{m}}}Z,\ \widehat{\theta}-f_{0}\bigr\rangle\space & \displaystyle\leq\ \tfrac{1}{4}\|\widehat{\theta}-f_{0}\|_{2}^{2}+4\,\|P_{U_{\widehat{m}}}Z\|_{2}^{2} \\ & \displaystyle\leq\ \tfrac{1}{4}\bigl(\|d\|_{2}+a\bigr)^{2}+16\sigma^{2}\bigl(d_{\widehat{m}}+\Delta_{\widehat{m}}+Z^{*}\bigr). \end{aligned}$$

By $d_{\widehat{m}}\leq D_{\widehat{m}}+D_{m_{0}}+1\text{,}$ the last term splits into $16\sigma^{2}(D_{\widehat{m}}+\Delta_{\widehat{m}})+16\sigma^{2}(D_{m_{0}}+1)+16\sigma^{2}Z^{*}\text{,}$ and on $E$ the penalty dominates its first piece: $\operatorname{pen}(\widehat{m})\geq 32\sigma^{2}(D_{\widehat{m}}+\Delta_{\widehat{m}})\text{.}$ Together with $\tfrac{1}{4}(\|d\|_{2}+a)^{2}\leq\tfrac{1}{2}\|d\|_{2}^{2}+\tfrac{1}{2}a^{2}\text{,}$ the expansion rearranges to

$$\|d\|_{2}^{2}\ \leq\ 3a^{2}+2\operatorname{pen}(m_{0})+32\sigma^{2}(D_{m_{0}}+1)+32\sigma^{2}Z^{*}\qquad\text{on }E\cap\{\text{a model is selected}\}.$$

(S.6.1)

The full model is selected. Then $\widehat{\theta}=Y$ and $\|d\|_{2}^{2}=\|Z\|_{2}^{2}\text{,}$ while the full model’s own residual vanishes, so the basic inequality against $(m_{0},f_{0})$ reads $\operatorname{pen}(\mathrm{full})\leq\|Y-f_{0}\|_{2}^{2}+\operatorname{pen}(m_{0})\leq 2a^{2}+2\|Z\|_{2}^{2}+\operatorname{pen}(m_{0})\text{;}$ on $E$ this gives $32\sigma^{2}N\leq 2a^{2}+2\|Z\|_{2}^{2}+\operatorname{pen}(m_{0})\text{.}$ On the event $\{\|Z\|_{2}^{2}\leq 2\sigma^{2}N\}\text{,}$ where $2\|Z\|_{2}^{2}\leq 4\sigma^{2}N\text{,}$ the preceding two inequalities force $28\sigma^{2}N\leq 2a^{2}+\operatorname{pen}(m_{0})\text{,}$ hence $\|Z\|_{2}^{2}\leq 2\sigma^{2}N\leq\tfrac{1}{14}\bigl(2a^{2}+\operatorname{pen}(m_{0})\bigr)\text{.}$ On the complement, the projection bound at $U=\mathbb{R}^{N}$ gives $\mathbb{P}(\|Z\|_{2}^{2}>2\sigma^{2}N)\leq 2^{N/2}e^{-N/2}\leq e^{-N/8}\text{;}$ moreover $\mathbb{E}\|Z\|_{2}^{4}=\sum_{i}\mathbb{E}Z_{i}^{4}+\sum_{i\neq j}\mathbb{E}Z_{i}^{2}\,\mathbb{E}Z_{j}^{2}\leq 3\sigma^{4}N+\sigma^{4}N(N-1)\leq 3\sigma^{4}N^{2}\text{,}$ the coordinates being independent with fourth moments at most $3\sigma^{4}\text{,}$ so the Cauchy–Schwarz inequality gives

$$\mathbb{E}\bigl[\|Z\|_{2}^{2}\,\mathbf{1}\{\|Z\|_{2}^{2}>2\sigma^{2}N\}\bigr]\ \leq\ \sqrt{3}\,\sigma^{2}N\,e^{-N/16}\ \leq\ C\sigma^{2},$$

the map $N\mapsto Ne^{-N/16}$ being bounded.

Assembly. Combining Equation S.6.1 with the two estimates of the full-model case and using $\mathbb{E}Z^{*}\leq 1$ and the bracketing of $\operatorname{pen}(m_{0})$ on $E\text{,}$

$$\mathbb{E}\bigl[\|\widehat{\theta}-\theta\|_{2}^{2}\,\mathbf{1}_{E}\bigr]\ \leq\ C\bigl(a^{2}+\bar{\kappa}\,\sigma^{2}(D_{m_{0}}+\Delta_{m_{0}}+1)\bigr)+C\sigma^{2},$$

and taking the infimum over $m_{0}$ gives the first branch of the minimum. For the second branch, compare with the full model instead: if the full model is selected, then $\mathbb{E}[\|d\|_{2}^{2}\,\mathbf{1}_{E}]\leq\mathbb{E}\|Z\|_{2}^{2}\leq\sigma^{2}N\text{;}$ if some $\widehat{m}$ is selected, the basic inequality against the full model gives $\|Y-\widehat{\theta}\|_{2}^{2}\leq\operatorname{pen}(\mathrm{full})\leq\bar{\kappa}\sigma^{2}N$ on $E\text{,}$ so $\|d\|_{2}^{2}\leq 2\|Y-\widehat{\theta}\|_{2}^{2}+2\|Z\|_{2}^{2}$ has expectation at most $(2\bar{\kappa}+2)\sigma^{2}N$ there. Taking the minimum of the two comparisons completes the proof. ∎

Three features are used below. The penalties may be random, only their bracketing on $E$ entering the argument. The coordinate variances may differ and may vanish. And a point $\nu$ is the affine model $\{\nu\}$ with $D_{m}=0\text{,}$ whose oracle value reads $\|\theta-\nu\|_{2}^{2}+\sigma^{2}(\Delta_{\{\nu\}}+1)\text{;}$ we write $w(\nu)$ for the weight of a point model. The display (4.8) is the deterministic special case: penalties $C_{2}\sigma^{2}(\dim m+\Delta_{m})$ and $C_{2}\sigma^{2}N$ satisfy the bracketing with $E$ the whole probability space and $\bar{\kappa}=C_{2}\text{.}$

The coverage argument will replace the unknown amplitude by a grid value and the unknown noise level by a rounded estimate. The next lemma prices both substitutions.

Lemma S.6.2 (Scale calculus).

For every finite rooted tree, $R^{*}_{T}(V,\sigma)$ is nondecreasing in $V$ and in $\sigma\text{,}$ and for every $c\geq 1\text{,}$

$$R^{*}_{T}(cV,\sigma)\ \leq\ c^{2}\,R^{*}_{T}(V,\sigma),\qquad R^{*}_{T}(V,c\sigma)\ \leq\ c^{2}\,R^{*}_{T}(V,\sigma).$$

Proof.

Monotonicity in $V\text{.}$ Let $V\leq V^{\prime}\text{.}$ The translation $\iota(\mu):=\mu+(V^{\prime}-V)p_{o}$ adds $V^{\prime}-V$ to the leak at the root and changes no other leak, so it maps $\mathcal{F}_{V}(T)$ into $\mathcal{F}_{V^{\prime}}(T)$ by Lemma 2.1. Given any estimator $\widehat{\mu}^{\prime}\text{,}$ define $\widehat{\mu}(Y):=\widehat{\mu}^{\prime}\bigl(Y+(V^{\prime}-V)p_{o}\bigr)-(V^{\prime}-V)p_{o}\text{.}$ Under $\mu\in\mathcal{F}_{V}(T)\text{,}$ the shifted data $Y+(V^{\prime}-V)p_{o}$ are distributed as an observation of $\iota(\mu)\text{,}$ and $\|\widehat{\mu}-\mu\|_{2}=\|\widehat{\mu}^{\prime}(Y+(V^{\prime}-V)p_{o})-\iota(\mu)\|_{2}\text{,}$ so the risk of $\widehat{\mu}$ at $\mu$ equals the risk of $\widehat{\mu}^{\prime}$ at $\iota(\mu)\in\mathcal{F}_{V^{\prime}}(T)\text{.}$ Taking the supremum over $\mu$ and then the infimum over $\widehat{\mu}^{\prime}$ gives $R^{*}_{T}(V,\sigma)\leq R^{*}_{T}(V^{\prime},\sigma)\text{.}$

Monotonicity in $\sigma\text{.}$ Let $\sigma\leq\sigma^{\prime}$ and let $W\sim N(0,(\sigma^{\prime 2}-\sigma^{2})I_{n})$ be independent of the data. For any estimator $\widehat{\mu}^{\prime}\text{,}$ the randomized procedure $\widehat{\mu}^{\prime}(Y+W)$ has, at every $\mu\text{,}$ the risk of $\widehat{\mu}^{\prime}$ under noise level $\sigma^{\prime}\text{;}$ replacing it by its average $\widehat{\mu}(Y):=\mathbb{E}_{W}\widehat{\mu}^{\prime}(Y+W)$ only lowers the squared-error risk, by Jensen’s inequality conditionally on $Y\text{,}$ and produces a deterministic estimator. Hence $R^{*}_{T}(V,\sigma)\leq R^{*}_{T}(V,\sigma^{\prime})\text{.}$

The quadratic bounds. Scaling the observation, the parameter, and the estimator by $a>0$ turns the experiment at $(V,\sigma)$ into the experiment at $(aV,a\sigma)\text{,}$ since $\mathcal{F}_{aV}(T)=a\,\mathcal{F}_{V}(T)\text{;}$ the squared error scales by $a^{2}\text{,}$ so

$$R^{*}_{T}(aV,a\sigma)\ =\ a^{2}\,R^{*}_{T}(V,\sigma)\qquad(a>0).$$

For $c\geq 1\text{,}$ combining this exact identity with the monotonicity just proved gives

$$\begin{aligned} \displaystyle R^{*}_{T}(cV,\sigma) & \displaystyle=\ c^{2}R^{*}_{T}(V,\sigma/c)\ \leq\ c^{2}R^{*}_{T}(V,\sigma), \\ \displaystyle R^{*}_{T}(V,c\sigma) & \displaystyle=\ c^{2}R^{*}_{T}(V/c,\sigma)\ \leq\ c^{2}R^{*}_{T}(V,\sigma), \end{aligned}$$

using $\sigma/c\leq\sigma$ and $V/c\leq V\text{.}$ ∎

Two consequences are used repeatedly. First, for every amplitude $v>0$ and every working scale $s\in[\sigma/2,4\sigma]\text{,}$ monotonicity and the quadratic bound at $c=4$ give

$$R^{*}_{T}(v,s)\ \leq\ R^{*}_{T}(v,4\sigma)\ \leq\ 16\,R^{*}_{T}(v,\sigma).$$

(S.6.2)

Second, $k_{\mathrm{alg}}(T,v,s)\leq k_{0}(T,v,s)$ by Lemma 2.3(3), and the middle term below is at most $CR^{*}_{T}(v,s)$ by Theorem 1 at $(v,s)\text{,}$ so Equation S.6.2 gives

$$\min\bigl\{v^{2}H_{T},\ s^{2}k_{\mathrm{alg}}(T,v,s)\bigr\}\ \leq\ \min\bigl\{v^{2}H_{T},\ s^{2}k_{0}(T,v,s)\bigr\}\ \leq\ C\,R^{*}_{T}(v,\sigma),$$

(S.6.3)

valid for all $v>0$ and $s\in[\sigma/2,4\sigma]\text{.}$

The amplitude-free half of the candidate system rests on the subspace approximation (4.7), verified next with an explicit constant.

Lemma S.6.3 (Subspace approximation).

For every $\mu\in\mathcal{F}_{V}(T)$ and every integer $2\leq k\leq n/C_{\star}$ there is an admissible support $S$ with

$$\operatorname{dist}^{2}\bigl(\mu,\ \mathcal{V}_{k,S}\bigr)\ \leq\ 22\,V^{2}\overline{\delta}_{k}(T).$$

Proof.

Run the proof of Proposition 3.1 on the normalized signal $f:=\mu/V\text{,}$ retaining its objects: the terminal cells $L$ with masses $\lambda_{L}$ and exits $q_{L}\text{;}$ the collapse $g$ with $\|f-g\|_{2}^{2}\leq 3\bar{\alpha}/k$ by (3.4); the skeleton $g_{0}=\sum_{h\in U}b_{h}\,p_{h}\text{,}$ supported on the set $U$ consisting of $R_{0}$ and the heavy roots, with $g=g_{0}+\sum_{L}\lambda_{L}q_{L}\text{;}$ and the exit sampling, fixed at a realization with $W\leq 2k$ and $\|\sum_{L}(Z_{L}-\lambda_{L})q_{L}\|_{2}^{2}\leq 8\bar{\alpha}/k\text{.}$ Keep the skeleton unrounded and set

$$\nu\ :=\ g_{0}+\sum_{L}Z_{L}q_{L},$$

a real linear combination of $\{p_{v}:\ v\in U\cup\{a_{L}:Z_{L}\neq 0\}\}\text{.}$ The set $S:=\bigl(U\cup\{a_{L}:Z_{L}\neq 0\}\bigr)\setminus R_{0}$ is admissible: the heavy roots carry total activation charge below $2k$ by Lemma S.2.3(3); the selected exit roots are pairwise distinct with $\widetilde{\omega}(a_{L})\leq m(L)\text{,}$ so their charges total at most $W\leq 2k\text{;}$ and a union is charged at most the sum of its parts. Hence $\sum_{v\in S}\widetilde{\omega}(v)\leq 4k\text{.}$ Since $\nu\in\mathcal{V}_{k,S}$ and $f-\nu=(f-g)+\sum_{L}(\lambda_{L}-Z_{L})q_{L}\text{,}$

$$\operatorname{dist}^{2}\bigl(f,\ \mathcal{V}_{k,S}\bigr)\ \leq\ \|f-\nu\|_{2}^{2}\ \leq\ 2\,\frac{3\bar{\alpha}}{k}+2\,\frac{8\bar{\alpha}}{k}\ =\ 22\,\frac{\bar{\alpha}}{k}\ =\ 22\,\overline{\delta}_{k}(T).$$

Restoring the amplitude multiplies the squared distance by $V^{2}\text{.}$ ∎

Selecting over the capped subspaces requires listing them; this is the enumeration claim of Section 4.3.

Lemma S.6.4 (Support enumeration).

For every integer $2\leq k\leq n/C_{\star}\text{,}$ the admissible supports at budget $k$ number at most $e^{C_{1}k}\text{,}$ and they can be listed in $O(|A_{k}|\,e^{C_{1}k})$ operations.

Proof.

An admissible support is a subset of $A_{k}\setminus R_{0}$ of total activation charge at most $4k\leq 9k\text{,}$ hence one of the charged parts counted in the proof of Lemma S.2.5, where the generating-function estimate bounds their number by $e^{19k}\leq e^{C_{1}k}\text{.}$ To enumerate, fix a linear order on $A_{k}\setminus R_{0}$ and run a depth-first search over subsets that appends only higher-indexed vertices and recurses only while the accumulated charge remains at most $4k\text{.}$ Every node visited by the search is an admissible support, every admissible support is visited exactly once, along the insertion order of its elements, and testing all extensions of a visited support costs $O(|A_{k}|)$ operations. ∎

The candidate system. The estimator of Theorem 3 is the penalized selection of Lemma S.6.1 over the candidates described in Section 4.3, which we now fix precisely. The system is parameterized by a working scale $s\text{,}$ equal to $\sigma$ in Theorem 3 and to the rounded noise estimate in Theorem 4. Fix a universal threshold $n_{\star}$ such that

$$\Bigl\lfloor\frac{\log n}{2C_{1}}\Bigr\rfloor\ \geq\ 2\qquad\text{and}\qquad\Bigl\lfloor\frac{\log n}{2C_{1}}\Bigr\rfloor\ \leq\ \Bigl\lfloor\frac{n}{C_{\star}}\Bigr\rfloor\qquad\text{for every }n\geq n_{\star}.$$

For $n<n_{\star}$ the estimator returns $Y\text{,}$ whose risk $n\sigma^{2}\leq n_{\star}\sigma^{2}$ is absorbed by the additive $\sigma^{2}$ after enlarging the universal constant; assume $n\geq n_{\star}$ from here on, and set $k^{\star}:=\lfloor\log n/(2C_{1})\rfloor\text{,}$ so that $2\leq k^{\star}\leq K\text{.}$ Write

$$v_{i}:=(Y_{o})_{+}+2^{i}s\ \ (0\leq i\leq J^{+}),\qquad v^{-}_{j}:=2^{-j}s\ \ (1\leq j\leq J^{-}),$$

with $J^{+}=J^{-}:=\lceil\log_{2}(n+1)\rceil+2\text{,}$ for the root-anchored and the low-amplitude grids. Throughout, $\mathcal{X}_{k}$ is the class (4.1) of integer states of Section 4.1. The candidates are:

•

the subspaces $\mathcal{V}_{k,S}$ for dyadic $2\leq k\leq k^{\star}$ and admissible $S\text{,}$ with weights $\Delta_{k,S}:=(C_{1}+1)k\text{,}$ together with $\operatorname{span}(p_{o})$ at weight $1\text{;}$
•

the point models $\nu_{i,k,x}:=(v_{i}/k)\,x$ for $x\in\mathcal{X}_{k}$ and dyadic $2\leq k\leq K\text{,}$ at weights $w(\nu_{i,k,x}):=2\Gamma(x)+k+i+\log_{2}k+4\text{,}$ and the root points $\nu^{0}_{i}:=v_{i}\,p_{o}$ at weights $i+4\text{;}$ their low-amplitude counterparts $\nu^{-}_{j,k,x}:=(v^{-}_{j}/k)\,x$ at weights $2\Gamma(x)+k+2\log(1+j)+\log_{2}k+4\text{,}$ and $\nu^{0-}_{j}:=v^{-}_{j}\,p_{o}$ at weights $2\log(1+j)+4\text{;}$
•

the full model.

The penalties are $\operatorname{pen}(m):=C_{2}s^{2}(D_{m}+\Delta_{m})$ for the affine and point models, with $\Delta_{m}$ the weight just listed, and $\operatorname{pen}(\mathrm{full}):=C_{2}s^{2}n\text{.}$ The supports $A_{k}\text{,}$ the charges $\widetilde{\omega}\text{,}$ and hence $\Gamma$ and $\mathcal{X}_{k}$ depend only on $(T,k)$ (Proposition 3.1); a grid index enters only through its amplitude and its weight, and all tie breaks are fixed, so the system and its penalties are deterministic given $(Y_{o},s)\text{.}$

The weights are Kraft-summable with room to spare. For the subspaces, Lemma S.6.4 gives

$$\sum_{\begin{subarray}{c}k\ \mathrm{dyadic}\\ 2\leq k\leq k^{\star}\end{subarray}}\ \sum_{S\ \mathrm{admissible}}e^{-(C_{1}+1)k}\ +\ e^{-1}\ \leq\ \sum_{\begin{subarray}{c}k\ \mathrm{dyadic}\\ k\geq 2\end{subarray}}e^{-k}+e^{-1}\ <\ 0.16+0.37\ =\ 0.53.$$

For the grid family, Lemma S.5.1 gives $\sum_{x\in\mathcal{X}_{k}}e^{-2\Gamma(x)-k}\leq 1$ for every dyadic $k=2^{\ell}\text{,}$ so, once the price of the grid index is factored out, each amplitude contributes at most $1$ from its root candidate and $\sum_{\ell\geq 1}e^{-\ell}$ from its flow candidates, and

$$\sum_{\text{grid models}}e^{-w}\ \leq\ \Bigl(\sum_{i\geq 0}e^{-i-4}+\sum_{j\geq 1}\frac{e^{-4}}{(1+j)^{2}}\Bigr)\Bigl(1+\sum_{\ell\geq 1}e^{-\ell}\Bigr)\\ =\ e^{-4}\Bigl(\frac{e}{e-1}+\frac{\pi^{2}}{6}-1\Bigr)\frac{e}{e-1}\ <\ \frac{1}{8}.$$

(S.6.4)

The total is below $1\text{,}$ as Lemma S.6.1 requires.

The root-anchored grid is finite, and the truth can exceed its top value only after a downward root-noise fluctuation of order $n\text{.}$ The next lemma quantifies both halves of this statement; its event $\mathcal{E}_{+}$ is the one announced in Section 4.3.

Lemma S.6.5 (Anchor event).

Let $s\in[\sigma/2,4\sigma]\text{,}$ extend the anchored grid to $v_{i}=(Y_{o})_{+}+2^{i}s$ for all integers $i\geq 0\text{,}$ and set

$$i^{*}\ :=\ \min\{i\geq 0:\ v_{i}\geq V\},\qquad\mathcal{E}_{+}\ :=\ \{i^{*}\leq J^{+}\},\qquad Z_{o}\ :=\ Z(o).$$

Then:

1.

pointwise, $2^{i^{*}}s\leq 2s+2\sigma|Z_{o}|$, and consequently

$$i^{*}\leq 1+\log_{2}\bigl(1+2|Z_{o}|\bigr),\qquad v_{i^{*}}\leq V+2s+3\sigma|Z_{o}|,\qquad 0\ \leq\ v_{i^{*}}-Y_{o}\ \leq\ 2s+3\sigma|Z_{o}|,$$

and in expectation $\mathbb{E}\bigl[\sigma^{2}(1+i^{*})+(Y_{o}-v_{i^{*}})^{2}\bigr]\leq C\sigma^{2}$;
2.

$\mathcal{E}_{+}^{\,c}\subseteq\{Z_{o}<-2(n+1)\}$;
3.

every penalized selection of this section, the full model carrying penalty $C_{2}s^{2}n$ and all penalties being nonnegative, satisfies pathwise $\|\widehat{\mu}-\mu\|_{2}^{2}\leq 2C_{2}s^{2}n+2\sigma^{2}\|Z\|_{2}^{2}$, and therefore $\mathbb{E}\bigl[\|\widehat{\mu}-\mu\|_{2}^{2}\,\mathbf{1}_{\mathcal{E}_{+}^{c}}\bigr]\leq C\sigma^{2}$.

Proof.

Part (1). Since $Y_{o}=V+\sigma Z_{o}$ and $(Y_{o})_{+}\geq Y_{o}\text{,}$

$$V-(Y_{o})_{+}\ \leq\ (V-Y_{o})_{+}\ =\ (-\sigma Z_{o})_{+}\ \leq\ \sigma|Z_{o}|.$$

If $\sigma|Z_{o}|\leq s\text{,}$ then $v_{0}=(Y_{o})_{+}+s\geq V-\sigma|Z_{o}|+s\geq V\text{,}$ so $i^{*}=0$ and $2^{i^{*}}s=s\text{.}$ If $\sigma|Z_{o}|>s$ and $i^{*}\geq 1\text{,}$ minimality gives $v_{i^{*}-1}<V\text{,}$ that is, $2^{i^{*}-1}s<V-(Y_{o})_{+}\leq\sigma|Z_{o}|\text{,}$ so $2^{i^{*}}s<2\sigma|Z_{o}|\text{;}$ and if $\sigma|Z_{o}|>s$ and $i^{*}=0\text{,}$ then $2^{i^{*}}s=s<2\sigma|Z_{o}|\text{.}$ In every case $2^{i^{*}}s\leq 2s+2\sigma|Z_{o}|\text{,}$ and $i^{*}$ is finite, the grid values increasing to infinity. Dividing by $s\geq\sigma/2$ gives $2^{i^{*}}\leq 2+4|Z_{o}|=2(1+2|Z_{o}|)\text{,}$ which is the bound on $i^{*}\text{.}$ Next, $(Y_{o})_{+}\leq V+\sigma|Z_{o}|\text{,}$ since $Y_{o}\leq V+\sigma|Z_{o}|$ and $0\leq V\text{,}$ so $v_{i^{*}}=(Y_{o})_{+}+2^{i^{*}}s\leq V+\sigma|Z_{o}|+2s+2\sigma|Z_{o}|\text{.}$ For the last pointwise claim, $v_{i^{*}}\geq Y_{o}$ because $(Y_{o})_{+}\geq Y_{o}\text{,}$ and

$$v_{i^{*}}-Y_{o}\ =\ \bigl((Y_{o})_{+}-Y_{o}\bigr)+2^{i^{*}}s\ \leq\ (-Y_{o})_{+}+2s+2\sigma|Z_{o}|\ \leq\ 2s+3\sigma|Z_{o}|,$$

since $-Y_{o}=-V-\sigma Z_{o}\leq\sigma|Z_{o}|\text{.}$ In expectation, $\mathbb{E}\,i^{*}\leq 1+\log_{2}(1+2\,\mathbb{E}|Z_{o}|)\leq 3$ by concavity of $t\mapsto\log_{2}(1+2t)$ and $\mathbb{E}|Z_{o}|\leq 1\text{,}$ while $\mathbb{E}(Y_{o}-v_{i^{*}})^{2}\leq 2(2s)^{2}+18\sigma^{2}\mathbb{E}Z_{o}^{2}\leq 146\sigma^{2}\text{,}$ using $s\leq 4\sigma\text{.}$

Part (2). On $\mathcal{E}_{+}^{\,c}\text{,}$ $i^{*}>J^{+}\text{,}$ so $v_{J^{+}}<V$ and

$$(-\sigma Z_{o})_{+}\ \geq\ V-(Y_{o})_{+}\ >\ 2^{J^{+}}s\ \geq\ 4(n+1)\cdot\frac{\sigma}{2}\ =\ 2(n+1)\,\sigma,$$

using $2^{J^{+}}\geq 2^{\log_{2}(n+1)+2}=4(n+1)$ and $s\geq\sigma/2\text{;}$ the positive part is positive here, so it equals $-\sigma Z_{o}\text{,}$ and $Z_{o}<-2(n+1)\text{.}$

Part (3). The selected criterion value is at most the full model’s, whose fit is $Y$ itself, so $\|Y-\widehat{\mu}\|_{2}^{2}\leq\|Y-\widehat{\mu}\|_{2}^{2}+\operatorname{pen}(\widehat{m})\leq\operatorname{pen}(\mathrm{full})=C_{2}s^{2}n\text{,}$ and $\|\widehat{\mu}-\mu\|_{2}^{2}\leq 2\|\widehat{\mu}-Y\|_{2}^{2}+2\|Y-\mu\|_{2}^{2}\leq 2C_{2}s^{2}n+2\sigma^{2}\|Z\|_{2}^{2}\text{.}$ Put $a:=2(n+1)\text{.}$ By part (2) and the independence of $Z_{o}$ from the nonroot coordinates,

$$\begin{aligned} \displaystyle\mathbb{E}\bigl[\|Z\|_{2}^{2}\,\mathbf{1}_{\mathcal{E}_{+}^{c}}\bigr]\space & \displaystyle\leq\ (n-1)\,\mathbb{P}(Z_{o}<-a)+\mathbb{E}\bigl[Z_{o}^{2}\,\mathbf{1}\{Z_{o}<-a\}\bigr] \\ & \displaystyle\leq\ (n-1)\,e^{-a^{2}/2}+a\,\phi_{\mathcal{N}}(a)+\Phi_{\mathcal{N}}(-a)\ \leq\ C, \end{aligned}$$

where $\phi_{\mathcal{N}}$ is the standard normal density, the middle step by the Gaussian tail bound and one integration by parts. With $s^{2}\leq 16\sigma^{2}$ and $n\,e^{-a^{2}/2}\leq C\text{,}$ the pathwise bound integrates to $\mathbb{E}[\|\widehat{\mu}-\mu\|_{2}^{2}\mathbf{1}_{\mathcal{E}_{+}^{c}}]\leq C\sigma^{2}\text{.}$ ∎

With the anchor in place, some grid candidate always sits close to the truth at a price the selection can afford. The lemma is stated conditionally on the root observation, matching the conditional analysis of the selection to come.

Lemma S.6.6 (Grid coverage).

Condition on $Y_{o}=y_{0}\text{,}$ let $s\in[\sigma/2,4\sigma]\text{,}$ suppose $\mathcal{E}_{+}$ holds, and write $v:=v_{i^{*}}\text{,}$ $k^{\prime}:=k_{\mathrm{alg}}(T,v,s)\text{,}$ and $\widetilde{\theta}:=(y_{0},\mu_{-o})\text{.}$ Let $\mathfrak{O}$ be the minimum, over the point models of the candidate system and the full model, of the oracle values of Lemma S.6.1 at mean $\widetilde{\theta}\text{:}$

$$\mathfrak{O}\ :=\ \min\Bigl\{\ \inf_{\nu}\bigl(\|\widetilde{\theta}-\nu\|_{2}^{2}+\sigma^{2}(w(\nu)+1)\bigr),\ \ \sigma^{2}n\ \Bigr\}.$$

Then, for every realization,

$$\mathfrak{O}\ \leq\ C\Bigl(R^{*}_{T}(v,\sigma)+(y_{0}-v)^{2}+\sigma^{2}(1+i^{*})\Bigr).$$

(S.6.5)

If moreover $V<s/2\text{,}$ set $j^{*}:=\lfloor\log_{2}(s/V)\rfloor\geq 1\text{,}$ so that $v^{-}_{j^{*}}\in[V,2V]\text{;}$ if $j^{*}\leq J^{-}\text{,}$ then also

$$\mathfrak{O}\ \leq\ C\Bigl(R^{*}_{T}\bigl(v^{-}_{j^{*}},\sigma\bigr)+\bigl(y_{0}-v^{-}_{j^{*}}\bigr)^{2}+\sigma^{2}\bigl(1+\log(1+j^{*})\bigr)\Bigr).$$

(S.6.6)

Proof.

Every candidate of anchored index $i^{*}$ has root coordinate exactly $v\text{:}$ the flow candidates because their root state is $k\text{,}$ so $\nu_{i^{*},k,x}(o)=(v/k)\,k=v\text{,}$ and the root candidates trivially. Since $v\geq V\text{,}$ the translate $\mu^{\prime}:=\mu+(v-V)p_{o}$ lies in $\mathcal{F}_{v}(T)$ by Lemma 2.1 and agrees with $\mu$ off the root, with $\mu^{\prime}(o)=v\text{;}$ hence, for every such candidate $\nu\text{,}$

$$\|\widetilde{\theta}-\nu\|_{2}^{2}\ =\ (y_{0}-v)^{2}+\|\mu^{\prime}-\nu\|_{2}^{2}.$$

(S.6.7)

Let $k_{\max}$ be the largest dyadic integer at most $K\text{,}$ defined when $K\geq 2\text{,}$ so that $2k_{\max}>K\text{.}$ There are three exhaustive cases.

Case (a): $s^{2}k^{\prime}\leq v^{2}H_{T}\text{,}$ $K\geq 2\text{,}$ and $k^{\prime}\leq k_{\max}\text{.}$ Choose the smallest dyadic $k\geq\max\{k^{\prime},2\}\text{;}$ then $k\leq 2k^{\prime}$ and $k\leq k_{\max}\leq K\text{,}$ since $k_{\max}$ is itself a dyadic integer at least $\max\{k^{\prime},2\}\text{.}$ Proposition 3.1(1), applied to $\mu^{\prime}\in\mathcal{F}_{v}(T)$ at budget $k$ and taken with the explicit constant of its error assembly in Section S.2, supplies $x^{\star}\in\mathcal{X}_{k}$ with $\Gamma(x^{\star})\leq 9k$ and

$$\|\mu^{\prime}-\nu_{i^{*},k,x^{\star}}\|_{2}^{2}\ \leq\ 51\,v^{2}\overline{\delta}_{k}(T)\ \leq\ 51\,v^{2}\overline{\delta}_{k^{\prime}}(T)\ \leq\ 102\,A_{0}\,s^{2}k^{\prime},$$

the second step by the monotonicity of $\overline{\delta}$ (Lemma 2.3(3)) and $k\geq k^{\prime}\text{,}$ the third because the dimension clause of (2.2) is silent at $k^{\prime}\leq K\text{,}$ so its surrogate crossing clause fired there: $v^{2}\overline{\delta}_{k^{\prime}}\leq 2A_{0}s^{2}k^{\prime}\text{.}$ The weight obeys

$$w(\nu_{i^{*},k,x^{\star}})\ =\ 2\Gamma(x^{\star})+k+i^{*}+\log_{2}k+4\ \leq\ 19k+i^{*}+\log_{2}k+4\ \leq\ C(k^{\prime}+i^{*}+1).$$

By the case hypothesis and Equation S.6.3, $s^{2}k^{\prime}=\min\{v^{2}H_{T},s^{2}k^{\prime}\}\leq CR^{*}_{T}(v,\sigma)\text{,}$ and $\sigma^{2}k^{\prime}\leq 4s^{2}k^{\prime}\text{;}$ with Equation S.6.7, the candidate’s oracle value is therefore at most $(y_{0}-v)^{2}+CR^{*}_{T}(v,\sigma)+C\sigma^{2}(1+i^{*})\text{,}$ which is Equation S.6.5.

Case (b): $v^{2}H_{T}<s^{2}k^{\prime}\text{.}$ Take the root candidate $\nu^{0}_{i^{*}}\text{,}$ whose nonroot coordinates vanish. By Equation S.6.7 and the convexity bound $\|\mu-Vp_{o}\|_{2}\leq V\sqrt{h_{T}}$ established in the proof of Lemma S.5.2,

$$\|\widetilde{\theta}-\nu^{0}_{i^{*}}\|_{2}^{2}\ =\ (y_{0}-v)^{2}+\|\mu-Vp_{o}\|_{2}^{2}\ \leq\ (y_{0}-v)^{2}+V^{2}h_{T},$$

and $V^{2}h_{T}\leq v^{2}H_{T}=\min\{v^{2}H_{T},s^{2}k^{\prime}\}\leq CR^{*}_{T}(v,\sigma)$ by Equation S.6.3. The weight is $i^{*}+4\text{,}$ so the oracle value is at most $(y_{0}-v)^{2}+CR^{*}_{T}(v,\sigma)+\sigma^{2}(i^{*}+5)\text{.}$

Case (c): $s^{2}k^{\prime}\leq v^{2}H_{T}\text{,}$ and $K\leq 1$ or $k^{\prime}>k_{\max}\text{.}$ If $K\geq 2$ and $k^{\prime}>k_{\max}\text{,}$ then $K<2k_{\max}<2k^{\prime}$ and $n<C_{\star}(K+1)\leq 2C_{\star}K<4C_{\star}k^{\prime}\text{,}$ so

$$\sigma^{2}n\ \leq\ 4s^{2}n\ <\ 16\,C_{\star}\,s^{2}k^{\prime}\ \leq\ C\,R^{*}_{T}(v,\sigma)$$

by the case hypothesis and Equation S.6.3: the full-model branch of $\mathfrak{O}$ qualifies. If $K\leq 1\text{,}$ then $n<2C_{\star}\text{,}$ and $\sigma^{2}n\leq 2C_{\star}\sigma^{2}$ is absorbed by the term $\sigma^{2}(1+i^{*})$ of Equation S.6.5.

The low-amplitude bound. Let $V<s/2$ and $j^{*}\leq J^{-}\text{.}$ From $2^{j^{*}}\leq s/V<2^{j^{*}+1}\text{,}$ the amplitude $v^{-}_{j^{*}}=2^{-j^{*}}s$ lies in $[V,2V)\text{,}$ the translate $\mu+(v^{-}_{j^{*}}-V)p_{o}$ lies in $\mathcal{F}_{v^{-}_{j^{*}}}(T)$ and agrees with $\mu$ off the root, and every candidate of low-amplitude index $j^{*}$ has root coordinate exactly $v^{-}_{j^{*}}\text{,}$ so Equation S.6.7 holds with $v^{-}_{j^{*}}$ in place of $v\text{.}$ The three cases above run verbatim at the amplitude $v^{-}_{j^{*}}$ and its crossing index $k_{\mathrm{alg}}(T,v^{-}_{j^{*}},s)\text{,}$ the weight contributions $i^{*}+\log_{2}k+4$ and $i^{*}+4$ replaced by $2\log(1+j^{*})+\log_{2}k+4$ and $2\log(1+j^{*})+4\text{;}$ they yield Equation S.6.6. ∎

One computational ingredient remains before the assembly: the inner minimization of the selection criterion over an integer-state class. It is the min-plus analogue of the evaluation of Lemma S.5.4, on the same one-dimensional state and with its own overflow scalar.

Lemma S.6.7 (Min-plus evaluation).

Fix an amplitude $v>0\text{,}$ a dyadic budget $2\leq k\leq K\text{,}$ and a constant $c\geq 0\text{.}$ The minimum

$$m_{v,k}\ :=\ \min_{x\in\mathcal{X}_{k}}\Bigl[\bigl\|Y-\tfrac{v}{k}\,x\bigr\|_{2}^{2}+c\,\Gamma(x)\Bigr]$$

and one minimizer are computable exactly in $O(nk^{2})$ operations.

Proof.

Write $b:=v/k$ and $D:=3k\text{.}$ The class is nonempty: the state vector with $x_{o}=k$ and all other states zero belongs to $\mathcal{X}_{k}\text{,}$ its one leak sitting at the root. Losses and charges are vertex-additive, and the admissible completions on the subtrees of distinct children are independent given the state at their parent, so the minimum is computed by a postorder dynamic program over one state per vertex.

As in Lemma S.5.5, every $x\in\mathcal{X}_{k}$ vanishes on a subtree disjoint from $A_{k}\text{,}$ so such a subtree contributes one state-independent constant, the sum of $Y_{w}^{2}$ over its vertices, to every candidate; and a maximal chain of vertices off $A_{k}$ with a single child subtree meeting $A_{k}$ carries one common state. Call the tree obtained by pruning the zero subtrees and contracting these chains the reduced tree. The cost of such a chain is the explicit quadratic $x\mapsto\sum_{u}(Y_{u}-bx)^{2}$ in the state $x$ at the reduced-tree vertex below it, generated in $O(k)$ operations from the chain’s length, sum, and sum of squares. Unlike its sum-product counterpart, the min-plus recursion retains these additive constants: the values $m_{v,k}$ of different candidates are later compared, so nothing may be dropped. The reduction partitions the vertices: each vertex $u$ of the reduced tree owns itself, the suppressed chain entering it, and the zero subtrees pruned at $u$ or along that chain. Write $\widehat{T}_{u}$ for the union of the blocks owned by $u$ and by its descendants in the reduced tree, so that $\widehat{T}_{o}=\mathsf{V}\text{,}$ and define at each such $u$ the table

$$m_{u}(x)\ :=\ \min\Bigl\{\sum_{w\in\widehat{T}_{u}}(Y_{w}-bx^{\prime}_{w})^{2}+c\,\Gamma\bigl(z^{\prime}|_{\widehat{T}_{u}}\bigr)\Bigr\}\qquad(0\leq x\leq D),$$

the minimum over the state vectors $x^{\prime}\in\{0,\dots,D\}^{\widehat{T}_{u}}$ with $x^{\prime}_{u}=x$ whose leaks $z^{\prime}$ vanish off $A_{k}\text{.}$

Overflow. Child states are individually at most $D\text{,}$ but their sum is not. A partial min-plus product of child tables, with conceptual value $A(t)$ at child sum $t\geq 0\text{,}$ is carried as its truncation $A(0),\dots,A(D)$ together with the overflow scalar

$$\tau_{A}\ :=\ \min_{t>D}\ \bigl\{A(t)+c\,(t-D)\bigr\},$$

the minimum of an empty set being $+\infty\text{.}$ For the min-plus convolution of two partial products, $H(q)=\min_{i+j=q}\{A(i)+B(j)\}\text{,}$ the values $H(0),\dots,H(D)$ cost $O(D^{2})$ directly, and partitioning the pairs $(i,j)$ with $i+j>D$ according to whether each index exceeds $D$ gives the exact overflow

$$\tau_{H}\ =\ \min\Bigl\{\min_{\begin{subarray}{c}0\leq i,j\leq D\\ i+j>D\end{subarray}}\{A(i)+B(j)+c(i+j-D)\},\\ \tau_{A}+\min_{0\leq j\leq D}\{B(j)+cj\},\ \ \tau_{B}+\min_{0\leq i\leq D}\{A(i)+ci\},\ \ \tau_{A}+\tau_{B}+cD\Bigr\},$$

(S.6.8)

since $A(i)+c(i-D)$ ranges over the overflowed side: for instance, $i>D$ and $j\leq D$ give $A(i)+B(j)+c(i+j-D)=[A(i)+c(i-D)]+[B(j)+cj]\text{,}$ minimized by the second block. Each merge therefore costs $O(D^{2})=O(k^{2})\text{,}$ and there are fewer than $n$ merges in the whole tree.

The local step. Let $(Q(0),\dots,Q(D),\tau_{Q})$ be the assembled child-sum table at $u\text{,}$ an empty product having $Q(0)=0\text{,}$ $Q(t)=+\infty$ for $t\geq 1\text{,}$ and $\tau_{Q}=+\infty\text{,}$ and let $\chi_{u}(x)$ be the cost of the rest of $u$’s own block: the quadratic cost of the suppressed chain entering $u\text{,}$ all of whose vertices carry the state $x\text{,}$ plus the constants of the block’s pruned subtrees, zero if the block is the singleton $\{u\}\text{.}$ If $u\notin A_{k}\text{,}$ conservation forces the child sum to equal $x\leq D\text{,}$ the overflow is irrelevant, and $m_{u}(x)=(Y_{u}-bx)^{2}+\chi_{u}(x)+Q(x)\text{.}$ If $u\in A_{k}\text{,}$ splitting on $z_{u}=0$ against $z_{u}\neq 0$ gives

$$m_{u}(x)\ =\ (Y_{u}-bx)^{2}+\chi_{u}(x)+\min\Bigl\{Q(x),\ \ c\,\widetilde{\omega}(u)+\min_{t\geq 0}\bigl[Q(t)+c\,|x-t|\bigr]\Bigr\};$$

including $t=x$ in the charged branch only adds the option $Q(x)+c\widetilde{\omega}(u)\text{,}$ dominated by the first branch, so the split is exact. The inner minimum is a one-dimensional distance transform, the min-plus mirror of the recurrences in Lemma S.5.4: with

$$\begin{gathered} \displaystyle F(0):=Q(0),\qquad F(x):=\min\{Q(x),\,F(x-1)+c\}; \\ \displaystyle B(D):=\min\{Q(D),\,\tau_{Q}\},\qquad B(x):=\min\{Q(x),\,B(x+1)+c\}, \end{gathered}$$

one has $F(x)=\min_{t\leq x}\{Q(t)+c(x-t)\}$ and $B(x)=\min_{t\geq x}\{Q(t)+c(t-x)\}\text{,}$ the seed at $D$ covering the overflowed sums because $\min_{t>D}\{Q(t)+c(t-x)\}=\tau_{Q}+c(D-x)$ for $x\leq D\text{;}$ hence $\min_{t\geq 0}\{Q(t)+c|x-t|\}=\min\{F(x),B(x)\}\text{,}$ at cost $O(k)$ for the two passes. At the root, $x_{o}=k$ is pinned and $\widetilde{\omega}(o)=0\text{,}$ so $m_{v,k}$ is the local step evaluated at $x=k\text{.}$ Storing, at every minimum, an attaining index, an attaining block of Equation S.6.8, and an attaining overflow sum, and backtracking from the root, recovers a minimizer $x\in\mathcal{X}_{k}\text{.}$ The total is $O(k^{2})$ per merge and $O(k)$ per vertex, hence $O(nk^{2})$ operations. ∎

Completion of the proof of Theorem 3, sufficiency.

The estimator is the penalized selection of Lemma S.6.1 over the candidate system at working scale $s=\sigma\text{,}$ with penalties $C_{2}\sigma^{2}(D_{m}+\Delta_{m})$ and $C_{2}\sigma^{2}n\text{;}$ for $n<n_{\star}$ it returns $Y\text{,}$ a case already absorbed. Condition on $Y_{o}=y_{0}\text{:}$ the candidate system is deterministic, the data decompose as $Y=\widetilde{\theta}+(0,\sigma Z_{-o})$ with $\widetilde{\theta}=(y_{0},\mu_{-o})\text{,}$ the noise being centered Gaussian with independent coordinates of variances at most $\sigma^{2}\text{,}$ the root variance zero, and the penalties are deterministic, so Lemma S.6.1 applies conditionally with $E$ the whole space, $\bar{\kappa}=C_{2}\text{,}$ and $N=n\text{.}$ Since $\widetilde{\theta}-\mu=(Y_{o}-V)p_{o}\text{,}$ the pathwise root replacement

$$\|\widehat{\mu}-\mu\|_{2}^{2}\ \leq\ 2\,\|\widehat{\mu}-\widetilde{\theta}\|_{2}^{2}+2\,(Y_{o}-V)^{2},\qquad\mathbb{E}(Y_{o}-V)^{2}=\sigma^{2},$$

(S.6.9)

converts conditional bounds on $\|\widehat{\mu}-\widetilde{\theta}\|_{2}^{2}$ into risk bounds. Write $k_{a}:=k_{\mathrm{alg}}(T,V,\sigma)$ and $r:=\min\{V^{2}H_{T},\sigma^{2}k_{a}\}\text{;}$ by Theorem 1 and the sandwich $k_{a}\leq k_{0}\leq 2k_{a}$ of Lemma 2.3(3), $r\asymp R^{*}_{T}(V,\sigma)\text{.}$ The anchor event $\mathcal{E}_{+}$ is determined by $Y_{o}\text{.}$ Its complement contributes at most $C\sigma^{2}$ to the risk, by Lemma S.6.5(3); on $\mathcal{E}_{+}\text{,}$ taking conditional expectations in Equation S.6.9 and applying Lemma S.6.1 conditionally bounds the contribution by $C\,\mathbb{E}[\mathfrak{O}^{\prime}\,\mathbf{1}_{\mathcal{E}_{+}}]+C\sigma^{2}\text{,}$ where $\mathfrak{O}^{\prime}$ is the conditional oracle value of Lemma S.6.1, the minimum over all candidates including the subspaces. It therefore suffices to prove

$$\mathbb{E}\bigl[\mathfrak{O}^{\prime}\,\mathbf{1}_{\mathcal{E}_{+}}\bigr]\ \leq\ C\,(r+\sigma^{2}).$$

(S.6.10)

There are three exhaustive regimes; in the first two the bound is pointwise.

Diameter regime: $V^{2}H_{T}\leq\sigma^{2}k_{a}\text{.}$ The model $\operatorname{span}(p_{o})$ contains $y_{0}p_{o}\text{,}$ and $\widetilde{\theta}-y_{0}p_{o}$ vanishes at the root and equals $\mu$ off it, so

$$\operatorname{dist}^{2}\bigl(\widetilde{\theta},\operatorname{span}(p_{o})\bigr)\ \leq\ \|\mu-Vp_{o}\|_{2}^{2}\ \leq\ V^{2}h_{T}\ \leq\ r,$$

by convexity as in Lemma S.5.2. Dimension and weight are one each, so $\mathfrak{O}^{\prime}\leq r+3\sigma^{2}\text{.}$

Entropy regime with small crossing: $\sigma^{2}k_{a}<V^{2}H_{T}$ and $k_{a}\leq k^{\star}/2\text{.}$ Choose the smallest dyadic $k\geq\max\{k_{a},2\}\text{,}$ so that $k\leq\max\{2k_{a},2\}\leq k^{\star}$ and $k\leq 4k_{a}\text{.}$ Since $k_{a}\leq k^{\star}/2\leq K/2\text{,}$ the dimension clause of (2.2) is silent at $k_{a}\text{,}$ so its surrogate crossing clause fired there: $V^{2}\overline{\delta}_{k_{a}}\leq 2A_{0}\sigma^{2}k_{a}\text{.}$ Lemma S.6.3 at budget $k$ and the monotonicity of $\overline{\delta}$ (Lemma 2.3(3)) give an admissible $S$ with

$$\operatorname{dist}^{2}\bigl(\mu,\ \mathcal{V}_{k,S}\bigr)\ \leq\ 22\,V^{2}\overline{\delta}_{k}\ \leq\ 22\,V^{2}\overline{\delta}_{k_{a}}\ \leq\ 44\,A_{0}\,\sigma^{2}k_{a}.$$

The root indicator $p_{o}$ lies in every profile subspace, $o$ being in $R_{0}\text{,}$ so translating an approximant of $\mu$ by $(y_{0}-V)p_{o}$ shows $\operatorname{dist}(\widetilde{\theta},\mathcal{V}_{k,S})\leq\operatorname{dist}(\mu,\mathcal{V}_{k,S})\text{.}$ With $\dim\mathcal{V}_{k,S}\leq|R_{0}|+|S|\leq(e+4)k$ and $\Delta_{k,S}=(C_{1}+1)k\text{,}$ the oracle value of this subspace is at most $44A_{0}\sigma^{2}k_{a}+C\sigma^{2}k\leq C\sigma^{2}k_{a}=Cr\text{.}$

Entropy regime with large crossing: $\sigma^{2}k_{a}<V^{2}H_{T}$ and $k_{a}>k^{\star}/2\text{.}$ Suppose first $V\geq\sigma/2\text{.}$ On $\mathcal{E}_{+}\text{,}$ Lemma S.6.5(1) with $s=\sigma$ and $\sigma/V\leq 2$ gives, pointwise,

$$\frac{v_{i^{*}}}{V}\ \leq\ 1+\frac{2\sigma}{V}+\frac{3\sigma|Z_{o}|}{V}\ \leq\ 5+6|Z_{o}|,$$

so Lemma S.6.2, at the factor $c=v_{i^{*}}/V\geq 1\text{,}$ gives $R^{*}_{T}(v_{i^{*}},\sigma)\leq(5+6|Z_{o}|)^{2}R^{*}_{T}(V,\sigma)$ pointwise, of expectation at most $CR^{*}_{T}(V,\sigma)\text{.}$ Since $\mathfrak{O}^{\prime}\leq\mathfrak{O}\text{,}$ the infimum running over more candidates, Equation S.6.5 and Lemma S.6.5(1) together prove Equation S.6.10 in this regime. Suppose now $V<\sigma/2\text{.}$ The regime inequality and $H_{T}<n$ give $\sigma^{2}k_{a}<V^{2}H_{T}\leq V^{2}n\text{,}$ so $\sigma/V<\sqrt{n/k_{a}}\leq\sqrt{n}$ and $j^{*}=\lfloor\log_{2}(\sigma/V)\rfloor$ satisfies $1\leq j^{*}\leq\tfrac{1}{2}\log_{2}n\leq J^{-}\text{.}$ The low-amplitude bound Equation S.6.6 applies, and its three terms are handled in turn: $v^{-}_{j^{*}}\in[V,2V]$ and Lemma S.6.2 give $R^{*}_{T}(v^{-}_{j^{*}},\sigma)\leq 4R^{*}_{T}(V,\sigma)\text{;}$ the amplitude $v^{-}_{j^{*}}=2^{-j^{*}}\sigma$ is deterministic, so

$$\mathbb{E}\bigl(Y_{o}-v^{-}_{j^{*}}\bigr)^{2}\ \leq\ 2\sigma^{2}\,\mathbb{E}Z_{o}^{2}+2\bigl(V-v^{-}_{j^{*}}\bigr)^{2}\ \leq\ 2\sigma^{2}+2V^{2}\ \leq\ 3\sigma^{2},$$

using $|V-v^{-}_{j^{*}}|\leq V<\sigma/2\text{;}$ and, since $\log(1+t)\leq t$ and $k_{a}>k^{\star}/2\geq\log n/(8C_{1})$ for $n\geq n_{\star}\text{,}$

$$\sigma^{2}\log(1+j^{*})\ \leq\ \sigma^{2}\,\frac{\log_{2}n}{2}\ \leq\ C\,\sigma^{2}k_{a}\ =\ C\,r.$$

This proves Equation S.6.10 in every regime, hence the risk bound of Theorem 3.

Running time. The dyadic budgets $k\leq k^{\star}$ carry $\sum_{k}e^{C_{1}k}\leq 2e^{C_{1}k^{\star}}\leq 2\sqrt{n}$ admissible supports in total, the terms of the sum growing at least geometrically; by Lemma S.6.4 they are listed in $O(n\sqrt{n})$ operations. One $O(n\log n)$ preprocessing pass stores every vertex’s $2^{j}$-th ancestor for $j\leq\lceil\log_{2}n\rceil$ and every root-path sum of $Y\text{;}$ lifting the deeper endpoint and then both endpoints finds the deepest common ancestor of any pair in $O(\log n)$ operations, and, by the ancestor count recorded with Lemma 2.2, $\langle p_{u},p_{w}\rangle=\operatorname{depth}(u\wedge w)+1\text{,}$ while $\langle Y,p_{u}\rangle$ is the stored root-path sum at $u\text{.}$ Forming and solving the normal equations of one subspace, of dimension $O(\log n)\text{,}$ therefore costs $O(\log^{3}n)$ operations after the preprocessing, and the subspace half costs $O(n^{3/2})$ in total, up to logarithmic factors. The grid half has $O(\log n)$ amplitudes, $O(\log n)$ budgets, and, for each pair, one run of Lemma S.6.7 with $c=2C_{2}\sigma^{2}\text{:}$ the remaining weight $C_{2}\sigma^{2}(k+i+\log_{2}k+4)$ of a flow candidate is constant across $x\in\mathcal{X}_{k}\text{,}$ so the penalized criterion is minimized by the returned minimizer. With $k\leq n\text{,}$ the grid half costs $O(n^{3}\log^{2}n)$ operations, a conservative bound, and the final comparison across all candidates is dominated by it. All tie breaks are fixed, so the estimator is a deterministic function of $(T,\sigma,Y)\text{,}$ computable in polynomial time. ∎

It remains to remove the noise level; the difference statistics and their contamination bound come first. Write $\operatorname{hc}(u)$ for the size-heavy child of an internal vertex $u\text{,}$ fixed in Section 4.3, pair the children of each branching vertex by a fixed rule, one child left unpaired when their number is odd, and take

$$X_{u}\ :=\ Y_{u}-Y_{\operatorname{hc}(u)}\ \ (u\ \text{internal}),\qquad X_{c,c^{\prime}}\ :=\ Y_{c}-Y_{c^{\prime}}\ \ (\{c,c^{\prime}\}\ \text{a pair of children}),$$

(S.6.11)

at least $(n-1)/2$ statistics in all, each a Gaussian of variance $2\sigma^{2}$ around a signal difference. For every $\varepsilon>0\text{,}$ the contamination bound announced in the main body reads

$$\#\bigl\{u:\ \mathbb{E}X_{u}\geq\varepsilon\sigma\bigr\}\ +\ \#\bigl\{\{c,c^{\prime}\}:\ |\mathbb{E}X_{c,c^{\prime}}|\geq\varepsilon\sigma\bigr\}\ \leq\ \frac{V\,w_{T}}{\varepsilon\sigma}\ .$$

(S.6.12)

Proof of (S.6.12).

Fix $\varepsilon>0\text{.}$ Both counts rest on the identity, valid for every vertex set $B$ by the subtree sums of Lemma 2.1,

$$\sum_{a\in B}\mu(a)\ =\ \sum_{a\in B}\ \sum_{u:\,u\succeq a}s_{u}\ =\ \sum_{u\in\mathsf{V}}s_{u}\,\#\{a\in B:\ a\preceq u\}\ ;$$

(S.6.13)

when every root path meets $B$ in at most $w$ vertices, the count on the right is at most $w\text{,}$ and the leaks total $V\text{,}$ so $\sum_{a\in B}\mu(a)\leq Vw\text{.}$

Heavy differences. Every internal vertex carries exactly one heavy edge, and every vertex is the size-heavy child of at most one vertex, its parent, so the heavy edges partition $\mathsf{V}$ into vertex-disjoint paths; the top of such a path is the root or the lower endpoint of a light edge. The flow form of Lemma 2.1 makes every mean $\mathbb{E}X_{u}=\mu(u)-\mu(\operatorname{hc}(u))$ nonnegative, and along one heavy path the means telescope to $\mu(\mathrm{top})-\mu(\mathrm{bottom})\leq\mu(\mathrm{top})\text{.}$ A root path contains at most $w_{\mathrm{lt}}$ tops, one for the root and one per light edge it crosses, so Equation S.6.13, applied to the set of tops, gives

$$\varepsilon\sigma\,\#\bigl\{u:\ \mathbb{E}X_{u}\geq\varepsilon\sigma\bigr\}\ \leq\ \sum_{u\ \mathrm{internal}}\mathbb{E}X_{u}\ \leq\ \sum_{\mathrm{tops}\ a}\mu(a)\ \leq\ V\,w_{\mathrm{lt}}\ .$$

Sibling pairs. The coordinates $\mu(c)$ of the children of a branching vertex $u$ are nonnegative, so a pair with $|\mathbb{E}X_{c,c^{\prime}}|=|\mu(c)-\mu(c^{\prime})|\geq\varepsilon\sigma$ contains a child whose coordinate is at least $\varepsilon\sigma\text{;}$ such children number at most $\mu(u)/(\varepsilon\sigma)\text{,}$ since $\sum_{c\in\operatorname{ch}(u)}\mu(c)\leq\mu(u)\text{,}$ and distinct pairs contain distinct children. Summing over the branching vertices, of which a root path contains at most $w_{\mathrm{br}}\text{,}$ and applying Equation S.6.13 to their set,

$$\#\bigl\{\{c,c^{\prime}\}:\ |\mathbb{E}X_{c,c^{\prime}}|\geq\varepsilon\sigma\bigr\}\ \leq\ \frac{1}{\varepsilon\sigma}\sum_{u\ \mathrm{branching}}\mu(u)\ \leq\ \frac{V\,w_{\mathrm{br}}}{\varepsilon\sigma}\ .$$

Adding the two displays gives Equation S.6.12, the width being $w_{T}=w_{\mathrm{lt}}+w_{\mathrm{br}}\text{.}$ ∎

The statistic family Equation S.6.11 has at least $(n-1)/2$ members: an internal vertex with $g$ children contributes the difference along its heavy edge together with $\lfloor g/2\rfloor$ sibling pairs, $1+\lfloor g/2\rfloor\geq(g+1)/2\text{,}$ and the child counts of the internal vertices sum to $n-1\text{.}$ Each vertex $u$ enters at most three statistics: its own difference $X_{u}$ when internal, the difference $X_{\operatorname{pa}(u)}$ when it is the size-heavy child of its parent, and at most one pair. Scan the family in a fixed order and retain every statistic both of whose coordinates are unused by the statistics retained so far: a retained statistic blocks at most four others, two through each coordinate, so at least a fifth of the family is retained. This produces a deterministic subfamily, relabeled $X_{1},\dots,X_{N}\text{,}$ with $N\geq(n-1)/10$ and pairwise disjoint coordinate pairs; the $X_{i}$ are therefore independent, with $X_{i}-\mathbb{E}X_{i}\sim N(0,2\sigma^{2})\text{.}$

Lemma S.6.8 (Noise estimate).

Let $n\geq 2\text{,}$ write $\operatorname{med}\{|X_{1}|,\dots,|X_{N}|\}$ for the $\lceil N/2\rceil$-th smallest of the absolute values, and set

$$\widehat{\sigma}\ :=\ \frac{\operatorname{med}\{|X_{1}|,\dots,|X_{N}|\}}{\sqrt{2}\,\Phi_{\mathcal{N}}^{-1}(3/4)},$$

the denominator being the median of $|N(0,2)|\text{,}$ with $\Phi_{\mathcal{N}}$ the standard normal distribution function. If $V\,w_{T}\leq\sigma n/1280\text{,}$ then

$$\mathbb{P}\bigl(\widehat{\sigma}\in[\sigma/2,\ 2\sigma]\bigr)\ \geq\ 1-2e^{-c(n-1)}$$

with a universal $c>0\text{.}$

Proof.

Call $X_{i}$ clean when $|\mathbb{E}X_{i}|\leq\sigma/8\text{.}$ The contamination count Equation S.6.12 at $\varepsilon=1/8\text{,}$ applied to the full family and hence to the subfamily, bounds the number of statistics that are not clean by $8Vw_{T}/\sigma\leq n/160\leq N/8\text{,}$ the last step because $N\geq(n-1)/10\geq n/20$ for $n\geq 2\text{.}$ A clean statistic satisfies, for every $t\geq 0\text{,}$

$$\mathbb{P}\bigl(|X_{i}|\leq t\bigr)\ \leq\ \mathbb{P}\Bigl(|N(0,2)|\leq\frac{t}{\sigma}+\frac{1}{8}\Bigr),\qquad\mathbb{P}\bigl(|X_{i}|>t\bigr)\ \leq\ \mathbb{P}\Bigl(|N(0,2)|>\frac{t}{\sigma}-\frac{1}{8}\Bigr),$$

and two numerical evaluations of the standard normal distribution function give

$$\mathbb{P}\bigl(|N(0,2)|\leq 0.625\bigr)\ <\ 0.35\ <\ \tfrac{3}{8},\qquad\mathbb{P}\bigl(|N(0,2)|>1.675\bigr)\ <\ 0.24\ <\ \tfrac{3}{8}.$$

If $\operatorname{med}|X|<\sigma/2\text{,}$ at least $\lceil N/2\rceil$ of the absolute values are below $\sigma/2\text{,}$ of which at least $N/2-N/8=3N/8$ belong to clean statistics, and each clean statistic falls below $\sigma/2$ with probability less than $0.35\text{.}$ If $\operatorname{med}|X|>9\sigma/5\text{,}$ at least $N-\lceil N/2\rceil+1\geq N/2$ of the absolute values exceed $9\sigma/5\text{,}$ at least $3N/8$ of them clean, each clean statistic exceeding $9\sigma/5$ with probability less than $0.24\text{.}$ In either case, a count of at most $N$ independent indicators, each of success probability at most a fixed $p<3/8\text{,}$ must reach $3N/8\text{.}$ For $\lambda>0\text{,}$ Markov’s inequality applied to the exponential of $\lambda$ times this count bounds the probability by $[e^{-3\lambda/8}(1-p+pe^{\lambda})]^{N}\text{,}$ each missing indicator only lowering the moment generating function below the factor $1-p+pe^{\lambda}>1\text{;}$ the bracket equals $1$ at $\lambda=0$ with derivative $p-\tfrac{3}{8}<0\text{,}$ so a fixed small $\lambda$ makes it $e^{-c}<1\text{,}$ and each probability is at most $e^{-cN}\leq e^{-c(n-1)/10}\text{;}$ renaming $c\text{,}$ the two events together have probability at most $2e^{-c(n-1)}\text{,}$ the claimed exceptional probability. On the complement of the two events, $\operatorname{med}|X|\in[\sigma/2,\,9\sigma/5]\text{;}$ since $\Phi_{\mathcal{N}}(0.6718)<\tfrac{3}{4}<\Phi_{\mathcal{N}}(0.6788)$ by a numerical evaluation, the denominator $\sqrt{2}\,\Phi_{\mathcal{N}}^{-1}(3/4)$ lies in $(0.95,\,0.96)\text{,}$ and

$$\widehat{\sigma}\ \in\ \Bigl[\frac{\sigma/2}{0.96},\ \frac{9\sigma/5}{0.95}\Bigr]\ \subseteq\ [\sigma/2,\ 2\sigma].$$

Completion of the proof of Theorem 4.

For $n<n_{\star}$ return $Y\text{,}$ whose risk $n\sigma^{2}\leq n_{\star}\sigma^{2}$ is absorbed as before; assume $n\geq n_{\star}\text{.}$ The estimator computes $\widehat{\sigma}\text{;}$ on the null event $\{\widehat{\sigma}=0\}$ it returns $Y\text{,}$ and otherwise it rounds upward to $\widetilde{\sigma}:=2^{\lceil\log_{2}\widehat{\sigma}\rceil}$ and runs the selection of Theorem 3 at working scale $s=\widetilde{\sigma}\text{,}$ with the weight of every subspace and point model increased by two and the penalties built from these shifted weights at scale $\widetilde{\sigma}\text{.}$ The estimator depends only on $(T,Y)\text{,}$ and its running time adds $O(n\log n)$ for the statistics and the median to the polynomial cost already accounted. We prove the risk bound with $c=1/1280$ in the regime condition.

Let $\widehat{E}:=\{\widehat{\sigma}\in[\sigma/2,2\sigma]\}\text{,}$ so that $\mathbb{P}(\widehat{E}^{c})\leq 2e^{-c(n-1)}$ by Lemma S.6.8 under the regime condition. On $\widehat{E}\text{,}$ the rounding gives $\widetilde{\sigma}\in[\sigma/2,4\sigma]\text{,}$ and $\widetilde{\sigma}$ is a power of two in that interval, hence takes at most four values determined by $\sigma\text{.}$

On $\widehat{E}\text{.}$ Condition on $Y_{o}=y_{0}\text{.}$ The realized candidate-and-penalty system is one of the four deterministic systems indexed by the possible values of $\widetilde{\sigma}\text{;}$ the subspace candidates are common to all four, while the grids differ. The penalties satisfy the bracketing of Lemma S.6.1 with respect to the true $\sigma$ on $\widehat{E}\text{:}$ with shifted weights $\Delta_{m}+2\text{,}$ since $\widetilde{\sigma}^{2}\in[\sigma^{2}/4,16\sigma^{2}]$ and $C_{2}/4=32\text{,}$

$$32\,\sigma^{2}(D_{m}+\Delta_{m}+2)\ \leq\ C_{2}\widetilde{\sigma}^{2}(D_{m}+\Delta_{m}+2)\ \leq\ 16C_{2}\,\sigma^{2}(D_{m}+\Delta_{m}+2),$$

and likewise for the full model, so $E=\widehat{E}$ and $\bar{\kappa}=16C_{2}\text{.}$ The proof of Lemma S.6.1 is repeated with one change: the deviation variable is the supremum over the four systems, each paired with its own comparison model. The weight shift by two keeps the combined Kraft sum below one, since $4e^{-2}\bigl(0.53+\tfrac{1}{8}\bigr)<1$ by Equation S.6.4 and the subspace sum, so the expectation of the deviation variable is still at most one; the basic inequality is unchanged, the selected and the comparison model both lying in the realized subsystem. The conditional oracle bound of Lemma S.6.1 therefore holds on $\widehat{E}$ for the realized system.

Coverage now runs as in the completion of Theorem 3, at the realized scale $s=\widetilde{\sigma}\in[\sigma/2,4\sigma]\text{,}$ with $k_{a}(s):=k_{\mathrm{alg}}(T,V,s)$ and $r_{s}:=\min\{V^{2}H_{T},s^{2}k_{a}(s)\}\text{;}$ by Equation S.6.3, $r_{s}\leq CR^{*}_{T}(V,\sigma)$ uniformly over the four values, and the weight shift adds $2\sigma^{2}$ to every oracle value, absorbed by the additive term. In the diameter regime, the pointwise bound through $\operatorname{span}(p_{o})$ is unchanged. In the entropy regime with $k_{a}(s)\leq k^{\star}/2\text{,}$ the surrogate crossing at scale $s$ gives $V^{2}\overline{\delta}_{k_{a}(s)}\leq 2A_{0}s^{2}k_{a}(s)\text{,}$ and the subspace bound reads $22V^{2}\overline{\delta}_{k}\leq 44A_{0}s^{2}k_{a}(s)\leq Cr_{s}\text{.}$ In the entropy regime with $k_{a}(s)>k^{\star}/2\text{:}$ for $V\geq s/2\text{,}$ Lemma S.6.5(1) gives, pointwise,

$$\frac{v_{i^{*}}}{V}\ \leq\ 1+\frac{2s}{V}+\frac{3\sigma|Z_{o}|}{V}\ \leq\ 5+12\,|Z_{o}|,$$

using $s/V\leq 2$ and $\sigma\leq 2s\text{,}$ so Equation S.6.5, Lemma S.6.2, and Lemma S.6.5(1) give the bound $C(R^{*}_{T}(V,\sigma)+\sigma^{2})$ in expectation as before. For $V<s/2\text{,}$ the index $j^{*}=\lfloor\log_{2}(s/V)\rfloor$ again lies in $\{1,\dots,J^{-}\}\text{,}$ since $s^{2}k_{a}(s)<V^{2}H_{T}\leq V^{2}n\text{;}$ the amplitude $v^{-}_{j^{*}}=2^{-j^{*}}\widetilde{\sigma}$ now depends on the data through $\widetilde{\sigma}\text{,}$ so no independence from $Z_{o}$ is available, and the root-coordinate term of Equation S.6.6 is bounded pathwise on $\widehat{E}$ instead:

$$\bigl(Y_{o}-v^{-}_{j^{*}}\bigr)^{2}\ \leq\ 2\sigma^{2}Z_{o}^{2}+2\bigl(V-v^{-}_{j^{*}}\bigr)^{2}\ \leq\ 2\sigma^{2}Z_{o}^{2}+2V^{2}\ \leq\ 2\sigma^{2}Z_{o}^{2}+8\sigma^{2},$$

using $v^{-}_{j^{*}}\in[V,2V]$ and $V<s/2\leq 2\sigma\text{,}$ of expectation $O(\sigma^{2})\text{;}$ the weight term $\sigma^{2}\log(1+j^{*})\leq C\sigma^{2}k_{a}(s)=Cr_{s}$ as before. Finally, for every realized $s$ the omitted-anchor event is contained in $\{Z_{o}<-2(n+1)\}$ by Lemma S.6.5(2), and Lemma S.6.5(3) with $s^{2}\leq 16\sigma^{2}$ bounds its loss contribution by $C\sigma^{2}\text{.}$ Splitting along the realized anchor event and applying the pathwise root replacement Equation S.6.9 on it, as in the completion of Theorem 3, gives $\mathbb{E}[\|\widehat{\mu}-\mu\|_{2}^{2}\,\mathbf{1}_{\widehat{E}}]\leq C(R^{*}_{T}(V,\sigma)+\sigma^{2})\text{.}$

Off $\widehat{E}\text{.}$ On $\{\widehat{\sigma}=0\}$ the loss is $\sigma^{2}\|Z\|_{2}^{2}\text{.}$ On $\{\widehat{\sigma}>0\}\cap\widehat{E}^{c}\text{,}$ compare the selected criterion with $\operatorname{span}(p_{o})\text{,}$ of dimension one and shifted weight three:

$$\|Y-\widehat{\mu}\|_{2}^{2}\ \leq\ \bigl\|Y-P_{\operatorname{span}(p_{o})}Y\bigr\|_{2}^{2}+4C_{2}\widetilde{\sigma}^{2}\ \leq\ \|Y\|_{2}^{2}+4C_{2}\widetilde{\sigma}^{2},$$

so $\|\widehat{\mu}-\mu\|_{2}\leq\|\widehat{\mu}-Y\|_{2}+\|Y-\mu\|_{2}\leq 2\|Y\|_{2}+\|\mu\|_{2}+2\sqrt{C_{2}}\,\widetilde{\sigma}\text{.}$ Pathwise, $\widetilde{\sigma}\leq 2\widehat{\sigma}\leq 3\operatorname{med}|X|\leq 3\max_{i}|X_{i}|\leq 6\|Y\|_{\infty}\leq 6\|Y\|_{2}\text{,}$ each $X_{i}$ being a difference of two coordinates of $Y\text{,}$ so all losses off $\widehat{E}$ obey $\|\widehat{\mu}-\mu\|_{2}\leq C(\|Y\|_{2}+\|\mu\|_{2})\text{.}$ For the moments: $\mu(u)\leq V$ at every vertex and $\sum_{u}\mu(u)=\sum_{x}s_{x}(\operatorname{depth}(x)+1)\leq Vn$ by Lemma 2.1, so $\|\mu\|_{2}^{2}\leq V^{2}n\text{;}$ $\mathbb{E}\|Z\|_{2}^{4}\leq 3n^{2}$ gives $\mathbb{E}\|Y\|_{2}^{4}\leq 8(\|\mu\|_{2}^{4}+\sigma^{4}\mathbb{E}\|Z\|_{2}^{4})\leq 8(V^{4}n^{2}+3\sigma^{4}n^{2})\text{;}$ and the regime condition with $w_{T}\geq 1$ gives $V\leq\sigma n/1280\text{,}$ hence $V^{4}n^{2}\leq\sigma^{4}n^{6}\text{.}$ The squared loss therefore has second moment at most $C\sigma^{4}n^{6}$ off $\widehat{E}\text{,}$ and the Cauchy–Schwarz inequality gives

$$\mathbb{E}\bigl[\|\widehat{\mu}-\mu\|_{2}^{2}\,\mathbf{1}_{\widehat{E}^{c}}\bigr]\ \leq\ \bigl(C\sigma^{4}n^{6}\bigr)^{1/2}\bigl(2e^{-c(n-1)}\bigr)^{1/2}\ \leq\ C\sigma^{2}n^{3}e^{-c(n-1)/2}\ \leq\ C\sigma^{2}.$$

Adding the two contributions proves Theorem 4. ∎

The width of the benchmark families is read off the definition in Section 4.3. A path has no light edge and no branching vertex, so $w_{T}=1\text{.}$ On the complete binary tree of height $h\text{,}$ every internal vertex has one light child edge, and a root-to-leaf path may cross a light edge and a branching vertex at each of its $h$ internal levels, so $w_{T}=(1+h)+h=2h+1\text{.}$ On a star with $m\geq 2$ leaves, and on the broom of Theorem 5, a root-to-leaf path crosses at most one light edge and one branching vertex, so $w_{T}\leq 3\text{.}$

The regime of Theorem 4 cannot be removed entirely by a better noise estimate. The following converse, promised at the end of Section 4.3, shows that once budgets of order $\sigma n\sqrt{\log n}$ are admitted, no estimator lands strictly within a factor $\sqrt{2}$ of the noise level, uniformly, on any tree.

Proposition S.6.9 (Noise-estimation ceiling).

For every finite rooted tree and every $\sigma>0\text{,}$ put $\lambda_{n}:=\log\bigl(4(n+1)\bigr)$ and $V_{1}:=6\,\sigma n\sqrt{\lambda_{n}}\text{.}$ In the model $Y=\mu+sZ\text{,}$ $Z\sim N(0,I_{n})\text{,}$ where $\mu$ is a monotone flow of root value at most $V_{1}\text{,}$ that is, $\mu\in\bigcup_{0\leq V^{\prime}\leq V_{1}}\mathcal{F}_{V^{\prime}}(T)$ with $\mathcal{F}_{0}(T):=\{0\}\text{,}$ and $s\in\{\sigma,2\sigma\}\text{,}$ every estimator $\widehat{s}=\widehat{s}(Y)$ satisfies

$$\sup_{(\mu,s)}\ \mathbb{P}_{\mu,s}\Bigl(\widehat{s}\notin\bigl(s/\sqrt{2},\ \sqrt{2}\,s\bigr)\Bigr)\ \geq\ \frac{3}{8},$$

the supremum running over the admitted pairs. The same bound holds for randomized estimators, by averaging over their internal randomness.

Proof.

The mechanism is a scale mixture: doubling the noise of a Gaussian shift is a random perturbation of its mean, and a signal whose flow inequalities all carry enough slack absorbs the perturbation without leaving the cone.

Step 1: the mixture identity. In one coordinate, with $a:=3\sigma^{2}$ and $b:=\sigma^{2}\text{,}$ the exponent of the convolution integrand of $N(0,a)$ and $N(0,b)$ completes the square as

$$\frac{w^{2}}{2a}+\frac{(y-w)^{2}}{2b}\ =\ \frac{a+b}{2ab}\Bigl(w-\frac{ay}{a+b}\Bigr)^{2}+\frac{y^{2}}{2(a+b)}\ ,$$

and integrating out $w$ leaves a Gaussian integral at variance $ab/(a+b)\text{;}$ the normalizers combine into that of $N(0,a+b)\text{,}$ so the convolution is $N(0,4\sigma^{2})\text{.}$ Coordinates being independent, with $\gamma:=N(0,3\sigma^{2}I_{n})$ and any fixed $\mu_{0}\in\mathbb{R}^{n}\text{,}$

$$N\bigl(\mu_{0},\,4\sigma^{2}I_{n}\bigr)\ =\ \int N\bigl(\mu_{0}+w,\ \sigma^{2}I_{n}\bigr)\,d\gamma(w):$$

(S.6.14)

the law of $Y$ at signal $\mu_{0}$ and noise $2\sigma$ is a $\gamma$-mixture of laws at noise $\sigma$ and shifted signals.

Step 2: a signal with slack. Define $\mu_{0}$ through its leaks,

$$s_{v}\ :=\ \sigma\sqrt{6\,(1+|\operatorname{ch}(v)|)\,\lambda_{n}}\qquad(v\in\mathsf{V}),\qquad V_{0}\ :=\ \sum_{v}s_{v},$$

so that $\mu_{0}\in\mathcal{F}_{V_{0}}(T)$ by Lemma 2.1. The Cauchy–Schwarz inequality and $\sum_{v}(1+|\operatorname{ch}(v)|)=n+(n-1)\leq 2n$ give $V_{0}\leq\sigma\sqrt{6\lambda_{n}}\,\sqrt{n}\,\sqrt{2n}=\sigma n\sqrt{12\lambda_{n}}\leq V_{1}\text{,}$ so $\mu_{0}$ is admitted. Define the legality event

$$A\ :=\ \Bigl\{w\in\mathbb{R}^{\mathsf{V}}:\ w_{v}-\textstyle\sum_{c\in\operatorname{ch}(v)}w_{c}\ \geq\ -s_{v}\ \text{ for all }v\in\mathsf{V}\Bigr\}\ \cap\ \bigl\{w_{o}\leq\sigma\sqrt{6\lambda_{n}}\bigr\}.$$

The leak map is linear, so on $A$ the leak of $\mu_{0}+w$ at $v$ is $s_{v}+(w_{v}-\sum_{c}w_{c})\geq 0\text{:}$ the shifted signal is a monotone flow, of root value

$$V_{0}+w_{o}\ \leq\ \sigma n\sqrt{12\lambda_{n}}+\sigma\sqrt{6\lambda_{n}}\ \leq\ \bigl(\sqrt{12}+\sqrt{6}\bigr)\,\sigma n\sqrt{\lambda_{n}}\ \leq\ V_{1},$$

hence admitted. Under $\gamma\text{,}$ the constraint at $v$ fails when a centered Gaussian of variance $3\sigma^{2}(1+|\operatorname{ch}(v)|)$ exceeds $s_{v}\text{,}$ and the root cap fails when a centered Gaussian of variance $3\sigma^{2}$ exceeds $\sigma\sqrt{6\lambda_{n}}\text{;}$ by the Gaussian tail bound each failure has probability at most $e^{-\lambda_{n}}\text{,}$ so

$$\gamma(A^{c})\ \leq\ (n+1)\,e^{-\lambda_{n}}\ =\ \frac{1}{4}.$$

Step 3: two inseparable hypotheses. Let $P:=N(\mu_{0},4\sigma^{2}I_{n})\text{,}$ the law of $Y$ at the admitted pair $(\mu_{0},2\sigma)\text{,}$ and let

$$Q_{A}\ :=\ \frac{1}{\gamma(A)}\int_{A}N\bigl(\mu_{0}+w,\ \sigma^{2}I_{n}\bigr)\,d\gamma(w),$$

a mixture of laws at admitted pairs with noise level $\sigma\text{.}$ Splitting Equation S.6.14 over $A$ and $A^{c}$ gives $P=\gamma(A)Q_{A}+\gamma(A^{c})Q_{A^{c}}\text{,}$ so every event $B$ satisfies

$$\bigl|P(B)-Q_{A}(B)\bigr|\ =\ \gamma(A^{c})\,\bigl|Q_{A^{c}}(B)-Q_{A}(B)\bigr|\ \leq\ \frac{1}{4}.$$

Given $\widehat{s}\text{,}$ let $B:=\{\widehat{s}<\sqrt{2}\,\sigma\}\text{.}$ Then $Q_{A}(B^{c})+P(B)=1-[Q_{A}(B)-P(B)]\geq\tfrac{3}{4}\text{,}$ so at least one term is at least $\tfrac{3}{8}\text{.}$ If $P(B)\geq\tfrac{3}{8}\text{:}$ at $(\mu_{0},2\sigma)$ the event $B$ reads $\{\widehat{s}<s/\sqrt{2}\}\text{,}$ outside the window. If $Q_{A}(B^{c})\geq\tfrac{3}{8}\text{:}$ a mixture average is at least $\tfrac{3}{8}\text{,}$ so some $w\in A$ has $\mathbb{P}_{\mu_{0}+w,\,\sigma}(B^{c})\geq\tfrac{3}{8}\text{,}$ and at $(\mu_{0}+w,\sigma)$ the event $B^{c}$ reads $\{\widehat{s}\geq\sqrt{2}\,s\}\text{,}$ outside the window. A randomized estimator is handled by averaging the display over its internal randomness. ∎

The statements deferred from Section 4.3 are now all in place.

S.7 The Universal Upper Bound for Least Squares

This section proves the upper bound of Theorem 5, following the three blocks sketched in Section 5: the localized projection principle (Lemma S.7.1), the truth-localized width bound (5.10) (Lemma S.7.4), which rests on a noise-compatible choice of centers in the coded cover (Lemma S.7.3), and the closure of the exponent, for which Lemma S.7.5 supplies the height branch. The capped order-statistics bound of Lemma S.7.2 does the repeated work inside Lemma S.7.3.

Throughout, $Z\sim N(0,I_{n})$ is the noise, $\ell_{n}:=1+\log(en)\text{,}$ and, for an integer $2\leq k\leq n/C_{\star}\text{,}$ the construction of Proposition 3.1 is in force with the notation and of Section S.2: the working scale $\bar{\alpha}=k\overline{\delta}_{k}(T)\text{,}$ the nets $R_{j}$ with level weights $m_{j}=2^{j}$ and deepest-ancestor maps $a_{j}(\cdot)\text{,}$ and, for a signal $f\in\mathcal{F}_{1}(T)\text{,}$ the terminal cells $L$ with masses $\lambda_{L}$ and levels $m(L)\text{,}$ the collapse $g\text{,}$ the skeleton $g_{0}$ supported on $U\text{,}$ its rounding $\bar{g}_{0}\text{,}$ the exits $q_{L}\text{,}$ and the sampling variables $Z_{L}$ with selection weight $W\text{,}$ a notation fixed in the proof of Proposition 3.1; the unsubscripted $Z$ is always the noise. Root coordinates never contribute: every point of $\mathcal{F}_{V}(T)$ has root coordinate $V\text{,}$ so every difference below vanishes there. The Gaussian tail bound $\mathbb{P}(N(0,\tau^{2})\geq u)\leq e^{-u^{2}/(2\tau^{2})}\text{,}$ valid for $u\geq 0\text{,}$ and Gaussian concentration for Lipschitz functions (Borell, 1975; Cirel’son et al., 1976), namely $\mathbb{P}\bigl(F(Z)\geq\mathbb{E}F(Z)+u\bigr)\leq e^{-u^{2}/(2\Lambda^{2})}$ for $\Lambda$-Lipschitz $F:\mathbb{R}^{n}\to\mathbb{R}\text{,}$ are used without further comment.

The first lemma turns a bound on the localized Gaussian width into a risk bound for the projection. It is the fixed-point principle of Chatterjee’s analysis of least squares (Chatterjee, 2014), in the form quoted in Section 5; nothing in it refers to trees.

Lemma S.7.1 (Localized projection).

Let $C\subseteq\mathbb{R}^{N}$ be compact and convex, let $\mu\in C\text{,}$ let $Y=\mu+\sigma Z$ with $Z\sim N(0,I_{N})\text{,}$ and let $\widehat{\mu}:=\Pi_{C}(Y)$ be the Euclidean projection of $Y$ onto $C\text{.}$ Suppose that $t\geq\sigma$ and that for every $s\geq t$

$$\sigma\,\mathbb{E}\,\sup\bigl\{\langle Z,x-\mu\rangle:\ x\in C,\ \|x-\mu\|_{2}\leq s\bigr\}\ \leq\ \frac{s^{2}}{4}\,.$$

(S.7.1)

Then $\mathbb{E}\|\widehat{\mu}-\mu\|_{2}^{2}\leq 33\,t^{2}\text{.}$

Proof.

Write $\widehat{z}:=\widehat{\mu}-\mu$ and, for $s>0\text{,}$ let $X_{s}$ denote the supremum in Equation S.7.1, so that the hypothesis reads $\mathbb{E}X_{s}\leq s^{2}/(4\sigma)$ for $s\geq t\text{.}$ The projection is the nearest point of $C$ and $\mu\in C\text{,}$ so $\|Y-\widehat{\mu}\|_{2}^{2}\leq\|Y-\mu\|_{2}^{2}\text{;}$ expanding both sides at $Y=\mu+\sigma Z$ and cancelling $\sigma^{2}\|Z\|_{2}^{2}$ gives

$$\|\widehat{z}\|_{2}^{2}\ \leq\ 2\sigma\,\langle Z,\widehat{z}\rangle.$$

(S.7.2)

Fix $s\geq t$ and suppose $\|\widehat{z}\|_{2}\geq s\text{.}$ The point $\mu+s\widehat{z}/\|\widehat{z}\|_{2}$ lies in $C$ by convexity, on the segment from $\mu$ to $\widehat{\mu}\text{,}$ and within distance $s$ of $\mu\text{,}$ so

$$X_{s}\ \geq\ \frac{s}{\|\widehat{z}\|_{2}}\,\langle Z,\widehat{z}\rangle\ \geq\ \frac{s\,\|\widehat{z}\|_{2}}{2\sigma}\ \geq\ \frac{s^{2}}{2\sigma}\,,$$

the middle inequality by Equation S.7.2. As a function of $Z$ the supremum $X_{s}$ is $s$-Lipschitz, being a maximum of linear functionals with gradients of norm at most $s\text{,}$ so concentration and $\mathbb{E}X_{s}\leq s^{2}/(4\sigma)$ give

$$\mathbb{P}\bigl(\|\widehat{z}\|_{2}\geq s\bigr)\ \leq\ \mathbb{P}\Bigl(X_{s}\geq\mathbb{E}X_{s}+\frac{s^{2}}{4\sigma}\Bigr)\ \leq\ \exp\Bigl(-\frac{(s^{2}/4\sigma)^{2}}{2s^{2}}\Bigr)\ =\ e^{-s^{2}/(32\sigma^{2})}\qquad(s\geq t).$$

The layer-cake formula integrates the tail:

$$\mathbb{E}\|\widehat{z}\|_{2}^{2}\ \leq\ t^{2}+\int_{t}^{\infty}2s\,e^{-s^{2}/(32\sigma^{2})}\,ds\ =\ t^{2}+32\sigma^{2}e^{-t^{2}/(32\sigma^{2})}\ \leq\ t^{2}+32\sigma^{2}\ \leq\ 33\,t^{2},$$

the last step by $t\geq\sigma\text{.}$ ∎

For $C=\mathcal{F}_{V}(T)$ the expectation in Equation S.7.1 is the localized width $w_{\mu}(s)$ of Section 5, and the lemma is the principle stated there. Bounding $w_{\mu}$ requires expected maxima of many correlated Gaussian terms whose weights are capped individually and in total; the next lemma is the tool. For $a\in(0,1]$ and $y\in\mathbb{R}^{M}$ define

$$\mathfrak{M}_{a}(y)\ :=\ \sup\Bigl\{\,\sum_{i=1}^{M}w_{i}y_{i}:\ 0\leq w_{i}\leq a\ \text{ for all }i,\ \ \sum_{i=1}^{M}w_{i}\leq 1\Bigr\},$$

(S.7.3)

the largest weighted sum under a cap on each weight and on the total.

Lemma S.7.2 (Capped order statistics).

Let $G_{1},\dots,G_{M}$ be jointly Gaussian, each centered with variance at most $\tau^{2}\text{,}$ not necessarily independent, and let $a\in(0,1]\text{.}$ Then

$$\begin{gathered} \displaystyle\mathbb{E}\,\mathfrak{M}_{a}(G_{1},\dots,G_{M})\ \leq\ C\tau\sqrt{1+[\log(Ma)]_{+}}\,, \\ \displaystyle\mathbb{E}\,\mathfrak{M}_{a}\bigl(G_{1}^{2},\dots,G_{M}^{2}\bigr)\ \leq\ C\tau^{2}\bigl(1+[\log(Ma)]_{+}\bigr). \end{gathered}$$

Proof.

Write $G_{(1)}^{+}\geq G_{(2)}^{+}\geq\cdots$ for the decreasing rearrangement of the positive parts $[G_{i}]_{+}$ and $Q_{(1)}\geq Q_{(2)}\geq\cdots$ for that of the squares $G_{i}^{2}\text{,}$ and set $q:=\min\{\lceil 1/a\rceil,M\}\text{.}$

Reduction to order statistics. A feasible weight vector puts total weight at most $1\text{,}$ each entry at most $a\text{,}$ and gains nothing from coordinates where $G_{i}<0\text{;}$ moving weight from a smaller to a larger positive coordinate only increases the sum, so the supremum is attained by filling the largest positive coordinates to capacity $a\text{,}$ with at most one fractional entry. Since $aq\geq 1$ when $q=\lceil 1/a\rceil\text{,}$ while all $M$ coordinates are already covered when $q=M\text{,}$

$$\mathfrak{M}_{a}(G)\ \leq\ a\sum_{j\leq q}G_{(j)}^{+},\qquad\mathfrak{M}_{a}(G_{1}^{2},\dots,G_{M}^{2})\ \leq\ a\sum_{j\leq q}Q_{(j)}.$$

(S.7.4)

Tails of the order statistics. If $G_{(j)}^{+}\geq u$ for some $u>0\text{,}$ at least $j$ of the $G_{i}$ are at least $u\text{;}$ Markov’s inequality applied to the count, whose mean is at most $Me^{-u^{2}/(2\tau^{2})}$ by the Gaussian tail bound, gives

$$\mathbb{P}\bigl(G_{(j)}^{+}\geq u\bigr)\ \leq\ \min\Bigl\{1,\ \frac{M}{j}\,e^{-u^{2}/(2\tau^{2})}\Bigr\},\qquad\mathbb{P}\bigl(Q_{(j)}\geq u^{2}\bigr)\ \leq\ \min\Bigl\{1,\ \frac{2M}{j}\,e^{-u^{2}/(2\tau^{2})}\Bigr\},$$

the second by the same argument applied to $|G_{i}|\text{.}$ Splitting the first tail at $u_{j}\text{,}$ where $u_{j}^{2}:=2\tau^{2}\log(eM/j)\text{,}$ gives

$\mathbb{E}\,G_{(j)}^{+}\ \leq\ u_{j}+\frac{M}{j}\int_{u_{j}}^{\infty}e^{-u^{2}/(2\tau^{2})}\,du\\ \leq\ u_{j}+\frac{M}{j}\cdot\frac{\tau^{2}}{u_{j}}\,e^{-u_{j}^{2}/(2\tau^{2})}\ =\ u_{j}+\frac{\tau^{2}}{e\,u_{j}}\ \leq\ C\tau\sqrt{\log\frac{eM}{j}}\,,$

where the integrand is at most $(u/u_{j})\,e^{-u^{2}/(2\tau^{2})}\text{.}$ Splitting the second tail at $2\tau^{2}\log(2eM/j)$ gives

$$\mathbb{E}\,Q_{(j)}\ \leq\ 2\tau^{2}\log\frac{2eM}{j}+\frac{2\tau^{2}}{e}\ \leq\ C\tau^{2}\log\frac{eM}{j}\,.$$

Summation. Summing the first bound over $j\leq q$ and applying the Cauchy–Schwarz inequality to the average,

$$a\sum_{j\leq q}\mathbb{E}\,G_{(j)}^{+}\ \leq\ Ca\tau\,q\,\Bigl(\frac{1}{q}\sum_{j\leq q}\log\frac{eM}{j}\Bigr)^{1/2}\ \leq\ Ca\tau\,q\sqrt{2+\log(M/q)}\,,$$

using $\sum_{j\leq q}\log(eM/j)=q\log(eM)-\log q!\leq q\bigl(2+\log(M/q)\bigr)\text{,}$ by $\log q!\geq q\log q-q\text{.}$ Now $aq\leq a\lceil 1/a\rceil\leq 1+a\leq 2\text{;}$ and either $q=\lceil 1/a\rceil\text{,}$ in which case $M/q\leq Ma\text{,}$ or $q=M\text{,}$ in which case $\log(M/q)=0\text{.}$ In both cases the right side is at most $C\tau\sqrt{1+[\log(Ma)]_{+}}\text{,}$ which with Equation S.7.4 proves the first claim. The second is identical with $\tau^{2}\log(eM/j)$ in place of $\tau\sqrt{\log(eM/j)}\text{,}$ without the Cauchy–Schwarz step. ∎

The heart of the matter is the next lemma: for each signal and noise realization, take the best accurate member of the coded cover; the resulting pairing with the noise, maximized over the body, is small in expectation. Its proof prices the three residuals of the construction separately, and only the last one, the sampled exits, needs the noise-dependent choice.

Lemma S.7.3 (Noise-compatible centers).

Fix an integer $2\leq k\leq n/C_{\star}$ and let $\mathcal{D}_{k}$ be the coded cover at amplitude $V=1\text{.}$ For $f\in\mathcal{F}_{1}(T)$ and a noise realization $Z$ set

$$\mathfrak{r}_{Z}(f)\ :=\ \inf\Bigl\{\langle Z,\ f-\nu\rangle:\ \nu\in\mathcal{D}_{k},\ \|f-\nu\|_{2}^{2}\leq 51\,\overline{\delta}_{k}(T)\Bigr\}.$$

(S.7.5)

The infimum runs over a nonempty set for every $f$ and every $Z\text{,}$ and

$$\mathbb{E}\,\sup_{f\in\mathcal{F}_{1}(T)}\mathfrak{r}_{Z}(f)\ \leq\ C\Bigl[\sqrt{\bar{\alpha}}\,\log(2+\ell_{n})+\sqrt{\overline{\delta}_{k}(T)\,\ell_{n}}\,\Bigr].$$

(S.7.6)

Proof.

Fix $f$ and run the proof of Proposition 3.1 on it, retaining its objects. Each realization of the sampling defines the candidate $\nu:=\bar{g}_{0}+\sum_{L}Z_{L}q_{L}\text{,}$ and the error decomposes as there:

$$f-\nu\ =\ (f-g)\ +\ (g_{0}-\bar{g}_{0})\ +\ \sum_{L}(\lambda_{L}-Z_{L})\,q_{L}.$$

(S.7.7)

On the sampling event of that proof, call it $A\text{,}$ of probability at least $\tfrac{1}{4}\text{,}$ where $W\leq 2k$ and $\|\sum_{L}(Z_{L}-\lambda_{L})q_{L}\|_{2}^{2}\leq 8\bar{\alpha}/k\text{,}$ the candidate lies in $\mathcal{D}_{k}\text{,}$ and the three residuals assemble, as in Section S.2, to $\|f-\nu\|_{2}^{2}\leq 3(3+6+8)\bar{\alpha}/k=51\,\overline{\delta}_{k}(T)\text{:}$ the feasible set of Equation S.7.5 is nonempty, for every $f$ and irrespective of $Z\text{.}$ Pairing Equation S.7.7 with the noise and taking on $A$ the realization that minimizes the third term,

$$\mathfrak{r}_{Z}(f)\ \leq\ \langle Z,f-g\rangle+\langle Z,g_{0}-\bar{g}_{0}\rangle+\min_{A}\,\Bigl\langle Z,\sum_{L}(\lambda_{L}-Z_{L})q_{L}\Bigr\rangle.$$

(S.7.8)

The three terms are bounded uniformly over $f$ in turn; the first carries the main contribution, the second is a union bound over few subspaces, and the third is where choosing the center after the noise pays.

The collapse residual. Let $\tau(u)$ be the level of the terminal cell containing $u\text{,}$ so that $g=\sum_{u}\lambda_{u}\,p_{a_{\tau(u)}(u)}$ and the difference telescopes through the levels:

$$\begin{gathered} \displaystyle f-g\ =\ \sum_{u}\lambda_{u}\bigl(p_{u}-p_{a_{\tau(u)}(u)}\bigr), \\ \displaystyle p_{u}-p_{a_{\tau(u)}(u)}\ =\ \sum_{j=\tau(u)}^{J-1}\bigl(p_{a_{j+1}(u)}-p_{a_{j}(u)}\bigr)+\bigl(p_{u}-p_{a_{J}(u)}\bigr). \end{gathered}$$

Fix a level $j<J$ and group the contributions by the level-$(j{+}1)$ root: by Lemma S.2.2(3), $a_{j}(u)=a_{j}(b)$ whenever $a_{j+1}(u)=b\text{,}$ so the level-$j$ increment of every $u$ in the group of $b$ is the same vector $p_{b}-p_{a_{j}(b)}\text{,}$ of squared norm at most $d_{T}(a_{j}(b),b)\leq\bar{\alpha}/m_{j}$ by Lemma S.2.2(4). The group mass

$$\lambda_{b}\ :=\ \sum\bigl\{\lambda_{u}:\ a_{j+1}(u)=b,\ \tau(u)\leq j\bigr\}$$

is capped: each member of the group lies in the level-$(j{+}1)$ cell of $b$ and in its own terminal cell, which is coarser and hence contains that whole cell; two terminal cells with a common vertex coincide, the terminal cells being a partition, so the group shares one terminal cell, stopped at a level at most $j<J$ and hence light, whence $\lambda_{b}\leq m_{j}/k$ by the monotonicity of the level weights. The group masses are nonnegative and total at most one, each $\lambda_{u}$ entering at most one group, so, with $M_{j}\leq\min\{n,\,|R_{j+1}|\}\leq\min\{n,\,2ke^{2m_{j}}\}$ possible groups (Lemma S.2.1(3)),

$$\mathbb{E}\,\sup_{f}\,\sum_{b}\lambda_{b}\,\langle Z,p_{b}-p_{a_{j}(b)}\rangle\leq\mathbb{E}\,\mathfrak{M}_{m_{j}/k}\Bigl(\bigl(\langle Z,p_{b}-p_{a_{j}(b)}\rangle\bigr)_{b}\Bigr)\leq C\sqrt{\bar{\alpha}}\,\min\bigl\{1,\sqrt{\ell_{n}/m_{j}}\bigr\},$$

by Lemma S.7.2 with variance bound $\bar{\alpha}/m_{j}$ and cap $m_{j}/k\text{:}$ its logarithmic factor $1+[\log(M_{j}m_{j}/k)]_{+}$ is at most $\min\{5m_{j},\,\ell_{n}\}\text{,}$ being at most $1+\log 2+\log m_{j}+2m_{j}\leq 5m_{j}$ from $M_{j}m_{j}/k\leq 2m_{j}e^{2m_{j}}\text{,}$ and at most $1+\log n\leq\ell_{n}$ from $M_{j}m_{j}/k\leq n\text{.}$ Summing the dyadic levels: those with $m_{j}\leq\ell_{n}$ number at most $C\log(2+\ell_{n})$ and contribute $C\sqrt{\bar{\alpha}}$ each, while over the rest $\sqrt{\ell_{n}/m_{j}}$ is geometrically decaying from below one, and sums to a constant. The final increments $p_{u}-p_{a_{J}(u)}$ number at most $n$ and have squared norms at most $\bar{\alpha}/m_{J}\leq 2\overline{\delta}_{k}(T)\text{,}$ and their mass-weighted sum is a feasible value of $\mathfrak{M}_{1}\text{;}$ so Lemma S.7.2 with cap $1$ and variance bound $2\overline{\delta}_{k}(T)$ bounds its expected supremum by $C\sqrt{\overline{\delta}_{k}(T)}\,\sqrt{1+\log n}\leq C\sqrt{\overline{\delta}_{k}(T)\,\ell_{n}}\text{.}$ Altogether

$$\mathbb{E}\,\sup_{f}\ \langle Z,\ f-g\rangle\ \leq\ C\Bigl[\sqrt{\bar{\alpha}}\,\log(2+\ell_{n})+\sqrt{\overline{\delta}_{k}(T)\,\ell_{n}}\,\Bigr].$$

(S.7.9)

The rounding residual. The difference $g_{0}-\bar{g}_{0}$ has squared norm at most $6\bar{\alpha}/k\text{,}$ by the constant recorded after Lemma S.2.3, and lies in the span of $\{p_{h}:h\in U\}\text{.}$ The set $U\setminus R_{0}$ consists of heavy roots, of total activation charge below $2k$ by Lemma S.2.3(3); it is therefore one of the charged parts counted in the proof of Lemma S.2.5, so at most $e^{19k}$ sets $U\text{,}$ hence at most $e^{19k}$ subspaces $V_{U}:=\operatorname{span}\{p_{h}:h\in U\}\text{,}$ occur as $f$ varies. Each has dimension $|U|\leq|R_{0}|+k\leq ek+k\leq 4k\text{,}$ since a heavy root outside $R_{0}$ carries charge at least $m_{1}=2\text{.}$ For any fixed $U\text{,}$

$$\sup\bigl\{\langle Z,z\rangle:\ z\in V_{U},\ \|z\|_{2}^{2}\leq 6\bar{\alpha}/k\bigr\}\ =\ \sqrt{6\bar{\alpha}/k}\,\bigl\|P_{V_{U}}Z\bigr\|_{2},$$

and $\|P_{V_{U}}Z\|_{2}$ is a $1$-Lipschitz function of $Z$ with mean at most $\sqrt{\dim V_{U}}\leq 2\sqrt{k}\text{.}$ Concentration and a union bound give, for $u\geq 0\text{,}$ $\mathbb{P}(\max_{U}\|P_{V_{U}}Z\|_{2}\geq 2\sqrt{k}+\sqrt{38k}+u)\leq e^{19k}e^{-(\sqrt{38k}+u)^{2}/2}\leq e^{-u^{2}/2}\text{,}$ so $\mathbb{E}\max_{U}\|P_{V_{U}}Z\|_{2}\leq 2\sqrt{k}+\sqrt{38k}+2\leq C\sqrt{k}$ and

$$\mathbb{E}\,\sup_{f}\ \langle Z,\ g_{0}-\bar{g}_{0}\rangle\ \leq\ \sqrt{6\bar{\alpha}/k}\cdot C\sqrt{k}\ =\ C\sqrt{\bar{\alpha}}\,.$$

(S.7.10)

The exit pairing. Condition on $Z$ and on $f\text{,}$ so that only the sampling is random, and write $X:=\langle Z,\sum_{L}(\lambda_{L}-Z_{L})q_{L}\rangle\text{.}$ The sampling variables are independent with $\mathbb{E}Z_{L}=\lambda_{L}$ and $\operatorname{Var}(Z_{L})\leq\lambda_{L}m(L)/k\text{,}$ so $X$ is centered with

$$\mathbb{E}\,X^{2}\ =\ \sum_{L}\operatorname{Var}(Z_{L})\,\langle Z,q_{L}\rangle^{2}\ \leq\ \sum_{L}\lambda_{L}\,\frac{m(L)}{k}\,\langle Z,q_{L}\rangle^{2}\ =:\ S_{f}(Z)^{2}.$$

Centeredness and the Cauchy–Schwarz inequality give

$$\mathbb{E}\bigl[X\ \big|\ A\bigr]\ =\ -\frac{\mathbb{E}\bigl[X\,\mathbf{1}_{A^{c}}\bigr]}{\mathbb{P}(A)}\ \leq\ \frac{(\mathbb{E}X^{2})^{1/2}\,\mathbb{P}(A^{c})^{1/2}}{\mathbb{P}(A)}\ \leq\ 4\,S_{f}(Z),$$

and some realization in $A$ is no larger than this conditional expectation. Hence the third term of Equation S.7.8 is at most $4S_{f}(Z)\text{,}$ and it remains to prove

$$\mathbb{E}\,\sup_{f}\ S_{f}(Z)^{2}\ \leq\ C\bar{\alpha},$$

(S.7.11)

which gives $\mathbb{E}\sup_{f}S_{f}(Z)\leq\sqrt{\mathbb{E}\sup_{f}S_{f}(Z)^{2}}\leq C\sqrt{\bar{\alpha}}$ by Jensen’s inequality. An exit $q_{L}$ is determined by the root $a_{L}$ of its cell: its other endpoint is the root of the parent cell (Lemma S.2.2(3)), so at level $m$ the possible exits number at most $\min\{n,2ke^{m}\}\text{,}$ with $\|q_{L}\|_{2}^{2}\leq 2\bar{\alpha}/m$ by the parent cell’s radius. Split $S_{f}(Z)^{2}$ by levels. At a level $m\leq\ell_{n}\text{,}$ the weights $\lambda_{L}\leq m/k$ are again capped with total at most one, so Lemma S.7.2, applied to the squares with variance bound $2\bar{\alpha}/m$ and cap $m/k\text{,}$ bounds the expected supremum of the level’s sum by $C(2\bar{\alpha}/m)\bigl(1+[\log(2me^{m})]_{+}\bigr)\leq C\bar{\alpha}\text{,}$ since $1+\log(2me^{m})\leq 4m\text{;}$ with the prefactor $m/k$ the level contributes $C\bar{\alpha}\,m/k\text{,}$ and the dyadic sum over $m<k$ is at most $C\bar{\alpha}\text{.}$ At the levels $m>\ell_{n}\text{,}$ the prefactor obeys $m/k\leq 1$ and the weights total at most one, so the combined contribution is a feasible value of $\mathfrak{M}_{1}$ over the possible exits at these levels, at most $n(1+\log_{2}k)$ vectors of variance at most $2\bar{\alpha}/\ell_{n}\text{;}$ Lemma S.7.2 with cap $1\text{,}$ applied to the squares, bounds its expectation by $C(\bar{\alpha}/\ell_{n})\bigl(1+\log(n(1+\log_{2}k))\bigr)\leq C\bar{\alpha}\text{,}$ since $1+\log\bigl(n(1+\log_{2}k)\bigr)\leq 2\ell_{n}\text{.}$ This proves Equation S.7.11.

Combining Equation S.7.8 with Equations S.7.9, S.7.10 and S.7.11,

$$\mathbb{E}\,\sup_{f}\mathfrak{r}_{Z}(f)\ \leq\ C\Bigl[\sqrt{\bar{\alpha}}\,\log(2+\ell_{n})+\sqrt{\overline{\delta}_{k}(T)\,\ell_{n}}\,\Bigr]+C\sqrt{\bar{\alpha}}+4\,C\sqrt{\bar{\alpha}}\,,$$

and $\sqrt{\bar{\alpha}}\leq\sqrt{\bar{\alpha}}\log(2+\ell_{n})$ absorbs the last two terms into the first. ∎

The width bound follows by recentering the local ball at a deterministic member of the cover.

Lemma S.7.4 (Truth-localized width).

For every integer $2\leq k\leq n/C_{\star}\text{,}$ every $\mu\in\mathcal{F}_{V}(T)\text{,}$ and every $t>0\text{,}$

$$w_{\mu}(t)\ \leq\ C\,\bigl(t+V\sqrt{\overline{\delta}_{k}(T)}\,\bigr)\sqrt{k}\ +\ C\,V\Bigl[\sqrt{k\overline{\delta}_{k}(T)}\,\log(2+\ell_{n})+\sqrt{\overline{\delta}_{k}(T)\,\ell_{n}}\,\Bigr],$$

with $w_{\mu}$ the localized Gaussian width defined in Section 5; this is the display (5.10) there.

Proof.

Abbreviate $\overline{\delta}:=\overline{\delta}_{k}(T)\text{.}$ Amplitudes rescale: for $\theta\in\mathcal{F}_{V}(T)$ the normalized signal $\theta/V$ lies in $\mathcal{F}_{1}(T)\text{,}$ and multiplying a member of the unit-amplitude cover by $V$ gives a member of $\mathcal{D}_{k}$ at amplitude $V\text{.}$ The feasible set of Equation S.7.5 does not depend on $Z$ and is never empty, so fix $\nu_{\mu}\text{,}$ $V$ times one of its members at $\mu/V\text{:}$ then $\|\mu-\nu_{\mu}\|_{2}\leq V\sqrt{51\overline{\delta}}$ and $\nu_{\mu}$ is deterministic. For each $\theta$ with $\|\theta-\mu\|_{2}\leq t$ and each realization of $Z\text{,}$ let $\nu_{\theta}$ be $V$ times a minimizer of Equation S.7.5 at $\theta/V\text{,}$ so that $\|\theta-\nu_{\theta}\|_{2}\leq V\sqrt{51\overline{\delta}}$ and $\langle Z,\theta-\nu_{\theta}\rangle=V\,\mathfrak{r}_{Z}(\theta/V)\text{.}$ Decompose

$$\langle Z,\ \theta-\mu\rangle\ =\ \langle Z,\ \nu_{\theta}-\nu_{\mu}\rangle+\langle Z,\ \theta-\nu_{\theta}\rangle-\langle Z,\ \mu-\nu_{\mu}\rangle.$$

The last term does not depend on $\theta$ and has expectation zero, $\nu_{\mu}$ being deterministic, so it drops from the expected supremum. The middle term is at most $V\sup_{f\in\mathcal{F}_{1}(T)}\mathfrak{r}_{Z}(f)\text{,}$ whose expectation Lemma S.7.3 bounds by the second group of the claim. For the first term, the triangle inequality confines every center met to a deterministic ball:

$$\|\nu_{\theta}-\nu_{\mu}\|_{2}\ \leq\ \|\nu_{\theta}-\theta\|_{2}+\|\theta-\mu\|_{2}+\|\mu-\nu_{\mu}\|_{2}\ \leq\ t+2V\sqrt{51\overline{\delta}}\ =:\ \rho,$$

so the first term is at most the maximum of $\langle Z,\nu-\nu_{\mu}\rangle$ over $\{\nu\in\mathcal{D}_{k}:\|\nu-\nu_{\mu}\|_{2}\leq\rho\}\text{,}$ a fixed set of at most $e^{C_{1}k}$ centered Gaussians of standard deviation at most $\rho\text{.}$ The maximum is a feasible value of $\mathfrak{M}_{1}\text{,}$ so Lemma S.7.2 with cap $1$ bounds its expectation by $C\rho\sqrt{1+C_{1}k}\leq C(t+V\sqrt{\overline{\delta}}\,)\sqrt{k}\text{,}$ the first group of the claim. ∎

The second comparison branch of Section 5 rests on the profile remembering the height of the tree.

Lemma S.7.5 (Height floor).

For every finite rooted tree and every integer $1\leq k\leq n/C_{\star}\text{,}$

$$\delta_{k}(T)\ \geq\ \frac{\log 2}{8}\cdot\frac{H_{T}}{k^{2}}\,.$$

Proof.

Write $h:=h_{T}\text{,}$ so that $H_{T}\leq 2h\text{,}$ and fix a root-to-leaf path with $h$ edges; every ancestor of a vertex of the path lies on the path, so ancestor covering restricted to it is interval covering of its $h+1$ vertices, as on the path of Lemma S.4.1.

If $h\geq 4k\text{,}$ set $q:=\lfloor h/(2k)\rfloor-1\geq 1\text{.}$ A center covers at most $q+1$ consecutive path vertices, so

$$N^{\uparrow}_{T}(q)\ \geq\ \frac{h+1}{q+1}\ =\ \frac{h+1}{\lfloor h/(2k)\rfloor}\ \geq\ 2k,$$

whence $[\log(N^{\uparrow}_{T}(q)/k)]_{+}\geq\log 2$ and the term of the discrete formula (2.3) at $q$ is at least $(q+1)\log 2\text{.}$ Since $q+1=\lfloor h/(2k)\rfloor>h/(2k)-1\geq h/(4k)\text{,}$

$$\delta_{k}\ \geq\ \frac{(q+1)\log 2}{k}\ \geq\ \frac{h\log 2}{4k^{2}}\ \geq\ \frac{H_{T}\log 2}{8k^{2}}\,.$$

If $h<4k\text{,}$ the term of (2.3) at $q=0$ is $\min\{k,[\log(n/k)]_{+}\}\geq\min\{k,2\}\geq 1\text{,}$ since $n/k\geq C_{\star}=e^{2}\text{;}$ hence $\delta_{k}\geq 1/k>H_{T}\log 2/(8k^{2})\text{,}$ using $H_{T}\leq 2h<8k$ and $\log 2<1\text{.}$ ∎

The blocks are in place, and it remains to assemble them along the case analysis of Section 5.

Completion of Theorem 5, upper bound.

The skeleton, the meeting of the two branches at $k_{0}=\ell_{n}^{1/5}\text{,}$ and the resulting bound $C(1+\log(en))^{2/5}$ are in Section 5; deferred here are the fixed-point verification behind (5.11), the height branch (5.12), and the endpoint cases.

The interior case: the width branch (5.11). Let $2\leq k_{0}\leq K:=\lfloor n/C_{\star}\rfloor\text{,}$ so that the crossing clause fired at $k_{0}\text{:}$ $V^{2}\delta_{k_{0}}\leq A_{0}\sigma^{2}k_{0}\text{,}$ and $R^{*}_{T}\asymp\sigma^{2}k_{0}$ by the interior case of Section 3.3. Lemma 2.3 gives $\overline{\delta}_{k_{0}}\leq 2\delta_{k_{0}}\text{,}$ so

$$V\sqrt{\overline{\delta}_{k_{0}}}\ \leq\ \sqrt{2A_{0}}\,\sigma\sqrt{k_{0}}\,,\qquad V\sqrt{k_{0}\overline{\delta}_{k_{0}}}\ \leq\ \sqrt{2A_{0}}\,\sigma\,k_{0}\,,$$

and Lemma S.7.4 at $k=k_{0}$ yields, for every $s>0\text{,}$

$$w_{\mu}(s)\ \leq\ Cs\sqrt{k_{0}}\ +\ C\sigma B,\qquad B\ :=\ k_{0}\log(2+\ell_{n})+\sqrt{k_{0}\,\ell_{n}}\,,$$

with one universal constant $C\text{,}$ uniformly over $\mu\in\mathcal{F}_{V}(T)\text{:}$ the term $CV\sqrt{\overline{\delta}_{k_{0}}}\,\sqrt{k_{0}}\leq C^{\prime}\sigma k_{0}$ from the first group, and both terms of the second group after the two displayed substitutions, are absorbed into $C\sigma B\text{,}$ since $k_{0}\leq B/\log 2\text{.}$ Set $t^{2}:=C_{\mathrm{loc}}\sigma^{2}B$ with $C_{\mathrm{loc}}:=64C^{2}/\log 2+8C+1\text{.}$ Then $t\geq\sigma\text{,}$ since $B\geq k_{0}\log 2\geq 2\log 2>1\text{;}$ and for every $s\geq t\text{,}$

$$\sigma\,w_{\mu}(s)\ \leq\ C\sigma s\sqrt{k_{0}}+C\sigma^{2}B\ \leq\ \frac{s^{2}}{8}+\frac{s^{2}}{8}\ =\ \frac{s^{2}}{4}\,,$$

the first term because $s\geq t\geq\sqrt{C_{\mathrm{loc}}k_{0}\log 2}\,\sigma\geq 8C\sigma\sqrt{k_{0}}\text{,}$ and the second because $C\sigma^{2}B=Ct^{2}/C_{\mathrm{loc}}\leq t^{2}/8\leq s^{2}/8\text{.}$ Lemma S.7.1 applies at every $\mu$ and gives

$$R^{\mathrm{worst}}_{\mathrm{LSE}}(T,V,\sigma)\ \leq\ 33\,t^{2}\ =\ 33\,C_{\mathrm{loc}}\,\sigma^{2}\bigl[k_{0}\log(2+\ell_{n})+\sqrt{k_{0}\,\ell_{n}}\,\bigr];$$

dividing by $R^{*}_{T}\asymp\sigma^{2}k_{0}$ proves (5.11).

The interior case: the height branch (5.12). Still for $2\leq k_{0}\leq K\text{,}$ Lemma S.7.5 applies at $k_{0}$ and combines with the crossing:

$$\frac{\log 2}{8}\cdot\frac{V^{2}H_{T}}{k_{0}^{2}}\ \leq\ V^{2}\delta_{k_{0}}\ \leq\ A_{0}\sigma^{2}k_{0},\qquad\text{so}\qquad V^{2}H_{T}\ \leq\ \frac{8A_{0}}{\log 2}\,\sigma^{2}k_{0}^{3}.$$

Both $\mu$ and its projection lie in the body, so $\|\widehat{\mu}_{\mathrm{LSE}}-\mu\|_{2}^{2}\leq\operatorname{diam}^{2}\mathcal{F}_{V}(T)=V^{2}H_{T}$ pathwise, and dividing by $R^{*}_{T}\asymp\sigma^{2}k_{0}$ proves (5.12).

The endpoints. The projection onto a convex set is a contraction, so $\|\widehat{\mu}_{\mathrm{LSE}}-\mu\|_{2}=\|\Pi_{\mathcal{F}_{V}(T)}(Y)-\Pi_{\mathcal{F}_{V}(T)}(\mu)\|_{2}\leq\|Y-\mu\|_{2}$ and $R^{\mathrm{worst}}_{\mathrm{LSE}}\leq\sigma^{2}n$ always; together with the diameter,

$$R^{\mathrm{worst}}_{\mathrm{LSE}}(T,V,\sigma)\ \leq\ \min\bigl\{V^{2}H_{T},\ \sigma^{2}n\bigr\}.$$

(S.7.12)

If $k_{0}=K+1$ with $n\geq C_{\star}\text{,}$ the dimension case of Section 3.3 gives $R^{*}_{T}\asymp\sigma^{2}n\text{,}$ and Equation S.7.12 bounds the ratio by a constant. If $k_{0}=1\text{,}$ the two-point bound (3.8) gives $R^{*}_{T}\geq c\min\{V^{2}H_{T},\sigma^{2}\}$ for $n\geq 2\text{,}$ and there are two subcases. When $n<C_{\star}\text{,}$ the right side of Equation S.7.12 is at most $C_{\star}\min\{V^{2}H_{T},\sigma^{2}\}\text{:}$ its second branch is $\sigma^{2}n\leq C_{\star}\sigma^{2}\text{,}$ and its first is unchanged. When the crossing clause fired at $k=1\text{,}$ the second case of Section 3.3 gives $V^{2}H_{T}\leq 2V^{2}h_{T}\leq(2A_{0}/\log 2)\,\sigma^{2}\text{,}$ so the right side of Equation S.7.12 is again at most $C\min\{V^{2}H_{T},\sigma^{2}\}\text{:}$ if $V^{2}H_{T}\leq\sigma^{2}$ it equals the first branch, and otherwise it is at most $C\sigma^{2}\text{.}$ In both subcases $R^{\mathrm{worst}}_{\mathrm{LSE}}\leq C\,R^{*}_{T}\text{.}$ The three cases are exhaustive: $k_{0}=K+1$ with $n<C_{\star}$ means $K=0$ and $k_{0}=1\text{,}$ which the last case covers. Together with the two interior branches and the case analysis of Section 5, the upper bound of Theorem 5 is proved; with the lower bound established there, so is the theorem. ∎

References

Abowd et al. (2022) J. M. Abowd, R. Ashmead, R. Cumings-Menon, et al. The 2020 Census disclosure avoidance system TopDown algorithm. Harv. Data Sci. Rev.. Note: Special Issue 2 Cited by: §1.
Aolaritei et al. (2025) L. Aolaritei, M. I. Jordan, R. Pathak, and A. Ulichney Revisiting mean estimation over $\ell_{p}$ balls: is the MLE optimal?. Note: arXiv:2506.10354 Cited by: §1.3.
Barron and Cover (1991) A. R. Barron and T. M. Cover Minimum complexity density estimation. IEEE Trans. Inf. Theory 37 (4), pp. 1034–1054. Cited by: §1.3.
Bellec (2018) P. C. Bellec Sharp oracle inequalities for least squares estimators in shape restricted regression. Ann. Statist. 46 (2), pp. 745–780. Cited by: §1.3.
Benabbas et al. (2011) S. Benabbas, H. C. Lee, J. Oren, and Y. Ye Efficient sum-based hierarchical smoothing under $\ell_{1}$-norm. Note: arXiv:1108.1751 Cited by: §1.3.
Birgé and Massart (2001) L. Birgé and P. Massart Gaussian model selection. J. Eur. Math. Soc. 3 (3), pp. 203–268. Cited by: §1.1, §1, §S.3, §4.3.
Borell (1975) C. Borell The Brunn-Minkowski inequality in Gauss space. Invent. Math. 30 (2), pp. 207–216. Cited by: §S.7.
Boucheron et al. (2013) S. Boucheron, G. Lugosi, and P. Massart Concentration inequalities: a nonasymptotic theory of independence. Oxford University Press. Cited by: §4.1.
Chatterjee et al. (2015) S. Chatterjee, A. Guntuboyina, and B. Sen On risk bounds in isotonic and other shape restricted regression problems. Ann. Statist. 43 (4), pp. 1774–1800. Cited by: §1.3, §1.
Chatterjee and Lafferty (2018) S. Chatterjee and J. Lafferty Denoising flows on trees. IEEE Trans. Inf. Theory 64 (3), pp. 1767–1783. Cited by: §1.1, §1.1, §1.1, §1.1, §1.2, §1.2, §1.3, §1, §S.1, §S.1, §2.1.
Chatterjee (2014) S. Chatterjee A new perspective on least squares under convex constraint. Ann. Statist. 42 (6), pp. 2340–2381. Cited by: §1.1, §1.2, §1.3, §5, §S.7.
Cirel’son et al. (1976) B. S. Cirel’son, I. A. Ibragimov, and V. N. Sudakov Norms of Gaussian sample functions. In Proceedings of the Third Japan-USSR Symposium on Probability Theory, Lecture Notes in Mathematics, Vol. 550, pp. 20–41. Cited by: §S.7.
Dalalyan and Tsybakov (2008) A. S. Dalalyan and A. B. Tsybakov Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity. Mach. Learn. 72, pp. 39–61. Cited by: §1.3.
Darwiche (2003) A. Darwiche A differential approach to inference in Bayesian networks. J. ACM 50 (3), pp. 280–305. Cited by: §1.3.
Donoho and Johnstone (1994) D. L. Donoho and I. M. Johnstone Minimax risk over $\ell_{p}$-balls for $\ell_{q}$-error. Probab. Theory Related Fields 99 (2), pp. 277–303. Cited by: §1.1, §1.
Giraud (2008) C. Giraud Mixing least-squares estimators when the variance is unknown. Bernoulli 14 (4), pp. 1089–1107. Cited by: §1.3.
Graham et al. (1982) S. L. Graham, P. B. Kessler, and M. K. McKusick Gprof: a call graph execution profiler. SIGPLAN Not. 17 (6), pp. 120–126. Cited by: §1.
Guntuboyina and Sen (2018) A. Guntuboyina and B. Sen Nonparametric shape-restricted regression. Statist. Sci. 33 (4), pp. 568–594. Cited by: §1.3.
Han et al. (2019) Q. Han, T. Wang, S. Chatterjee, and R. J. Samworth Isotonic regression in general dimensions. Ann. Statist. 47 (5), pp. 2440–2471. Cited by: §1.3.
Hay et al. (2010) M. Hay, V. Rastogi, G. Miklau, and D. Suciu Boosting the accuracy of differentially private histograms through consistency. Proc. VLDB Endow. 3 (1), pp. 1021–1032. Cited by: §1.
Kur et al. (2024) G. Kur, F. Gao, A. Guntuboyina, and B. Sen Convex regression in multidimensions: suboptimality of least squares estimators. Ann. Statist. 52 (6), pp. 2791–2815. Cited by: §1.1, §1.3.
Kyng et al. (2015) R. Kyng, A. Rao, and S. Sachdeva Fast, provable algorithms for isotonic regression in all $\ell_{p}$-norms. In Adv. Neural Inf. Process. Syst. 28, pp. 2719–2727. Cited by: §1.3.
Leung and Barron (2006) G. Leung and A. R. Barron Information theory and mixing least-squares regressions. IEEE Trans. Inf. Theory 52 (8), pp. 3396–3410. Cited by: §1.3, §4.1.
Lifshits and Linde (2011a) M. Lifshits and W. Linde Compactness properties of weighted summation operators on trees. Studia Math. 202 (1), pp. 17–47. Cited by: §1.3.
Lifshits and Linde (2011b) M. Lifshits and W. Linde Compactness properties of weighted summation operators on trees—the critical case. Studia Math. 206 (1), pp. 75–96. Cited by: §1.3.
Luss and Rosset (2017) R. Luss and S. Rosset Bounded isotonic regression. Electron. J. Stat. 11 (2), pp. 4488–4514. Cited by: §1.3.
Massart (2007) P. Massart Concentration inequalities and model selection. Lecture Notes in Mathematics, Vol. 1896, Springer. Cited by: §S.3.
Neykov (2023) M. Neykov On the minimax rate of the Gaussian sequence model under bounded convex constraints. IEEE Trans. Inf. Theory 69 (2), pp. 1244–1260. Cited by: §1.1, §1.3, §1.
Neykov (2026) M. Neykov Fast near-optimal estimation over symmetric norm balls. Note: arXiv:2606.01554 Cited by: §1.3.
Prasadan and Neykov (2025) A. Prasadan and M. Neykov Some facts about the optimality of the LSE in the Gaussian sequence model with convex constraint. IEEE Trans. Inf. Theory 71 (11), pp. 8928–8958. Cited by: §1.3.
Rigollet and Tsybakov (2011) P. Rigollet and A. B. Tsybakov Exponential screening and optimal rates of sparse estimation. Ann. Statist. 39 (2), pp. 731–771. Cited by: §1.3.
Robertson et al. (1988) T. Robertson, F. T. Wright, and R. L. Dykstra Order restricted statistical inference. Wiley. Cited by: §1.3.
Stein (1981) C. M. Stein Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 (6), pp. 1135–1151. Cited by: §4.1, §S.5.
Tsybakov (2009) A. B. Tsybakov Introduction to nonparametric estimation. Springer. Cited by: §3.2, §S.3.
Vaškevičius and Zhivotovskiy (2023) T. Vaškevičius and N. Zhivotovskiy Suboptimality of constrained least squares and improvements via non-linear predictors. Bernoulli 29 (1), pp. 473–495. Cited by: §1.3.
von zur Gathen and Gerhard (2013) J. von zur Gathen and J. Gerhard Modern computer algebra. 3rd edition, Cambridge Univ. Press. Cited by: §2.1, §S.5.
Wei et al. (2019) Y. Wei, M. J. Wainwright, and A. Guntuboyina The geometry of hypothesis testing over convex cones: generalized likelihood ratio tests and minimax radii. Ann. Statist. 47 (2), pp. 994–1024. Cited by: §1.3.
Willems et al. (1995) F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens The context-tree weighting method: basic properties. IEEE Trans. Inf. Theory 41 (3), pp. 653–664. Cited by: §1.3.
Yang and Barron (1999) Y. Yang and A. Barron Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27 (5), pp. 1564–1599. Cited by: §1.1, §1.3.
Zhang (2002) C. Zhang Risk bounds in isotonic regression. Ann. Statist. 30 (2), pp. 528–555. Cited by: §1.1, §1.3, §1.
Zhang (2013) L. Zhang Nearly optimal minimax estimator for high-dimensional sparse linear regression. Ann. Statist. 41 (4), pp. 2149–2175. Cited by: §1.3.

The Price of Order in the Logarithmic Method

Sun, 26 Jul 2026 00:00:00 +0000

1 Introduction

A classical theme in dynamic data structures is to obtain dynamism by rebuilding static structures in batches. The logarithmic method of Bentley and Saxe keeps several static structures of different ages and merges them as updates arrive [BS80]; lower-bound and structural versions go back to Mehlhorn and Overmars [MEH81, OVE83]. This is also a natural abstraction of write-optimized ordered storage. The LSM-tree was introduced to make high-rate updates sequential and cheap [OCG+96]; sorted-run architectures appear in large-scale storage systems and key-value stores such as Bigtable, Cassandra, MyRocks/RocksDB, WiscKey, and PebblesDB [CDG+08, LM10, MDL20, LPG+17, RKC+17], and in cache-oblivious write-optimized search structures [BFF+07].

The broad area is active because write optimization creates a persistent tension between update cost, point lookup cost, range-query cost, and space. The systems literature formulates this tension explicitly through the RUM tradeoff, optimizes merge policies and filter allocation for point lookups, and designs range filters or range-query mechanisms for LSM-style stores [AKM+16, DAI17, DI18, MLI25, ZLL+18, LCK+20, ZCW+21]. From a theory point of view, however, these systems raise a more basic question: what does the logarithmic method itself do to the complexity of ordered-search queries?

At a fixed time, a logarithmic-method structure is not one search tree. It is a set of independently built sorted components. A point lookup can often be certified locally in each component, for instance by a hash table or filter. An exact ordered query is different: predecessor, rank, and exact range queries must know where the query key, or the interval endpoints, fall inside the relevant components. Thus the issue is not only the number of components, but also what kind of information the query must recover inside each component.

Question. For each query family $Q$, what is the optimal product of amortized write cost and worst-case read cost for a logarithmic-method ordered set? Equivalently, if $E$ is the total amount of data written and $\widehat{R}_{Q}$ is the worst read cost for $Q$, we study

$$P_{Q}(n)=\inf_{\mathcal{A}}\left(\frac{E_{\mathcal{A}}(n)}{n}\right)\widehat{R}_{\mathcal{A},Q}(n),$$

and throughout the paper we report the unnormalized product $E\widehat{R}_{Q}$.

This question is not answered by the closest existing lines of work. Classical static-to-dynamic transformations analyze the overhead of maintaining many static structures, but do not separate query families that all have logarithmic static search cost [BS80, MEH81, OVE83, MRY+21]. LSM and write-optimized-storage work optimizes concrete tradeoffs for point and range queries, but does not give a matching query-by-query lower-bound classification under a single merge history [AKM+16, DAI17, DI18, ZLL+18, LCK+20, ZCW+21]. Fractional cascading explains how repeated searches across related catalogs can be shared [CG86, MN90, AFS21], but assumes that the cross-catalog links needed by the search are actually available. Lower bounds in cell-probe and external-memory models study different access regimes, where memory cells or blocks can be probed by address [FS89, BF03, LAR12, PĂT11]. Our focus is the restricted but natural merge-stack regime in which components are written by sequential merges and later read once in chronological order.

The main obstruction is easy to see in a small example. Suppose the current set is stored in three sorted components $C_{1},C_{2},C_{3}$, scanned in that order. For membership of a key $x$, each component only has to certify whether $x$ occurs there. For predecessor or rank of $x$, the query must learn the local gap of $x$ inside each $C_{i}$. A search in $C_{1}$ does not reveal the gap in $C_{2}$ or $C_{3}$ unless some cross-component search shortcut has already been materialized. The natural algorithmic objection is fractional cascading: repeated searches in many sorted catalogs are precisely what fractional cascading was designed to remove [CG86], and dynamic variants maintain such links under updates [MN90, AFS21].

In a materialized one-way merge stack, however, the shortcut has the wrong time direction. The query scans older components before younger ones. A bridge into a younger component would have to be encountered in an older component, but the younger component did not exist when the older one was written. Rewriting the older component later changes its birth time and moves it behind the younger one in the scan. Thus the components remain independent sorted catalogs for the purpose of exact order localization.

We formalize this obstruction in a merge-stack model. An insertion-only ordered set is stored as immutable sorted components. Updates append singleton components and rebuild suffixes by sequential merges. Queries make one forward pass over the live components, from old to young, and cannot return to a component after passing it. Our main theorem is for abstract keys, which can be compared and hashed but carry no word-RAM coordinate revealing their rank, and for strongly materialized components, which are self-contained encodings of their own keys rather than directories for other components. These assumptions are the clean setting in which the missing fractional cascade is exposed. The select query gives the forward scan even less information: it supplies only a rank $K$, not a target key value, and the threshold key may be in a component that has already passed under the head.

1.1 Our Results

The main result is a tight, query-sensitive classification of the logarithmic method in this model.

Theorem 1.1 (Main Classification).

In the strongly materialized merge-stack model over abstract keys, with oblivious merge schedules and bounded-error randomized query evaluation, the optimal write-read product satisfies

$$\min E\widehat{R}_{Q}=\begin{cases}\Theta(n\log^{2}n),&Q\text{ is locally certifiable in each component},\\[2.84526pt] \Theta(n\log^{3}n),&Q\text{ is a target-key order or exact range query},\\[2.84526pt] \Theta(n^{2}),&Q\text{ is select}.\end{cases}$$

Here the first line includes membership, point-emptiness, and global minimum/maximum. The second line includes predecessor, successor, rank, range-count, exact range-sum with unit weights, range-minimum, range-maximum, and bounded-interval emptiness.

The minimum ranges over all algorithms in the model. The upper bounds are explicit merge schedules with forward-readable static indexes. The lower bounds are worst-case over insertion sequences and query instances.

family	examples	information available before the scan	product
local certificates	membership, point-emptiness, global min/max	a key or constant-size certificate checked component by component	$\Theta(n\log^{2}n)$
target-key order/range	predecessor, rank, range-count, exact range aggregates	key values or interval endpoints whose local gaps must be found	$\Theta(n\log^{3}n)$
selection	select	only a rank $K$, with no target key value	$\Theta(n^{2})$

Theorem 1.1 is not an answer-size separation. Membership, rank, and select all have $\Theta(\log n)$-bit answers on the hard instances, and rank and select are inverse operations. The difference is what the scan knows before it starts. Membership names a key and equality can be checked locally. Predecessor, rank, and range queries name key values, but exactness requires locating those values inside each component. Select names no key at all; the answer value is unknown while early components pass under the one-way scan.

The assumptions in Theorem 1.1 are also close to the boundary of the phenomenon. Integer keys reduce the cost of searching within one component; random access removes the no-backseek difficulty behind select; and non-materialized cross-component directories can reintroduce fractional-cascading information. We treat these variants after the main proof, in Section 7 and Appendix A. They are not part of the headline model, but they help separate which restriction is responsible for which line of the classification.

1.2 Overview

The proof has a simple architecture. For each query family we prove a read barrier at a fixed state, and then combine it with an accounting lemma for all suffix-merge histories.

For membership, the fixed-state barrier is the number of live components. Let $\widehat{W}$ be the maximum number of live components, and let $\widetilde{D}$ be the average number of times an item is written, including its initial write. The edit floor and the merge-forest counting lemma give

$$E=\Omega(n\widetilde{D}),\qquad\widetilde{D}\widehat{W}=\Omega(\log^{2}n).$$

Some membership query must inspect all live components, so $\widehat{R}=\Omega(\widehat{W})$. This gives $E\widehat{R}=\Omega(n\log^{2}n)$, matched by the balanced Bentley–Saxe schedule with local hashes.

For target-key order and range queries, the read barrier is the summed static search cost in the live components. At a state with component sizes $m_{i}$, we prove

$$\widehat{R}_{\mathrm{ord}}=\Omega\!\left(\sum_{i}\log(m_{i}+1)\right).$$

The terms add because strong materialization forbids a read in one component from revealing the private search position of the query key in another component. The merge-history side then proves the size-diversity law

$$E\cdot\max_{t}\sum_{C\text{ live at }t}\log(|C|+1)=\Omega(n\log^{3}n).$$

This is the only extra logarithm in the middle line of Theorem 1.1. Once the predecessor lower bound is established, direct-sum arguments transfer it to rank, range-count, exact range-sum, range-minimum, range-maximum, and bounded-interval emptiness.

For select, the query is target-free. Consider a boundary in the scan with $P$ stored keys before it and $U$ stored keys after it. We construct a rank query whose answer is one of $\min\{P,U\}$ possible prefix keys, but the index of the right key is determined only by suffix information. Since the scan cannot go back, it must read

$$\Omega(\min\{P,U\})$$

words before crossing the boundary. If the read budget is small, nearly all live keys must repeatedly sit in one huge component; immutability then forces that component to be rebuilt many times. This yields $E=\Omega(n^{2}/\widehat{R})$ and hence $E\widehat{R}=\Omega(n^{2})$.

Section 2 defines the model, costs, query families, and key assumptions. Section 3 proves the membership line. Section 4 proves the no-cascade read floor, the size-diversity law, and the order/range line. Section 5 proves the cut lower bound and the quadratic select line. Section 6 gives a general static-to-dynamic lifting principle for additive queries. Section 7 records access-model variants, and Appendix A studies cascades beyond strong materialization.

The closest prior work explains pieces of the landscape: dynamization by merging, write-optimized storage, fractional cascading, and lower bounds in richer memory models. The following table summarizes why none of these directions already implies Theorem 1.1.

line of work	what it explains	difference from this paper
logarithmic method	how to maintain many rebuilt static structures	no classification separating query families with similar static costs
LSM and write-optimized storage	practical read/write/space tradeoffs for sorted runs	not a matching lower-bound theorem for membership, order/range, and select under one merge history
fractional cascading	how to share repeated searches across related catalogs	the useful bridge has the wrong chronological direction in a materialized one-way merge stack
cell-probe and external-memory lower bounds	tradeoffs under random access to cells or blocks	our restriction is not cell access but the order in which materialized information can be read and rewritten

Static-to-Dynamic Transformations.

Bentley and Saxe introduced the logarithmic method for decomposable searching problems [BS80]. Mehlhorn proved lower bounds for such transformations [MEH81], and Overmars developed a systematic theory of dynamic structures built from static ones [OVE83]. Recent competitive dynamization studies merge policies under nonuniform inputs and read/write ratios [MRY+21]. These works explain the overhead of maintaining many static structures. They do not distinguish query types whose static problems are all logarithmic; the extra logarithm in Theorem 1.1 is caused by exact order localization across materialized components.

Write-Optimized Storage.

The LSM-tree was introduced to make high-rate updates sequential and cheap [OCG+96]. LSM-style sorted runs appear in large-scale storage systems and key-value stores [CDG+08, LM10, MDL20, LPG+17, RKC+17]. The systems literature maps the read/update/storage tradeoff [AKM+16], optimizes merge policies and filter allocation for point lookups [DAI17, DI18, MLI25], and builds succinct or probabilistic range filters and range-query mechanisms [ZLL+18, LCK+20, ZCW+21]. This work motivates the access discipline, but it does not prove a tight query-by-query product classification under the same merge histories.

Fractional Cascading.

Fractional cascading removes repeated binary searches across related catalogs in static structures [CG86]; dynamic fractional cascading studies how to maintain such links under updates [MN90, AFS21]. Our obstruction is orthogonal. The issue is not only the cost of updating a cascade, but whether the bridge can appear before the component it accelerates is created. In the strongly materialized one-way model it cannot; Appendix A examines what changes when such cross-component data are allowed.

Lower Bounds and Restricted Access.

External-memory dictionary lower bounds study update/query tradeoffs under random block access [BF03, DIN22]. Cell-probe lower bounds for dynamic data structures, including dynamic range counting, use a more powerful random-access memory model [FS89, LAR12, PĂT11, KO25]. Our bounds are orthogonal: component encodings are static and transparent, but the read and rewrite order is restricted. The lifting theorem in Section 6 connects our merge-stack bounds to static predecessor and range-searching lower bounds [PT06, PT07, PĂT07, CW13]. The merge-history accounting is closest in spirit to list labeling and file maintenance [IKR81, BKS12, BCF+22, BCF+24]; the select lower bound is related to selection under restricted access [MP80, FJ84, KKZ+19, BDH+23].

2 Preliminaries

This section fixes the model used in the main classification. The objects are merge histories of immutable sorted components, the costs are total rewritten mass and prefix-worst read cost, and the key assumptions isolate the absence of cross-component order information.

2.1 Merge Histories and Components

A merge stack maintains a sequence of immutable sorted components $C_{1},\dots,C_{W}$, oldest at the bottom. Each component stores a sorted block of key-value samples together with any static index built from that block alone. Three operations maintain it:

•

insert$(x,y)$ pushes a new singleton component $\{(x,y)\}$ on top;
•

merge pops a suffix $C_{i},\dots,C_{W}$ and pushes a single component holding the union of their samples in sorted order;
•

query makes one forward pass over $C_{1},C_{2},\dots$ in stack order, reading component data; after an $O(1)$ header it may skip a component’s body, but it never returns to an earlier component.

Inserts and merges interleave according to a schedule of the algorithm’s choice. The main lower bounds consider schedules that are oblivious to the key values; this condition is part of strong materialization below.

2.2 Cost Measures

The write cost $E$ is the total component data created or rewritten over the execution. The read cost of a query is the data it reads, and $\widehat{R}$ is its prefix-worst value: the maximum, over every time $t$ and every query, of the cost of answering that query against the state after step $t$. Making $\widehat{R}$ prefix-worst keeps the read cost from being amortized away by periodic global rebuilds, since every intermediate state must remain queryable. For a randomized evaluator the cost of a query is its expected number of charged reads, and $\widehat{R}$ is again prefix-worst. The complexity measure throughout is the product $E\widehat{R}$.

A token is a machine word of $w=\Theta(\log n)$ bits, and keys and values are drawn from universes of size $\mathrm{poly}(n)$, so a sample occupies $O(1)$ tokens and a component of $m$ samples occupies $\Theta(m)$. We use the following notation.

symbol	meaning
$E$	total rewritten component data
$\widehat{R}_{Q}$	prefix-worst read cost for query family $Q$
$\widehat{W}$	maximum number of live components over the execution
$\widetilde{D}$	average item write depth, including the initial write
$L$	$\log_{2}n$
$\lambda$	$\log_{2}\log_{2}n$

2.3 Query Families

Three query types drive the paper. Membership is the exact-match point query: given $x$, return its value if $x$ is live and a default otherwise. It is locally certifiable because equality can be settled inside each component by a hash or perfect dictionary. Predecessor is the basic target-key order query: given $x$, return the value of the largest live key at most $x$. Successor, rank, and exact range queries – count, sum, minimum, maximum, and emptiness over an interval – are target-key order or range queries of the same kind, since each must locate a key value or endpoint in the live order. Membership and these order queries are decomposable: an answer combines fixed per-component contributions. Select is different. It is given a rank $K$ and returns the $K$-th smallest live key; no target key value is supplied to the scan. Inserted keys are distinct, and the lower bounds use distinct-key instances.

2.4 Key Models and Materialization

Keys come in two models, and the order separation depends on which one is used. An abstract key is the model for string, tuple, or opaque record keys: it is hashable and comparable, but carries no integer structure. We read it through an oracle. A key has an opaque equality label and an unknown position in the total order, and neither the label nor its hash reveals that position. A single read returns at most one stored key handle together with $O(1)$ control words, and the handle’s rank among those already read is learned only by comparison. The encoder cannot manufacture an order-bearing separator it has not read, and an answer must be a key or value decoded from the words read, not a pointer into an unread cell. The $O(1)$ control words therefore cannot stand in for the order of a whole block. An integer key is an integer in a $\mathrm{poly}(n)$ universe, with the word-RAM available: a van Emde Boas tree or y-fast trie locates the predecessor inside a component in $O(\lambda)$ reads, independent of its size, and the order gap narrows accordingly (Section 7).

The main theorems hold in executions realized by size-based log-structured merging. Ordinary materialization, which constrains only a component’s contents, is not enough: once the merge schedule may depend on the data, the partition into components itself carries order information across them, and an unread block is no longer free for an adversary to complete. We use the following stronger condition.

Definition 2.1 (Strong Materialization).

An execution is strongly materialized if it satisfies the following.

•

Canonical updates: after each insertion the active components partition the inserted samples into contiguous blocks of consecutive insertion times, and each insertion performs at most one merge, replacing a suffix of the active components and the new singleton by a single component.
•

Oblivious schedule: the merged suffix at each step depends only on the step index and the current component sizes, not on the keys, values, query answers, or any coins.
•

Exact layout: a component of $m$ samples occupies a publicly fixed length $\ell(m)\in[c_{0}m,c_{1}m]$, so component boundaries, addresses, and the merge structure are functions of the size sequence alone.
•

Local encoding: a component’s tokens depend only on its own samples, their insertion order and values, and comparisons among its own keys; inputs that agree on every component’s internal order, labels, values, and insertion history, and differ only in how keys of distinct components interleave, induce identical component encodings.
•

No external state: no sample-dependent state persists outside the components, the query evaluator starts from a data-independent configuration with coins independent of the data, and every dependence on the data passes through reads of component tokens.

This is what an LSM run already is, a self-contained sorted file; Appendix A removes the condition and lets a young component share or copy order information from older ones.

The per-component arguments use one consequence. Call an execution no-cascade if a read in one component cannot shrink another component’s set of candidate answers. Strong materialization supplies it: changing one block leaves the partition, every other component’s encoding, and the layout untouched, so a read elsewhere says nothing about that block. A non-materialized global key order, searchable by random access, would violate this, the components then admitting a joint search in $O(\log n)$ reads; random access alone, with the components still materialized, does not. This locality is what Appendix A gives up.

The canonical-updates clause is without loss of generality: the merges performed between two insertions act on nested suffixes of the stack and coalesce into their final suffix merge, and collapsing them removes the intermediate writes and queryable states, so neither $E$ nor $\widehat{R}$ grows. Under exact layout the write cost is structural rather than informational: building a component of $m$ samples costs $\Theta(m)$ tokens whatever it holds, a fact the edit floor of Section 3 reads off directly.

2.5 Serialization and Forward Reads

The lower bounds are cleaner in a low-level view of the same execution. Serialize the components in stack order into a single token string, each component a contiguous block preceded by a length-prefix header. Edits become tail operations: an insert pushes one block at the tail, and a merge of a stack suffix pops the most recently pushed blocks and pushes one new block. Writing $\mathrm{LCP}$ for the longest common prefix, the edit $s_{t-1}\to s_{t}$ costs

$$d_{\mathrm{LIFO}}(s_{t-1},s_{t})=\bigl(|s_{t-1}|-|\mathrm{LCP}|\bigr)+\bigl(|s_{t}|-|\mathrm{LCP}|\bigr),$$

the number of tokens popped and pushed at the tail, so $E=\sum_{t}d_{\mathrm{LIFO}}(s_{t-1},s_{t})$; we call such tail-only edits LIFO. A query is run by a forward, no-backseek head: it starts at position $0$ and at each step either reads the current token, at unit cost, advancing one place, or skips forward at no cost by an amount it computes from its finite control, the query, and the tokens read so far. It never moves backward, working memory is free, and only reads are charged.

Lemma 2.2 (Serialization Equivalence).

For strongly materialized executions, the merge-stack model and the LIFO-token, forward-no-backseek model agree up to constant factors in $(E,\widehat{R})$.

Proof.

Serialize as above. An insert pushes a block of $\Theta(1)$ tokens at the tail, and a merge pops exactly the most recently pushed blocks and pushes one new block, so every edit is tail-only; the pushed token mass equals the rewritten component data up to the matching pops, a constant factor, so $E_{\mathrm{token}}=\Theta(E)$. The components occupy the string in stack order, which is birth order and the order a forward head reads them, and skipping a body after its header is a legal no-backseek pass, so $\widehat{R}_{\mathrm{token}}=\Theta(\widehat{R})$. Conversely, a tail-only token execution whose pushed blocks are local encodings of the unions of the popped blocks splits into contiguous blocks in birth order, since the tokens pushed at one step are consecutive and share a birth time, and is a merge-stack execution with the same merge structure. ∎

One may therefore keep the merge-stack picture throughout and read the lower-bound proofs as statements about the token string.

2.6 Baseline Upper Bounds

The serial Bentley–Saxe transform [BS80] supplies the upper bounds the lower bounds will match.

Proposition 2.3 (Bentley–Saxe Baseline).

A single schedule has $E=O(n\log n)$ with at most $\lceil\log_{2}(n+1)\rceil$ active components at all times, and answers membership in $\widehat{R}=O(\log n)$, abstract-key predecessor in $\widehat{R}=O(\log^{2}n)$, and integer predecessor in $\widehat{R}=O(\lambda\log n)$. The corresponding products are

$$E\widehat{R}=O(n\log^{2}n),\qquad O(n\log^{3}n),\qquad O(n\log^{2}n\,\lambda).$$

Proof.

Keep the components whose sizes are the set bits of the insert count, merging equal-sized ones on each insert. A component of size $2^{k}$ is built at most $n/2^{k}$ times, each at $\Theta(2^{k})$ tokens, so $E=O(n\log n)$, and at most $\lceil\log_{2}(n+1)\rceil$ components are live at any time. Each component carries a forward-readable static structure, length-prefixed so the head skips its body after one header read. A static perfect hash answers membership in $O(1)$ reads per component, even against an adversarial query, so $\widehat{R}=O(\log n)$. For abstract-key predecessor, a balanced search tree laid out in search-round order, all depth-one nodes, then all depth-two, and so on, is traversed root to leaf as a forward scan of $O(\log m_{k})$ comparison reads in component $k$, summing to $O(\log^{2}n)$ over the active components. For integer predecessor, a y-fast trie answers in $O(\lambda)$ reads per component under the same round-by-round layout, giving $\widehat{R}=O(\lambda\log n)$. ∎

The membership and predecessor products are the first the lower bounds match (Sections 3 and 4); select needs a different schedule and is polynomially larger (Section 5). How the predecessor bound responds when these conditions are relaxed, and why the relaxations are not interchangeable, is the subject of Section 7.

3 Membership

The Bentley–Saxe schedule of Section 2 answers membership with product $n\log^{2}n$. We show this is best possible, and in doing so establish the three structural facts the later lower bounds reuse: a level decomposition that the tail-only discipline forces, an edit floor on the write cost, and a counting bound that trades depth against frontier. Together they expose the mechanism the paper turns on, that $E\widehat{R}$ factors into a write budget and a sum of per-level read charges. Membership is the case where a hash holds every charge at $O(1)$; order and rank raise it.

3.1 Merge Forests

Fix an execution. A live token in $s_{t}$ has a birth step, the step at which it was last pushed, and tokens with a common birth step form a level; by Lemma 2.2 a level is just a component of Section 2 in token-string form. Write $W(t)$ for the number of live levels at time $t$.

Lemma 3.1 (Level Decomposition).

At every time $t$ the live levels are contiguous and ordered by birth, $s_{t}=L_{t_{1}}\cdots L_{t_{W}}$ with $t_{1}<\cdots<t_{W}$, and the edit $s_{t-1}\to s_{t}$ pops a suffix of them and pushes one new top level.

Proof.

A canonical update pops a suffix of the active components and pushes one merged component (Section 2). The tokens pushed at one step are consecutive at the tail and share a birth step, so each level is contiguous and older levels lie earlier; an edit pops a suffix of whole levels and makes the pushed tokens the new top level. ∎

The decomposition makes the execution a merge forest: an internal node for each step that merges, its children the levels popped there and the singleton inserted then, and a leaf for each insert. Leaf $j$ takes part in $d_{j}$ merges, one for each of its internal ancestors. The rewritten sample mass is therefore

$$M:=\sum_{j}d_{j}=\sum_{P}|P|,$$

$|P|$ being the number of leaves below the internal node $P$. The forest is fixed by the schedule alone, whatever the tokens mean, and the lower bounds read off it.

The hard instances carry their entropy in the values: each inserted value is independent and uniform over a $\mathrm{poly}(n)$ universe, so it is $\Theta(\log n)$ bits, the content of one token.

Lemma 3.2 (Incompressibility).

On the i.i.d.-value instance, a materialized level that is the sole source for $m$ of its samples occupies $\Omega(m)$ tokens.

Proof.

The state must determine those $m$ values, whose joint entropy is $\Theta(m\log n)$ bits, or $\Theta(m)$ tokens. Materialization places them in this level alone, so it occupies $\Omega(m)$ tokens. ∎

Corollary 3.3 (Edit Floor).

With augmented depth $\widetilde{D}:=(M+n)/n$, the write cost is $E=\Omega(n\widetilde{D})$.

Proof.

Under exact layout a level of $m$ samples costs $\Theta(m)$ tokens, so $E=\Theta(n+M)=\Theta(n\widetilde{D})$. With no layout fixed, Lemma 3.2 applies to each level when it is built, the sole source for its samples at that moment, and summing over the levels built, of total size $n+M$, gives $E\geq c(n+M)=cn\widetilde{D}$ for a constant $c\in(0,1]$. ∎

The third fact ties depth to frontier. Its counting argument is the online-labeling lemma of Bulánek, Koucký, and Saks [BKS12], recast in merge-forest terms. Write $\widehat{W}:=\max_{t}W(t)$ for the frontier width.

Lemma 3.4 (Frontier Width).

Every merge forest on $n$ leaves with augmented depth $\widetilde{D}$ and frontier width $\widehat{W}$ has $\widetilde{D}\widehat{W}=\Omega(\log^{2}n)$.

Proof.

With a dummy root above the final frontier, a leaf in $d_{j}$ merges has augmented depth $d_{j}+1$, of mean $\widetilde{D}$. Let $N(h,W)$ be the most leaves an ordered suffix-merge tree of height at most $h$ can have while its live frontier never exceeds $W$. Its root pops a suffix of $r\leq W$ children, each a completed subtree that holds a frontier slot until the merge, together with the singleton inserted then, a height-$0$ child holding no slot. With $N(0,W)=1$,

$$N(h,W)\ \leq\ 1+\sum_{q=1}^{W}N(h-1,q)\ \leq\ \sum_{q=0}^{W}\binom{h-1+q}{q}\ =\ \binom{h+W}{W},$$

the singleton being the $q=0$ term and the last step the hockey-stick identity. For $a\leq b$, $\log_{2}\binom{a+b}{a}\leq a\log_{2}\frac{e(a+b)}{a}=O(\sqrt{ab})$ using $\log_{2}u\leq 2\sqrt{u}$. By Markov at least $n/2$ leaves have augmented depth at most $2\widetilde{D}$; keeping them and suppressing empty internal nodes leaves an ordered tree with at least $n/2$ leaves, height at most $\lceil 2\widetilde{D}\rceil$, and frontier at most $\widehat{W}$, so

$$\log_{2}\tfrac{n}{2}\ \leq\ \log_{2}\binom{\lceil 2\widetilde{D}\rceil+\widehat{W}}{\widehat{W}}\ =\ O\!\bigl(\sqrt{\widetilde{D}\widehat{W}}\bigr),$$

that is $\widetilde{D}\widehat{W}=\Omega(\log^{2}n)$. ∎

3.2 The Membership Bound

The read side needs one fact: a forward head must visit every live level.

Lemma 3.5 (Sequential Access).

For an oblivious schedule, some strongly materialized realization forces $\widehat{R}\geq\widehat{W}$ on a fresh membership query against a deterministic evaluator, and $\widehat{R}\geq\widehat{W}/3$ against a bounded-error one.

Proof.

At a time of maximum frontier, take the realization in which every live level straddles a fresh key $x$ matching none of its samples. Each live level must be decided on its own, that $x$ is absent there, and under strong materialization no other level constrains it, so a level left unread can be filled with $x$ by the adversary, flipping the answer. A deterministic evaluator therefore reads all $\widehat{W}$ levels, so $\widehat{R}\geq\widehat{W}$. For a bounded-error one, couple this input with the one in which only level $i$ gains $x$: the transcripts agree until level $i$ is read, so skipping it makes the evaluator answer both alike and err on one, and the two error probabilities sum to at least the probability of skipping level $i$. Bounded error keeps that sum below $\tfrac{2}{3}$, so level $i$ is read with probability at least $\tfrac{1}{3}$ and $\widehat{R}\geq\widehat{W}/3$. A length header does not escape the charge, that read being the one counted. ∎

Theorem 3.6 (Tight Membership Bound).

Strongly materialized membership has $E\widehat{R}=\Theta(n\log^{2}n)$, attained at $(E,\widehat{R})=(\Theta(n\log n),\Theta(\log n))$ by hashing on the Bentley–Saxe schedule.

Proof.

The upper bound is Proposition 2.3. For the lower bound, Lemma 3.5 gives $\widehat{R}\geq\widehat{W}$, Corollary 3.3 gives $E=\Omega(n\widetilde{D})$, and Lemma 3.4 gives $\widetilde{D}\widehat{W}=\Omega(\log^{2}n)$, so $E\widehat{R}=\Omega(n\widetilde{D}\widehat{W})=\Omega(n\log^{2}n)$. ∎

Nothing in the lower bound used what membership computes, only that every live level must be decided, so $n\log^{2}n$ is a floor for the whole model, reached by membership. It is robust to randomization through Lemma 3.5, and it constrains the tradeoff and not just the product: $\widehat{R}=O(\log n)$ forces $\widehat{W}=O(\log n)$, hence $\widetilde{D}=\Omega(\log n)$ and $E=\Omega(n\log n)$. Membership is the hashable extreme, each level settled by one probe. The rest of the paper keeps this write budget and asks whether the read side can stay at $O(1)$ per level; once the query needs order, it cannot.

4 Order Queries

This section proves the middle line of Theorem 1.1. The proof has three steps. First, strong materialization and one-way access rule out the cross-component shortcut that fractional cascading would need. Second, once the components cannot share searches, a fixed state with component sizes $m_{i}$ forces $\Omega(\sum_{i}\log(m_{i}+1))$ reads for a hard predecessor query. Third, every cheap suffix-merge history exposes some state where this summed log-size is large, giving the product lower bound $\Omega(n\log^{3}n)$. The final subsection transfers predecessor to rank and exact range queries.

4.1 No Cascading Across Components

Membership spends $O(1)$ per level because a hash names where the answer sits. Order has no such shortcut and cannot build one by linking levels. Call a forward bridge from a level $L_{i}$ to a younger level $L_{j}$ any data stored in $L_{i}$ that lets the head, once it has read $L_{i}$, reach the query’s predecessor in $L_{j}$ within $O(1)$ further reads.

Lemma 4.1 (Causal-Bridge Obstruction).

Under forward, no-backseek access with LIFO edits, no forward bridge can be consulted when it would help.

Proof.

Levels are read oldest first (Lemma 3.1). A bridge that speeds up a younger level $L_{j}$ must be read before $L_{j}$, hence stored in an older level $L_{i}$; but its content depends on $L_{j}$, fixed only when $L_{j}$ is built, so its carrier must be younger than $L_{j}$. The two demands collide: when $L_{i}$ was built $L_{j}$ did not yet exist, and on adversarial independent keys no guess survives. Rebuilding $L_{i}$ after $L_{j}$ only pushes its birth past $L_{j}$, moving it behind in read order, and a younger level pointing into an older one is reached only once the older target has been passed. ∎

This is the obstruction behind Afshani’s dynamic fractional-cascading lower bound [AFS21], sharpened from a tax to an impossibility: the bridge a cascade needs would have to be written before its target exists. The other route, a young level carrying a directory into the older ones, is closed not by access but by strong materialization, each component encoding only its own samples (Section 2); Appendix A drops that assumption. What remains is the cost of searching one level alone.

Within a level a hash still names one specific key, while the predecessor is another, fixed by the level’s order, that the hash of $x$ does not reach. Locating it is a comparison search, and on a level of $m$ keys an adversarial query forces $\Omega(\log m)$ reads, made precise against a hard gap distribution in Lemma 4.2.

4.2 Per-State Search Lower Bound

Fix a time and write $\Phi(t)=\sum_{\text{active }L}\log_{2}|L|$ for the summed log-size of the live levels. Since the per-level searches do not combine (Lemma 4.1), the read cost is their sum. This sum is not forced at every state: with the live levels in disjoint key ranges a single header settles all but one. What does hold is that each size skeleton, the live levels’ sizes alone, admits an adversarial key assignment making the sum necessary, which under an oblivious schedule is all the lower bound needs.

Lemma 4.2 (Zero-Side Ordered Gap).

Fix stored keys $k_{1}<\cdots<k_{m}$ and handles $b<x$. For $0\leq j\leq m$ let $Z_{j}$ place $k_{1},\dots,k_{j}<b<x<k_{j+1},\dots,k_{m}$, and for $0\leq j<m$ let $H_{j}$ place $k_{1},\dots,k_{j}<b<k_{j+1}<x<k_{j+2},\dots,k_{m}$, so the predecessor of $x$ is $b$ on $Z_{j}$ and $k_{j+1}$ on $H_{j}$. Any randomized comparison tree reads $\Omega(\log(m+1))$ keys in expectation on $Z_{J}$, $J$ uniform, if it uses the known internal order of the $k_{r}$, learns the order of an unread key only by reading it, and errs with probability at most $\tfrac{1}{3}$ over $\{Z_{j},H_{j}\}$.

Proof.

Take a deterministic tree first. A read key has three relevant outcomes, $k_{r}<b$, $b<k_{r}<x$, and $x<k_{r}$. Let $E_{0}$ count the zero instances not answered $b$, $E_{1}$ the hot instances answered $b$, and $\Lambda$ the distinct $b$-leaves reached by correctly answered zero instances. The zero instances reaching one leaf form an interval: if $Z_{a}$ and $Z_{c}$ ($a<c$) share a $b$-leaf then no $k_{r}$ with $a<r\leq c$ was read, being above $x$ in $Z_{a}$ and below $b$ in $Z_{c}$, so every $H_{h}$ with $a\leq h<c$ follows the same transcript and is wrongly answered $b$. A leaf covering $z$ zero instances thus forces $z-1$ hot errors, whence $\Lambda\geq(m+1)-E_{0}-E_{1}$. Average error at most $\tfrac{5}{12}$ over the uniform mixture of the $2m+1$ instances gives $E_{0}+E_{1}\leq\tfrac{5}{12}(2m+1)$ and $\Lambda\geq(m+1)/6$. With one correct zero instance per leaf, at depths $d_{1},\dots,d_{\Lambda}$, the ternary Kraft inequality $\sum_{\ell}3^{-d_{\ell}}\leq 1$ forces mean depth $\geq\log_{3}\Lambda$, so

$$\mathbb{E}_{J}[\text{reads on }Z_{J}]\ \geq\ \tfrac{\Lambda}{m+1}\log_{3}\Lambda\ =\ \Omega(\log(m+1)).$$

For a randomized tree the coin-fixed error averages at most $\tfrac{1}{3}$, so by Markov a $\tfrac{1}{5}$ fraction of the fixings have error at most $\tfrac{5}{12}$, and averaging over them keeps $\Omega(\log(m+1))$. ∎

Lemma 4.3 (Anchor-and-Zones Floor).

Fix a public size vector $(m_{1},\dots,m_{W})$ of an oblivious schedule and an evaluator of error at most $\tfrac{1}{3}$. Some strongly materialized state with these sizes admits a query $x$ on which the evaluator has $\widehat{R}\geq c\sum_{i=1}^{W}\log_{2}(m_{i}+1)$, for an absolute $c>0$.

Proof.

We average over a distribution of states, so some realization meets the expectation, and obliviousness makes it carry the prescribed sizes. Take level $1$ as anchor: fix its internal order, draw the query gap uniformly among the $m_{1}$ gaps with a stored lower endpoint, and put $x$ there with $b$ that endpoint. Locating $b$ is a randomized predecessor search among equally likely gaps, one ternary split per read, so a bounded-error evaluator spends $\Omega(\log(m_{1}+1))$ reads, and an $O(1)$ header holds only constantly many handles and cannot help. For $i\geq 2$ fix three private zones, a low slab below every anchor key, a high slab above $x$, and a hot subzone inside $(b,x)$, all ordered publicly by level index. Draw $J_{i}$ uniform in $\{0,\dots,m_{i}\}$; the zero placement puts $k_{i,1},\dots,k_{i,J_{i}}$ in the low slab and the rest in the high slab, and the hot alternative, when $J_{i}<m_{i}$, moves $k_{i,J_{i}+1}$ into the hot subzone. Expose all outside information for free, $b$ included. Reading $k_{i,r}$ then reveals only its zone, comparisons within the level giving the known index order and comparisons across levels only the public zone order, so nothing finer about $J_{i}$ is learned without reading level $i$. The task is Lemma 4.2 with $m=m_{i}$ and inherits its error guarantee, costing $\Omega(\log(m_{i}+1))$. Summing over the anchor and the levels gives the bound. ∎

The floor is existential over the states realizing a skeleton, not a property of every state; specialized to one read per level it is the randomized membership floor $\widehat{R}\geq\widehat{W}/3$ of Theorem 3.6. On the hard realization $\widehat{R}\geq c\sum_{k}\log_{2}(m_{k}+1)\geq c\max\{\widehat{W},\Phi(t)\}$, and the separation turns on whether a schedule can hold $\Phi$ down throughout while editing for only $O(n\log n)$. It cannot.

4.3 Merge-History Size Diversity

A logarithm above membership needs $\Phi(t)=\Omega(\log^{2}n)$ at some moment, and no schedule holds $\Phi$ at the membership scale $O(\log n)$ throughout. The size-diversity law prices the time-averaged potential from below by the edit budget. It is an entropy-prefix argument on the merge forest, budget-free, the budget entering only through the augmented depth $\widetilde{D}$.

Add a dummy root above the final frontier, with $S_{v_{0}}=n$. For an internal node $v$ write $S_{v}$ for the leaves below it, $q_{v}=S_{v}/n$, and $z_{v}=\log_{2}S_{v}$; its children in birth order have sizes $s_{1},\dots,s_{r}$, weights $p_{i}=s_{i}/S_{v}$, entropy $h_{v}=\sum_{i}p_{i}\log_{2}(1/p_{i})$, and ordered prefix load $a_{v}=\sum_{j}p_{j}\sum_{i<j}\log_{2}s_{i}$. A merge coalesces an active suffix, so $r\leq\widehat{W}+1$.

Lemma 4.4 (Four Identities).

For every merge forest, exactly and with no budget hypothesis,

$$\mathrm{(I)}\ \tfrac{1}{n}\!\int_{0}^{n}\!\Phi=\sum_{v}q_{v}a_{v},\quad\mathrm{(II)}\ \sum_{v}q_{v}h_{v}=L,\quad\mathrm{(III)}\ \sum_{v}q_{v}h_{v}z_{v}\geq\tfrac{1}{2}L^{2},\quad\mathrm{(IV)}\ \sum_{v}q_{v}=\widetilde{D}.$$

Proof.

For (I), read the integral as a sum over the intervals between insertions; it is the lifetime identity $\int_{0}^{n}\Phi=\sum_{L}\tau_{L}\log_{2}|L|$, where $\tau_{L}$, the inserts during $L$’s life, equals the mass of $L$’s younger siblings at its parent, since a LIFO merge carries off exactly the levels younger than $L$. As $S_{v}a_{v}=\sum_{i}(\sum_{j>i}s_{j})\log_{2}s_{i}=\sum_{i}\tau_{v_{i}}\log_{2}s_{i}$, summing over $v$ and dividing by $n$ gives (I). For (II) and (III), sample a leaf uniformly and let $Z_{0}=L>Z_{1}>\cdots\geq 0$ be the log-sizes along its root-to-leaf path, with $X_{k}=Z_{k}-Z_{k+1}$ and $\sum_{k}X_{k}=L$. At a node $v$ on the path $\mathbb{E}[X_{k}\mid v]=h_{v}$ and $\Pr[v\text{ on path}]=q_{v}$, giving (II); and $\sum_{k}Z_{k}X_{k}=\tfrac{1}{2}(\sum_{k}X_{k})^{2}+\tfrac{1}{2}\sum_{k}X_{k}^{2}\geq\tfrac{1}{2}L^{2}$ with $\mathbb{E}[Z_{k}X_{k}\mid v]=z_{v}h_{v}$, giving (III). For (IV), $\sum_{v}q_{v}=\tfrac{1}{n}\sum_{\text{leaves}}(d_{j}+1)=\widetilde{D}$, since each leaf has $d_{j}+1$ internal ancestors. ∎

Identity (III) is the engine: entropy weighted by current log-size unavoidably reaches $\Omega(L^{2})$, exactly. Turning it into a bound on (I) is a local computation.

Lemma 4.5 (Local Prefix Bound).

Fix $\varepsilon\in(0,\tfrac{1}{4})$, $\gamma=\varepsilon^{2}$, $c_{0}=L^{-7}$, and write $\mu=G^{-1}$ for $G(m)=(m+1)\log_{2}(m+1)-m\log_{2}m$. Uniformly over internal nodes with $z_{v}\geq\varepsilon L$, $h_{v}\geq c_{0}$, and $r\leq n^{\gamma}+1$,

$$a_{v}\ \geq\ (1-\kappa)\,z_{v}\,\mu(h_{v}),\qquad\kappa\leq\varepsilon+o(1).$$

Proof.

Put $\theta=(\log_{2}r+9\log_{2}L)/(\varepsilon L)$, so $\theta=\varepsilon+o(1)$ under the hypotheses. Call a child good if $p_{i}\geq 2^{-\theta z_{v}}$, so $\log_{2}s_{i}=z_{v}+\log_{2}p_{i}\geq(1-\theta)z_{v}$. A bad child has $p_{i}<2^{-\theta\varepsilon L}=(rL^{9})^{-1}$, so the bad mass is at most $L^{-9}$, and renormalizing onto the good children shifts the entropy by $O(L^{-8})$ and hence $\mu$ by a factor $1-o(1)$: since $(\ln\mu)^{\prime}=O(1+1/h)$ and $h_{v}\geq c_{0}$, the shift in $\ln\mu$ is $O((1+L^{7})L^{-8})=O(L^{-1})$. The maximum-entropy bound, that a law on $\{0,1,2,\dots\}$ of entropy $h$ has mean at least $\mu(h)$, applied to a child drawn by $p_{j}$, gives at least $(1-o(1))\mu(h_{v})$ expected earlier good children; each adds $\geq(1-\theta)z_{v}$ to $a_{v}$, so $a_{v}\geq(1-\theta-o(1))z_{v}\mu(h_{v})$. ∎

Theorem 4.6 (Size-Diversity Law).

Every LIFO merge forest with $\widehat{W}\leq n^{\gamma}$ and $\widetilde{D}\leq L^{6}$ satisfies

$$\tfrac{1}{n}\!\int_{0}^{n}\!\Phi(t)\,dt\ \geq\ (1-\varepsilon-o(1))\,\widetilde{D}\,L\,\mu\!\Bigl(\tfrac{\beta L}{\widetilde{D}}\Bigr),\qquad\beta=\tfrac{1}{2}-\varepsilon-o(1),$$

so $\max_{t}\Phi(t)$ is at least the same value.

Proof.

Discard from (III) the nodes with $z_{v}<\varepsilon L$, contributing at most $\varepsilon L\sum_{v}q_{v}h_{v}=\varepsilon L^{2}$ by (II), and those with $h_{v}<c_{0}$, contributing at most $c_{0}L\widetilde{D}=o(L^{2})$ by (IV). The survivors have $z_{v}\geq\varepsilon L$, $h_{v}\geq c_{0}$, $r\leq\widehat{W}+1\leq n^{\gamma}+1$, and carry $\sum^{\mathrm{s}}q_{v}h_{v}z_{v}\geq\beta L^{2}$. Write $Y_{v}=q_{v}h_{v}z_{v}/L^{2}$, so $m_{Y}:=\sum^{\mathrm{s}}Y_{v}\geq\beta$ and $\sum^{\mathrm{s}}Y_{v}/h_{v}=L^{-2}\sum^{\mathrm{s}}q_{v}z_{v}\leq\widetilde{D}/L$ by (IV). Set $\varphi(u)=u\,\mu(1/u)$, convex and decreasing, and note $q_{v}z_{v}\mu(h_{v})=L^{2}Y_{v}\varphi(1/h_{v})$. Identity (I) and the local prefix bound give

$$\tfrac{1}{nL^{2}}\!\int\!\Phi\ \geq\ (1-\kappa)\,m_{Y}\,\mathbb{E}[\varphi(1/h)],$$

the expectation under $Y/m_{Y}$. Jensen and then the monotonicity of $\varphi$ at $\mathbb{E}[1/h]\leq\widetilde{D}/(Lm_{Y})$ give

$$m_{Y}\,\mathbb{E}[\varphi(1/h)]\ \geq\ m_{Y}\,\varphi\!\Bigl(\tfrac{\widetilde{D}}{Lm_{Y}}\Bigr)\ =\ \tfrac{\widetilde{D}}{L}\,\mu\!\Bigl(\tfrac{m_{Y}L}{\widetilde{D}}\Bigr)\ \geq\ \tfrac{\widetilde{D}}{L}\,\mu\!\Bigl(\tfrac{\beta L}{\widetilde{D}}\Bigr),$$

using $m_{Y}\geq\beta$ and $\mu$ increasing. Multiply by $L^{2}$. ∎

The frontier cap is necessary for the law, since the never-merge schedule has $\Phi\equiv 0$, but it does not constrain the predecessor bound: the assembly below routes wide-frontier executions through the read floors instead. The cap aside, the law is the right general form rather than a relaxation, asymptotically tight along the lazy base-$q$ counter, so the uniform reading $\max_{t}\Phi\geq c\log^{2}n$ for all schedules is false, that family sending $\max_{t}\Phi/\log^{2}n\to 0$ as $q$ grows. Since $\Phi$ depends only on the forest, the bound holds for every realization of a randomized algorithm.

Lemma 4.7 (Calculus Floor).

$\min_{h>0}\mu(h)/h^{2}=\tfrac{1}{4}$, attained only at $h=2$; equivalently $G(m)\leq 2\sqrt{m}$ with equality iff $m=1$. Hence $\widetilde{D}^{2}\,\mu(\beta L/\widetilde{D})\geq(\beta^{2}/4)L^{2}$.

Proof.

Substituting $m=\mu(h)$, the claim $\mu(h)\geq h^{2}/4$ reads $G(m)\leq 2\sqrt{m}$. Let $F(m)=2\sqrt{m}-G(m)$, so $F(0^{+})=F(1)=0$ and $F^{\prime}(m)=m^{-1/2}-\log_{2}(1+1/m)$, of the sign of $1-u(m)$ for $u(m)=\sqrt{m}\,\log_{2}(1+1/m)$. Here $u$ is unimodal, with $u^{\prime}(m)=0$ at a single point since $(1+1/m)^{m+1}$ decreases strictly to $e$, and with $u\to 0$ at both ends and $u(1)=1$ it exceeds $1$ on exactly one interval $(m_{1},1)$. So $F$ rises on $(0,m_{1})$, returns to $F(1)=0$, then rises, giving $F\geq 0$ with equality only at $m\in\{0,1\}$. The lone interior contact $m=1$ gives $h=G(1)=2$ and $\mu(2)=1$, so the minimum is $\tfrac{1}{4}$; substituting $h=\beta L/\widetilde{D}$ gives the last inequality. ∎

4.4 The Predecessor Bound

Theorem 4.8 (Tight Predecessor Bound).

Every strongly materialized algorithm with a bounded-error evaluator, under no hypothesis on the frontier $\widehat{W}$ or the budget $M$, has predecessor $E\widehat{R}=\Omega(n\log^{3}n)$, which binary Bentley–Saxe attains. Predecessor is $\Theta(n\log^{3}n)$, one clean logarithm above membership.

Proof.

The upper bound is Proposition 2.3. Fix the algorithm; its oblivious schedule fixes the size trajectory and $E$ independently of the keys. Fix $\varepsilon=10^{-2}$, $\gamma=\varepsilon^{2}$, $\widetilde{D}=(M+n)/n$, and split on the schedule.

(i) $\widehat{W}>n^{\gamma}$. Lemma 4.3 at a maximum-frontier skeleton gives $\widehat{R}=\Omega(\widehat{W})>\Omega(n^{\gamma})$, and $E\geq n$, so $E\widehat{R}=\Omega(n^{1+\gamma})\gg n\log^{3}n$.

(ii) $\widehat{W}\leq n^{\gamma}$, $\widetilde{D}>L^{6}$. The edit floor gives $E=\Omega(n\widetilde{D})>\Omega(nL^{6})$ and Lemma 4.3 at the final skeleton, where $\sum_{k}\log_{2}(m_{k}+1)\geq\log_{2}(n+1)$, gives $\widehat{R}=\Omega(L)$, so $E\widehat{R}=\Omega(nL^{7})\gg n\log^{3}n$.

(iii) $\widehat{W}\leq n^{\gamma}$, $\widetilde{D}\leq L^{6}$. The size-diversity law gives a time $t^{\star}$ with $\Phi(t^{\star})\geq(1-\varepsilon-o(1))\widetilde{D}L\,\mu(\beta L/\widetilde{D})$; Lemma 4.3 there gives $\widehat{R}=\Omega(\Phi(t^{\star}))$, the edit floor gives $E=\Omega(n\widetilde{D})$, and the calculus floor closes it:

$$E\widehat{R}\ \geq\ \Omega\bigl(nL\cdot\widetilde{D}^{2}\mu(\beta L/\widetilde{D})\bigr)\ \geq\ \Omega\bigl(\tfrac{\beta^{2}}{4}\,nL^{3}\bigr)=\Omega(nL^{3}).$$

The cases are exhaustive. Obliviousness makes the size trajectory the same on every input, so the skeleton each case invokes occurs along the execution, and Lemma 4.3 exhibits a hard realization there; the product is the algorithm’s worst case over inputs, in expectation over its coins. ∎

The bound $\widehat{R}\geq\widehat{W}$ alone yields only $n\log^{2}n$, one logarithm short; the extra logarithm needs $\widehat{R}=\Omega(\Phi)$ at a high-$\Phi$ time, which the size-diversity law supplies. Predecessor pays for size diversity, not merely frontier width: membership is priced by $\widehat{W}$, order by $\Phi$.

4.5 Extensions to Rank and Range Queries

Predecessor is one order query among many. Whether a decomposable query pays the extra logarithm is read off the combiner, from how much of a single block’s contribution can reach the global answer, and for natural exact queries this takes one of two sizes. It is $O(1)$ when an $O(1)$-read witness settles a block’s contribution, as for membership, point-emptiness, and the global minimum or maximum. It is $\Theta(\log m)$ when the contribution turns on $q$’s position among a block’s $m$ sorted keys, as for predecessor, successor, rank, range-count, range-sum, range-minimum, range-maximum, and bounded-interval emptiness. The bridge to a read floor in the second case is a per-level direct sum, supplied for predecessor and successor by Lemma 4.3 and for the rest by the two lemmas below.

Call $Q$ additive if $Q(S,q)=\sum_{k}\varphi(L_{k},q)$ in an abelian group, each contribution $\varphi(B,q)$ a function of $q$’s position in $B$; rank, range-count, and unit-weight range-sum qualify. Let $g_{Q}(m)$ be the worst-case reads to compute $\varphi(B,q)$ on one static block of $m$ samples under a hard distribution $\mathcal{D}_{m}$, and call the family $\{\mathcal{D}_{m}\}$ embeddable when every active size vector is realized by independent private blocks and one shared query whose marginal at level $i$ is $\mathcal{D}_{m_{i}}$.

Lemma 4.9 (Additive Direct Sum).

For an additive $Q$ with an embeddable hard family of per-level complexity $g_{Q}$ under an oblivious schedule, the hard realization at time $t$ forces $\widehat{R}_{Q}\geq\Omega\bigl(\sum_{k}g_{Q}(m_{k})\bigr)$, deterministically and, with $g_{Q}$ the randomized static complexity, against bounded-error evaluators.

Proof.

Draw the embedded product instance and fix a level $i$. Run the evaluator from $B_{i}$’s draw alone, charging its reads inside $L_{i}$ and answering each read outside $L_{i}$ for free from a block drawn on independent coins from its law given $q$, which strong materialization and no-cascade make independent of $B_{i}$. The evaluator returns $Q(S,q)$, and subtracting the $\sum_{j\neq i}\varphi(L_{j},q)$ just sampled leaves $\varphi(L_{i},q)$ by the group inverse. This is a single-block evaluator for $\mathcal{D}_{m_{i}}$ whose outside answers are advice independent of $B_{i}$, so it cannot read $L_{i}$ below the static floor $g_{Q}(m_{i})$. The blocks are disjoint, so the reads sum. ∎

For abstract keys $g_{Q}(m)=\Theta(\log(m+1))$: computing $q$’s position in a block is the comparison search of Lemma 4.2, and a balanced tree matches it. The lemma then gives $\widehat{R}_{Q}\geq c\sum_{k}\log_{2}(m_{k}+1)$ for rank, range-count, and unit-weight range-sum, the floor predecessor pays; Section 6 reuses it with the integer and two-dimensional complexities.

Lemma 4.10 (Idempotent Direct Sum).

Fix a public size vector of an oblivious schedule and a bounded-error evaluator. For bounded-interval emptiness, range-minimum, and range-maximum, each with a fixed empty-interval default, some strongly materialized state with these sizes admits an interval $I$, empty in every level, on which the evaluator has $\widehat{R}_{Q}\geq c\sum_{i}\log_{2}(m_{i}+1)$, for an absolute $c>0$.

Proof.

Give each level $i$ a low slab below $I$, a high slab above $I$, and a hot subinterval inside $I$, the zones disjoint and ordered publicly by level index. Draw $J_{i}$ uniform in $\{0,\dots,m_{i}\}$, placing the first $J_{i}$ keys of level $i$ in its low slab and the rest in its high slab, so $I$ holds no key and $Q$ returns its default; the hot alternative, when $J_{i}<m_{i}$, moves $k_{i,J_{i}+1}$ into the hot subinterval, making $I$ nonempty and changing the answer while every other encoding stays fixed. Expose every level but $i$ in its empty state for free; deciding the global answer is then deciding whether level $i$ leaves $I$ empty, the task of Lemma 4.2 with $m=m_{i}$ and the endpoints of $I$ as the two handles, at $\Omega(\log(m_{i}+1))$ reads. By no-cascade, reads elsewhere do not help, the per-level costs sum, and some all-empty realization meets the bound. ∎

Theorem 4.11 (Order/Range Dichotomy).

A decomposable query has $E\widehat{R}_{Q}=\Theta(n\log^{2}n)$ when an $O(1)$-read per-level certificate settles it and some unread live level can change its answer, as with membership, point-emptiness, and the global extrema. When instead a level’s contribution turns on $q$’s position among its keys, as for predecessor and successor, the additive rank, range-count, and range-sum, and the idempotent range-minimum, range-maximum, and bounded-interval emptiness, the cost is $E\widehat{R}_{Q}=\Theta(n\log^{3}n)$, with no hypothesis on frontier or budget.

Proof.

On the cheap side an $O(1)$-read witness per level answers $Q$ in $\widehat{R}=O(W)=O(\log n)$, hence $E\widehat{R}=O(n\log^{2}n)$, matched below by the membership floor of Section 3, whose argument needs only that an unread live level can flip the answer. On the order-localizing side the direct sums give $\widehat{R}_{Q}\geq c\sum_{k}\log_{2}(m_{k}+1)\geq c\max\{\widehat{W},\Phi(t)\}$ on the hard skeleton, predecessor’s own floor: Lemma 4.3 for predecessor and, reversing the order, successor; Lemma 4.9 for rank, range-count, and unit-weight range-sum; and Lemma 4.10 for the idempotent range queries, where an empty query interval keeps every level pivotal. The three-case assembly of Theorem 4.8 then gives $\Omega(n\log^{3}n)$, attained by binary Bentley–Saxe carrying the matching static structure, a balanced tree with subtree counts, prefix sums, or subtree extrema, at $O(\log m_{k})$ reads per level. ∎

The line is hashing against within-level order localization: membership and point-emptiness fall to a hash and the global extrema to one stored value per level, all staying at the membership scale, while predecessor, successor, rank, and the range queries turn on $q$’s position in each block and pay the logarithm. Total count is cheaper still, fixed by the public size sequence at $\Theta(n)$ and lying outside the trichotomy. What the logarithm buys is exactness of the order. The idempotent range queries carry no group inverse, yet an empty interval makes every level pivotal, a single hidden key overturning the empty default (Lemma 4.10). The logarithm is specific to abstract keys, and integer keys shrink it (Section 7).

Remark 4.12.

Approximation refunds the logarithm. A constant additive error $\pm\varepsilon n$ in rank or range-count, or a constant-precision quantile, separates only $O(1)$ buckets and returns the query to $\Theta(n\log^{2}n)$; a multiplicative error keeps the $\Theta(\log n)$ scales and does not.

5 Select

Select proves the third line of Theorem 1.1. Rank is a key-to-count query: the query names a key, and each component contributes a count. Select is the inverse, count-to-key query: the query names only a rank $K$. In a one-way scan the key that will become the answer is not known while the early components are being read, and those components cannot be revisited once later components reveal the threshold. The proof first turns this target-free difficulty into a cut lower bound, and then shows that any structure with small read cost must repeatedly rebuild a large component.

5.1 The Cut Lower Bound

Comparisons among keys already read are free, so the lower bound is on token reads and the adversary attacks the unread keys. Two lemmas carry it: a bound on what a single read can learn, and a cut floor built from it.

Lemma 5.1 (Layout Channel).

Fix the query $q$, its coins $\rho$, and the data-independent initial configuration $\sigma_{0}$. Condition on the public layout $G$, the size sequence together with the boundaries and addresses it determines. Let $X$ be the stored keys and values. For one run of the forward head let $T$ be its number of charged reads and $\Pi$ its transcript, the words it reads together with its working memory. Then

$$I(X;\Pi\mid G,q,\rho,\sigma_{0})\ \leq\ b\,\mathbb{E}[T\mid G,q,\rho,\sigma_{0}]\ =\ O\bigl(\mathbb{E}[T]\log n\bigr),\qquad b=\Theta(\log n).$$

Proof.

Assume $\mathbb{E}[T]<\infty$, the bound being vacuous otherwise, and condition throughout on $G,q,\rho,\sigma_{0}$. Record the $k$-th read as $V_{k}=(A_{k},Y_{k})$ when $T\geq k$ and $V_{k}=\bot$ otherwise, where $A_{k}$ is the address read and $Y_{k}$ the returned word together with the rank of its handle among those already read. A query reads at most the $O(n)$ live tokens, so that rank fits in $O(\log n)$ bits and $Y_{k}$ carries $b=\Theta(\log n)$ bits. The head is deterministic given the conditioning and the reads so far, so $\{T\geq k\}$ is a function of $V_{1},\dots,V_{k-1}$, and on that event so is $A_{k}$, by Definition 2.1; hence $I(X;A_{k}\mid V_{<k})=0$ and

$$I(X;V_{k}\mid V_{<k})\ \leq\ H(Y_{k}\mid V_{<k},A_{k})\ \leq\ b.$$

Since $V_{k}=\bot$ when $T<k$, chain rule over $k\leq K$ gives $I(X;V_{1},\dots,V_{K})\leq b\,\mathbb{E}[\min\{T,K\}]$, and $K\to\infty$ gives $I(X;V^{\infty})\leq b\,\mathbb{E}[T]$. The transcript $\Pi$ is a function of the conditioning and $V^{\infty}$ by the no-external-state clause, so data processing closes it. ∎

For a randomized evaluator the coins $\rho$ are independent of the data, so the bound holds with $\Pi$ replaced by $(\Pi,\rho)$ and averaged over $\rho$.

Lemma 5.2 (Cut Floor).

At a boundary between two active levels holding $P$ samples before it and $U$ after, with $r=\min\{P,U\}$, some input family with the fixed rank $K=r$ forces every bounded-error forward evaluator to read $\Omega(r)$ words before the head crosses the boundary. Hence $\widehat{R}(t)\geq c\min\{P,U\}$ for an absolute $c>0$.

Proof.

Place $r$ disjoint rank slabs $S_{1}<\cdots<S_{r}$ in the prefix, separated by fixed guards, each slab a public range of $Q=\mathrm{poly}(n)$ candidate positions. Draw $a_{i}$ uniformly and independently from $S_{i}$ and let $Z_{i}$ name which candidate it is, so the $a_{i}$ carry $\Theta(\log n)$ independent bits while their order is fixed by the slabs; put the remaining $P-r$ prefix keys in a guard above $S_{r}$. The suffix encodes an index $j$: place $r-j$ of its keys in a guard below $S_{1}$ and the rest in a guard above every slab. Then exactly $r-j$ suffix keys and the $j-1$ slab keys $a_{1},\dots,a_{j-1}$ lie below $a_{j}$, so $a_{j}$ is the $r$-th smallest of the union, while the prefix and the rank $K=r$ are the same for every $j$.

Draw $J$ uniform on $\{1,\dots,r\}$, independent of $Z=(Z_{1},\dots,Z_{r})$, and let $\Pi$ be the transcript and coins at the crossing. The prefix is read before the crossing and does not depend on $J$, so $(Z,\Pi)$ is independent of $J$, while the suffix determines $J$ and carries nothing about $Z$. A correct answer is $a_{J}$, which the head must decode from $\Pi$ and $J$ since it cannot return to the prefix; so by Fano $I(Z_{j};\Pi\mid J=j)=\Omega(\log Q)$, equal to $I(Z_{j};\Pi)$ by that independence. As the $Z_{i}$ are independent, $I(Z;\Pi)\geq\sum_{i}I(Z_{i};\Pi)=\Omega(r\log n)$. Since $Z$ is a function of $X$ and $\Pi$ records the $T$ prefix reads, Lemma 5.1 gives $\Omega(r\log n)\leq I(X;\Pi)\leq O(\mathbb{E}[T]\log n)$, whence $\mathbb{E}[T]=\Omega(r)$ and $\widehat{R}(t)\geq\mathbb{E}[T]=\Omega(r)$. ∎

The guards keep the suffix clear of $Z$: its keys sit in fixed slabs, never at a data-dependent value such as $a_{1}-1$, so reading them reveals only $j$. The floor rests on a single boundary rather than a sum over levels, but that is enough: a low read cost forces almost all the live mass into one level, and building such a level again and again is the write cost select cannot escape.

5.2 The Quadratic Product Bound

Theorem 5.3 (Quadratic Select Bound).

Strongly materialized select has $\min E\widehat{R}=\Theta(n^{2})$, with no hypothesis on the frontier or the budget, and the lower bound holds against bounded-error query randomization.

Proof.

Write $R=\widehat{R}$ and $s=R/c$ with $c$ the constant of Lemma 5.2. The live mass at time $t$ is $t$, and at every boundary $\min\{P,U\}\leq R/c=s$, so each boundary mass is $\leq s$ or $\geq t-s$ and the level spanning $(s,t-s)$ has size $\geq t-2s$. Suppose $R<cn/8$, so $s<n/8$ and this level exceeds $n/4$ throughout $t\in[n/2,n]$. Take times $t_{0}<\cdots<t_{K}$ spaced by $\Delta=\lceil 2s\rceil+1$ across $[n/2,n]$, so $K=\Omega(n/(R+1))$, and let $B_{i}$ be such a level at $t_{i}$. Being materialized, $B_{i}$ is static after birth, so $|B_{i}|\leq t_{i}$, while $|B_{i+1}|\geq t_{i+1}-2s>t_{i}\geq|B_{i}|$; the $B_{i}$ are therefore distinct components, each costing $\Omega(n)$ tokens. Hence $E=\Omega(n^{2}/(R+1))$, and as $R\geq 1$, $E\widehat{R}=\Omega(n^{2})$. If instead $R\geq cn/8$, then $E\geq n$ gives $E\widehat{R}\geq nR=\Omega(n^{2})$.

For the upper bound, keep one large level together with a suffix of at most $s$ singletons, merging the whole stack into a fresh large level every $s$ inserts. Rebuilding the large level at sizes $s,2s,\dots,n$ costs $E=\Theta(n^{2}/s)$. The large level is a sorted array at the bottom of the stack, so the head jumps by a computed offset to its rank window $[K-s,K]$, reads those $O(s)$ keys and the $\leq s$ singletons above it, and selects the global $K$-th among these candidates; the suffix shifts the answer’s rank in the large level by at most $s$, so the window holds it, and $\widehat{R}=O(s)$. The product is $\Theta(n^{2})$ at every $s$. ∎

The three query classes now stand in a line, governed by what the query hands the forward head:

$$\underbrace{\Theta(n\log^{2}n)}_{\text{hashable witness}}\ <\ \underbrace{\Theta(n\log^{3}n)}_{\text{known key value}}\ \ll\ \underbrace{\Theta(n^{2})}_{\text{rank, no value}}.$$

The first gap is one logarithm, order against hashing; the second is a polynomial. Rank and select are inverse operations, value to count and count to value, and select’s answer is itself only $O(\log n)$ bits, yet the two lie polynomially apart: the gap is made by the access discipline, not by the information in the answer. Select gives the head no value to aim at, and that is what carries it past predecessor. The cost is a property of the forward head, not of the key model, since the lower bound forces its reads by completing cells the head has not visited and never inspects a key’s representation; it therefore persists for integer keys, where the word-RAM that narrows the order gap (Section 7) still leaves select at $\Theta(n^{2})$.

Remark 5.4.

Constant-precision approximate select returns to $\Theta(n\log^{2}n)$: $O(1)$ samples at evenly spaced ranks per level form a forward quantile sketch that needs no refinement of the exact threshold. Exactness is again what the cost buys.

Remark 5.5.

A cross-level rank catalog would answer select in $O(1)$ reads, but maintaining one under LIFO appears to cost $\Omega(n^{2})$ edits, which would carry the $\Theta(n^{2})$ past strong materialization, as Appendix A carries the predecessor bound. We do not pursue this here.

6 Static-to-Dynamic Lifting

Every query that pays the order logarithm lands at the same $n\log^{3}n$, and a single reduction explains why. Each pays, per level, the static cost of locating the query there; the model forbids sharing those costs across levels, so the read cost is their sum. For an additive query this turns the dynamization optimum into the edit budget times a sum of static per-level complexities, a dynamic lower bound read off a static one. The logarithm of order is then no artifact of maintenance but the faithful lift of a static search cost.

Recall the additive queries of Section 4: $Q(S,q)=\sum_{k}\varphi(L_{k},q)$ in an abelian group, with per-level static complexity $g_{Q}(m)$, the reads to compute one block’s contribution $\varphi(B,q)$, and embeddable hard families that plant an independent instance in every level (Lemma 4.9).

Theorem 6.1 (Static-to-Dynamic Lifting).

Let $Q$ be additive with an embeddable hard family and per-level static complexity $g_{Q}$: a forward-readable static structure attains $O(g_{Q}(m))$ reads on a block of $m$ samples, and the hard family forces $\Omega(g_{Q}(m))$. Suppose $g_{Q}$ is nondecreasing with $g_{Q}(1)\geq 1$ and of linear rate $g_{Q}(2^{k})\geq\zeta_{Q}k$ for some $\zeta_{Q}\in(0,1]$, and $\Sigma_{Q}:=\sum_{k=0}^{L}g_{Q}(2^{k})=\Theta(\zeta_{Q}L^{2})$. Then every strongly materialized algorithm with a bounded-error evaluator has $E\widehat{R}_{Q}=\Omega(nL\,\Sigma_{Q})$, which binary Bentley–Saxe attains, so $\min E\widehat{R}_{Q}=\Theta(nL\,\Sigma_{Q})$.

Proof.

For the upper bound, the schedule of Proposition 2.3 carries that static structure at each level, a balanced search tree with subtree aggregates for the additive queries. Then $E=O(nL)$, and the active sizes being distinct powers of two, $\widehat{R}=\sum_{\text{active}}g_{Q}(m_{k})\leq\Sigma_{Q}$.

For the lower bound, Lemma 4.9 gives $\widehat{R}_{Q}(t)=\Omega(\sum_{k}g_{Q}(m_{k}))$; the linear rate and the floor $g_{Q}(1)\geq 1$ make $g_{Q}(m)=\Omega(\zeta_{Q}\log_{2}(m+1))$, so $\widehat{R}_{Q}(t)=\Omega(\zeta_{Q}\sum_{k}\log_{2}(m_{k}+1))$. Since $\sum_{k}\log_{2}(m_{k}+1)$ is at least $\max\{W(t),\Phi(t)\}$ at all times and at least $\log_{2}(n+1)$ at the final skeleton, this is the read floor of Theorem 4.8 scaled by $\zeta_{Q}$. Its three-case assembly then applies: the wide-frontier and deep cases exceed $nL\,\Sigma_{Q}$, while the size-diversity case multiplies $E=\Omega(n\widetilde{D})$ by $\widehat{R}_{Q}=\Omega(\zeta_{Q}\Phi(t^{\star}))$ and closes with the calculus floor, giving $E\widehat{R}_{Q}=\Omega(\zeta_{Q}nL^{3})$. By $\Sigma_{Q}=\Theta(\zeta_{Q}L^{2})$ this is $\Omega(nL\,\Sigma_{Q})$. ∎

The middle row of the dichotomy is now one line: abstract-key rank, range-count, and range-sum have $g_{Q}(m)=\Theta(\log(m+1))$, hence $\zeta_{Q}=\Theta(1)$, $\Sigma_{Q}=\Theta(L^{2})$, and $E\widehat{R}_{Q}=\Theta(nL^{3})$. Beyond recovering the additive row of Theorem 4.11, the lift names the structure of the cost. The read bound is a sum of per-level static search complexities, one term per live level, because maintenance does not inflate a single level and cascading cannot amortize across them. That this sum cannot stay below $L^{2}$ is the size-diversity of Section 4: that section forces the sum up to $L^{2}$, and this one explains why the cost is that sum to begin with. The two are the lower-bound face and the structural face of the same $n\log^{3}n$.

Remark 6.2.

Because $g_{Q}$ is a static complexity, the lift inherits its robustness. Integer keys shrink the per-level cost and carry rank, range-count, and range-sum to $\Theta(n\log^{2}n\,\lambda)$, while in two dimensions the separation survives the word-RAM, with range-counting lifting to $\Theta(n\log^{3}n/\lambda)$ and no van Emde Boas collapse. Both are taken up in Section 7; the integer bounds there are deterministic and Las Vegas cell-probe bounds, and a bounded-error version would need a randomized static one. Integer predecessor, not an additive query, stays open between $1$ and $\lambda$.

7 Access-Model Variants

The classification of Theorem 1.1 is stated for abstract keys, strong materialization, and one-way scans. This section records what happens when one restriction at a time is relaxed. These variants are not needed for the main lower bounds; their role is to identify which assumption supports which separation. Integer keys reduce the cost of searching inside one component, random access removes the no-backseek obstruction for select, and non-materialized cascade data can remove the abstract-key predecessor gap only when it can be placed in the right scan direction.

7.1 Integer Keys

The order logarithm is a property of abstract keys. An integer key in a $\mathrm{poly}(n)$ universe is searchable by the word-RAM: a y-fast trie [WIL83] finds a run’s predecessor in $O(\lambda)$ reads and a fusion tree [FW93] in $O(\log_{w}m)$, both forward-readable under the round-by-round layout of Proposition 2.3. Predecessor then costs $O(n\log^{2}n\,\lambda)$, its gap over membership shrinking from $\log n$ to $\lambda$, the way radix sort undercuts the comparison-sorting bound without refuting it.

The collapse is not uniform, and the reason is what makes abstract keys hard. An additive query reads a contribution from every run, so each must still be searched, now at integer cost: a rank among $m$ integers is a predecessor search of static complexity $g_{Q}(m)=\Theta(\max\{1,\min\{\log_{w}m,\lambda\}\})$, the y-fast or fusion layout above, matched from below by Pătraşcu and Thorup [PT06, PT07], deterministic and Las Vegas alike. Its binary sum is $\Sigma_{Q}=\Theta(L\lambda)$, so Theorem 6.1 places rank, range-count, and range-sum at $\Theta(n\log^{2}n\,\lambda)$, exactly $\Theta(\lambda)$ above membership.

Predecessor behaves differently, because it is settled by one run and the rest need only be ruled out. Ruling run $i$ out asks whether its keys meet the open interval between the running best and the query, a range-emptiness test the word-RAM answers in $O(1)$ [ABR01] but abstract keys answer only by localizing the query inside the run, the $\Omega(\log m)$ step behind the separation. The per-run floor the additive queries enjoy has no integer counterpart here, so the exact integer cost of predecessor stays open between $\Theta(n\log^{2}n)$ and $\Theta(n\log^{2}n\,\lambda)$: a lower bound would need a per-run test that stays $\Omega(\lambda)$-hard under a shared query, and the natural one, whether a run holds the global predecessor, is the range-emptiness integers trivialize. The key models part exactly here.

The collapse is one-dimensional. Two-dimensional range-counting has no van Emde Boas speedup, its cell-probe complexity $\Omega(\log m/\lambda)$ at linear space [PĂT07] matched by a range tree [CW13], so lifting $g_{Q}(m)=\Theta(\max\{1,\log m/\lambda\})$ gives $\Theta(n\log^{3}n/\lambda)$, above membership by $\Theta(\log n/\log\log n)$. The separation is no artifact of unstructured keys; it survives the word-RAM as soon as the queries are two-dimensional, the regime of multi-attribute range search.

7.2 Random Access

Select’s polynomial cost is the signature of the no-backseek head, and it vanishes the moment the head may return. The obstacle of Section 5 was an answer threshold fixed by all runs jointly, unknown while the early runs are read and out of reach once they are passed. Random access dissolves it: selecting the $K$-th smallest among $W$ sorted runs is classical, $O(W+\sum_{i}\log(k_{i}+1))$ comparisons with $k_{i}$ run $i$’s share of the $K$ smallest [FJ84, KKZ+19], which is $O(\log^{2}n)$ at a diverse frontier. Random access brings select to $O(n\log^{3}n)$, the value-query scale, with the exact optimum open.

It does no more. Predecessor stays at $n\log^{3}n$ under random access, since every run must still be searched and no run bridges another; what random access removes is not the per-run search but the forward head’s inability to revisit an internal threshold. The polynomial gap was never in the answer, only $O(\log n)$ bits, but in that head. Integer keys touch only the order gap and random access only select’s, each isolating the restriction its cost depends on.

7.3 Deque Edits

Two restrictions guard the cascade together: a merge stack writes only at the tail, behind the forward head, and its runs are strongly materialized, so no run holds order information about another. Relax both, writing at the head’s entrance and letting a young run carry a bridge into an older one, and the separation falls. Model the relaxation by deque edits, popping and pushing at both ends of the token string while keeping the same forward, no-backseek head, so a component may be placed at position $0$, ahead of all the head will read; dropping materialization lets a young component store forward handles into the older ones, charged to itself and leaving them unedited.

Theorem 7.1 (Deque Collapse).

Under deque edits with non-materialized runs, abstract-key predecessor admits $E=O(n\log n)$ and $\widehat{R}=O(\log n)$, so $E\widehat{R}=O(n\log^{2}n)$ and the order separation does not survive.

Proof.

A useful bridge from $U$ to $V$ needs $U$ read before $V$ and $V$ born before $U$ (Lemma 4.1), and tail-only editing forces birth order to follow read order and kills it. Deque editing breaks the coupling: lay the active components youngest first, $C_{1},\dots,C_{W}$ read from the entrance with sizes increasing, so $C_{i}$ is read before $C_{i+1}$ while $C_{i+1}$ is the older of the two, the very order a cascade needs.

Run the Bentley–Saxe counter mirrored at the entrance: each insert pushes a singleton at the front and, while the two front components share a size, pops both, merges, and pushes one new front component. A component of $m_{i}$ keys stores its block as a forward-readable balanced tree together with a sparse cascade into the next-older $C_{i+1}$, cutting $C_{i+1}$’s order into $O(m_{i})$ intervals of span $O(m_{i+1}/m_{i})$ and keeping, per gap, its own predecessor and a forward handle to the precomputed subtree root for that interval, a node already inside $C_{i+1}$’s own tree once $C_{i+1}$ is serialized. Each older component is serialized once with its roots in place, so a younger one adds only its $O(m_{i})$ handles and never edits it; hence $|C_{i}|=O(m_{i})$, and a size-$2^{k}$ component is built $O(n/2^{k})$ times at $O(2^{k})$ tokens, giving $E=O(n\log n)$.

A query searches $C_{1}$ in $O(\log m_{1})$ reads, then at each step follows the stored handle into a span-$O(1+m_{i+1}/m_{i})$ interval of $C_{i+1}$, searches its precomputed subtree, updates the best candidate, and moves on without a backseek. The reads telescope,

$$\widehat{R}=O\Bigl(W+\log m_{1}+\sum_{i\geq 2}\log\tfrac{m_{i}}{m_{i-1}}\Bigr)=O(W+\log m_{W})=O(\log n).$$

The collapse is one of orientation, not of two-endedness. A queue, pushing at the tail and popping at the head, still builds its components at the scan’s exit and does not collapse; what predecessor pays for under tail-only editing is exactly that new connectivity must be written behind the forward scan. The gap sits at the missing cascade, and the cascade the deque builds is non-materialized, a young run pointing into an older one, so the collapse leaves the materialized model and the tail-only one together. This is why random access alone, the runs still materialized, does not collapse predecessor. The matching $\Omega(n\log^{2}n)$ would follow from a membership floor under deque edits, a minor open point since the universal floor of Section 3 uses the tail-only frontier-width bound; the collapse out of the $n\log^{3}n$ class is unconditional. Afshani’s dynamic fractional-cascading lower bound [AFS21] forces either fully dynamic updates or worst-case update time, while the construction here is insertion-only with amortized rebuilding, and escapes both.

7.4 LSM Read Amplification

The read gap reaches past edit histories to the runs in place at query time, though only under a hypothesis. A run can hold its median and successor in an $O(1)$ header and certify a query’s predecessor between them in two reads, so overlap by itself does not force $\Omega(\log m)$. What forces it is a query distribution that, after all outside information is exposed, still leaves $\Omega(\log m_{i})$ of conditional gap entropy in each overlapped run, exactly the hard family of Lemma 4.3.

Proposition 7.2 (Range Read Amplification).

Let $R_{1},\dots,R_{W}$ be active sorted runs, pairwise no-cascade and carrying no cross-run order catalog. Under a query distribution leaving $\Omega(\log m_{i})$ conditional gap entropy in each run it crosses, an exact range or order query has read cost $\Omega(\sum_{i\,\text{crossed}}\log m_{i})$, while a point query is settled in $O(1)$ reads per run.

Proof.

The runs being mutually no-cascade, a read in one cannot shrink another’s candidate set, so the per-run argument of Lemma 4.3 applies to each crossed run on its own: the hypothesis leaves $\Omega(\log m_{i})$ gap entropy there, and localizing the endpoint costs $\Omega(\log m_{i})$ reads. Carrying no cross-run catalog, the runs do not share the cost, so it sums. A point query is hashable, settled per run by a filter in $O(1)$. ∎

The bound is on the runs in place, however built or read, so it holds in any leveled sorted-run store. In a random-access store the per-run filters let a point query skip the runs that miss it, leaving $O(1)$; a range or order query has no such escape and localizes in every run it spans, $\Theta(\log_{T}n)$ at size-ratio $T$, each localization irreducible by Proposition 7.2. This range-over-point read amplification closes only two ways: a word-RAM integer index, which serves one dimension but not string keys or multi-attribute ranges, where the two-dimensional separation persists; or a cross-run order catalog, which appears to need $\Omega(n^{2})$ edits to maintain under append-only merging (Section 5). For write-optimized, string-keyed, multi-attribute engines in the insertion-only regime the model captures, the range read amplification is intrinsic, not an implementation artifact [OCG+96, DAI17, DI18].

7.5 Intermediate Queries

Order at $n\log^{3}n$ and select at $n^{2}$ are the two ends of one scale, set by how much the query reveals about its answer’s value. A $w$-windowed select, the $K$-th smallest under the promise that the answer lies in a bracket of at most $w$ live keys, interpolates between them: at $w=O(1)$ the bracket pins the value and the query is an order query, at $w=N$ the promise is empty and it is select. The cut floor restricted to the bracket gives $\widehat{R}\geq c\min\{w,P,U\}$, only the $\leq w$ keys inside it carrying the ambiguity. The matching write tradeoff across the interior is open, and we leave it.

References

[AFS21] P. Afshani (2021) A lower bound for dynamic fractional cascading. In SODA, pp. 2229–2248. Note: arXiv:2011.00503 External Links: Document Cited by: Remark A.5, §1.3, §1, §1, §4.1, §7.3.
[ABR01] S. Alstrup, G. S. Brodal, and T. Rauhe (2001) Optimal static range reporting in one dimension. In STOC, pp. 476–482. External Links: Document Cited by: §7.1.
[AKM+16] M. Athanassoulis, M. S. Kester, L. M. Maas, R. Stoica, S. Idreos, A. Ailamaki, and M. Callaghan (2016) Designing access methods: the RUM conjecture. In EDBT, pp. 461–466. External Links: Document Cited by: §1.3, §1, §1.
[BCF+24] M. A. Bender, A. Conway, M. Farach-Colton, H. Komlós, M. Koucký, W. Kuszmaul, and M. Saks (2024) Nearly optimal list labeling. In FOCS, pp. 2253–2274. Note: arXiv:2405.00807 External Links: Document Cited by: §1.3.
[BCF+22] M. A. Bender, A. Conway, M. Farach-Colton, H. Komlós, W. Kuszmaul, and N. Wein (2022) Online list labeling: breaking the $\log^{2}n$ barrier. In FOCS, pp. 980–990. Note: arXiv:2203.02763 External Links: Document Cited by: Appendix A, §1.3.
[BFF+07] M. A. Bender, M. Farach-Colton, J. T. Fineman, Y. R. Fogel, B. C. Kuszmaul, and J. Nelson (2007) Cache-oblivious streaming B-trees. In SPAA, pp. 81–92. External Links: Document Cited by: Remark A.5, §1.
[BS80] J. L. Bentley and J. B. Saxe (1980) Decomposable searching problems I: static-to-dynamic transformation. Journal of Algorithms 1 (4), pp. 301–358. External Links: Document Cited by: §1.3, §1, §1, §2.6.
[BDH+23] S. Borst, D. Dadush, S. Huiberts, and D. Kashaev (2023) A nearly optimal randomized algorithm for explorable heap selection. In IPCO, Lecture Notes in Computer Science, Vol. 13904, pp. 29–43. Note: arXiv:2210.05982 External Links: Document Cited by: §1.3.
[BF03] G. S. Brodal and R. Fagerberg (2003) Lower bounds for external memory dictionaries. In SODA, pp. 546–554. External Links: Document Cited by: §1.3, §1.
[BKS12] J. Bulánek, M. Koucký, and M. Saks (2012) Tight lower bounds for the online labeling problem. In STOC, pp. 1185–1198. Note: arXiv:1112.5636 External Links: Document Cited by: Appendix A, §1.3, §3.1.
[CW13] T. M. Chan and B. T. Wilkinson (2013) Adaptive and approximate orthogonal range counting. In SODA, pp. 241–251. External Links: Document Cited by: §1.3, §7.1.
[CDG+08] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber (2008) Bigtable: a distributed storage system for structured data. ACM Transactions on Computer Systems 26 (2), pp. 1–26. External Links: Document Cited by: §1.3, §1.
[CG86] B. Chazelle and L. J. Guibas (1986) Fractional cascading: I. A data structuring technique. Algorithmica 1 (2), pp. 133–162. External Links: Document Cited by: §1.3, §1, §1.
[DIN22] R. Das, J. Iacono, and Y. Nekrich (2022) External-memory dictionaries with worst-case update cost. In ISAAC, LIPIcs, Vol. 248, pp. 21:1–21:13. Note: arXiv:2211.06044 External Links: Document Cited by: §1.3.
[DAI17] N. Dayan, M. Athanassoulis, and S. Idreos (2017) Monkey: optimal navigable key-value store. In SIGMOD, pp. 79–94. External Links: Document Cited by: §1.3, §1, §1, §7.4.
[DI18] N. Dayan and S. Idreos (2018) Dostoevsky: better space-time trade-offs for LSM-tree based key-value stores via adaptive removal of superfluous merging. In SIGMOD, External Links: Document Cited by: §1.3, §1, §1, §7.4.
[FJ84] G. N. Frederickson and D. B. Johnson (1984) Generalized selection and ranking: sorted matrices. SIAM Journal on Computing 13 (1), pp. 14–30. External Links: Document Cited by: §1.3, §7.2.
[FS89] M. L. Fredman and M. E. Saks (1989) The cell probe complexity of dynamic data structures. In STOC, pp. 345–354. External Links: Document Cited by: §1.3, §1.
[FW93] M. L. Fredman and D. E. Willard (1993) Surpassing the information theoretic bound with fusion trees. Journal of Computer and System Sciences 47 (3), pp. 424–436. External Links: Document Cited by: §7.1.
[IKR81] A. Itai, A. G. Konheim, and M. Rodeh (1981) A sparse table implementation of priority queues. In ICALP, Lecture Notes in Computer Science, Vol. 115, pp. 417–431. External Links: Document Cited by: §1.3.
[KKZ+19] H. Kaplan, L. Kozma, O. Zamir, and U. Zwick (2019) Selection from heaps, row-sorted matrices, and $X+Y$ using soft heaps. In SOSA, OASIcs, Vol. 69, pp. 5:1–5:21. External Links: Document Cited by: §1.3, §7.2.
[KO25] Y. K. Ko (2025) Unifying the landscape of super-logarithmic dynamic cell-probe lower bounds. Note: arXiv:2510.17717; ECCC TR25-156 Cited by: §1.3.
[LM10] A. Lakshman and P. Malik (2010) Cassandra: a decentralized structured storage system. ACM SIGOPS Operating Systems Review 44 (2), pp. 35–40. External Links: Document Cited by: §1.3, §1.
[LAR12] K. G. Larsen (2012) The cell probe complexity of dynamic range counting. In STOC, pp. 85–94. Note: arXiv:1105.5933 External Links: Document Cited by: §1.3, §1.
[LPG+17] L. Lu, T. S. Pillai, H. Gopalakrishnan, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau (2017) WiscKey: separating keys from values in SSD-conscious storage. ACM Transactions on Storage 13 (1), pp. 1–28. External Links: Document Cited by: §1.3, §1.
[LCK+20] S. Luo, S. Chatterjee, R. Ketsetsidis, N. Dayan, W. Qin, and S. Idreos (2020) Rosetta: a robust space-time optimized range filter for key-value stores. In SIGMOD, pp. 2071–2086. External Links: Document Cited by: §1.3, §1, §1.
[MRY+21] C. Mathieu, R. Rajaraman, N. E. Young, and A. Yousefi (2021) Competitive data-structure dynamization. In SODA, pp. 2269–2287. Note: arXiv:2011.02615 External Links: Document Cited by: §1.3, §1.
[MDL20] Y. Matsunobu, S. Dong, and H. Lee (2020) MyRocks: LSM-tree database storage engine serving facebook’s social graph. Proceedings of the VLDB Endowment 13 (12), pp. 3217–3230. External Links: Document Cited by: §1.3, §1.
[MN90] K. Mehlhorn and S. Näher (1990) Dynamic fractional cascading. Algorithmica 5 (2), pp. 215–241. External Links: Document Cited by: §1.3, §1, §1.
[MEH81] K. Mehlhorn (1981) Lower bounds on the efficiency of transforming static data structures into dynamic structures. Mathematical Systems Theory 15 (1), pp. 1–16. External Links: Document Cited by: §1.3, §1, §1.
[MLI25] D. Mo, S. Luo, and S. Idreos (2025) How to grow an LSM-tree? Towards bridging the gap between theory and practice. Proceedings of the ACM on Management of Data 3 (3), pp. 173:1–173:25. Note: arXiv:2504.17178 External Links: Document Cited by: §1.3, §1.
[MP80] J. I. Munro and M. S. Paterson (1980) Selection and sorting with limited storage. Theoretical Computer Science 12 (3), pp. 315–323. External Links: Document Cited by: §1.3.
[OCG+96] P. O’Neil, E. Cheng, D. Gawlick, and E. O’Neil (1996) The log-structured merge-tree (LSM-tree). Acta Informatica 33 (4), pp. 351–385. External Links: Document Cited by: §1.3, §1, §7.4.
[OVE83] M. H. Overmars (1983) The design of dynamic data structures. Lecture Notes in Computer Science, Vol. 156, Springer. External Links: Document Cited by: §1.3, §1, §1.
[PT06] M. Pătraşcu and M. Thorup (2006) Time-space trade-offs for predecessor search. In STOC, pp. 232–240. Note: arXiv:cs/0603043 External Links: Document Cited by: §1.3, §7.1.
[PT07] M. Pătraşcu and M. Thorup (2007) Randomization does not help searching predecessors. In SODA, pp. 555–564. External Links: Document Cited by: §1.3, §7.1.
[PĂT07] M. Pătraşcu (2007) Lower bounds for 2-dimensional range counting. In STOC, pp. 40–46. External Links: Document Cited by: §1.3, §7.1.
[PĂT11] M. Pătraşcu (2011) Unifying the landscape of cell-probe lower bounds. SIAM Journal on Computing 40 (3), pp. 827–847. External Links: Document Cited by: §1.3, §1.
[RKC+17] P. Raju, R. Kadekodi, V. Chidambaram, and I. Abraham (2017) PebblesDB: building key-value stores using fragmented log-structured merge trees. In SOSP, pp. 497–514. External Links: Document Cited by: §1.3, §1.
[WIL83] D. E. Willard (1983) Log-logarithmic worst-case range queries are possible in space $\Theta(N)$. Information Processing Letters 17 (2), pp. 81–84. External Links: Document Cited by: §7.1.
[ZLL+18] H. Zhang, H. Lim, V. Leis, D. G. Andersen, M. Kaminsky, K. Keeton, and A. Pavlo (2018) SuRF: practical range query filtering with fast succinct tries. In SIGMOD, pp. 323–336. External Links: Document Cited by: §1.3, §1, §1.
[ZCW+21] W. Zhong, C. Chen, X. Wu, and S. Jiang (2021) REMIX: efficient range query for LSM-trees. In USENIX FAST, pp. 51–64. Note: arXiv:2010.12734 Cited by: §1.3, §1, §1.