Knowledge Discovery Efficiency (KEDE) and Ashby's Law of Requisite Variety

Abstract

Ashby's Law of Requisite Variety is rarely applied quantitatively, because a regulator's internal variety and knowledge cannot be observed directly. We adopt Ashby's black-box stance and ask what a system's external record reveals about the knowledge it lacked: the responses it commits, which of them survive evaluation against its goal, and the capacity spent producing them. We read regulation as a query problem. Each disturbance poses a relation between disturbances and acceptable responses, and the regulator must ask enough questions to commit a response that the relation admits. Building on the information-theoretic form of the law, we define Knowledge To Be Discovered (KTD) as the least conditional entropy, given the disturbance information available to the regulator, of a response class that successful regulation admits. The definition needs no unique target. KTD is the floor of a three-rung ladder: it bounds below the least expected number of binary questions any procedure needs, which in turn bounds below the regulator's actual protocol. The gaps between the rungs separate what the regulator does not know, what it cannot ask, and how it asks; a regulator can fail Ashby's requirement for lack of questions as well as for lack of knowledge. We model the staged selection that reduces KTD as a Knowledge Discovery Process, distinguish within-episode discovery from learning retained across episodes, and account for both in operational and knowledge ledgers. We then show that KTD, though latent, can be estimated from closure counts alone. For certainty-terminated binary protocols, the ratio of counted capacity to surviving closures gives, with high probability, an upper bound on average KTD; under bounded search it gives a two-sided bracket, and in an ideal regime it converges to KTD. Its bounded transform, Knowledge-Discovery Efficiency (KEDE), takes values in (0, 1]. KEDE makes requisite knowledge observable and factorises along the ladder into coding, question, and protocol efficiencies that indicate which of knowledge, questions, or protocol limits a regulator. We illustrate KEDE on typing, software development, a published learning curve, intelligence testing, basketball, and manual and industrial assembly, with a worked robot example in the appendix.

Introduction

The Law of Requisite Variety, formulated by W. Ross Ashby, states that for a system to effectively regulate its environment, it must have at least as much variety/complexity as its environment[1]. This principle is foundational in disciplines such as cybernetics, control theory, and machine learning.

The concept of requisite variety has since been applied across diverse domains, including organizational theory, ecology, and information systems. It underscores the necessity for systems to adapt to environmental complexity in order to maintain stability and achieve intended outcomes.

Real-world attempts to apply Ashby's Law of Requisite Variety face three persistent obstacles. (i) Combinatorial explosion: enumerating all relevant states of a system and its environment quickly becomes intractable, especially when hidden or unmeasured variables are present. (ii) Dual control dilemma: a regulator must simultaneously amplify its own control variety and attenuate external variety - an optimization that is delicate in multiscale, hierarchical, and time-varying settings such as digital ecosystems or military command structures. (iii) Resource constraints: limited data, computational power, and organisational capacity often preclude sophisticated control architectures. Existing remedies - markup-language state catalogues, iterative multidimensional sampling, and distributed self-organising controllers - mitigate but do not eliminate these limitations.

This article takes a different route around those obstacles. Instead of counting the states of a regulator, it counts the questions the regulator must ask. Ashby described selection as something that may proceed in stages, each stage removing alternatives[2], and he described intelligence, as measured by tests, as the “power of appropriate selection”, limited by the subject's capacity as a transducer, per second or per question[3]. Both remarks are statements about query complexity: how many discriminating operations it takes to settle on a response that works. Query complexity has a well-developed theory, with a worst-case setting, a distributional setting, and an information-theoretic lower bound that links the expected number of questions to entropy. We use that theory to turn the information-theoretic form of Ashby's law into a measurement.

The resulting question is dynamic. The static reading of the law asks whether a regulator has enough response variety. The query reading asks whether it is asking few enough, and good enough, questions to keep up with the variety of the disturbances it meets, and, when it is not, whether the shortfall lies in what it knows, in the questions available to it, or in how it uses them.

In section 2, we review the primary challenges in applying Ashby's Law to real-world systems, and propose a solution. It uses externally recorded closure and invalidation histories, together with a declared action-unit capacity convention, to count the questions a regulator spent and, under explicit bridging assumptions, to bound the knowledge it lacked.

In section 3, we present the Law of Requisite Variety in its set-based formulation, covering the Table of Outcomes, requisite variety, goal revision, and behavioral rework, and in its information-theoretic formulation, covering entropy, probabilistic success, residual variety bounds, response-equivalence, and the regulator's learned law of action. The section ends by collecting the success conditions into a regulatory relation, the problem the regulator's questions must solve, and by defining Knowledge To Be Discovered relative to it. Its worst-case and distributional query complexities, and the ladder they form with Knowledge To Be Discovered, are introduced there.

In section 4, we establish the Knowledge Discovery Process as a query process: an episode is one path through a decision procedure for the regulatory relation. We develop it in three layers: a set-based formulation of staged selection, covering episodes, closures, action units as queries, and rule regulation; an information-theoretic formulation, covering the expected and realized reduction of Knowledge To Be Discovered and the query ladder; and learning across episodes, covering comparable episodes, the Learning Axiom, the posterior-becomes-prior rule, and three adaptation channels, through knowledge, questions, and protocol.

In section 5, we develop the operational ledger and the knowledge ledger, which records each rung of the ladder at every closure, together with window-level accounting measures and a feedback-loop reading of all revisions. In section 6, we show how to bound Knowledge To Be Discovered from observable outcomes alone. We read the observed record as a program in two instructions, output and non-output, whose length is the counted capacity N(I) of the observation window, and derive the operational one-bit effective-depth estimator, the number of non-output instructions per surviving closure, together with its relationship to the latent entropy. In section 7, we generalize the Knowledge-Discovery Efficiency (KEDE), a scalar metric in (0, 1] that quantifies how efficiently a system closes the gap between the variety demanded by its environment and the variety embodied in its prior knowledge, and show that it factorises along the query ladder. In section 8, we reattach elapsed time and obtain discovery rates.

Finally, in section 9, we explore applications of KEDE across typing, software development, learning curves, intelligence testing, sports performance, and manufacturing. Each application declares what counts as one output and what counts as one unit of capacity. The applications differ in how their non-output units are observed: one by one, inferred from elapsed time, counted as failed attempts, or inferred from a rated capacity.

Core challenges in applying Ashby's Law to real systems

We conducted a literature review aimed at identifying the primary challenges and limitations associated with applying Ashby's Law in real-world systems.

A central challenge that emerges is the measurement of variety. In most of the reviewed literature, the concept of variety is either poorly defined or not explicitly measured, resulting in ambiguity and potential misinterpretation of the law's implications. Key obstacles to effective measurement include:

  • The direct measurement of variety is fundamentally incomputable for all but the simplest systems [14].
  • Hidden variables introduce uncertainty and complicate measurement efforts [15].
  • Trade-offs often arise between variety at different scales [16].
  • A combinatorial explosion occurs when attempting to enumerate all possible system states [15,16].
  • Resource limitations constrain the feasibility of comprehensive measurement [20].
  • Environmental complexity is frequently “unknowable,” preventing complete assessment [25].
  • Most studies lack explicit or standardized methods for quantifying variety [14,17'-20,25,27].
  • Existing approaches often lack rigorous quantitative validation [17].

Several measurement methods have been proposed, including:

  • Markup language-based variety estimation [18],
  • Iterative sampling techniques [21],
  • Entropy and determinism metrics to evaluate communication complexity, where greater variety was correlated with improved effectiveness [22],
  • Social network and cluster analysis to assess resilience [23], and
  • Multiple Correspondence Analysis (MCA) for capturing organizational complexity [24].

In addition, a subset of studies estimate variety through observed performance rather than structural attributes. Notable examples include:

  • Communication-based performance measures, employing determinism metrics to evaluate repeatable patterns in team behavior [22];
  • Team performance assessments, using task-based surveys to evaluate an organization's risk-handling capabilities [23];
  • Leadership behavior analysis, based on actual behavioral responses to simulated scenarios [26]; and
  • Relative performance comparisons, assessing organizational effectiveness across contexts using perception-based rather than absolute metrics [14].

While these performance-based approaches provide practical insights, they often rely on subjective or indirect indicators of variety, which may introduce biases and limit their generalizability. For example, performance outcomes may fail to account for hidden variables or the underlying complexity of the system [15]. Moreover, these approaches remain underrepresented in the literature, where structural and theoretical analyses still dominate.

In summary, although numerous methods for measuring variety have been proposed, no single comprehensive or universally accepted solution has emerged. Quantification remains a persistent challenge in the application of Ashby's Law to complex real-world systems.

Solution

These challenges significantly hinder the practical application of Ashby's Law. Whether considering a human, an AI model, or an organization, we are typically limited to observing external behavior rather than internal mechanisms - unless we are able to "open the box."

Ashby’s Law of Requisite Variety asks whether a regulator can respond adequately to the disturbances it faces[2]. He argued that while black boxes mimic the behavior of real objects, in practice, real objects are black boxes: we have always interacted with systems whose internal workings are, to some extent, unknown.

We propose treating the system as a black box, recording which committed responses produce outcomes that remain acceptable and how much counted capacity was spent producing them, and from that record bounding the knowledge the regulator lacked when it began each episode.

We cover staged selection within an episode and learning across episodes, including comparable episodes and the idea that a posterior model becomes a prior for subsequent episodes.

This is valuable because Ashby's Law can otherwise be read statically:

Does the regulator currently have enough response variety?
Instead, we support a dynamic question:
Is the regulator acquiring effective variety quickly enough to cope with the variety it encounters?
Each meaningful reduction in uncertainty represents discovered knowledge. Learning occurs when the outcome of one episode changes the system's prior knowledge for later, comparable episodes.

Regulation as a query problem

The obstacles above share one root: a regulator's variety and knowledge cannot be observed directly. Ashby's answer was that every real system is a black box [2]. We keep that stance and make it quantitative. We record the outcomes a regulator commits, which of them survive evaluation against its goal, and how much counted capacity it spent before committing them, and from this record alone we bound the knowledge it lacked. The static reading of Ashby's law asks whether a regulator has enough response variety. The measurement answers a dynamic question:

Is the regulator asking few enough, and good enough, questions to keep up with the variety it meets? If not, is the shortfall in what it knows, in the questions available to it, or in how it uses them?
Query complexity as a measurement frame

A body of work outside the cybernetics literature measures exactly the quantity these approaches lack: the number of discriminating questions needed to determine an answer. In decision-tree or query complexity, an algorithm learns about an unknown input only by querying it, and its cost is the number of queries. The theory distinguishes a worst-case measure, over every input, from a distributional measure, the expected number of queries under a known input distribution, and relates both to certificate complexity and related measures 74. The distributional measure has an information-theoretic floor: the expected number of binary questions is at least the entropy of what must be determined, and Huffman coding attains the floor to within one question, which is the “twenty questions” correspondence between search and source coding [Cover & Thomas §5.7]. Yao's minimax principle relates the distributional measure to the worst-case cost of randomized algorithms[75].

Three further strands bear on the present article. Information-based complexity measures the cost of solving a problem from partial information obtained by a declared repertoire of information operations, and so makes the choice of admissible questions part of the problem[76]. Searching with lies, which grew out of the Rényi–Ulam game, measures how many additional questions are needed when some answers may be false [77]. And Kolmogorov-complexity arguments give lower bounds on query complexity by counting the information a query algorithm must extract [78; 79], with a close and well-mapped relation between Kolmogorov complexity and Shannon entropy[80].

Two levels of questioning are involved, and they should not be confused. The observer questions the black box by perturbing it. The regulator, inside the box, questions its environment before it commits each response. The observer sees only the outcomes of the regulator's questioning, never the questions themselves, and the article's results bound the regulator's questions from what the observer can count.

1. The regulatory problem is a relation (Section 3)

We keep Ashby's Table of Outcomes T:D×R→Z, the map φ from outcomes to essential variables, and the acceptable region ηt. The regulatory relation Rt pairs each disturbance with every response whose outcome is acceptable. It is a relation, not a function: several responses may succeed, and the regulator needs only one. Responses are grouped into response-equivalence classes at the coarsest granularity that never changes success or failure, so the goal, not the observer, decides which distinctions count.

2. What is missing: Knowledge To Be Discovered (Section 3)

Each episode starts with an observation Y=y which represents the disturbance information available to the regulator and Mt its knowledge state, which supplies the probabilities. Knowledge To Be Discovered (KTD) is the minimum conditional entropy of the class to be fixed, under that knowledge state, over every admissible selector χ of an acceptable response class to a possible disturbance:

KTDrel(y)=minχ∈Sel(Rt)HMt(χ(Dt,y)∣Yt=y),KTDrel=∑yP(Y=y)KTDrel(y).

With a unique target X*, KTD is H(X*∣Y), and H(X*)=I(X*;Y)+H(X*∣Y) shows it as the part of the required response variety that the regulator's knowledge does not yet supply. KTD is zero exactly when one response class works for every disturbance still possible.

3. The query ladder (Sections 3–4)

A query is a map q:D→ Uq into a finite answer alphabet, binary unless stated otherwise. The query family Qt is the set of queries the regulator can execute. Answers here are deterministic functions of the disturbance; noisy answers and realized signal paths are treated with staged selection.

A decision procedure over Qt for observation y is a finite rooted tree ϑ whose internal nodes are labelled by queries in Qt , whose edges are labelled by answers, and whose leaves are labelled by response classes in Xt . Each d∈ Dt(y) follows the path its answers determine. Write ℓϑ(d) for the number of queries on that path and ϑ(d) for the class at its leaf.

A query is a discriminating operation, binary unless stated otherwise, and the query family Qt is the set the regulator can execute. Each query returns a signal about the current disturbance. The regulator uses that signal to refine its assessment of which response classes are acceptable. For deterministic binary queries, Theorem Q places KTD at the bottom of a ladder. The next rung is the least expected number of binary questions any protocol needs. Above that is the least number using only the regulator's own questions. At the top is the expected length of its actual protocol:

KTDrel(y)≤DTμy(Rt∣y)≤DTμyQt(Rt∣y)≤𝔼[K∣Y=y].

The gaps have distinct causes (Proposition Q2). The coding gap, below one question, exists because a question cannot ask for a fraction of a bit. A query-family gap means a question the regulator lacks. A protocol gap means questions it asks badly. A regulator can therefore fail Ashby's requirement in three ways: for lack of knowledge, for lack of questions, or by asking poorly. For example, a robot with posterior (12,14,18,18) over four moves needs 1.75 bits. If it always asks two questions, the extra 0.25 is a protocol gap when any question is available. It is a query-family gap when the sensors can only ask “vertical or horizontal?” and then “which of the two?”. The count is the same in both cases; the diagnosis and the remedy differ.

4. The Knowledge Discovery Process (Section 4)

An episode is one path through a decision procedure for the regulatory relation Rt, from the first query to the committed response. Each action unit (a reading, a look, an act of recall, a trial) instantiates a query and exposes a signal Ui. A realized episode has an associated conditioning path Y=y, U1=u1 , … , Uk=uk . . The episode ends at a closure, the commitment of a response. Three conditions hold. P1: each signal has at most m values. P2: the next query depends only on what has been observed so far. P3: the number of stages is a stopping time. Within an episode, KTD is the uncertainty stock present at the beginning of the realized conditioning path. Realized changes telescope exactly along the path (Lemma 0), and their expectation is the information the signals carry about the required class. A binary unit carries at most one bit in expectation (Proposition 0). So a protocol that stops only when its response is safe needs, on average, at least as many binary stages as there are bits to discover (Corollary 0′, Proposition Q1′):

KTDrel(y)≤𝔼[K∣Y=y]⋅log2m.

This is Ashby's additivity of selection[2] made exact, and the whole measurement rests on it.

Across episodes, the Learning Axiom defines learning as a change of knowledge state after which the next comparable episode starts with less KTD; in its strong form, a posterior becomes the next prior. Learning is one of three adaptation channels, one per rung: L2a knowledge (KTD falls), L2b questions (the query family grows), and L2c protocol (the procedure improves). Loop L3 revises the goal or the regulator's model of it. Loop L1 is selection within an episode.

5. Two ledgers (Section 5)

The Knowledge Discovery process is recorded in two ledgers that share the closure event but are never merged. One ledger records what was produced and survived, the other how much ignorance is carried forward.

Operational Ledger

The operational ledger counts closure events in a window I. A counted atomic unit is either a non-closure action unit q t , i inside episode t, or a closure unit κ t of episode t. The episode protocol Q t := ( q t , 1 , … , q t , k t ) contains only non-closure action units. The closure unit is the terminal commitment event and is counted separately.

For a window of closed episodes I , the gross action-unit count is:

Q (I) := ∑ t∈I | Qt |

A closure acceptable when produced may later be invalidated. The cause is either a change of goal or a correction to the regulator's incorrect understanding of the goal. A later correction is a new closure, and the earlier effort remains in the record. The invalidation convention charges invalidated episodes to the effective load. If a rejection supplies evidence for a later closure on the same target, its query cost belongs to that surviving episode.

Q eff ( I ) := Q ( I ) + W obs ( I ) .

Gross closures count equals the surviving count plus the invalidated count:

S gross ( I ) = S net ( I ) + W obs ( I ) .

Combining both we get:

Q eff ( I ) + S net ( I ) = Q ( I ) + S gross ( I )

In the physical partition, each invalidated closure sits on the closure side, inside Sgross(I) . In the effective partition, that same closure has been removed from the surviving-closure side — it is not in Snet(I) — and re-booked as a non-progress debit inside Qeff(I) . No new units are introduced; the invalidated closures are simply re-attributed from “closure” to “load.”

In the query reading, every non-closure action unit instantiates a query from the regulator's family, with action units that expose nothing counted as constant queries. The gross action-unit count Q(I) is therefore the number of queries asked in the window, and |Qt| is the realized path length of episode t.

Knowledge Ledger

All action units are a count of acts, not of bits. The knowledge ledger gives each closure a row in bits. The row holds KTD at the start of the episode, the residual at its end, and KTD at the start of the next comparable episode. It also holds the query rung and the realized number of action units. Discovery within the episode, Bt, is kept apart from retained learning, which obeys a stock-flow identity with gain GtK and loss LtK:

Ht+1start=Htstart−GtK+LtK,Bt=Htstart−Htterm=IMt(X;Ut∣Y).

The query rung has a parallel identity, whose change splits exactly into a knowledge part (L2a) and a question part (L2b). An invalidation may trigger learning, but it is not learning.

The measurement targets the ex-ante average starting KTD over the closures that survive the window:

KTD avg start,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net H u start .

The realized counterpart uses the actual disturbance state yu encountered by each surviving closure:

K ∗ := KTD avg start-real,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net H u start,real .

The relation between these two window averages, and the conditions under which the realized average can represent the expected quantity, are developed later in the quantification layer.

6. The record as a program with two instructions (Section 6)

The observer sees only closures, their timestamps, and invalidations. A unit convention must first be declared. Units are either counted directly, or inferred from time by tiling the window into slots of length Δ, the duration of one closure act, so that N(I)=|I|/Δ. The capacity N(I) is fixed before the closures are seen. Every unit is a closure unit or a non-closure unit, and waits count as non-closure units. That gives the physical partition. Re-booking each invalidated closure as load gives the effective partition:

N(I)=Q(I)+Sgross(I)=Qeff(I)+Snet(I),Qeff:=Q+Wobs.

Written in time order, with a 1 for each surviving closure and a 0 for every other unit, the window becomes a binary string of length N: the program. A 1 is an output instruction, which emits an acceptable closure that survives. A 0 is a non-output instruction. The ledger knows four kinds of unit: emit, find out, wait, and retract. The program folds the last three into 0. It records which instructions ran, not what they returned. The answers form a separate record, the answer transcript, and one 0, such as one look at a script, may return many bits. A bit of program is therefore not a bit of information; the program's length measures effort, not content. Every call to an oracle is counted, which makes the program's length a query complexity. The estimator is the effective depth: the number of 0s per surviving closure:

K^:=QeffSnet=NSnet−1.

It is exact by accounting. Invalidation raises it convexly (Theorem 1): the same capacity is spread over fewer surviving closures, so each further invalidation costs more than the one before it.

7. Assumptions, and the bridge from counts to bits (Section 6)

What the count says about the latent bits depends on assumptions that must be declared:

Declares
A1–A4Operational: fixed counted capacity N(I); every unit is a closure or a non-closure unit (waits folded in); net closures are the gross closures that survive; one unit per gross closure.
A5Every non-closure unit is a binary query, and P1–P3 hold with m=2. Checkable at design time.
B1Bounded search gap: query-family gap at most rQ, protocol gap at most rP, bounded unit-inference error. Declared; not observable.
B2The protocol closes only at certainty, and every discriminating stage is counted.
B3, B4Units spent on invalidated closures are counted; the counted closures are the knowledge-ledger rows.
B5Sequential calibration: given the past, stage counts are unbiased, within a declared range d.
CΔWhen units are inferred from time: at most one query per slot.
H1Human regulators only: at most Ch bits per second taken in about the required class.

Theorem 2 relates effective depth with K∗, the average starting KTD evaluated at the actual starting observation and knowledge state of each surviving episode. Population matching (B4) ensures that the counted closures and the knowledge-ledger rows refer to those same episodes. The difference is a path fluctuation, a search excess, an invalidation term, a unit-inference error, and a boundary load:

K^−K*=ζ+εred+εinv+εΔ+εbd.

Its claims are graded, and each level adds assumptions to the one before:

  • Under A2–A4 and B4, the decomposition is an identity.
  • Adding A5, B2 and CΔ, the corrected K^ is an upper bound on K*, up to the path fluctuation. This follows from the entropy bound on prefix codes. Inefficient search only loosens the bound.
  • Adding B5, Corollary 2 controls that fluctuation at fixed capacity even though the number of completed episodes is random. In a window with directly counted stages and zero invalidation and boundary load, a two-sided fluctuation allowance Zα yields, with probability at least 1−α,

    K^−(1+r)−Zα≤K∗≤K^+Zα.

    For this clean window, Corollary 2 permits Zα=d(N+1)ln(2/α)2Snet. The observer supplies the counts. The model supplies the calibration premise and the declared range. A larger window can reduce sampling uncertainty when closures grow in proportion to capacity. The declared search allowance sets a separate contribution to the bracket’s width.

  • Adding B1 makes the bracket two-sided, of width about 1+rQ+rP plus the measurement terms.
  • With direct counting, even splits and no invalidation, K^ converges to K* at rate N−1/2.
  • When a human's lookup returns more than a binary answer, A5 fails. H1 then restores the upper bound, scaled by ch=ChΔ (Corollary H), without CΔ.

Each departure costs something specific. A wasteful protocol or a restricted query family loses only the lower side. Answers from an alphabet of size m keep the upper side, scaled by log2m. Several stages in one slot lose the upper side. Counted units that are not stages of the episode give no bound in either direction.

A separate upper bound for human regulators

A human can get several letters from one lookup or ask several questions during one time slot. The human bound uses the same ledger and compares information directly with counted non-closure time. For episodes with a unique required class, Lemma H uses P2–P3, B2 and both clauses of H1. Corollary H applies the result to the matched surviving population under A2–A4 and B4.

H1-a. Information intake capacity
For each episode-start state, the information about the required class in the complete non-closure transcript is at most a declared throughput Ch times the expected attributed non-closure time. The transcript includes informative timing or stopping. The bound is conditional on the starting state and holds in expectation. The throughput must apply to the task being measured.
H1-b. Commitment on information already registered
Any new information about the episode’s own required class arrives during its attributed non-closure time. The regulator already has the necessary information when it performs the closure act. Information learned earlier about a later target enters that later episode’s starting information, even if it was acquired while committing an earlier response.
B5Δ. Sequential calibration of interval counts
Before each gross episode’s path is realized, the predicted number of non-closure slots equals the conditional mean of the observed slot count, given the prior history and current starting state. For Proposition H‴, declare either a predictable range for that count or a bound on the average conditional second moment over the completed episodes.

With certainty at commitment, the transcript resolves the starting uncertainty. H1 bounds that information by expected non-closure time multiplied by Ch. A slot therefore contributes an allowance of ChΔ bits in expectation. This route applies to wider answer alphabets and to slots that contain several questions. In a fixed-capacity window without invalidations or boundary load, Proposition H‴ gives the one-sided bound

K∗≤ChΔ(K^+ΛαSnet),

where Λα is the one-sided fluctuation allowance for the window. The capacity premise gives an upper bound on KTD and hence a lower bound on latent KEDE. For a relational goal with several acceptable classes, the relational extension strengthens the intake premise to information about the disturbance and requires the transcript to determine the selected class, which must itself be determined by the disturbance conditional on the starting state.

8. Knowledge-Discovery Efficiency and discovery rates (Sections 7–8)

KEDE maps depth into (0, 1]. The operational value is the share of the program that is output. The latent value is the efficiency that the regulator's missing knowledge alone would allow. Between them, KEDE factors along the ladder:

KEDE^=11+K^=SnetN,KEDE*=11+K*,KEDE^=KEDE*⋅ωcod⋅ωQ⋅ωP⋅ωM.

Corollary 3 gives the latent value exactly as 11+K^−Γ, where Γ is the gap of Theorem 2. Under the conditions of the upper bound, the observed value is therefore a conservative reading of latent KEDE, with high probability from observed and declared quantities (Corollary 4). The factors name the cause of a low value, and each cause has its own remedy. Missing knowledge calls for learning (L2a), missing questions for new questions (L2b), and wasted questions for a better protocol (L2c). Invalidation calls for aligning the goal model (L3). Coding slack needs no remedy; its factor never falls below one half. Only missing knowledge and invalidation are visible in the ledger alone. Telling missing questions from wasted ones requires declaring the query family. Reattaching elapsed time splits the unit rate into a closure rate and a load rate, νact=νcl+νdisc. The latent discovery rate νcl⋅K* has the form of Shannon's source-rate identity, and it answers the dynamic question directly.

Applications (Section 9)

Each application declares what a 1 is, what one unit of capacity is, and what a 0 holds, and this fixes which of the claims above it supports. In typing, the 0s are looks at a script, observed one by one. One look returns many letters, so A5 fails and H1 gives an upper bound. In software development, the 0s are 0.288-second slots without a typed symbol. H1 again gives an upper bound, but with no declared query family the factorisation is unavailable. The robot of the appendix declares everything and shows every rung and gap. In a published learning curve CΔ fails, so the result is a relative efficiency only. In intelligence testing and basketball, the 0s are failed attempts, and the bound shifts by the uncertainty left at commitment. In screw tightening and assembly lines, the 0s are rated slots, and waiting loosens the bound.

In short, missing knowledge cannot be seen, but finding it out costs questions, and questions cost capacity. Capacity spent per surviving output therefore bounds what had to be found out. Declaring the regulator's questions shows whether the shortfall lies in its knowledge, its questions, or its protocol.

3. The Law of Requisite Variety

For a system to effectively regulate its environment, it must have at least as much variety as its environment.

Set-based formulation

Regulation achieves a goal by selecting responses against disturbances.

E = ( E1 , … , En ) is the essential-variable vector. The essential variables E1 , … , En are the goal-relevant state components. Ei is the value set of Ei . The essential variables define the dimensions; their value sets define the coordinate domains of the essential-variable space E = E1 × E2 × ... × En , which is the set of all possible combinations of their values[29].

An essential state e ∈ E is one particular point in the essential-variable space, i.e. a tuple of values of these dimensions, e.g. e = ( 36.8 ° C , 92 mg / dL , … ) .

It is assumed that the goal has already been determined: the acceptable essential states are given by an acceptable region η ⊆ E in the essential-variable space. Thus, the goal is a region in a multidimensional essential-variable space E .

Let D, R, and Z denote, respectively, the disturbance variable, regulatory-response variable, and outcome variable. Let their corresponding sets of possible values be D , R , and Z . Thus d ∈ D , r ∈ R , and z ∈ Z are particular values.

In the general case, the outcome-value space may itself be multidimensional. Let Z = ( Z1 , … , Zm ) be the outcome-variable vector. The outcome variables Z1 , … , Zm are the components used to describe the concrete or functional result produced by a disturbance-response pair. Zj is the value set of Zj . Thus the outcome-value space is Z = Z1 × Z2 × ... × Zm .

An outcome-value z ∈ Z is one particular point in the outcome-value space. In the multidimensional case, it has the form z = ( z1 , … , zm ) . In the Appendix we have an example of a typing task, where an outcome-value may record both the intended target position and the typed symbol: z = ( p , a ) .

The number of outcome dimensions m need not equal the number of essential-variable dimensions n. The outcome-value space Z describes the result produced by the system, while the essential-variable space E describes the goal-relevant state used to judge that result. The mapping from outcomes to essential-variable values will be introduced below.

The scalar outcome-value case is recovered when m = 1 . In that special case, each outcome-value may be treated as an unstructured value. More generally, each table cell still contains one outcome-value, but that outcome-value may itself be internally structured as a tuple.

We use roman E for the essential-variable vector and script E for its value space. Likewise, we use roman Z for the outcome-variable vector and script Z for its value space.

The Table of Outcomes is the map

T : D × R → Z

so that for each disturbance-value d and response-value r , the value T(d,r) is the outcome-value z∈Z .

Thus the table T is two-dimensional in its indexing by disturbance-response pairs (d,r), while each cell-entry may be a scalar outcome-value or a structured outcome-vector. The scalar-outcome case is recovered when Z = Z1 .

T R
r₁ r₂ r₃ ...
D d₁ z₁₁ z₁₂ z₁₃ ...
d₂ z₂₁ z₂₂ z₂₃ ...
d₃ z₃₁ z₃₂ z₃₃ ...
d₄ z₄₁ z₄₂ z₄₃ ...
... ... ... ... ...

Two distinct disturbance-response pairs may yield the same outcome-value:

T(d1,r1) = T(d2,r2) = z

This means that repetition belongs to the mapping T, not to the set Z . A set contains each of its elements only once. So one should not say that Z contains repeated values. What repeats is the same outcome-value appearing as the image of multiple table-cells.

Correspondence between outcome-values and essential-variable values. The “outcomes” in the Table of Outcomes are simple outcome-values, without any implication of desirability. An outcome-value z∈Z is a concrete or functional result of a disturbance-response pair. An essential-variable value e∈E is the goal-relevant state description used to judge whether the result falls inside or outside the acceptable region.

The correspondence between outcome-values and essential-variable values is therefore not automatic. It is part of the modeling frame of the regulatory problem. To use a function from outcomes to essential-variable values, the outcome-description must be specified at a level of detail sufficient to determine the relevant essential-variable value. In this article we assume that this has been done. Thus each outcome-value under the adopted description has a determinate associated essential-variable value.

φ : Z → E

so that φ(z)=e . This means that the function φ is a modeling map from the outcome-description used in the table to the essential-variable description used for regulation. It should not be read as saying that every possible description of an outcome would automatically determine an essential-variable value. If the outcome-description were too coarse to determine the relevant essential-variable value, then φ would have to be replaced by a relation, a refinement of Z , or a probabilistic mapping.

The reduced, bijective, and many-to-one cases discussed below are therefore not alternatives to this modeling assumption. They are different ways in which the adopted outcome-description may correspond to the essential-variable description. In the reduced case, outcome-values already are essential-variable values. In the bijective case, outcome-values and essential-variable values are distinct descriptions but correspond one-for-one. In the many-to-one case, several finer-grained outcome-values correspond to the same essential-variable value.

Reduced correspondence. The special case is the reduced representation in which the table T directly contains values of the essential variables, i.e.

Z = E and φ = idE
so that the table entries are already values of the essential variables.[1] More generally, however, φ may be bijective or many-to-one, depending on the level of description adopted.

Bijective correspondence between outcome-values and essential-variable values. This is the case in which each relevant outcome-value maps to a unique essential-variable value, and each relevant essential-variable value is represented by exactly one outcome-value. In the bijective case, φ is one-to-one and onto between the relevant outcome-values and the relevant essential-variable values represented in the regulatory model, not bijective over the entire possible outcome and essential-variable spaces. Then Z and E are conceptually distinct but informationally equivalent: φ : Z → E with z 1 ≠ z 2 → φ ( z 1 ) ≠ φ ( z 2 ) and every essential-variable value relevant to the regulatory model e ∈ E has exactly one corresponding z ∈ Z . On this reading, Z may still be understood as the outcome-value space and E as the essential-variable space, but every relevant distinction in outcomes is mirrored one-for-one at the level of the essential variables. This is the form used in later reformulations such as D → R → Y ↔ E with an explicit bijection between the outcome-value variable and the essential variable E. In that literature Y corresponds to our outcome variable Z, or to the induced outcome variable generated through T. Aulin-Ahmavaara's formulation uses a one-to-one mapping between outcome Y and essential variable E, so our φ : Z → E is a generalization of that case[31]. See the illustrative example in the Appendix.

Many-to-one correspondence between outcome-values and essential-variable values. This is the case in which multiple distinct outcome-values map to the same essential-variable value. Ashby says that a particular outcome can be treated “as unit with unit,” but “in another context, [it] may be analysed more finely.[1]” That means an outcome can be used as a single table-entry in the regulatory schema, even though in a more detailed analysis it may unfold into a whole trajectory, microstate, or process.[1] Then different outcome-values can collapse to the same essential-variable value: z 1 ≠ z 2 ∧ φ ( z 1 ) = φ ( z 2 ) = e .

On this reading, Z contains finer-grained outcomes, while E contains the coarser essential-variable values that matter for survival or goal-attainment. The many-to-one relation therefore expresses that several distinct outcome-states may be equivalent from the standpoint of regulation. Successful regulation is still judged at the level of the essential variables: distinct trajectory histories may all count as equally successful if they yield the same relevant value of E and hence fall within the acceptable region η⊆E. This is a refinement of Ashby's basic regulatory schema, but it is an added modeling layer[1]. See the illustrative example in the Appendix.

In this article formulas below are written in the general φ : Z → E form. Most later calculations use the general φ -form; where individual outcome deletion/addition is interpreted literally, the reduced or bijective case is assumed unless stated otherwise.

Requisite variety

The regulator does not choose outcome-values directly. It selects a response-value in the presence of a disturbance-value, and thereby determines an outcome-value.

Strictly speaking, there may be many disturbance-values that are conceivable in the world but not part of the regulatory problem currently being modeled. Let Dall denote the larger set of all conceivable disturbance-values. In the present formulation, however, we restrict attention to the disturbance-values under consideration in the regulatory problem. For notational economy, we write this context-restricted disturbance set simply as D , where D ⊆ Dall .

This restriction is part of the modeling frame, not an effect of regulation. It does not mean that the regulator has already reduced the disturbance variety. It means only that success is being evaluated over the disturbance-values included in the present regulatory problem. Thus, from this point onward, D means the disturbance set under consideration.

In Ashby's table formulation, the “actual outcomes” is the rule-induced actual outcome-set selected from the possible outcome table by the regulator's response choices. They are actual in contrast to the full set of possible outcomes Z, not necessarily actual in the narrower historical sense of having occurred in a single observed run[1][8].

Let the regulator use a response rule

ρ : D → R

This induces the actual outcome map

yρ : D → Z , yρ(d) := T(d,ρ(d))

and its image:

yρ [D] = { yρ(d) : d∈D } ⊆ Z

The image yρ [D] is the rule-induced actual outcome-set under the response rule ρ . It contains the outcome-values selected from the possible outcome table when each disturbance-value under consideration is paired with the response chosen for it by the rule ρ . In this Ashby-style sense, these are the rule-induced actual outcomes: actual relative to the regulator's response rule, as distinct from the full space of possible outcomes Z .

This should be distinguished from the historically realized outcome-set in one particular run. If only some disturbance-values actually occur, let Dr ⊆ D be the historically realized disturbance subset. Then the historically realized outcome-set is

Oρr := yρ [Dr] ⊆ yρ [D] ⊆ Z .

Thus Z is the possible outcome-value space; yρ [D] is the actual outcome-set over the modeled disturbance domain; and Oρr is the historically realized outcome-set in the actual case.

Successful regulation requires that the essential-variable values corresponding to all rule-induced actual outcomes yρ [D] lie in the acceptable region: φ [ yρ [D] ] ⊆ η for any A ⊆ Z , φ [ A ] := { φ ( z ) : z ∈ A } .

This is a universal rule-level success condition over the disturbance set under consideration. At a particular moment only one disturbance may occur, but the response rule is judged by what it would produce for every disturbance in the modeled set.

This set-based representation tracks which outcome-values can occur under the policy, but it does not track multiplicity, frequency, or probability. Its role here is to support a universal success condition over the disturbance set.

Equivalently, successful regulation requires that every disturbance-value under consideration be mapped, through the selected response rule, into an acceptable outcome-value:

yρ [D] ⊆ φ−1 [η]

Here, the acceptable outcome-set φ−1 [η] denotes the preimage of the set η , not a two-sided inverse function. Also φ−1 [η] is a subset of Z , so it is at the outcome-value level.

This subset relation is the primary success condition. It says that every actual outcome produced under the response rule ρ must lie in the acceptable outcome-set. Any variety inequality derived from it is only a necessary numerical consequence, not the defining criterion of success. In Ashby-style terms, regulation succeeds when the rule-induced actual outcomes remain within the goal subset[1].

In the vocabulary of query complexity, this universal rule-level condition is the success notion of the worst-case setting: a response rule is judged by every disturbance-value in D, not only by the disturbances that happen to be likely. The probability-one condition introduced in the information-theoretic formulation is the corresponding success notion of the distributional setting. The two settings recur throughout the article.

Count-Variety Formulation

In this subsection, V denotes count-variety: for any finite set A , we define V (A) := |A| , the number of distinguishable values/states in A under the adopted classification. The following count-variety statements assume that the relevant spaces have been discretized into distinguishable classes under the adopted modeling resolution. If η is an interval in a continuous space, plain cardinality becomes unhelpful because many regions may have the same infinite cardinality. For continuous spaces, the same role would have to be played by a measure, entropy, or another explicitly chosen variety measure. In the information-theoretic formulation this role is played by entropy over the regulatory partition. Thus all formulas written with V are cardinality statements. Therefore, at the level of variety, success implies the following necessary numerical bound:

V ( φ [ yρ [D] ] ) ≤ V ( η )

Similarly, from yρ [D] ⊆ φ−1 [η] we obtain the necessary bound

V ( yρ [D] ) ≤ V ( φ − 1 [ η ] )

These variety inequalities are consequences of the subset condition. They are not, by themselves, equivalent to it: an actual outcome-set yρ [D] may have variety no greater than that of an acceptable set and still fail to be a subset of that set. So success is fundamentally a matter of set inclusion, while the variety inequalities are derived necessary conditions. They are not, by themselves, equivalent to it.

Under the reduced representation in which Z = E and φ = id E , the outcome-values are already values of the essential variables, so the success condition reduces to

yρ [D] ⊆ η

and therefore implies the numerical bound

V ( yρ [D] ) ≤ V ( η )

This numerical compatibility condition is still only necessary, not sufficient. The decisive success criterion remains: φ [ yρ [D] ] ⊆ η .

Thus, in this set-based formulation, regulatory success is stated at the level of the rule-induced actual outcome-set yρ [D] , not at the level of a single actual outcome zt which is used to state success at a particular time.

A special finite-table lower-bound result. Under Ashby's finite table T idealization, a special count-variety consequence of the law of requisite variety can be stated precisely[1][8]. Suppose the table T has finitely many disturbance-values and response-values, and suppose that no response-column contains a repeated outcome-value. Equivalently, for every fixed response-value r∈R , the map d↦T(d,r) is injective. Thus a single unchanged response-value cannot by itself collapse different disturbance-values into the same outcome-value.

Let the regulator select one response-value for each disturbance-value by a response rule ρ:D→R . This selects one cell from each disturbance-row of the table and induces the actual outcome-set

yρ [D] = { T(d,ρ(d)) : d∈D } ⊆ Z .

The response rule may use only a subset of the available response repertoire. Define the used response-set as

Rρ := ρ [D] ⊆ R .

Because no response-column contains a repeated outcome-value, any one outcome-value can appear among the selected cells at most once per used response-column. Since the rule uses V(Rρ) response-columns, any one outcome-value can cover at most V(Rρ) selected disturbance-rows. But the response rule selects one cell from each of the V(D) disturbance-rows. Assume D is non-empty, so that Rρ is also finite and non-empty. Therefore the selected cells cannot collapse into fewer than the ceiling of disturbance-rows divided by used response-columns:

V ( yρ [D] ) ≥ ⌈ V (D) V (Rρ) ⌉ .

Since Rρ ⊆ R , we also have V (Rρ) ≤ V (R) . Therefore the weaker repertoire-level lower bound is:

V ( yρ [D] ) ≥ ⌈ V (D) V (R) ⌉ .

This is a finite-table, count-variety quotient analogue of the Ashby-style requisite-variety bound. It is not Ashby's general entropy formulation. It is not derived from the goal-subset condition alone. It follows from the additional table assumption that no response-column already contains repeated outcome-values. If the table itself contains such repetitions, then some disturbance variety has already been collapsed by the table structure before the regulator's response rule is considered. In that case this quotient lower bound need not hold at the outcome-value level.

This is an outcome-value-level bound before applying φ . In the many-to-one case, φ may collapse several distinct outcome-values into one essential-variable value, so the same lower bound need not hold for V ( φ [ yρ [D] ] ) unless φ is injective on the selected outcome-values.

The quotient bound is a lower bound on the variety of the actual outcome-set selected by the response rule. Successful regulation, however, is still defined by set inclusion, not by the quotient alone. The success condition remains:

yρ [D] ⊆ φ −1 [η] .

Therefore a necessary outcome-level numerical compatibility condition for success is:

⌈ V (D) V (R) ⌉ ≤ V ( φ −1 [η] ) .

This condition is necessary, not sufficient. It says only that the acceptable outcome-set is numerically large enough to contain the unavoidable residual outcome variety. It does not guarantee that the table entries are arranged so that some response rule actually maps every disturbance-value into the acceptable outcome-set. Success remains a matter of the subset relation itself.

If one wants the numerical condition to count only outcome-values that are actually attainable from the table, define the table-image as

Im(T) := { T(d,r) : d∈D , r∈R } ⊆ Z .

Then the attainable acceptable outcome-set is

O acc att := Im(T) ∩ φ −1 [η] .

Using this stricter attainable set gives the sharper necessary condition:

⌈ V (D) V (R) ⌉ ≤ V ( O acc att ) .

In the reduced representation, where Z=E and φ=idE , the acceptable outcome-set is just the acceptable essential-variable region:

φ −1 [η] = η .

Therefore the necessary numerical condition becomes:

⌈ V (D) V (R) ⌉ ≤ V (η) .

The same reduction holds in the bijective case, provided φ preserves cardinality on the relevant acceptable sets. In the many-to-one case, however, V ( φ −1 [η] ) may be much larger than V(η) . Thus the outcome-level compatibility condition and the essential-variable-level compatibility condition coincide in the reduced or bijective case, but can diverge in the many-to-one case.

Outcome map induced by a response rule at time t

At time t , let the regulator use the response rule

ρt : D → R

The induced actual outcome map is

yt : D → Z , yt(d) := T(d,ρt(d))

where yt := yρt for notational economy.

The acceptable outcome-set at time t is not chosen directly in Z ; it is induced by pulling back the acceptable essential-variable region ηt⊆E along the outcome-to-essential-variable map φ .

Oacc,t := φ−1 [ηt] = { z∈Z : φ(z)∈ηt }

Thus the acceptable outcome-set is the pullback, or preimage, of the acceptable essential-variable region. An outcome-value is acceptable at time t exactly when its associated essential-variable value lies in ηt .

Successful regulation under the criterion prevailing at time t means that the response rule ρt maps every disturbance under consideration into the acceptable outcome-set for that time:

yt [D] ⊆ Oacc,t

Equivalently,

φ [ y t [ D ] ] ⊆ η t

Goal revision with fixed table and time-indexed acceptable outcomes

We consider a regulator acting against disturbances within a fixed system structure. The aim is to model the case in which the structure of the system does not change from time t1 to time t2, but the criterion of success does.

In this subsection, goal revision means a change in the acceptable region within a fixed modeling frame. The disturbance space D , the response space R , the possible outcome space Z , the essential-variable space E , the table T , and the outcome-to-essential-variable map φ remain fixed. Thus the same outcome-value keeps the same essential-variable interpretation. What changes is the acceptable region ηt1 ⊆ E to ηt2 ⊆ E and therefore the acceptable outcome-set changes from Oacc,t1 to Oacc,t2 inside the same essential-variable space E , not the meaning or dimensionality of the essential variables themselves.

If the goal revision changes which variables count as essential, or changes the level at which outcomes are mapped to essential-variable values, then E or φ must also be revised; that would be a different modeling case.

This is goal revision within a fixed repertoire, not structural adaptation. That distinction is important because stronger Ashby-style adaptation involves change in the system's structure, organization, or available repertoire, not merely a different rule selected from the same fixed table[1].

In a higher-order system, revising the acceptable set may itself be part of a meta-regulatory process[57]. Here, goal revision is treated as an external change in the criterion of success, not as regulation by the fixed first-order regulator. It could be modeled as higher-order regulation in a larger adaptive system, but that is outside the fixed-table case considered here.

A change in the acceptable set is not yet a change in actual behavior. It is a change in the criterion by which behavior is judged. A behavioral change occurs only if the regulator changes its response rule from ρ t 1 to ρ t 2 . Such a change is required only when the old rule no longer maps the disturbances under consideration into the revised acceptable outcome-set.

Thus the transition from ηt1 to ηt2 first changes the criterion of success. This induces a change from Oacc,t1 to Oacc,t2 . The old response rule ρt1 must then be tested against the revised acceptable outcome-set. If yt1 [D] ⊆ Oacc,t2 , then the old behavior remains successful under the new criterion. If not, regulation requires selecting a revised response rule ρt2 such that yt2 [D] ⊆ Oacc,t2 . Goal revision is not itself regulation. Continued regulation after goal revision requires that some response rule - possibly the old one, possibly a revised one - maps the disturbances under consideration into the revised acceptable outcome-set.

We are therefore tracking acceptability at the level of outcome-values in Z , via the pullback along φ. Acceptability-status is time-indexed, while outcome-identity in Z is not. So the change is not that outcome-values disappear from the table. The change is that some fixed outcome-values in Z may cease to be acceptable, while others may become acceptable. That is why it is correct to time-index Oacc,t.

Before defining deletion and addition at the level of outcome-values, one qualification is needed. If acceptability is defined by pulling back the acceptable essential-variable region along φ, then acceptability is constant on the fibers of φ.

Fe := φ−1 [{e}] = { z∈Z : φ(z)=e }

The set Fe is the fiber of outcome-values that all correspond to the same essential-variable value e . If Oacc,t := φ−1 [ηt] , then all members of the same fiber have the same acceptability-status at time t.

z,z′ ∈ Fe ⇒ z∈ Oacc,t ⇔ z′∈ Oacc,t

When acceptability is defined by pullback from ηt , acceptability cannot vary inside a fiber of φ. If two distinct outcome-values map to the same essential-variable value, they share the same acceptability-status at time t. Therefore, if the modeler wants two outcome-values in the same fiber to have different acceptability-statuses, then the map φ has collapsed a distinction that is relevant for regulation.

This is not a limitation of the set-theoretic construction; it is a consequence of the chosen level of description. If two outcome-values must be judged differently for purposes of regulation, then they cannot be treated as equivalent under the outcome-to-essential-variable map. In that case, φ is too coarse for the regulatory problem being modeled. The outcome-description must be refined, or the mapping must be replaced by a relation, a more detailed essential-variable space, or a probabilistic mapping.

Therefore, in the many-to-one case, deletion and addition of acceptable outcome-values is not truly individual. It happens fiber-wise. Because acceptability is defined by pulling back the acceptable essential-variable region ηt along φ , one cannot remove one member of a fiber from the acceptable set while leaving another member of the same fiber acceptable.

Revision of acceptability: deletion, addition, and substitution within the acceptable outcome-set

With this interpretation, the notions of deletion, addition, and substitution are legitimate, provided they are understood as operations on the essential-variable region Oacc,t, not on the table T itself. So no outcome-value is deleted from the table T. Rather, some fixed outcome-values may lose or gain acceptability.

An outcome-value z is deleted from the acceptable set between t1 and t2 iff

z ∈ O acc,t1 and z ∉ O acc,t2 .

This does not mean that z disappears from the table T. It means only that z, though still a possible outcome-value in T, is no longer acceptable under the revised goal.

These definitions concern changes in acceptability-status only. They do not imply that the regulator actually produces, stops producing, or replaces those outcome-values. Behavioral change is tracked separately by comparing y t 1 [ D ] and y t 2 [ D ] .

An outcome-value z is added to the acceptable set between t1 and t2 iff

z ∉ O acc,t1 and z ∈ O acc,t2 .

In the minimal set-theoretic sense, a weak substitution occurs when one acceptable outcome loses acceptability and another gains acceptability between t1 and t2. That is, there exist b, c ∈ Z such that

b ∈ O acc,t1 ∖ O acc,t2 , c ∈ O acc,t2 ∖ O acc,t1 .

Then one may say that b is removed from the acceptable set and c is admitted into it. This expresses substitution as coexistence of loss and gain; a stronger one-for-one replacement notion would require an additional correspondence between lost and gained elements.

It is useful to decompose the transition from t1 to t2 into three parts.

The persistently acceptable outcomes are

P12 := O acc,t1 ∩ O acc,t2 = φ−1 [ ηt1 ∩ ηt2 ] .

Equivalently, changes in acceptable outcome-values are the pullbacks of changes in acceptable essential-variable values. If Oacc,t := φ−1 [ηt] , then the outcome-values that lose acceptability between t1 and t2 are:

L12 := O acc,t1 ∖ O acc,t2 = φ−1 [ ηt1 ∖ ηt2 ] .

The outcome-values that gain acceptability between t1 and t2 are:

G12 := O acc,t2 ∖ O acc,t1 = φ−1 [ ηt2 ∖ ηt1 ] .

This makes the fiber-wise character explicit. If several distinct outcome-values z map to the same essential-variable value e , then they enter or leave the acceptable outcome-set together. The reason is that acceptability is assigned first to essential-variable values and only then transferred back to outcome-values through φ .

Thus, in the reduced or bijective case, each relevant fiber contains exactly one outcome-value, so deletion and addition can be read directly at the level of individual outcome-values. In the many-to-one case, however, deletion and addition should be read as operations on whole fibers of φ .

Then

O acc,t2 = ( O acc,t1 ∖ L12 ) ∪ G12 .

and the disjoint unions of the persistently acceptable outcomes and the outcomes that lose and gain acceptability are given by: O acc, t 1 = P 12 ∪˙ L 12 O acc, t 2 = P 12 ∪˙ G 12

Thus deletion, addition, and substitution are operations on acceptability-status within the fixed outcome-space Z , not on the existence of values in T itself.

Equivalently, each such revision changes the regulatory relation Rt defined at the end of this section. The table T and the repertoire R stay fixed; what changes is the problem the regulator must solve. The feedback-loop reading treats this kind of change as loop L3.

This models the attempt to maintain regulation under an externally revised criterion of success within a fixed repertoire. It is not structural adaptation in the stronger sense, because the regulator does not alter its own structure or acquire a new repertoire. Stronger adaptation enters only when the fixed repertoire is no longer sufficient and the system must reorganize itself.

Strict substitution

A strict substitution is stronger than the mere coexistence of loss and gain. Weak substitution says only that some outcome-values lose acceptability and some other outcome-values gain acceptability. Strict substitution says, in addition, that the model supplies a specified replacement correspondence between the lost and gained outcome-values.

Using the earlier definitions:

L12 := O acc,t1 ∖ O acc,t2
G12 := O acc,t2 ∖ O acc,t1

A strict substitution structure from t1 to t2 is an ordered triple

( L12 , G12 , σ12 )

such that:

L12 ≠ ∅ and G12 ≠ ∅

and the model supplies a specified bijection

λ12 : L12 → G12

where λ12 is not merely a bijection whose existence follows from equal cardinalities. It is part of the model. It specifies which lost acceptable outcome-value is treated as replaced by which newly acceptable outcome-value.

Thus,

λ12 (b) = c

means that the previously acceptable outcome-value b is replaced, under the revised criterion of success, by the newly acceptable outcome-value c.

Strict substitution is still a relation between acceptability-statuses. It says that the revised criterion treats newly acceptable outcome-value c as the specified replacement for previously acceptable outcome-value b. It does not, by itself, imply that the regulator has actually produced c as a behavioral replacement for b. That would require an additional closure event or a change in the response rule.

The existence of a bijection requires

|L12| = |G12|

but this equality is only a necessary cardinality condition. It does not determine which lost outcome-value corresponds to which gained outcome-value. The transition of acceptable sets determines only the lost set and the gained set. It does not, by itself, determine the replacement pairing.

Therefore, replacement is not objectively present in the set transition alone. Weak substitution is present in the set transition whenever loss and gain coexist. Strict substitution requires additional correspondence structure.

In the special one-for-one case, where exactly one outcome-value loses acceptability and exactly one outcome-value gains acceptability, strict substitution reduces to:

L12 = {b} and G12 = {c}

In that case there is only one possible bijection, so one may say that b is strictly substituted by c.

This defines strict substitution at the outcome-value level. In the many-to-one case, one may instead define strict substitution at the essential-variable level by pairing lost and gained elements of ηt1 ∖ ηt2 and ηt2 ∖ ηt1 . The outcome-level and essential-variable-level notions coincide in the reduced or bijective case. Strict substitution at the outcome-value level may fail in the many-to-one case even when strict substitution exists at the essential-variable level, because fibers φ may have unequal cardinalities.

Behavioral rework

The preceding definitions describe revision of acceptability-status. They do not yet describe behavioral rework. Behavioral rework concerns a change in the outcome-values actually selected by the regulator's response rule. Thus, an outcome-value z is behaviorally removed between t1 and t2 iff:

z ∈ y t1 [ D ] and z ∉ y t2 [ D ] .

It is behaviorally added iff:

z ∉ y t1 [ D ] and z ∈ y t2 [ D ] .

Therefore, goal revision changes the criterion of success; behavioral rework changes the regulator's selected outcomes. The two are related but not identical. A goal revision may require no behavioral rework if the old response rule remains successful under the revised acceptable outcome-set.

The regulatory relation

The set-based success conditions above can be collected into a single object. For each disturbance-value d, the acceptable response-set at time t is:

At (d) := { r∈R : φ(T(d,r)) ∈ ηt } .

The regulatory relation at time t pairs each disturbance-value with every response that regulates it:

Rt := { (d,r) ∈ D×R : r∈ At(d) } .

Its section at d is At(d) . The relation is not a function in general: several responses may map the same disturbance into the acceptable region, and the set-based formulation never requires one of them to be singled out.

A response rule ρ:D→R is universally successful exactly when it is a selector of the relation:

yρ [D] ⊆ Oacc,t ⇔ (d,ρ(d)) ∈ Rt for every d∈D .

Some successful rule exists exactly when every section is non-empty. The successful rule-set Πacc,t introduced with staged selection below is precisely the set of selectors of Rt .

A regulator that already holds such a selector and observes the disturbance exactly solves Rt without any further discrimination. A regulator that observes the disturbance only partially, or does not hold the selector, must first find out which acceptable response applies. The rest of the article treats that finding-out as a sequence of queries and measures its cost.

Information-Theoretic Formulation

The set-based formulation specifies the regulatory problem in terms of value spaces, the Table of Outcomes, the outcome-to-essential-variable map, and the acceptable region. The information-theoretic formulation keeps that same structure and adds probability distributions over it. It does not introduce a different regulatory model; it measures uncertainty, variety, buffering, and knowledge over the same disturbance-values, response-values, outcome-values, and essential-variable values.

Random variables over the set-based regulatory geometry

At time t, let Dt , Rt , Zt , and Et denote, respectively, the disturbance random variable, regulatory-response random variable, outcome-value random variable, and essential-state random variable. These random variables are functions into the value spaces introduced above:

Dt:Ωt→D , Rt:Ωt→R , Zt:Ωt→Z , Et:Ωt→E .

Thus their realized values satisfy d∈D , r∈R , z∈Z , and e∈E .

The disturbance distribution Pt(Dt) describes how probability mass is distributed over the disturbance-values under consideration. The regulator's response policy is represented by Pt(Rt | Dt). In the deterministic case, this policy reduces to a response rule:

ρt : D → R ,

meaning that Pt(Rt = r | Dt = d) = 1 exactly when r = ρt(d).

Given the Table of Outcomes T : 𝒟 × 𝓡 → 𝒵, the disturbance and response variables induce the outcome-value variable:

Zt = T ( Dt , Rt ) .

The corresponding essential-state variable is obtained by applying the outcome-to-essential-variable map:

Et = φ ( Zt ) = φ ( T ( Dt , Rt ) ) .

Under the fixed-table assumption used here, the system transition is deterministic: once d and r are fixed, z = T(d,r) and e = φ(z) are fixed. A stochastic plant would require replacing the indicator terms below with conditional probability kernels.

Equivalently, the joint distribution is concentrated on tuples satisfying the structural constraints imposed by T and φ:

Pt (d,r,z,e) = Pt(d) Pt(r|d) 1[z=T(d,r)] 1[e=φ(z)] .

Thus the information-theoretic formulation measures uncertainty over the same regulatory geometry already defined in the set-based formulation. The Table of Outcomes determines which outcome-value is produced by a disturbance-response pair, and φ determines which essential-variable value is associated with that outcome-value.

Entropy, variety, and the outcome-to-essential-variable map

Shannon entropy, denoted by H, measures uncertainty over distinguishable values of a random variable. For a finite random variable X with distribution P(X), entropy is:

H(X) = - ∑ x∈supp(X) P(x) log2 P(x) .

When all values in the finite support of X are equiprobable, entropy reduces to the logarithm of count-variety:

H(X) = log2 |supp(X)| .

More generally, entropy is the probabilistic analogue of variety. It measures how much uncertainty remains about which distinguishable value of a variable will be realized.

Because Et = φ(Zt), the essential-state variable is a function of the outcome-value variable. Therefore:

H(Et) ≤ H(Zt) .

More precisely:

H(Zt) = H(Et) + H(Zt|Et) .

In the reduced or bijective case, H(Zt|Et) = 0, so the outcome-value variable and the essential-state variable are informationally equivalent on the relevant support:

H(Zt) = H(Et) .

In the many-to-one case, φ collapses several distinct outcome-values into the same essential-variable value. Then outcome-level uncertainty may be greater than essential-variable uncertainty. Equality holds only when φ is one-to-one on the relevant outcome-values. This distinction matters because regulation is judged at the level of the essential variables, even though the Table of Outcomes produces outcome-values.

Probabilistic success and support-level success

The acceptable region at time t is the set:

ηt ⊆ E .

The corresponding acceptable outcome-set is the pullback of that acceptable essential-variable region along φ:

Oacc,t := φ−1 [ ηt ] = { z ∈ Z : φ (z) ∈ ηt } .

Strict probabilistic success at time t means that the essential-state variable lies inside the acceptable region with probability one:

Pr ( Et ∈ ηt ) = 1 .

Equivalently, at the outcome-value level:

Pr ( Zt ∈ Oacc,t ) = 1 .

In discrete support notation, this is:

supp ( Et ) ⊆ ηt

or equivalently:

supp ( Zt ) ⊆ Oacc,t .

This is the distribution-level probabilistic version of the set-based success condition. It checks the disturbance-values that have positive probability under the current disturbance distribution. If supp(Dt) = 𝒟, then in the deterministic rule-level case it reduces to the universal set-based condition that the rule-induced actual outcome-set lies inside the acceptable outcome-set. If supp(Dt) ⊂ 𝒟, then the probability-one condition is weaker: it guarantees success only over disturbance-values that have positive probability at time t.

For universal rule-level success over the whole modeled disturbance set, the policy must be successful for every disturbance-value under consideration. Here Pt(Rt|Dt=d) is treated as the regulator's policy kernel, defined for every modeled disturbance-value d ∈ 𝒟, not merely for disturbance-values with positive probability under the current disturbance distribution.

Define the acceptable concrete response-set for disturbance d as:

At (d) := { r ∈ R : φ ( T (d,r) ) ∈ ηt } .

Then universal rule-level success requires:

∀d∈D , supp ( Pt (Rt|Dt=d) ) ⊆ At (d) .

In the deterministic case, this condition reduces to:

∀d∈D , ρt (d) ∈ At (d) .

This distinction matters because probability-one success is a statement about the support of the current distribution, while universal set-based success is a statement about every disturbance-value included in the modeled regulatory problem.

Allowed residual variety

If the acceptable essential-variable region is finite, its maximum allowed essential-variable entropy is:

hη,t := log2 | ηt | .

Strict probabilistic success implies the necessary entropy condition:

H ( Et ) ≤ hη,t .

This entropy condition is necessary, not sufficient. A distribution can have low entropy while still placing probability mass outside the acceptable region. Successful regulation requires confinement within ηt, not merely low entropy.

Similarly, if the acceptable outcome-set is finite, its maximum allowed outcome-level entropy is:

hO,t := log2 | Oacc,t | .

Strict probabilistic success at the outcome-value level implies:

H ( Zt ) ≤ hO,t .

In the reduced representation, where 𝒵 = 𝓔 and φ = id𝓔, the acceptable outcome-set is just the acceptable essential-variable region, so hO,t = hη,t. In the many-to-one case, however, the outcome-level entropy allowance hO,t may be larger than the essential-variable entropy allowance hη,t, because the pullback may include whole fibers of φ back into acceptability.

Response-equivalence and regulatory ignorance

Aulin-Ahmavaara and Heylighen characterize the ignorance term H(R|D) at the level of disturbance, regulator action, and quality of regulation. They describe it as uncertainty about how to react correctly to a disturbance and how to use the available regulatory acts optimally - using "correctly" and "optimally" loosely as near-synonyms rather than as a deliberate two-level distinction. However, their prose does not fully fix the granularity of the response variable R. It leaves open whether the response denotes: (i) an equivalence class of concrete responses that achieve the same acceptable outcome, (ii) the exact concrete response emitted by the regulator, or (iii) the optimal response among several acceptable responses[29][31].

The term H(Rt|Dt) measures uncertainty over which concrete response-value the regulator will use after the disturbance is known. That is useful, but it can overcount ignorance if several concrete responses are equivalent for the adopted success criterion. Entropy only measures uncertainty over the variable supplied to it; it does not know which distinctions matter for regulation.

To avoid counting irrelevant implementation variation as ignorance, we introduce response-equivalence classes. “Response-equivalence” means that two concrete responses belong to the same response class if the model treats them as interchangeable for the regulatory purpose being analyzed. At the coarsest success-preserving granularity, two concrete responses are placed in different classes only if choosing between them can matter to whether the resulting outcome is acceptable under the adopted success criterion.

For a fixed time t, let gt : 𝓡 → 𝓧t map each concrete response-value to a response-equivalence class. Each element of 𝓧t is therefore a subset of 𝓡. The partition 𝓧t should be chosen at the coarsest granularity that still preserves distinctions relevant to the adopted valuation and success criterion. Note that this choice is what fixes the finite resolution at which KTD is measured. The response-class random variable is:

Xt := gt ( Rt ) .

One natural acceptability-based response-equivalence requirement is:

gt(r1) = gt(r2) ⇒ ∀d∈D [ φ(T(d,r1)) ∈ηt ⇔ φ(T(d,r2)) ∈ηt ] .

That is, two concrete responses may be placed in the same class only if replacing one with the other never changes success or failure under the adopted regulatory criterion. This weaker version is enough when regulation only asks: Did the outcome fall inside the acceptable region?

If the model needs to preserve exact essential-variable values rather than merely acceptability-status, a stronger valuation-based response-equivalence requirement may be used:

gt(r1) = gt(r2) ⇒ ∀d∈D [ φ(T(d,r1)) = φ(T(d,r2)) ] .

The weaker success-equivalence relation is appropriate when only membership in the acceptable region matters. The stronger valuation-equivalence relation is appropriate when different acceptable essential-variable values must still be distinguished. If the modeler wants the coarsest possible partition, the corresponding implication can be strengthened to a biconditional. “Response-equivalence” is important because it determines what uncertainty counts as regulatory ignorance.

Using the acceptable concrete response-set 𝒜t(d) defined above, the corresponding acceptable response-class set is:

X acc,t (d) := { x ∈ Xt : x ⊆ At (d) } .

In words, a response-class is acceptable for disturbance d when every concrete response in that class is acceptable for d. If a proposed class contains both acceptable and unacceptable concrete responses, then the class is too coarse for the adopted success criterion and must be refined.

Lifted to response classes, the regulatory relation becomes:

RtX := { (d,x) ∈ D×Xt : x∈ Xacc,t (d) } .

From here on, where no confusion arises, Rt denotes this class-level relation. The unique-target case used by much of the literature is the special case in which every section is a singleton, Xacc,t (d) = {qt(d)} , so that the relation is the graph of the target-class map.

The basins of the cycle branches are Cantor-like [66] (Fig. 6). Any finite bin near a basin boundary contains inputs leading to different branches. The refinement therefore may never finish, and a coarsest success-preserving partition may not exist at finite resolution. [66] Fig. 8 shows the number of certainly-reachable outcomes growing with r. Therefore, gt presupposes non-fractal acceptability boundaries.

Rough-set reading of response-equivalence

The partition Xt induced by gt is, in the language of Pawlak's rough set theory [67][68], an indiscernibility partition of the response space: two concrete responses in the same class cannot be told apart at the regulatory granularity being analyzed. For a disturbance-value d, the acceptable concrete response-set At(d) is then a target concept in R, and the classes of Xt approximate it from below and from above:

A¯ t (d) := ⋃ { x∈Xt : x⊆At(d) } ,
A¯ t (d) := ⋃ { x∈Xt : x∩At(d) ≠∅ } ,
BNDt(d) := A¯ t (d) ∖ A¯ t (d) .

The acceptable response-class set Xacc,t (d) defined above consists exactly of the classes that make up the lower approximation. Classes in the boundary region are the mixed classes that the preceding subsection rejects as too coarse: they contain both acceptable and unacceptable concrete responses for d. Classes outside the upper approximation are wholly unacceptable.

Proposition R1 — Success-equivalence is exactness. The partition Xt satisfies the acceptability-based response-equivalence requirement if and only if every acceptable response-set is exact with respect to it:

∀d∈D , BNDt(d) = ∅ .

Proof. If responses in the same class never differ in acceptability, each class lies wholly inside or wholly outside At(d) , so the two approximations coincide. Conversely, if r1 and r2 share a class x and differ in acceptability for some d, then x meets At(d) without being contained in it, so x⊆BNDt(d) . □

For finite R, or under an admissible size measure μ, Pawlak's accuracy and roughness give probability-free summaries of how far the partition is from success-preserving:

acct(d) := μ( A¯ t (d)) μ( A¯ t (d)) , rought(d) := 1− acct(d) ,

with acct(d):=1 when the upper approximation is empty (the acceptable set is then exact but empty, and no response regulates d). These are support-level quantities and belong to the set-based layer: they register whether mixed classes exist and how much of the response space they occupy, not how probability is distributed over them.

Decision-table reading of Knowledge To Be Discovered

The same vocabulary applies on the disturbance side, where the approximating partition is induced by the regulator's observation rather than by the goal. Anticipating the quantities defined in the subsection on Knowledge To Be Discovered below, take the modeled disturbance set D as the universe, the observation Yt as the condition attribute, and the required target class Xt* = qt(Dt) as the decision attribute. Assume here that the observation is a deterministic function of the disturbance, Yt= ot(Dt) , and that PMt (Dt=d)>0 for every d∈D ; the noisy case is treated where Dt(y) is introduced. The sets Dt(y) are then the indiscernibility classes of the observation, and the generalized decision of an observation-value y is the set of required classes still compatible with it:

∂t(y) := qt [ Dt(y) ] = { qt(d) : d∈ Dt(y) } .

The positive region is the set of disturbances whose required class is already fixed by observation alone, and its probability is the probability-weighted form of Pawlak's degree of dependency:

POSY (Xt*) := ⋃ { Dt(y) : |∂t(y)| =1 } , γt := PMt ( Dt∈ POSY (Xt*) ) .

Under a uniform distribution on D, γt reduces to Pawlak's |POSY| / |D| [68].

Proposition R2 — Support bound and consistency. For every observation-value y with positive probability, suppMt ( Xt* ∣ Yt=y ) = ∂t(y) , and therefore

HMt ( Xt* ∣ Yt=y ) ≤ log2 |∂t(y)| , HMt ( Xt* ∣ Yt ) ≤ EMt [ log2 |∂t(Yt)| ] .

Moreover, HMt ( Xt* ∣ Yt ) =0 if and only if γt=1 , that is, if and only if the decision table is consistent.

Proof. The posterior of Xt* given Yt=y is the push-forward under qt of the posterior of Dt , whose support is Dt(y) ; its support is therefore qt [Dt(y)] . A distribution on k points has entropy at most log2k , and zero entropy exactly when k=1 . Averaging over y gives the second inequality and the equivalence. □

The two layers therefore agree on where Knowledge To Be Discovered vanishes and differ on how large it is elsewhere. The positive region and the dependency degree record only which observation-values leave more than one required class possible; KTD additionally weights those candidates by the posterior. Two observation channels with the same generalized decisions have the same positive region and the same support bound, but in general different KTD. The bound log2 |∂t(y)| is attained exactly when the posterior is uniform over ∂t(y) , which is the count-variety special case of the set-based formulation. Finally, under the bridge condition of the staged-selection section, the generalized decision is the initial epistemic class-candidate set, C t,0 X (y) = ∂t(y) : it is the support from which staged selection starts.

Propositions R1 and R2 are support-level statements. Once queries are introduced below, they reappear as worst-case query bounds: the positive region is the set of disturbances that need no query at all (Proposition Q3), and a dependency degree of one, computed with the answers of the regulator's queries added to the observation, is exactly the condition that those queries can finish the job (Proposition Q4).

Regulator's learned law of action

The regulator's accumulated structure Mt is its learned law of action: its prior knowledge about disturbances, available responses, outcome consequences, and success-relevant distinctions. In the deterministic case, this learned structure may be represented as a response rule:

Mt : D → R .

In the general case of learning and uncertainty, Mt induces a response policy:

Mt ⇒ PMt ( Rt | Dt ) .

This policy describes how probability mass is allocated across possible response-values or response-classes for each disturbance-value, given the regulator's current knowledge state. If the Table of Outcomes T is the fixed space of possibilities, then Mt is the learned structure that induces a probability distribution over possible paths through that table.

Across time, Mt need not remain fixed. Changes in Mt alter which disturbance-response pairs become more or less probable, and therefore how the system's realized trajectories are distributed over the Table of Outcomes. Successful learning shifts probability mass away from ineffective responses and toward responses that better compensate the same disturbances.

All probabilities, entropies, and mutual informations used to describe the regulator's knowledge state can therefore be read epistemically, relative to Mt. When that dependence matters, we write PMt and HMt. When it is clear from context, the subscript is suppressed.

When the regulator's response variation is both disturbance-specific and correct for the Table of Outcomes, Mt reduces residual outcome or essential-variable variety. However, correctness is still judged by the set-based success condition:

Pr ( Zt ∈ Oacc,t ) = 1

or, equivalently:

Pr ( Et ∈ ηt ) = 1 .

The regulator's structure contains a second component besides its knowledge state: the questions it can ask. Write Qt for its query family, the set of discriminating operations it can perform on a disturbance before committing a response. Its formal definition is given with Knowledge To Be Discovered below.

The two components play different roles. Mt fixes what the regulator believes; Qt fixes what it can find out. Ashby's requirement applies to both: a regulator can fail for lack of knowledge, and it can also fail for lack of questions.

Knowledge To Be Discovered

Let Yt denote the disturbance information available to the regulator. In the simplest fully observed case, Yt = Dt. In a partially observed case, Yt may be only a signal, observation, or classification of the disturbance.

For a particular observation-value y, define the disturbance-values still possible under that observation as:

Dt (y) := { d ∈ D : PMt ( Dt = d | Yt = y ) > 0 } .

Then the acceptable response-class set under observation y is:

X acc,t (y) := ⋂ d∈Dt(y) X acc,t (d) .

Thus, under partial observation, a response-class is safely acceptable only if it works for every disturbance-value still possible under the regulator's observation. If this intersection is empty, then no response-class is guaranteed to succeed under that observation without additional information, additional buffering, or a richer response repertoire.

When the observation is a deterministic function of the disturbance, the sets Dt(y) partition the disturbances of positive prior probability, and the decision-table reading of the rough-set subsection above applies directly. When the observation is noisy, the posterior supports of different observation-values may overlap. The relation

d ≃t d′ :⟺ ∃y : d,d′ ∈ Dt(y)

is then reflexive and symmetric but in general not transitive: a tolerance relation rather than an equivalence. This is the structure studied for incomplete information systems, where a missing attribute value makes an object similar to several objects that are not similar to each other [69]; the sets Dt(y) play the role of tolerance classes. Lower approximations, positive regions and generalized decisions carry over with tolerance classes in place of equivalence classes, and the support bound of Proposition R2 continues to hold for each y, because its proof uses only the support of the posterior.

The tolerance reading also exposes the coarseness of the support-level condition above: Xacc,t (y) depends on Dt(y) and not on the posterior weights within it, so a disturbance-value of negligible posterior probability constrains the acceptable classes as much as a likely one. The rough-set literature on incomplete information systems makes the same criticism of rough entropy, which cannot distinguish systems that share the same tolerance classes, and responds with graded degrees of equivalence [72]. In the present formulation that grading is supplied directly by the posterior PMt (Dt∣Yt) , and the measure of remaining uncertainty is the conditional entropy HMt ( Xt* ∣ Yt ) , not a set-level surrogate for it.

Queries and decision procedures

A query is a map q:D→ Uq into a finite answer alphabet, binary unless stated otherwise. The query family Qt is the set of queries the regulator can execute. Answers here are deterministic functions of the disturbance; noisy answers and realized signal paths are treated with staged selection.

A decision procedure over Qt for observation y is a finite rooted tree ϑ whose internal nodes are labelled by queries in Qt , whose edges are labelled by answers, and whose leaves are labelled by response classes in Xt . Each d∈ Dt(y) follows the path its answers determine. Write ℓϑ(d) for the number of queries on that path and ϑ(d) for the class at its leaf. The procedure is certainty-terminated for y when every leaf it reaches is acceptable:

ϑ(d) ∈ Xacc,t (d) for every d∈ Dt(y) .

The worst-case and distributional decision-tree complexities of the relation under observation y are the least depths such procedures can achieve, with the minimum taken over certainty-terminated procedures over Qt :

DTwcQt (Rt∣y) := minϑ max d∈Dt(y) ℓϑ(d) ,
DTμyQt (Rt∣y) := minϑ ∑ d∈Dt(y) PMt (Dt=d∣Yt=y) ℓϑ(d) ,

where μy := PMt (Dt∣Yt=y) . Both are set to ∞ when no certainty-terminated procedure exists. When Qt contains every binary question about the disturbance, the superscript is dropped and the quantities are called unrestricted.

A procedure with no queries at all, a single leaf, is certainty-terminated exactly when the intersection Xacc,t (y) defined above is non-empty. The earlier statement that an empty intersection leaves no class guaranteed to succeed “without additional information” therefore reads: at least one query is needed.

Definition of Knowledge To Be Discovered

Knowledge To Be Discovered must not depend on a choice the regulatory problem does not make. When several response classes are acceptable for the same disturbance, the regulator is free to settle on any of them, and it may settle differently for different observations. The definition therefore minimizes over all such choices.

An admissible selector is a map χ that assigns to every observation-value y and every d∈ Dt(y) an acceptable class, χ(d,y) ∈ Xacc,t (d) . Write Sel(Rt) for the set of admissible selectors. The selector may depend on y because a decision procedure's output may.

The Knowledge To Be Discovered under observation y, is:

KTDrel(y) := min χ∈Sel(Rt) HMt ( χ(Dt,y) ∣ Yt=y ) .

, and its average over observations, is:

KTDrel := ∑y PMt (Yt=y) KTDrel(y) .

It is the remaining uncertainty, after the available disturbance information is known, about which acceptable response class the regulator will have to fix, evaluated for the most economical admissible way of fixing one. With a finite posterior support and finitely many classes, the minimum is attained.

If the regulatory problem requires a unique valuation-based class for each disturbance-value, every section of Rt is a singleton {qt(d)} , the only admissible selector is the target-class map, and the definition reduces to the conditional entropy of the required target class:

Xt* := qt(Dt) , KTDrel = HMt ( Xt* ∣ Yt ) .

This unique-target form is the one used in most of the article, and it is written H(X∣Y) when the dependence on Mt is clear. Every result stated for it holds for the relational form with Xt* replaced by a minimizing selector's class.

Proposition Q0 — Basic properties. For every observation-value y:

  1. KTDrel(y)=0 if and only if Xacc,t (y)≠∅ , that is, if and only if one class is acceptable for every disturbance still possible.
  2. For every map qt that picks an acceptable class for each disturbance, KTDrel(y) ≤ HMt (qt(Dt) ∣Yt=y) . Imposing a unique target never lowers the measured ignorance; it can only add uncertainty among classes that were equally acceptable.

Proof. (1) A selector's class has zero conditional entropy exactly when it is constant on the posterior support, and a constant admissible selector exists exactly when some class is acceptable for every d∈ Dt(y) . (2) Such a target map, read as χ(d,y):= qt(d) , is one admissible selector, and the minimum is no larger than any member. □

Proposition Q1 — Relational bracket. Let the posterior support Dt(y) be finite and every section of Rt on it non-empty. With unrestricted binary queries:

KTDrel(y) ≤ DTμy (Rt∣y) < KTDrel(y) +1 .

Averaging over observations gives the same bracket for KTDrel . In the unique-target case this is the one-bit bracket of Lemma 1.

Proof. Lower bound. Let ϑ be certainty-terminated. Its output χ(d,y):= ϑ(d) is an admissible selector, and it is a function of the leaf d reaches. The answer strings at the leaves form a binary prefix-free set, so by the Kraft inequality the expected depth is at least the entropy of the leaf reached, which is at least the entropy of the output class, which is at least KTDrel(y) . Upper bound. Let χ* attain the minimum, and take a Huffman code for the distribution of χ*(Dt,y) given Yt=y . Asking for the bits of the codeword one at a time is a sequence of binary questions about the disturbance; it stops at the class χ*(d,y) , which is acceptable, after fewer than KTDrel(y)+1 questions on average. □

A separate matter is whether the regulator's actual response-class variable Xt= gt(Rt) is aligned with what success requires. Its conditional entropy HMt (Xt∣Yt) can measure randomness, exploration, implementation variation, or consistently wrong behaviour, and is not Knowledge To Be Discovered unless the actual class is an admissible selection. A regulator may be perfectly selective and still wrong, so Knowledge To Be Discovered is paired throughout with a success condition:

Xt ∈ X acc,t (Dt) .

This is the key distinction: uncertainty over concrete responses is not automatically ignorance. It becomes regulatory ignorance only when the unresolved distinction matters for achieving an acceptable outcome under the adopted success criterion.

If Xt is a quotient of the concrete response variable Rt, then Xt is determined by Rt. Therefore:

HMt (Xt|Yt) ≤ HMt (Rt|Yt) .

More explicitly, since Xt = gt(Rt):

HMt (Rt|Yt) = HMt (Xt|Yt) + HMt (Rt|Xt,Yt) .

The second term, HMt (Rt|Xt,Yt) , is residual uncertainty over which concrete response inside the already-fixed response-equivalence class will be emitted. That residual uncertainty is not necessarily regulatory ignorance. It may simply be implementation variation.

For example, suppose the disturbance is “pay $10,” and both “use one $10 bill” and “use ten $1 bills” are equally acceptable from the valuation layer's point of view. These are different concrete response-values, but they belong to the same response-equivalence class if the adopted success criterion cares only that $10 is paid. In that model, uncertainty between those concrete responses should not increase Knowledge To Be Discovered.

If the environment later cares about speed, accounting policy, fraud risk, or change preservation, then “one $10 bill” and “ten $1 bills” may no longer be equivalent. They should then be separated into different response-equivalence classes. In that revised model, uncertainty over which bill combination to use is genuine lack of requisite knowledge.

Thus, Xt must be defined at the right granularity. If Xt is too fine, the model counts irrelevant implementation variation as ignorance. If Xt is too coarse, the model hides distinctions that matter for success. Knowledge To Be Discovered is meaningful only when the response-equivalence classes preserve exactly the distinctions required by the adopted regulatory criterion.

Therefore, response-equivalence is the bridge between the entropy formula and Ashby's regulatory success condition. Without it, H(X|Y) is merely uncertainty over a chosen response variable. With it, H(X|Y) is uncertainty over the response variable at the regulatory granularity being analyzed: the uncertainty that must still be resolved for successful regulation.

Resolution-relativity of Knowledge To Be Discovered

Knowledge To Be Discovered is not an absolute property of the underlying state space. It is a conditional entropy defined over the response-equivalence classes that the regulatory problem requires to be distinguished, conditioned on the disturbance distinctions available to the regulator. Regulatory granularity is set by the goal, not by the observer's measurement precision. It is how coarse the classes can be without mixing acceptable and unacceptable responses. Thus, the granularity of X is fixed by success-relevance i.e. the coarsest partition preserving acceptability (or valuation) under ηt, φ and T .

Physical resolution is how finely the regulator can discriminate disturbances and actuate responses. It determines whether that regulatory partition can actually be realized by the regulator's discrimination and actuation. Physical observations have finite precision. Continuous states are therefore represented by distinguishable bins, and Shannon information applies to those resulting symbols. More importantly, the amount of observable information depends explicitly on resolution[66].

KTD is relative to both the distinctions that matter for success, and to physical resolution, which constrains which of those distinctions the regulator can make. H(X | Y) depends on the resolution of both variables, in opposite directions. Refining X by splitting each class into at most k subclasses can raise H(X | Y) by at most log2 k [66]. That is our "implementation variation counted as ignorance" problem. Refining Y can never raise H(X | Y), because conditioning on a finer partition does not increase entropy.

This answers the question of why KTD is finite when the underlying variables are continuous? Differential entropy can be negative and changes under reparametrization of the response space by the expected log-Jacobian. Entropy over a fixed partition is invariant to any relabelling or change of units, so KTD does not depend on how the response parameter is coordinatized. The condition is that the success-preserving partition must be finite, or at least have finite entropy[66]. Metzler's fractal basins show this can fail. When the acceptability boundary is fractal, every finite cell near it is mixed, so the coarsest success-preserving partition does not exist at finite resolution. At the regulator's physical resolution some classes then straddle the boundary, and certainty-guaranteed success x ⊆ A t ( d ) } . is unattainable. Thus, KTD is finite whenever the regulatory problem admits a finite success-preserving partition at the regulator's physical resolution.

The rough-set reading of response-equivalence gives this a quantitative form. Write Xt(k) for the response partition realizable at physical resolution k, with Xt(k+1) refining Xt(k) . Refinement never shrinks a lower approximation and never enlarges an upper one, so acct(d) is non-decreasing and rought(d) non-increasing along the sequence [68]. By Proposition R1, a success-preserving partition exists at resolution k exactly when BNDt(d)=∅ for every d. When the acceptability boundary is fractal in the sense of [66], every finite cell near it is mixed, so the boundary region is non-empty at every finite resolution. Since each mixed cell has positive measure, rought(d) may decrease under refinement but stays strictly positive. Roughness then measures, at the regulator's physical resolution, how much of the candidate response space lies where certainty-guaranteed success is unavailable. The same monotonicity holds on the disturbance side: refining the observation partition never shrinks the positive region POSY(Xt*) , which is the support-level counterpart of the fact that refining Y never increases H(X∣Y) .

Where the acceptability boundary is too fine for any such partition - as with the fractal basins of the logistic map (Metzler, Bar-Yam & Kardar 2004) - some classes are necessarily mixed and certainty-guaranteed success is unavailable. The observer-relativity of variety was noted and deliberately set aside by Ashby [2], Bateson and Heylighen [29] said which distinctions should count, and Metzler et al.[66] show quantitatively how the numbers move with resolution: Id = Ic + log2 k . Response-equivalence is this article's way of taking up the complication Ashby deferred.

Decision-tree complexity inherits the same relativity. Refining the observation never raises the distributional complexity on average, because a procedure for the coarser observation remains available on each finer cell. Splitting each response class into at most k subclasses raises Knowledge To Be Discovered by at most log2k , and by Proposition Q1 raises the unrestricted complexity by less than log2k+1 . When the acceptability boundary is fractal, every query family of finite resolution leaves some cell mixed, so by Proposition Q4 the restricted complexity is infinite at every finite resolution.

Knowledge To Be Discovered and effective regulatory knowledge

The preceding definition of Knowledge To Be Discovered is not separate from the mutual-information terms used in the Ashby-style formulation below. It is the complementary uncertainty term in the same entropy decomposition.

Under the unique-target assumption, Xt* is the response-equivalence class that must be fixed for successful regulation. The total uncertainty over required response-classes is:

H(Xt*) = I(Xt*;Yt) + H(Xt*|Yt) .

The mutual-information term I(Xt*;Yt) measures how much the available observation reduces uncertainty about the required response-class. The conditional-entropy term H(Xt*|Yt) is Knowledge To Be Discovered: the remaining uncertainty about which response-class must be fixed.

Thus Knowledge To Be Discovered and effective regulatory knowledge are two sides of the same decomposition:

Knowledge To Be Discovered = H(Xt*) − I(Xt*;Yt) .

In the fully observed case, where Yt = Dt, this becomes:

H(Xt*|Dt) = H(Xt*) − I(Xt*;Dt) .

This is the response-class version of Aulin-Ahmavaara's ignorance term. At the correct regulatory granularity, the effective knowledge available to the regulator is the part of required response-class variety already determined by the disturbance information. The Knowledge To Be Discovered is the part not yet determined.

In query complexity terms, the mutual information is the part of the required response-class variety that the regulator's standing observation has answered before any query is asked. Knowledge To Be Discovered is the part that queries must still answer.

Therefore, the later mutual-information term I(St;Dt) should be read as an effective regulatory-knowledge term only when St is defined at the same success-relevant granularity as the required response-class variable. In the unique-target case, this means setting St = Xt*. If St denotes concrete responses or some other granularity, the mutual-information term measures disturbance-response coupling, but not necessarily Knowledge To Be Discovered.

Relation to rough entropy. Because the required class is a function of the disturbance, Xt*= qt(Dt) , we have H ( Xt* ∣ Dt,Yt ) =0 , and expanding H ( Dt,Xt* ∣ Yt ) by the chain rule in both orders gives

HMt (Dt∣Yt) = HMt ( Xt* ∣ Yt ) + HMt ( Dt ∣ Xt*, Yt ) .

The left-hand side is the regulator's residual uncertainty about the disturbance itself after observation. When the observation is a deterministic function Yt= ot(Dt) and the distribution on D is uniform, it coincides with the rough entropy of the knowledge induced by the observation,

H (Dt∣Yt) = 1 |D| ∑ d∈D log2 | Dt (ot(d)) | ,

the rough-set measure of how coarsely the observation granulates the disturbances [71]. The decomposition places Knowledge To Be Discovered inside it: KTD is the response-relevant part of the residual disturbance ambiguity. The second term, H ( Dt ∣ Xt*, Yt ) , is disturbance ambiguity that no longer matters once the required class is fixed; it is the disturbance-side counterpart of the implementation-variation term H ( Rt ∣ Xt, Yt ) on the response side. Hence H ( Xt* ∣ Yt ) ≤ H (Dt∣Yt) , with equality exactly when qt is injective on every posterior support Dt(y) , that is, when every disturbance distinction the regulator cannot yet make is one that matters for success. The decomposition holds for noisy observation as well, since it uses only that the required class is a function of the disturbance. A granularity measure of the observation alone, such as rough entropy, therefore overstates Knowledge To Be Discovered by exactly the regulation-irrelevant ambiguity.

Ashby-style residual-variety bounds

The preceding success conditions are exact confinement conditions. A variety inequality is not the definition of success; it is a necessary numerical consequence under additional assumptions.

In the idealized finite-table case, where disturbances and responses are treated as distinguishable possibilities, the Ashby-style count-variety lower bound can be written as:

V ( yt [D] ) ≥ ⌈ V (D) V (R) ⌉ .

In logarithmic form, and ignoring the ceiling for the idealized entropy analogue, this becomes:

log2 V ( yt [D] ) ≥ log2 V (D) − log2 V (R) .

This expression should be read as an idealized lower-bound form, not as a general theorem about every possible Table of Outcomes. It assumes that disturbance variety is not already collapsed by the table structure except for any explicitly modeled buffering or passive absorption. If the table itself collapses several disturbance-values into the same outcome-value, then the lower bound must be modified to reflect that additional collapse.

The preceding subsection defined Knowledge To Be Discovered at the level of the required response-equivalence class. The Ashby-style residual-variety bound can now be written using a general response variable St, where St is the response variable at the regulatory granularity being analyzed. If concrete response-values are relevant, then St = Rt. If response-equivalence classes are the relevant units, two cases must be distinguished. If St denotes the actual response-class emitted by the regulator, then St = Xt. If St denotes the required target response-class, then St = Xt*. Thus St is not a new kind of response; it is a placeholder for the response-level at which regulatory knowledge and residual variety are being measured.

Under the same idealizing assumptions as the finite-table quotient argument - no unmodeled table collapse except buffering K, a response variable St defined at the same regulatory granularity as the outcome variable being bounded, and correct disturbance-specific compensation - the following Shannon-style residual-variety analogue may be written:

H ( Zt ) ≥ [ H ( Dt ) + H ( St | Dt ) − H ( St ) − K ] + .

Here [a]+ := max(a,0), since residual entropy cannot be negative. The term K denotes buffering capacity: disturbance variety absorbed passively before active regulation is required. This equation should be read as an information-theoretic analogue of Ashby's finite-table lower-bound argument under the stated idealizing assumptions.

Since:

H ( St ) − H ( St | Dt ) = I ( St ; Dt ) ,

the same bound can be written compactly as:

H ( Zt ) ≥ [ H ( Dt ) − I ( St ; Dt ) − K ] + .

The mutual information I(St;Dt) measures how much the regulator's response variable, at the chosen regulatory granularity, is coupled to disturbance variation.

Heff ( St ) := I ( St ; Dt ) = H ( St ) − H ( St | Dt ) .

This is a measure of disturbance-specific response selection at the regulatory granularity being analyzed. By itself, however, it is not a guarantee of correctness. A response can be strongly coupled to a disturbance and still be the wrong response for the Table of Outcomes. Under the additional assumption that the coupling maps disturbances to appropriate compensating responses, it can be interpreted as requisite regulatory knowledge.

The outcome-level residual-variety bound can therefore be written as:

H ( Zt ) ≥ [ H ( Dt ) − Heff ( St ) − K ] + .

Combining this lower bound with the allowed outcome-level residual variety gives the following necessary feasibility condition for strict success:

Heff ( St ) + K ≥ H ( Dt ) − hO,t .

In the reduced or bijective case, where the acceptable outcome-set and the acceptable essential-variable region have the same relevant variety, this becomes:

Heff ( St ) + K ≥ H ( Dt ) − hη,t .

Expanded, this is:

H ( St ) ≥ H ( Dt ) + H ( St | Dt ) − K − hη,t .

If this necessary condition fails under the stated assumptions, strict success is impossible. If it holds, success is still not guaranteed; the response mapping must still be appropriate for the actual Table of Outcomes. The decisive success condition remains the confinement condition:

Pr ( Et ∈ ηt ) = 1 .

In the many-to-one case, the outcome-level and essential-variable-level statements must be kept distinct. A lower bound on H(Zt) does not automatically imply the same lower bound on H(Et), because φ may collapse outcome distinctions that are irrelevant at the essential-variable level. The safer formulation is therefore to write the bound first for Zt and then relate Zt to Et through Et = φ(Zt).

Point regulation

In the special case of point regulation, the acceptable region contains only one essential-variable state:

|ηt| = 1 and therefore hη,t = 0 .

Under the reduced or bijective representation, the necessary feasibility condition becomes:

Heff ( St ) + K ≥ H ( Dt ) .

Equivalently:

H ( St ) ≥ H ( Dt ) + H ( St | Dt ) − K .

In the ideal best-opportunity case, where buffering is ignored, the response policy is deterministic at the relevant regulatory granularity, and the response mapping is correct, this reduces to:

H ( St ) ≥ H ( Dt ) .

This should be read as an optimal-control limit, not as a blanket claim that every control problem is solved whenever response entropy is at least as large as disturbance entropy. The response mapping must also select the right response-values or response-classes for the actual Table of Outcomes, so that the induced outcome-values fall inside the acceptable outcome-set.

The deterministic-policy assumption means that the regulator's response variable at the chosen regulatory granularity is fully determined by the disturbance: H(St|Dt) = 0. This represents complete requisite knowledge only under the additional assumption that the learned response mapping sends each disturbance to an appropriate regulatory act. A regulator can deterministically choose the wrong response; therefore, zero conditional entropy alone does not prove correct regulation.

Successful essential-variable outcomes do not depend solely on the amount of response variety available to a regulator. The system must also be able to select the appropriate response for the given disturbance. Effective compensation of disturbances requires that the system possess a mapping from disturbances to appropriate responses within its repertoire. The absence or incompleteness of such knowledge can be represented by the conditional entropy H(St|Dt) only when St denotes the required response-class variable, or when the actual response variable is assumed to be aligned with the correct target mapping. Otherwise, zero conditional entropy means only that the response is determined by the disturbance; it does not by itself mean that the selected response is correct.

In other words, H(St | Dt) measures how much uncertainty remains about the regulator's response at the chosen regulatory granularity after the disturbance is known. Under the regulatory correctness assumption, this uncertainty corresponds to lack of requisite knowledge. Merely increasing response variety is therefore not sufficient. It must be complemented by a corresponding increase in selectivity: a reduction in H(St | Dt), i.e. an increase in knowledge. This requirement may be called the law of requisite knowledge[29].

The larger H(St | Dt) is, the less selectively the regulator's actions are determined by the disturbance. Under the assumption that each disturbance requires a limited set of appropriate responses at the chosen regulatory granularity, this increases the risk of selecting an ineffective response. Therefore, the term H(St | Dt) has a plus sign in the inequality: more uncertainty about how to respond increases the lower bound on residual outcome or essential-variable variety[54].

Knowledge To Be Discovered as a floor on query complexity

Knowledge To Be Discovered measures what must be found out; the decision-tree complexities measure how many questions finding it out takes. The two are ordered, and the gaps between them have distinct causes.

For any binary query family, every procedure over Qt is also an unrestricted procedure, so Proposition Q1 extends to a chain:

KTDrel(y) ≤ DTμy (Rt∣y) ≤ DTμyQt (Rt∣y) .

Two gaps follow. The coding gap δ(y) is the first difference; it lies in [0,1) by Proposition Q1 and vanishes for dyadic posteriors. The query-family gap

ψ(y) := DTμyQt (Rt∣y) − DTμy (Rt∣y) ≥ 0

is what the regulator's available questions force beyond the best unrestricted procedure. It is zero when Qt contains the questions of an optimal unrestricted procedure, and infinite when the family cannot certainty-terminate at all. A third rung, the expected length of the regulator's actual protocol, is added with staged selection; its excess over DTμyQt is the protocol gap.

For a query family with answer alphabets of size at most m, the first inequality weakens to KTDrel(y) ≤ log2m ⋅ DTμyQt (Rt∣y) , by the same Kraft argument with an m-ary prefix code.

Example. A robot must choose one of four moves with posterior (12,14, 18,18) for (up, down, left, right). Knowledge To Be Discovered is 1.75 bits, and the unrestricted complexity is also 1.75 queries, so δ=0 . If the robot's sensors can only answer “vertical or horizontal?” and then “which of the two?”, every procedure asks two questions, so ψ=0.25 . The robot lacks no knowledge; it lacks a question.

Proposition Q3 — Relational worst-case bound. Let cov(y) be the least number of response classes such that every d∈ Dt(y) has one of them acceptable, equivalently the least number of distinct classes an admissible selector can use on the support. With unrestricted binary queries:

KTDrel(y) ≤ log2cov(y) , DTwc (Rt∣y) = ⌈ log2cov(y) ⌉ ,

and every restricted binary family needs at least that many queries in the worst case. In the unique-target case cov(y)= |∂t(y)| , and the support bound of Proposition R2 becomes an exact worst-case depth. The value cov(y)=1 , depth zero, describes exactly the disturbances in the positive region.

Proof. A selector using c classes has entropy at most log2c ; apply this to a selector using cov(y) classes. A binary procedure of depth k has at most 2k leaves, so its output selector uses at most 2k classes, which forces 2k≥cov(y) . Conversely, halving a minimal cover with one binary question at a time reaches one class in ⌈log2cov(y)⌉ steps. □

Computing cov(y) is a minimum set-cover problem and is hard in general. The proposition is a characterization, not an algorithm.

Proposition Q4 — Query-family adequacy. Let Dt(y) be finite, and call two disturbances indiscernible by Qt when every query in the family gives them the same answer. Then DTμyQt (Rt∣y) <∞ if and only if every indiscernibility class within the support admits a common acceptable response class.

In rough-set terms: with the answers of Qt added to Yt as condition attributes, the relational positive region has posterior probability one. In the unique-target case this is the probability-weighted dependency degree of the decision-table reading, computed with the enlarged condition attributes, equal to one.

Proof. Necessity: any procedure over Qt sends indiscernible disturbances along the same path to the same leaf, so that leaf's class must be acceptable for all of them. Sufficiency: the support is finite, so finitely many queries in the family separate its indiscernibility classes; asking them in turn reaches a class, where its common acceptable response class is output. □

The ladder separates three questions that Ashby's law runs together: what must be known (Knowledge To Be Discovered), what can be asked (the query family), and how well it is asked (the protocol, added with staged selection). Every rung above the first is latent, like Knowledge To Be Discovered itself; its value is diagnostic, naming why the questions actually asked exceed the knowledge that was missing.

The information-theoretic formulation therefore does not replace the set-based success condition. It measures the uncertainty, variety, buffering, and knowledge involved in satisfying that condition. The set-based section gives the geometry of the regulatory problem; the information-theoretic section puts probability mass over that same geometry and measures the remaining uncertainty in bits.

4. Knowledge Discovery Process

The process of selection may be either more or less spread out in time. In particular, it may take place in discrete stages. What is fundamental quantitatively is that the overall selection achieved cannot be more than the sum (if measured logarithmically) of the separate selections. (Selection is measured by the fall in variety.) 13/17[2]

Ashby's selection in design and regulation share the same abstract selection schema: both reduce a space of possible alternatives by applying constraints, tests, rules, observations, or feedback. In regulation, however, the regulator does not select an outcome-value directly. It selects a response-value in the presence of a disturbance-value; the Table of Outcomes then determines the resulting outcome-value, and the outcome-to-essential-variable map determines whether that outcome is acceptable.

Importantly, so far we assumed that the regulator selects a response-value to a given disturbance-value in a single step. However, in many real-world regulatory problems, the regulator does not select a response-value in a single step. Instead, the regulator may select a response-value in stages, where each stage of selection may depend on the disturbance-value and on the response-values selected in previous stages. For example, a regulator may first select a coarse response-value, and then refine that coarse response-value in subsequent stages of selection. To capture this staged selection we now add time to the notation to indicate that the candidate response-sets are generated by a process of selection that unfolds over time.

Throughout this subsection, the index t denotes the staged selection trace index at which the disturbance, knowledge state, acceptable region, and committed response are evaluated. It does not index the internal stages of the staged selection trace. The internal stages of a staged selection trace are indexed by i=0,…,k . Thus a staged selection trace at time t may contain several internal selection stages, but those stages are represented by the stage index i , not by changing the external selection trace time index.

Unless explicitly stated otherwise, the acceptable region ηt and the knowledge state Mt are treated as the regulatory frame for the selection trace indexed by t .

Ashby's remark that selection may be spread out in stages is, in present-day terms, a statement about query complexity. Each stage is a query: a discriminating operation whose answer removes alternatives. An episode is one path through a decision procedure for the regulatory relation Rt, from the first query to the committed response. Ashby's claim that the overall selection cannot exceed the sum of the separate selections is the additivity of selection along that path, which the chain rule below makes exact for expected information, and his measure of selection by the fall in variety is the count of what each answer removes. The subsections below keep the existing set-based and information-theoretic accounts of an episode and read each of them as a property of that path or of the procedure that generates it.

Set-based Formulation of Staged Selection

Using the set-based formulation above, for a disturbance-value d∈D and a response-value r∈R , the outcome-value is T(d,r)∈Z . The corresponding essential-variable value is φ(T(d,r))∈E . At time t , this response is successful exactly when that essential-variable value lies in the acceptable region ηt⊆E .

Equivalently, the acceptable outcome-set at time t is the pullback of the acceptable essential-variable region:

Oacc,t := φ−1 [ηt] = { z∈Z : φ(z)∈ηt } .

For a given disturbance-value d , define the acceptable response-set at time t as:

At (d) := { r∈R : T(d,r) ∈ Oacc,t } .

This is just the pullback of the acceptable outcome-set through the row of the table T corresponding to disturbance d.

Equivalently:

At (d) = { r∈R : φ ( T(d,r) ) ∈ ηt } .

The set At(d) contains exactly those response-values that would produce an acceptable outcome-value for the disturbance-value d under the criterion of success prevailing at time t .

We can now define the set-based staged selection trace as the nested reduction of candidate response-values for a realized disturbance-value. For a particular disturbance-value d , let

∅ ≠ Ck (d) ⊆ … ⊆ C1 (d) ⊆ C0 (d) ⊆ R

be a nested sequence of candidate response-sets. Here, C0(d) is the initial set of response-values available or considered for disturbance d . Each stage applies a constraint, observation, test, rule, model, or feedback signal that removes response-values no longer admissible under the current regulatory problem.

The set-based trace is strongly successful when the remaining candidate response-values are all acceptable:

∅ ≠ Ck (d) ⊆ At (d) .

In the limiting case, the process fixes a single acceptable response-value:

Ck (d) = {rt} with rt ∈ At (d) .

Thus, the set-based staged selection trace is the staged reduction of candidate responses until the regulator can commit a response-value that maps the present disturbance into the acceptable outcome-set. The candidate response-set Ci (d) represents the remaining support of response-values still admissible after stage i for the realized disturbance-value d . This is a support-level description. It records which response-values remain possible.

In the language of decision procedures, the nested sequence C0(d)⊇C1(d)⊇…⊇Ck(d) is the record of one path: the set held at depth i is the live candidate set after i answers. Strong success says that every response still live at the end of the path regulates the disturbance, which in terms of the response-level relation of the Law of Requisite Variety section reads:

Ck(d)⊆At(d)⇔(d,r)∈Rtfor everyr∈Ck(d).

The trace therefore solves the relational problem without singling out a response. Any member of the final set may be committed, and the realized condition rt∈At(d) asks only that the one actually committed lie in the relation.

Rule regulation

The rule-level version is obtained by applying the same idea to response rules rather than individual responses. Let Π := RD be the set of possible response rules ρ:D→R .

For a rule ρ∈Π , define its induced outcome map by y ρ ( d ) := T ( d , ρ ( d ) ) and its image over the modeled disturbance set by yρ [D] := { T ( d , ρ (d) )   : d ∈ D }

At a particular moment, the regulator may face one realized disturbance-value. But a response rule is judged by what it would produce for every disturbance-value in the modeled disturbance set D .

Define the successful rule-set at time t as:

Πacc,t := { ρ∈Π : yρ [D] ⊆ Oacc,t } .

The rule-level condition is a universal guarantee over the modeled disturbance set. The episode-level condition is a pointwise condition for one realized disturbance ρ ( d ) ∈ At ( d )

Equivalently:

Πacc,t = { ρ∈Π : φ [ yρ [D] ] ⊆ ηt } .

This means a rule succeeds only if it produces acceptable outcomes for every modeled disturbance.

A rule-level staged selection trace is then a nested sequence of candidate rule-sets:

Π0 ⊇ Π1 ⊇ … ⊇ Πk

The process succeeds at the rule level when the remaining candidate rules are successful:

∅ ≠ Πk ⊆ Πacc,t .

In the limiting case, the process fixes a single successful response rule:

Πk = {ρt} with ρt ∈ Πacc,t .

This rule-level formulation is the universal version of the pointwise acceptable-response condition. Equivalently, a rule is successful when, for every modeled disturbance-value d , it selects a response-value in At(d) .

Therefore, staged response selection is not a process outside regulation. At the set level, it is the nested narrowing of candidate responses or candidate response rules over the disturbance-values under consideration

As noted with the regulatory relation, the successful rule-set Πacc,t is the set of selectors of the response-level relation. The two levels of staged selection therefore search different spaces. A response-level trace narrows the responses for one disturbance; a rule-level trace narrows the selectors, and so fixes in advance what every later response-level trace would have to find out.

This is why rule-level selection matters for query cost. A regulator that holds a successful rule and observes the disturbance exactly needs no query at all within an episode: the rule is a zero-query solution of Rt. The queries have not disappeared; they were spent when the rule was selected. Learning, treated below, is the process that moves query cost from inside episodes into the stored law of action.

Episodes and Closures

The set-based staged selection trace, an episode, and a closure describe three different aspects of the same regulatory event. The set-based staged selection trace is the nested reduction of candidate response-values for a realized disturbance-value. An episode is one realized run of that trace. A closure is the realized outcome-value produced when the episode terminates in a committed response-value.

For a selection trace at time t , suppose the realized disturbance-value is Dt=dt . An episode at time t is the realized set-based staged selection trace initiated for that disturbance-value. The staged candidate trace begins with an initial candidate response-set Ct,0 (dt) and proceeds through a sequence of internal staged reductions:

∅ ≠ Ck (d) ⊆ … ⊆ C1 (d) ⊆ C0 (d) ⊆ R .

Since t is fixed, we suppress the time index and write Ci (d) instead of Ct,i (dt) for readability.

The selection process terminates when the regulator commits a response-value rt ∈ Ck (d) . This commitment terminates the realized staged selection trace for that episode. The committed response-value then produces the realized outcome-value:

zt := T ( d , rt ) ∈ Z .

Thus, a closed episode can be represented by the formal tuple:

εt (d) := ( d , C0(d) , … , Ck(d) , rt , zt ) .

The closure cl ( εt (d) ) is what closes the episode εt (d) : the punctual act, committed at time tcommit , at which the regulator fixes the response rt and ends the episode's staged selection. It is the closure act, not its outcome-value, that individuates one counted closure unit.

The closure act produces a realized outcome-value through the Table of Outcomes, realized at time toutcome :

zt := out ( cl ( εt (d) ) ) = T ( d , rt ) .

The commitment and the outcome need not coincide in time. The half-open interval between them,

( tcommit , toutcome ] ↦ F ,

carries no response-relevant discrimination and no further commitment; it is booked as feedback. The closure unit is anchored and charged at tcommit , while the closure event ct is recorded at toutcome , when zt is realized.

An acceptable closure is a closure whose realized outcome-value belongs to the acceptable outcome-set prevailing at the outcome time toutcome :

out ( cl ( εt (d) ) ) ∈ O acc,toutcome iff zt ∈ O acc,toutcome .

Therefore, there are two different success conditions that should not be confused. A strongly successful staged selection trace terminates with a remaining candidate response-set containing only acceptable responses:

∅ ≠ C k ( d ) ⊆ A t ( d ) .

In that case, any response-value selected from the final candidate set will produce an acceptable closure.

The two conditions are equivalent, since zt = out ( cl ( εt (d) ) ) ; the right-hand form is the one used by the survival and invalidation accounting below. Because acceptability is a property of the outcome-value while counting is a property of the act, two distinct closure acts that produce the same outcome-value remain two counted closures.

A realized successful episode requires only that the response-value actually committed by the regulator be acceptable:

r t ∈ A t ( d ) .

Equivalently:

T ( d , r t ) ∈ O acc, t .

Equivalently, the episode has an acceptable closure exactly when the essential-variable value induced by the realized outcome lies inside the acceptable region:

φ ( T ( d , rt ) ) ∈ ηt .

The strong condition guarantees successful closure before final commitment. The realized condition judges the episode after the committed response has produced its outcome. Therefore, episode closure is defined at the outcome layer. It records that the regulator has committed a response-value and that this commitment has produced a realized outcome-value. Acceptable closure adds the further condition that the realized outcome-value is acceptable under the goal criterion prevailing at time t .

This distinction matters because response commitment and acceptable closure are not identical. A regulator may commit a response-value and still fail to produce an acceptable outcome-value. Such an unacceptable closure act still occupies capacity. The operational ledger of section 5 books it outside the gross closure count, and section 6 reads it as a non-output unit and, when it is a trial for the same target as a later acceptable closure, as a stage of that closure's episode (see “Failed and retracted outputs”). Conversely, an acceptable closure does not require that only one acceptable response-value was possible. Multiple distinct response-values may map the same disturbance-value into the acceptable outcome-set.

At the set level, staged selection is the process by which a regulator narrows candidate response-values until it can commit a response. An episode is one realized run of that staged selection trace. The episode closes when that committed response, together with the disturbance-value, produces a realized outcome-value. The closure is acceptable when that outcome lies in the acceptable outcome-set zt ∈ O acc,t . For an acceptable closure:

d → C0 (d) ⊇ C1 (d) ⊇ ⋯ ⊇ Ck (d) → rt → zt = T ( d , rt ) → zt ∈ O acc,t

The set-based staged selection trace does not by itself produce an outcome-value. It narrows the candidate response-set until the regulator can commit a response-value. The Table of Outcomes then maps that committed response-value, together with the disturbance-value, into a realized outcome-value. The episode has an acceptable closure exactly when that realized outcome-value belongs to the acceptable outcome-set prevailing at the time of closure.

In the query reading, an episode is one root-to-leaf path and its closure is the leaf: the committed response together with the outcome it produces. The closure act is not itself a query. It discriminates nothing; it ends discrimination and commits. That is why the protocol defined next contains only the internal action units, and why the closure is counted separately from them.

Action-unit protocols

The staged candidate trace can be generated by an ordered action-unit protocol. For the episode indexed by t , let

Qt := ( qt,1 , … , qt,kt )

denote the realized physical action-unit protocol inside that episode. Each qt,i is the concrete action-unit token that occurs at internal stage i : a test, observation, constraint application, sensor reading, feedback step, model application, partial execution, or other discriminating operation performed at internal stage i . The action unit may be externally visible or hidden from the observer, but it is treated as part of the realized physical process.

The protocol Q t is defined extensionally as the physical trace that occurs in the episode. This definition does not specify how the regulator chooses the action units, nor does it require the observer to identify the regulator's internal rule for selecting them. It only records that the episode contains an ordered sequence of physical action units. The formalism does not require a model of how the regulator selected that action unit. The action unit may have been fixed in advance, selected adaptively, or generated by an unobserved mechanism. Once it occurs in the realized trace, it is treated as the action unit for that stage.

At the set level, each physical action unit induces an update on the live candidate response-set. Let

Γ q t , i R : 𝒫 ( R ) → 𝒫 ( R )

be the set-update operator induced by the physical action unit q t , i . The update is eliminative or non-expansive:

Γ q t , i R ( C ) ⊆ C for every C ⊆ R .

Thus an action unit may remove candidate response-values, or it may leave the candidate set unchanged. The latter case represents a physical action unit that occurs but produces no set-level narrowing.

For a realized disturbance-value d , define the candidate response-set after stage i by:

C t , 0 R ( d ) := C 0 ( d )

and, for i = 1 , … , k t ,

C t , i R ( d ) := Γ q t , i R ( C t , i − 1 R ( d ) ) .

The resulting set-based staged selection trace is therefore:

∅ ≠ C t , k t R ( d ) ⊆ … ⊆ C t , 1 R ( d ) ⊆ C t , 0 R ( d ) ⊆ R .

The trace is strongly successful when the final candidate response-set is non-empty and every remaining candidate response-value is acceptable for the realized disturbance-value:

∅ ≠ C t , k t R ( d ) ⊆ A t ( d ) .

The protocol Q t contains only the internal physical action units of staged selection. The terminal commitment or emission of a response-value is a separate closure event, denoted: c t

When the regulator closes the episode, it commits a response-value from the final live candidate set:

r t ∈ C t , k t R ( d ) .

In the limiting case, the staged selection trace fixes a single response-value:

C t , k t R ( d ) = { r t } with r t ∈ A t ( d ) .

This is a purely set-based and physical description. It records the realized action-unit protocol and the corresponding reduction of candidate response-values. It does not yet introduce informational variables associated with the action units. Those informational variables are introduced only in the information-theoretic formulation, where the physical action-unit trace is interpreted as inducing epistemic distinctions under the knowledge state M t .

Action units as queries

The protocol Qt is a realized sequence; the query family Qt introduced with Knowledge To Be Discovered is the repertoire it draws on. Each action unit qt,i instantiates a query q∈Qt, and the answer the query returns for the realized disturbance is what the action unit makes available. The set-update then depends on the action unit only through the query and its answer:

Γqt,iR=Γq,q(d)R.

An action unit that discriminates nothing, such as a partial execution that exposes no new answer, is the constant query. Including constant queries in Qt changes no decision-tree complexity, because a constant query never shortens a procedure, so every action unit can be treated as a query without loss.

The protocol is defined extensionally and makes no assumption about how the next action unit is chosen. When that choice is a fixed function of the observation and of the answers already received, the protocols realized for the different disturbances compatible with an observation y are the paths of a single decision procedure ϑt over Qt, and the realized length of an episode is the depth of its path:

kt=ℓϑt(d).

If, in addition, the trace is strongly successful for every d∈Dt(y), the procedure solves the response-level relation on the whole support, with any response in its final candidate set as the output of each leaf. Its longest path is then at least the worst-case decision-tree complexity of that relation over Qt, defined as in the previous section with leaves labelled by responses instead of classes, the minimum being taken over procedures that solve it:

maxd∈Dt(y)kt(d)≥minϑmaxd∈Dt(y)ℓϑ(d).

The universal success condition of the set-based formulation is in this way the worst-case setting of query complexity, as anticipated in the Law of Requisite Variety section. The distributional setting is taken up in the information-theoretic formulation below.

Count-Variety Selection

The set-based formulation above defines the episode, the staged selection trace, and the closure. It records which candidate response-values or response rules remain possible after each stage. This is a support-level account. It can be given a logarithmic count-variety interpretation in the finite equiprobable case, but it does not yet define Knowledge To Be Discovered. Knowledge To Be Discovered requires a probability distribution over the required response-equivalence class and is therefore introduced in the information-theoretic formulation.

For finite equiprobable sets, Ashby[13/15 [2]] measures the amount of selection in bits as:

σ = log2 Vbefore Vafter .

In the present formulation, the “before” and “after” varieties are the cardinalities of explicitly defined candidate sets. For finite candidate response-sets, the count-variety selection achieved at stage i is:

σiset,R = log2 | Ci−1 (d) | | Ci (d) | .

This expression assumes that the remaining alternatives are being counted as distinguishable candidate response-values under the adopted modeling resolution i.e. is a concrete-level quantity rather than an class-level quantity. It measures support reduction, not probability-weighted uncertainty reduction. For the present set-based formulation, no probability distribution over Ci(d) is required.

The total count-variety selection achieved over a nested staged process is therefore:

σtotalset,R = ∑ i=1 k σiset,R = log2 | C0 (d) | | Ck (d) | .

This additive form is valid when the stages are nested refinements of the candidate set. Each stage must reduce the currently remaining alternatives, not independently recount the original space. If two stages constrain the same distinction in overlapping ways, the second stage's contribution must be counted relative to the alternatives that survived the first stage.

For finite equiprobable rule-sets, the corresponding rule-level count-variety selection at stage i is:

σirule = log2 | Πi−1 | | Πi | .

And the total rule-level count-variety selection is:

σtotalrule = ∑ i=1 k σirule = log2 |Π0| |Πk| .

This bridge establishes the count-variety special case. In the finite equiprobable case, and when selection operates purely by eliminating alternatives, logarithmic cardinality reduction coincides with entropy reduction. The information-theoretic formulation generalizes beyond that case by allowing non-uniform probabilities and probability redistribution among surviving candidates. In that formulation, the object being reduced is not merely the candidate set, but the regulator's Knowledge To Be Discovered: the conditional entropy of the required response-equivalence class.

Count-variety selection also has a worst-case reading. A binary action unit can be guaranteed to deliver at most one bit of count-variety selection, because one of its two answers always leaves at least half of the candidates live. The following proposition makes this precise.

Proposition Q3′ — Count-variety as worst-case depth. Let a decision procedure over binary queries generate the set-based traces for the disturbances in Dt(y), with answer-indexed eliminative updates as above. Suppose that at every node each live candidate survives the answer of at least one disturbance that reaches the node. If every path ends with at most c≥1 live candidates, then some disturbance in Dt(y) has a path of length

k(d)≥⌈log2|C0|c⌉,

and along that path every stage satisfies σiset,R≤1. For point selection, c=1, the bound is ⌈log2|C0|⌉.

Proof. At each node the live sets of the answers that occur cover the parent set, so at least one of them keeps at least half of it. Follow that answer at every node. The path exists, because each answer followed is given by some disturbance reaching the node, and after k steps it holds at least |C0|/2k candidates. Ending with at most c requires 2k≥|C0|/c. □

Realized paths can do better than this bound, because an answer that rules out more than half the candidates carries more than one bit of count-variety selection; the information-theoretic formulation returns to such lucky answers below. What a binary stage cannot do is guarantee more than one bit. At the class level, a counting argument of the same kind, with the cover number in place of the candidate count, gives the relational worst-case bound of Proposition Q3.

Information-Theoretic Formulation of Staged Selection

What the Information-Theoretic Model Adds

The information-theoretic formulation does not redefine an episode. An episode remains the realized set-based regulatory event defined above. What this subsection adds is a probability model over that episode.

The set-based formulation records the realized regulatory trace: a disturbance occurs, the regulator narrows candidate responses, fixes a response class, emits a concrete response, and produces a closure through the Table of Outcomes. The information-theoretic formulation adds an epistemic description of that trace: it represents uncertainty over the required response-equivalence class and how that uncertainty changes as information-bearing selection signals are obtained.

A realized episode has an associated conditioning path: Y=y, U1=u1 , … , Uk=uk . The uncertainty remaining about the required response-equivalence class X is its conditional entropy under the regulator's knowledge state Mt . This conditional entropy is the Knowledge To Be Discovered, measured in bits.

The information-theoretic description has two related but distinct levels. Before the signal-values are known, conditional entropy gives an expected KTD trace. Because conditioning reduces entropy on average, expected residual KTD is non-increasing as additional information-bearing signals are introduced. After particular signal-values have been observed, the episode has a realized posterior trace. A particular observation need not reduce entropy: it may concentrate the posterior, leave its entropy unchanged, or make the posterior more diffuse.

This distinction follows the specific-information analysis of DeWeese and Meister [61]. For a particular observation, they identify information gained with the change in entropy between the distribution before and after that observation. They show that this definition has the required additive property across successive observations and that, when averaged over possible observations, it recovers mutual information. Consequently, realized information gained at one stage may be positive, zero, or negative, even though its expectation is non-negative.

The organizing distinction is therefore: the episode is the realized set-based regulatory event; the conditioning path is the information-theoretic path through that episode; expected KTD describes uncertainty before future signal-values are known; realized specific information describes the signed change in uncertainty caused by each actually observed signal-value; and the closure is the outcome-value produced after final response commitment.

Notation and Modeling Assumptions

In this subsection, D denotes the actual disturbance-value that enters the Table of Outcomes, while Y denotes the disturbance information available to the regulator. In the fully observed case, Y = D. In the partially observed case, Y may be a signal, observation, classification, or task condition that leaves several actual disturbance-values possible.

Unless otherwise stated, X denotes the required response-equivalence class at the regulatory granularity being analyzed. In the unique-target case, this is the variable previously denoted Xt*. Thus X is not merely the regulator's actual response-class variation; it is the target response-class variable whose value must be fixed for successful regulation.

If several response-classes are equally acceptable for the same disturbance, X is read as the class chosen by a minimizing admissible selector, X=χ*(D,Y), so that H(X∣Y) is the relational Knowledge To Be Discovered KTDrel defined with the Law of Requisite Variety. No further selection criterion has to be supplied: uncertainty among classes that are equally acceptable is removed by the minimization, not by a modelling choice. In the unique-target case the only admissible selector is the target-class map and X is Xt* as before.

The regulator's knowledge state is denoted by Mt. All probabilities, entropies, and mutual informations in this subsection are evaluated relative to the epistemic distribution induced by Mt. When that dependence matters, it is written explicitly as PMt, HMt, and IMt.

As in the set-based episode definition, the episode index t is fixed in this subsection. The response-equivalence map gt : 𝓡 → 𝓧t, the acceptable region ηt, and the knowledge state Mt are evaluated at the same episode index.

Role Symbol Meaning in this subsection
Actual disturbance D The disturbance-value that combines with the final concrete response in the Table of Outcomes.
Available disturbance information Y The observation, signal, classification, or task condition available to the regulator.
Required response-equivalence class X The response class still unresolved for the regulator after observing Y.
Knowledge state Mt The epistemic state relative to which probabilities, entropies, and mutual informations are evaluated.
Action unit qi The operation performed at stage i.
Selection signal Ui The information-bearing result of action unit qi.
Query family 𝒬t The queries the regulator can execute; each action unit qi instantiates one of them.
Regulatory relation ℜt The pairs (d, x) such that class x regulates disturbance d; the problem the queries must solve.
Decision procedure ϑt The tree of queries whose paths are the episode protocols, when the next query depends only on Y and the answers so far.
Number of stages K The number of action units in the episode, a stopping time of the signal sequence; its realized value is k.
Actually fixed response class X̂k The response class selected by the regulator after k stages.
Concrete final response Rfinal A concrete response emitted from the fixed response class.
Outcome Z = T(D,Rfinal) The outcome produced after final response commitment.

Staged Action Units and Realized Signals

Within an episode indexed by t , the external regulatory frame is fixed: the knowledge state is Mt , the acceptable region is ηt , and the realized disturbance-value is d . The internal stages of the episode are indexed by i = 1 , … , kt .

The episode-level action-unit protocol is denoted by Qt . Its individual action units are denoted by qt,i . Let the action-unit prefix through stage i be:

Q t,≤i := ( qt,1 , … , qt,i ) .

The observable result produced by action unit qt,i is denoted by Ut,i , representing the epistemic signal made available by that unit under knowledge state M t . Let the corresponding realized signal prefix be:

U t,≤i := ( Ut,1 , … , Ut,i ) .

The result Ut,i , not the mere occurrence of the action unit, is the information-bearing signal associated with the physical action unit qt,i . On a realized episode path, Qt,≤i = qt,≤i and Ut,≤i = ut,≤i . Thus the action unit specifies the discriminating operation, while the realized signal selects which candidate responses remain compatible with the realized episode path.

This ensures a conceptual separation between the action unit and the realized signal. The protocol Qt structures the episode. The action unit qt,i specifies a discriminating operation. The signal Ut,i is the realized information-bearing result. The candidate-set update is the support-level consequence. Entropy reduction is the information-theoretic measure of how much uncertainty about the required response-equivalence class has been removed.

The action-unit protocol Q t is treated as the realized physical enactment of the regulator's learned law of action under the knowledge state M t . The protocol may be fixed, dynamic, or adaptive. However, its adaptivity is governed by the law of action already contained in M t and by the episode history already represented in Y t and the prior realized signals U t , <i . Therefore, q t , i is not treated as an additional information-bearing signal about X . It is the physical action-unit token that determines which signal map is applied at stage i . The informational update is carried by the realized signal U t , i = u t , i , not by conditioning on the action-unit token itself.

We assume the selected action protocol Q ≤ k is fixed in advance or is a deterministic function of information already contained in Y under the frame Mt, That means: knowing the physical protocol Q t tells us nothing about X beyond what Y=y already tells us, thus conditioning on Q t adds no further information about X.

Therefore, after conditioning on the initial observation and realized signals, the protocol path adds no further information about the required response-equivalence class:

I ( X ; Q ≤k ∣ Y , U ≤k ) = 0 .

Equivalently:

H Mt ( X ∣ Y , U ≤k , Q ≤k ) = H Mt ( X ∣ Y , U ≤k ) .

This is the standard assumption of adaptive decision trees: the next query may depend on everything already answered, but the choice of query reveals nothing that the answers do not. When, in addition, each answer is a deterministic function of the disturbance, the protocol is a decision procedure over Qt in the sense of the previous section, and its expected length can be compared with the decision-tree complexities defined there. That comparison is made after Corollary 0′ below.

Expected Staged Reduction of Knowledge To Be Discovered

The initial Knowledge To Be Discovered is the regulator's remaining epistemic uncertainty, under Mt , about which required response-equivalence class must be fixed after the available disturbance information Y is known:

KTDt,0 := HMt ( X | Y ) .

Each information-bearing signal Ui conditions that uncertainty further. After i stages, the expected residual Knowledge To Be Discovered under the selected action-unit protocol is:

KTDi := H Mt ( X | Y , U1 , … , Ui ) .

Equivalently, the residual Knowledge To Be Discovered after i stages is the initial Knowledge To Be Discovered minus the information gained from the first i selection signals:

KTDi = H Mt ( X | Y ) − I Mt ( X ; U1 , … , Ui | Y ) .

A Knowledge Discovery Process is the sequence:

KTD0 → KTD1 → … → KTDk .

The expected reduction achieved by stage i is the difference between the residual Knowledge To Be Discovered before and after that stage:

KTDi−1 − KTDi = I Mt ( X ; Ui | Y , U<i ) .

The total expected selection achieved over all k stages is:

KTD0 − KTDk = I Mt ( X ; U1 , … , Uk | Y ) .

Shannon's chain rule for mutual information gives the operational decomposition of that total reduction:

I Mt X ; U1 , … , Uk | Y = ∑ i = 1 k I Mt X ; Ui | Y , U < i .

The chain-rule identity holds regardless of dependence among the signals U1, ..., Uk. It does not require the stage signals to be independent or conditionally independent. Dependence among stages is handled by conditioning each stage-wise term on the earlier signal-values.

Therefore the per-stage contributions must be conditional and incremental. If stages share information or impose overlapping constraints, summing their marginal reductions I(X;Ui|Y) can overcount. The chain-rule decomposition avoids that overcounting by crediting each stage only for the reduction of residual uncertainty left by previous stages.

Realized Conditioning Path Inside One Episode

The quantities KTDi defined above are expected conditional entropies under Mt. A realized episode, however, follows one concrete conditioning path: Y=y, U1=u1, …, Uk=uk , explicitly layered on top of the physical protocol Q t , ≤ k . After the signal-values are observed, the regulator's uncertainty is represented by conditioning on those realized values.

For the fixed episode t , let the regulator's posterior distribution over required response-equivalence classes immediately before stage i be:

p i − 1 (x) := P Mt ( X=x ∣ Y=y , U <i = u <i ) .

In particular, at i=1 the conditioning prefix is empty, so the entry state of the realized path is the posterior induced by the initial disturbance information alone:

p0 (x) := P Mt ( X=x ∣ Y=y ) , H ( p0 ) = H Mt ( X ∣ Y=y ) .

After observing the realized stage signal Ui = ui , the posterior becomes:

pi (x) := P Mt ( X=x ∣ Y=y , U ≤i = u ≤i ) .

The question is now what scalar quantity should be assigned to the change p i−1 → pi . Shannon mutual information determines the information gained on average over possible observations, but by itself does not uniquely determine the information carried by one particular observed signal-value. DeWeese and Meister show that infinitely many functionals of the prior and posterior can have the correct mutual-information average, so additional requirements are needed to identify the event-specific information measure [61].

Form of the realized stage-information functional

Let F [ p , q ] denote the information gained in a stage that changes the regulator's belief over X from a distribution p to a distribution q . The realized staged Knowledge Discovery Process requires four properties.

D1 — Belief-pair sufficiency. The information gained in a stage depends only on the belief state immediately before and immediately after that stage:

F = F [ p , q ] .

The prior and posterior jointly represent what the observation did to the regulator's uncertainty about the required response-equivalence class. No additional feature of the physical execution trace or elapsed time enters the event-specific information functional. F is defined on every ordered pair of distributions over the response-equivalence classes, not only on pairs that happen to be realized in a given episode.

D2 — Stage additivity. If two successive observations move the regulator through belief states p → q → r , then the information gained across the two stages together must equal the sum of the information gained at each stage:

F[p,r] = F[p,q] + F[q,r] .

Thus the information attributed to successive stages must sum to the information attributed to the corresponding combined observation. The requirement is imposed for every ordered triple of belief states, not only for triples arising along one particular conditioning path.

D3 — Averaging consistency. For the episode prefix Y=y , U<i = u<i , let the posterior that would result from a possible next signal-value Ui=v be:

p i (v) (x) := P Mt ( X=x ∣ Y=y , U<i = u<i , Ui = v ) .

The realized posterior is therefore pi = p i ( ui ) . Averaging the event-specific information over the possible next signal-values must recover the corresponding conditional mutual information:

∑ v P Mt ( Ui = v ∣ Y=y , U<i = u<i ) F [ p i−1 , p i (v) ] = I Mt ( X ; Ui ∣ Y=y , U<i = u<i ) .

This is required to hold at every stage, for every realized prefix, and for every candidate signal variable the regulator could observe at that stage under Mt — not only for the signal actually realized. Averaging consistency is a condition on the measure, not a property of one episode, so it must constrain the counterfactual stages the regulator could have run as well as the stage it did run.

D4 — Resolution normalization. When a stage identifies the required response-equivalence class exactly, the resulting belief state is scored identically whichever class it turned out to be. Writing δx for the point mass on class x , the requirement is that

F [ p , δx ] does not depend on x .

Once the required class has been pinned down, nothing remains to be discovered, and it should not matter which class it was. D4 is what distinguishes a measure of removed ignorance from a measure that also scores the identity of the answer.

These four requirements are precisely the properties needed for a realized staged information quantity: it must be determined by the before-and-after beliefs, it must compose across stages, its average must agree with Shannon mutual information, and it must not attach residual value to the identity of a fully determined class.

Lemma 0 — Uniqueness of the entropy-difference form. Under D1–D4, the event-specific stage functional is forced:

F [ p , q ] = H [p] − H [q] ,

where H[⋅] is the Shannon entropy functional.

Proof. Step 1: separation. By D1, every transition between two belief states is scored by the same functional. Applying D2 with q=r=p gives F[p,p] = 2 F[p,p] , hence F[p,p] = 0 . Applying D2 to the chain p→q→ r0 for an arbitrary fixed reference distribution r0 gives:

F [p,q] = F [p,r0] − F [q,r0] .

Setting G[p] := F[p,r0] , the functional separates into the difference of a single state functional:

F [p,q] = G [p] − G [q] .

The choice of r0 shifts G by a constant and leaves every difference unchanged.

Step 2: identification. By the quantifier attached to D3, the requirement applies in particular to a candidate stage whose observation determines X exactly. Every resulting posterior is then a point mass δx , and by D4 G [ δx ] = K takes one and the same value for every class x . The right-hand side of D3 is the mutual information of a fully resolving observation, so it collapses to the prior entropy:

I ( X;V ) = H[p] − H ( X∣V ) = H[p] − 0 .

The left-hand side of D3 is G[p] − K , so D3 reduces to:

G [p] − K = H [p] ,

which gives:

G [p] = H [p] + K .

This holds for every belief state p , because a fully resolving candidate observation exists at every belief state. The additive constant cancels in every difference, leaving:

F [p,q] = H [p] − H [q] .

This is the DeWeese–Meister result for their additive event-specific information I2 [61]. □

Remark — D4 is not decorative. Belief-pair sufficiency, stage additivity, and averaging consistency do not by themselves force the entropy difference. For any fixed function c : 𝒳 → ℝ on the response-equivalence classes, the functional

Fc [p,q] := H[p] − H[q] + ∑ x ( p(x) − q(x) ) c(x)

satisfies D1 by inspection and D2 because the added term telescopes. It also satisfies D3, because posteriors average back to the prior,

∑ v P Mt ( Ui=v ∣ prefix ) p i (v) = p i−1 ,

so the added term has zero average at every stage. Yet Fc differs from the entropy difference whenever c is non-constant, and it assigns c(x) to a fully resolved belief state, so that identifying one class would be scored differently from identifying another. D4 excludes exactly this family and nothing else that matters here.

Realized stage information

Lemma 0 fixes the realized entropy-based selection measure. For the actual transition p i−1 → pi , define:

σ i ent := F [ p i−1 , pi ] = H ( p i−1 ) − H ( pi ) .

Equivalently, σ i ent is the DeWeese–Meister event-specific information carried by the realized signal Ui = ui given the already realized episode prefix:

σ i ent = I2 ( X ; Ui = ui ∣ Y=y , U<i = u<i ) .

This quantity tracks the probability-weighted change in uncertainty, including changes caused only by redistribution of probability mass when the support itself does not change. Its entropy-difference form is therefore not a modelling convenience: under D1–D4, no alternative event-specific functional is available.

In a realized episode, σient may be positive, zero, or negative. It is positive when the observed signal reduces posterior entropy, negative when the observed signal makes the posterior more diffuse, and zero when entropy is unchanged. Non-negativity holds only after averaging over the possible observations.

For the fixed realized prefix Y=y , U<i = u<i , D3 evaluated at the entropy-difference form gives:

𝔼 [ σ i ent ∣ Y=y , U<i = u<i ] = I Mt ( X ; Ui ∣ Y=y , U<i = u<i ) ≥ 0 .

Fully averaging over the initial disturbance information and all earlier signal-values gives:

𝔼 [ σ i ent ] = I Mt ( X ; Ui ∣ Y , U<i ) ≥ 0 .

Where the number of stages k is itself path-dependent, this outer expectation is taken over the realizations in which stage i occurs. The per-prefix identity above is unaffected, since it conditions on a prefix that has already reached stage i .

Why specific surprise is not the stage-flow measure

DeWeese and Meister contrast the additive event-specific information above with the non-negative Kullback–Leibler divergence from the prior to the posterior, which they call specific surprise [61]. For the realized stage, this statistic is:

σ i KL := ∑ x pi (x) log2 ⁡ pi (x) p i−1 (x) .

The superscript distinguishes it from the additive stage measure; the symbol S is reserved throughout for closure counts.

This statistic is always non-negative, and its average over possible signal-values also recovers the corresponding mutual information. It therefore satisfies belief-pair sufficiency and averaging consistency. It does not, however, satisfy stage additivity. It measures how strongly the realized observation displaced the regulator's belief, not the signed amount by which the observation reduced uncertainty. Under the requirements stated by DeWeese and Meister, the entropy-difference form is the unique additive event-specific information measure, while the Kullback–Leibler form is the corresponding non-negative one; the two properties cannot be had together [61].

Corollary 0 — Negative realized stage information is unavoidable. Suppose two successive observations return the regulator to the belief state from which they started, which ordinary conditioning within Mt permits. D2 then requires their contributions to sum to zero, so the two contributions are equal and opposite. If either is nonzero, the other is negative. No additive event-specific measure can therefore be non-negative, and the sign behaviour of σient is a consequence of stage accounting rather than a defect of the definition [61].

Proposition 0 - Unit information capacity. Fix the episode t , the prefix Y = y , U < i = u < i , and let the stage- i signal take values in a finite alphabet 𝒰 i . Then

𝔼 [ σ i ent ∣ Y = y , U < i = u < i ] = I M t ( X ; U i ∣ Y = y , U < i = u < i ) ≤ H ( U i ∣ Y = y , U < i = u < i ) ≤ log 2 | 𝒰 i | .

In particular a binary unit conveys at most one bit in expectation over its possible signal-values. No such bound holds for the realized flow σ i ent : e.g. with X uniform on eight classes and U 1 the binary signal "is it class 1", the realized answer yes carries three bits, and Corollary 0 shows the realized flow may also be negative.

Proof. The equality is D3 evaluated at the entropy-difference form (Lemma 0); the first inequality is I ( X ; U ∣ ⋅ ) ≤ H ( U ∣ ⋅ ) ; the second is the uniform bound on the entropy of a finite alphabet. □

Standing conditions on the protocol:

  • (P1) Each stage signal takes values in an alphabet of at most m values; the default is m=2;
  • (P2) Query choice is a function of past answers. The choice of the next stage is a fixed function of the information already conditioned on, so Q ≤ i adds nothing about X beyond Y , U ≤ i .
  • (P3) The number of units in an episode is a stopping time of the signal sequence.

Corollary 0′ — Path capacity (query lower bound). Let the number of stages K be a stopping time of the signal sequence (the regulator decides to close on the basis of what it has observed) and let every stage alphabet have at most m values. Then

I M t ( X ; U ≤ K ∣ Y = y ) ≤ H ( U ≤ K ∣ Y = y ) ≤ 𝔼 [ K ∣ Y = y ] ⋅ log 2 m .

Proof. Because K is a stopping time, no realized signal string is a proper prefix of another, so the strings satisfy the m -ary Kraft inequality and the entropy of a prefix-free set of variable-length strings is at most its expected length times log 2 m [Cover & Thomas, Thm 5.3.1; McMillan]. □

The query ladder

Corollary 0′ is the information-theoretic lower bound of distributional query complexity: no protocol with answers of at most m values can, on average, convey more than log2m bits per action unit. Combined with the definitions of the previous section it places the expected length of the regulator's actual protocol at the top of a chain whose bottom is Knowledge To Be Discovered.

A closure must be safe for every disturbance the regulator still considers possible when it commits. Call the protocol relationally certainty-terminated under Y=y when, on every path it can take, the class it fixes is acceptable for every disturbance in the posterior support of that path:

X^K∈Xacc,t(d)for everyd∈suppMt(D∣Y=y,U≤k=u≤k).

In the unique-target case this is termination at certainty about the required class, the condition used by Lemma 1′ and by assumption B2 below.

Theorem Q — The query ladder. Let the protocol satisfy P1–P3 with m=2, let every action unit instantiate a query in Qt whose answer is a deterministic function of the disturbance, and let the protocol be relationally certainty-terminated under Y=y. Then:

KTDrel(y)≤DTμy(Rt∣y)≤DTμyQt(Rt∣y)≤𝔼[K∣Y=y].

Proof. The first two inequalities are Proposition Q1 and the remark that every procedure over Qt is also an unrestricted procedure. For the third, P2 and deterministic answers make the protocol a decision procedure over Qt, P3 makes its depth the number of action units, and relational certainty termination makes it certainty-terminated in the sense of the previous section. Its expected depth under μy is 𝔼[K∣Y=y], which is therefore one of the values over which DTμyQt(Rt∣y) is the minimum. □

The expectation here is taken under the regulator's knowledge state, as all quantities of this section are. How a counted average estimates it is the subject of the quantification section.

The three differences along the chain have distinct causes, and each has a name. The coding gap δ(y) and the query-family gap ψ(y) were defined with the Law of Requisite Variety. The protocol gap is

g(y):=𝔼[K∣Y=y]−DTμyQt(Rt∣y)≥0.

Proposition Q2 — Three-term search excess. Under the hypotheses of Theorem Q,

𝔼[K∣Y=y]−KTDrel(y)=δ(y)+ψ(y)+g(y),

with δ(y)∈[0,1), 0≤ψ(y)<∞, and g(y)≥0. The coding gap is zero for dyadic posteriors, the query-family gap is zero when Qt contains the questions of an optimal unrestricted procedure, and the protocol gap is zero when the regulator uses an optimal procedure over the questions it has.

Proof. The identity is the telescoping of the chain in Theorem Q. The range of δ(y) is Proposition Q1, and the signs of ψ(y) and g(y) are the second and third inequalities of the chain. ψ(y) is finite because a certainty-terminated procedure over Qt exists, namely the protocol itself. □

The search excess of Lemma 1′ below splits into a coding term and a protocol term. With a restricted query family, its protocol term is ψ+g: part of what looks like inefficient questioning may be forced by the questions available. The two parts call for different remedies, which is why they are kept apart here.

Without determinism of the answers, the protocol is no longer a decision procedure for the disturbance, and the middle of the chain is undefined. Its two ends remain comparable.

Proposition Q1′ — Relational path capacity. Let the protocol satisfy P1–P3, with answers that may depend on the disturbance through noise, and let it be relationally certainty-terminated under Y=y. Then

KTDrel(y)≤HMt(X^K∣Y=y)≤𝔼[K∣Y=y]⋅log2m.

Proof. The second inequality holds because X^K is a function of the realized signal string, whose entropy Corollary 0′ bounds. For the first, list the classes x1,x2,… in decreasing order of PMt(X^K=xj∣Y=y), and let χ(d,y) be the first class in that list that the protocol fixes with positive probability when D=d. Relational certainty termination makes every such class acceptable for d, so χ is an admissible selector. For every j, the event χ(D,y)∈{x1,…,xj} contains the event X^K∈{x1,…,xj}, so the distribution of χ(D,y) majorizes that of X^K, and entropy is Schur-concave. Hence KTDrel(y)≤HMt(χ(D,y)∣Y=y)≤HMt(X^K∣Y=y). □

Proposition Q1′ is the relational form of the bound HMt(X∣Y=y)≤𝔼[K∣Y=y]⋅log2m derived below for paths that terminate at certainty. It holds without a unique target and without deterministic answers, and it is the inequality on which the operational estimator ultimately rests.

Terminal realized Knowledge To Be Discovered

At the terminal point of the realized conditioning path, the Knowledge To Be Discovered remaining in episode t is the posterior entropy left after the actual signal-values have been observed:

KTD term real ( y , u ≤k ) := H Mt ( X ∣ Y=y , U1 = u1 , … , Uk = uk ) = H ( pk ) .

The realized reduction in Knowledge To Be Discovered along the complete path is:

ΔKTD path real ( y , u ≤k ) := H Mt ( X ∣ Y=y ) − KTD term real ( y , u ≤k ) .

Because every realized stage contribution is the difference of the same entropy functional, the stage contributions telescope:

∑ i=1 k σ i ent = H ( p0 ) − H ( pk ) = ΔKTD path real ( y , u ≤k ) .

Thus the realized net information gained over an episode depends only on the initial and terminal posterior entropies. The conditioning path may contain many stages, and individual stages may contribute positive or negative amounts, but their sum is exactly the net reduction in the realized Knowledge To Be Discovered. Segmenting one informational update into finer stages cannot change that total.

In this sense, the realized starting Knowledge To Be Discovered H Mt ( X ∣ Y=y ) is the uncertainty stock present at the beginning of the realized conditioning path. Each observed selection signal contributes a signed event-specific information flow, and the sum of those flows equals the reduction of that stock between the beginning and end of the episode. This is the realized counterpart of the expected episode-start quantity H Mt ( X ∣ Y ) , which averages it over the disturbance law; the qualifier realized is retained throughout to keep the two apart.

In the unique-target case, if the realized conditioning path makes the required response-equivalence class posteriorly certain, then:

KTD term real ( y , u ≤k ) = 0 .

In that special case, the realized conditioning path has removed all of the Knowledge To Be Discovered present after the initial disturbance information was observed:

ΔKTD path real ( y , u ≤k ) = H Mt ( X ∣ Y=y ) .

Combining the telescoped identity with Corollary 0': when the path terminates at certainty, the realized reduction equals H ( p 0 ) on every path, so its expectation over paths is H ( p 0 ) = I M t ( X ; U ≤ K ∣ Y = y ) , and therefore

H M t ( X ∣ Y = y ) ≤ 𝔼 [ K ∣ Y = y ] ⋅ log 2 m .

The realized count k on one path is not bounded below by H ( p 0 ) ; a lucky early termination can resolve three bits of starting uncertainty with one binary stage. What the starting uncertainty bounds is the expected length of the path.

This terminal quantity is still a posterior uncertainty measure, not by itself a success condition. Zero terminal realized Knowledge To Be Discovered means that the required response-equivalence class has become identifiable under the model. Successful regulation additionally requires that the class actually fixed by the regulator be the required class.

Fixed-model scope of the specific-information result

The DeWeese–Meister result is applied here only within one fixed epistemic model. Throughout this section the episode index t is fixed, so every posterior p0 , … , pk is obtained by conditioning the same knowledge state Mt . In particular:

pi (x) = p i−1 ( x ∣ Ui = ui ) ,

so the chaining condition required for stage additivity holds by construction. No stationarity assumption is required, and the stage signals need not be independent. They may be statistically dependent in arbitrary ways, provided that the successive posteriors remain conditionals of the same Mt .

The result does not apply across a change of epistemic model such as:

Mt → M t+ .

Such a transition replaces or revises the model under which the entropy is evaluated and therefore is not, in general, an ordinary conditioning step inside the original Mt . The additive specific-information identities established here apply only to fixed-model conditioning segments.

Lemma 0 therefore fixes the informational functional used by the realized Knowledge Discovery Process. It does not yet make that latent entropy reduction observable or countable: connecting the bit-valued epistemic quantity to operational execution units is a separate measurement problem.

Bridge to the Set-Based Candidate Trace

The set-based staged selection trace introduced above is naturally indexed by the realized disturbance-value. That trace is an ex post description of what happened in the episode. For a realized disturbance-value d , write the actual response-candidate trace as:

∅ ≠ C t,k R,act (d) ⊆ … ⊆ C t,1 R,act (d) ⊆ C t,0 R,act (d) ⊆ R .

This actual trace is disturbance-indexed. It records the nested response-values that remain compatible with the realized disturbance and the realized physical action-unit path. However, in a partially observed regulatory episode, the regulator does not generally condition on the realized disturbance-value itself. It conditions on the observation available at the start of the episode and on the realized action-unit and signal history. Therefore, the information-theoretic bridge must use an epistemic candidate trace.

The regulator's information state after stage i is ( Y=y , U ≤i = u ≤i ) . The realized action-unit path Q ≤i = q ≤i is protocol context: it identifies which discriminating operations were applied. It is not, under the default protocol assumption, an additional information-bearing signal about X .

For an information state ( Y=y , U ≤i = u ≤i ) , write the epistemic response-candidate set as:

C t,i R,epi ( y , q ≤i , u ≤i ) ⊆ R .

This set contains the response-values still live from the regulator's point of view after the observed information path. The actual trace answers the ex post question, “Which responses remained compatible with the realized disturbance?” The epistemic trace answers the regulatory question, “Which responses remain possible under the information currently available to the regulator?” The two traces coincide only in the fully observed case, where the initial observation fixes the disturbance-value.

Y = D   ⇒   C t,i R,epi ( d , q ≤i , u ≤i ) = C t,i R,act (d) .

Bridge Lemma: epistemic support and terminal real KTD

Let Mt be the knowledge state governing the episode. Let X denote the required response-equivalence class. Let gt : R → Xt map each response-value to its response-equivalence class under the regulatory frame at time t . The epistemic class-candidate set induced by the response-candidate set is:

C t,i X ( y , q ≤i , u ≤i ) := gt [ C t,i R,epi ( y , q ≤i , u ≤i ) ] .

The bridge condition is that this induced class-candidate set equals the probabilistic support of the required response-equivalence class under the same information state:

C t,i X ( y , q ≤i , u ≤i ) = supp Mt ( X ∣ Y=y , Q ≤i = q ≤i , U ≤i = u ≤i ) .

Define the terminal real Knowledge To Be Discovered after the realized protocol and signal history as:

KTD term real ( y , u ≤k ) := H Mt ( X ∣ Y=y , U ≤k = u ≤k ) .

Under the bridge condition, terminal real KTD is zero exactly when the terminal epistemic trace leaves only one required response-equivalence class possible:

KTD term real ( y , u ≤k ) = 0 ⇔ | C t,k X ( y , q ≤k , u ≤k ) | = 1 .

Equivalently, if the terminal epistemic response-candidate set may still contain several response-values, terminal real KTD is nevertheless zero whenever all those response-values belong to the same required response-equivalence class:

gt [ C t,k R,epi ( y , q ≤k , u ≤k ) ] = {xt} ⇒ KTD term real ( y , u ≤k ) = 0 .

Proof sketch. By the bridge condition, the class-candidate set induced by the epistemic response trace is exactly the conditional support of X under the realized information state. For a finite random variable with positive probabilities on its support, conditional entropy is zero exactly when the conditional support is a singleton. Therefore, terminal real KTD is zero exactly when the terminal epistemic trace has reduced the possible required response-equivalence classes to one. This does not require the regulator to have fixed a unique response-value. It requires only that all remaining response-values be equivalent for the regulatory problem.

If the selected action-unit sequence carries information about X beyond the realized signals, then it must remain inside the conditioning set as shown above. If the protocol is fixed in advance, or is conditionally determined by information already contained in Y and U ≤k , then the simpler expression omitting Q ≤k is equivalent.

Relational version of the Bridge Lemma

The Bridge Lemma characterizes zero terminal Knowledge To Be Discovered in the unique-target case: one required class remains. When several classes may be acceptable for the same disturbance, a singleton is more than closing requires. What closing safely requires is a class acceptable for every disturbance still possible after the path. Write

Xacc,t(y,u≤k):={x∈Xt:x⊆At(d)for everyd∈suppMt(D∣Y=y,U≤k=u≤k)},

which is the acceptable class-set Xacc,t(y) of the success subsection evaluated at the information state reached by the path. The terminal relational Knowledge To Be Discovered is the relational Knowledge To Be Discovered of that information state:

KTDrel,termreal(y,u≤k):=minχHMt(χ(D)∣Y=y,U≤k=u≤k),

with the minimum over admissible selectors on the posterior support of the path.

Bridge Lemma (relational). For every realized path,

KTDrel,termreal(y,u≤k)=0⇔Xacc,t(y,u≤k)≠∅.

A protocol is relationally certainty-terminated exactly when, on every path, the class it fixes lies in Xacc,t(y,u≤k). In the unique-target case, Xacc,t(y,u≤k) is non-empty exactly when the required class is constant on the posterior support, and under the bridge condition that is the singleton condition |Ct,kX|=1 of the Bridge Lemma.

Proof. The information state (Y=y,U≤k=u≤k) is an observation in its own right, and Proposition Q0 applied to it gives the equivalence. The unique-target statement is Proposition Q0 with singleton acceptable sets, combined with the Bridge Lemma. □

As in the unique-target case, the condition says only that a safe class exists after the path, not that the regulator fixed one. The success subsection below keeps the two apart.

Action Units, Signals, and Response-Class Commitment

An action unit qi is an operational intervention performed during an episode. The intervention may produce or expose a signal Ui , whose realized value ui updates the set or probability distribution of responses still considered possible. The action unit and the signal should therefore be kept conceptually distinct: the action unit is what the regulator does, whereas the signal is the evidence made available by that action.

qi ⇝ Ui ⇝ C i−1 R → CiR .

The first implication is operational rather than deterministic: an action unit specifies how evidence is sought, but the resulting signal-value may depend on the disturbance, the system, and the current knowledge state. The second implication represents the epistemic effect of the realized evidence on the live candidate-response set.

Let CiR (d) denote the live concrete candidate-response set after the i -th action unit in the realized set-based trace. Let μ be the count measure, or another admissible size measure, on candidate-response sets. Provided both sets have positive measure, define the concrete-response amount of selection contributed at stage i as:

σ i set,R := log2 ( μ ( C i−1 R (d) ) μ ( CiR (d) ) ) bits.

In the finite case with the count measure, this becomes:

σ i set,R = log2 ( | C i−1 R (d) | | CiR (d) | ) .

This measure records the logarithmic reduction in the variety of live concrete responses. Within an ordinary fixed-knowledge-state episode:

  • σ i set,R = 0 when no concrete candidate response is eliminated;
  • 0 < σ i set,R < 1 when the update eliminates some candidates but leaves more than half of the previous measure;
  • σ i set,R = 1 when the update halves the measure of the live candidate set; and
  • σ i set,R > 1 when the update leaves less than half of the previous measure.

The concrete-response measure does not yet establish how much uncertainty about the required response-equivalence class has been removed. Several concrete responses may belong to the same response-equivalence class and may therefore be interchangeable for the regulatory problem. Let gt : R → Xt map concrete responses to their response-equivalence classes. Using the response-candidate trace defined above, the corresponding class-projected candidate set is:

CiX := gt [ CiR ] .

For finite class-candidate sets, define the class-level set-based amount of selection as:

σ i set,X := log2 ( | C i−1 X | | CiX | ) .

The concrete-response and class-level amounts of selection need not coincide:

σ i set,R ≠ σ i set,X in general.

For example, an action unit may eliminate several concrete responses while leaving every previously possible response class represented. In that case, concrete-response variety has been reduced but no required response-equivalence class has yet been ruled out. Conversely, a small concrete-set reduction may eliminate an entire response class and therefore produce a larger proportional reduction at the class level. The two measures coincide only under additional structural conditions on the projection gt , such as when it is injective on the relevant live candidate responses.

Set reduction must also be distinguished from probability redistribution. Let pi denote the posterior probability distribution over the required response-equivalence classes after observing the information available through stage i . An action unit may affect the episode in four distinct ways:

Case Support effect Probability effect Interpretation
Pure support reduction CiX ⊂ C i−1 X Relative probability ratios among the surviving classes are unchanged. The signal rules out one or more candidate response classes.
Probability redistribution only CiX = C i−1 X pi ≠ p i−1 The same classes remain possible, but their relative plausibilities change.
Mixed update CiX ⊂ C i−1 X pi ≠ p i−1 on the surviving support. The signal both eliminates classes and redistributes probability mass among the survivors.
Epistemically null update CiX = C i−1 X pi = p i−1 The signal contributes no information about the required response class.

Here the realized entropy-based selection measure σ i ent is the entropy-based analogue of σ i set,X , and measures realized event-specific information, including redistribution of probability mass even when support does not change.

In a probability-redistribution-only update, no response class has been ruled out, and therefore: σ i set,X = 0 . . The realized entropy-based selection may nevertheless be positive, zero, or negative because the posterior distribution over the surviving classes has changed.

In a mixed update, σ i set,X > 0 because the class support has shrunk. The realized entropy-based selection may still be positive, zero, or negative, depending on how probability mass is redistributed among the surviving classes. Non-negativity applies to the expected information gain, not necessarily to every realized entropy change.

In an epistemically null update, both the posterior support and the posterior distribution remain unchanged:

CiX = C i−1 X , pi = p i−1 , σ i set,X = σ i ent = 0 .

Thus, an action unit may narrow an episode by eliminating concrete responses, by eliminating response-equivalence classes, by concentrating probability mass among classes that remain possible, by combining these effects, or by contributing no information about the required response class. The concrete set-based measure observes reduction in response variety. The class-level set-based measure observes elimination of response-equivalence classes. The entropy-based measure additionally observes probability redistribution. These three effects are related, but they are not identical.

The epistemically null update has a precise meaning for a query. It takes one of two forms. Either every disturbance still possible gives the same answer, so the query is determined by the prefix and its node has a single live child; or the answer does split the disturbances, but not in a way that changes the posterior over classes. In both forms the stage adds one to the path length and nothing to the information. The first form is the Kraft-slack case of Lemma 1″ below, which charges exactly these stages. The second can arise only when answers depend on the disturbance beyond its class, which lies outside the deterministic-tree setting of that lemma; in the relational accounting of Proposition Q2 it is part of the protocol gap, or of the query-family gap when the family offers no better question.

Under a fixed knowledge state Mt , ordinary conditioning on a signal-value that had positive probability cannot broaden the support of possible response classes:

CiX ⊆ C i−1 X .

If an action unit reveals that a response class previously assigned zero support must be reconsidered, the event is not ordinary staged conditioning under the unchanged knowledge state Mt . It is a model-revision, reopening, or rework event and should be represented by an updated knowledge state Mt+ or by the start of a new episode.

At some stage, the regulator commits to a response-equivalence class. Let Xt be the set of response-equivalence classes over which X ranges. Let Xk ^ ∈ Xt denote the response class actually fixed by the regulator after k stages of selection. This commitment may be represented by a decision rule:

Xk ^ := δk ( Y , U1 , … , Uk ) .

Response-class commitment is an act of the regulator and does not, by itself, prove that all uncertainty has been removed. A regulator may commit while several response classes remain possible. The special case of uniquely determined commitment occurs when the terminal class-candidate set is a singleton:

| CkX | = 1 ⇒ Xk ^ = the unique element of CkX .

The actually fixed class Xk ^ is distinct from X , which denotes the required response-class uncertainty being reduced by the staged-selection process. In a well-aligned successful case, the regulator fixes the class required by the regulatory problem. The notation keeps the required class and the actually fixed class separate because a regulator may become confident, commit, and still select the wrong class.

A realized set-based episode may therefore be represented as:

D = d → Y = y → q1 → U1 = u1 → ⋯ → qk → Uk = uk → Xk ^ → Rfinal → Z = T ( d , Rfinal ) .

This display describes the realized set-based episode. The corresponding information-theoretic conditioning path is:

Y = y , U1 = u1 , … , Uk = uk .

If the sequence of action units q1 , … , qk is fixed in advance, or is generated by a policy already included in the regulator's prior structure Mt , the signal sequence is sufficient to represent the staged conditioning path. If the choice of the next action unit is itself adaptive and may convey information about X , the selected action unit must also be included in the information sequence. The stage-level information contribution is then represented by:

I Mt ( X ; Qi , Ui ∣ Y , Q <i , U <i ) .

Success, Closure, and Policy-Level Success

At the response-class level, success requires that the actually fixed class lie inside the acceptable response-class set for the available disturbance information:

Xk ^ ∈ Xacc,t (Y) .

In the partially observed case, a response class is acceptable under Y = y only if every concrete response in that class is acceptable for every actual disturbance-value still possible under Y = y:

Xacc,t (y) := { x ∈ Xt : ∀ d ∈ supp (D|Y=y) , x ⊆ At (d) } .

This is a robust class-level acceptability condition. It is stronger than realized closure success, because it requires success for every disturbance still possible under the observation, not merely for the actual disturbance that occurs.

Once the response class is fixed, a concrete response is emitted from that class:

Rfinal ∈ Xk ^ .

The Table of Outcomes then produces the closure:

Z = T ( D , Rfinal ) .

Closure does not, by itself, imply that the terminal realized Knowledge To Be Discovered is zero. Closure means that the regulator has committed a final response and that the Table of Outcomes has produced an outcome-value. It does not necessarily mean that the realized conditioning path has eliminated all posterior uncertainty about the required response-equivalence class.

Z = T ( D , Rfinal ) ⇏ KTD term real ( y , u ≤k ) = 0 .

A closed episode may therefore still end with positive terminal realized Knowledge To Be Discovered:

KTD term real ( y , u ≤k ) > 0 .

This is not a contradiction. It means only that the episode has produced a closure while the model still represents residual uncertainty over the required response-equivalence class. Whether that residual uncertainty counts as remaining Knowledge To Be Discovered depends on the response-class granularity and on whether the model requires one specific target class to be fixed.

At the level of a realized episode, success is judged by the actual closure:

φ ( T ( d , rt ) ) ∈ ηt .

The closure is acceptable exactly when its induced essential-variable value lies in the acceptable region:

φ (Z) = φ ( T ( D , Rfinal ) ) ∈ ηt .

At the probabilistic policy level, a stronger almost-sure success condition is:

Pr ( φ ( T ( D , Rfinal ) ) ∈ ηt ) = 1 .

These are different levels of evaluation. The first judges one realized episode. The second judges the regulator's policy across the modeled distribution of episodes.

The levels match the settings of query complexity. Acceptable realized closure is a correct leaf on one path. Probability-one policy success is a procedure that is certainty-terminated on the posterior support, a selector of Rt on that support: the distributional setting. The universal rule-level condition of the set-based formulation is the same requirement over every modeled disturbance: the worst-case setting.

In the relational form, the analogue of the unique-target requirement below is that the class fixed by the regulator be acceptable for the actual disturbance, X^k∈Xacc,t(D); when several classes are acceptable, it need not equal any particular one of them.

In the unique-target case, successful selection requires that the residual uncertainty about the required response-equivalence class approach zero: HMt(X | Y, U1, ..., Uk) → 0. However, zero residual entropy means only that the regulator has enough information to identify the required class. It does not by itself mean the regulator actually selects it. Successful selection additionally requires that the class actually fixed by the regulator equals the required class: X̂k = X.

More generally, regulation does not require that only one concrete response remain possible. It requires that the concrete response finally emitted by the regulator belong to a response class that induces an outcome inside the acceptable region.

In the fully observed special case, where Y = D, the closure may also be written as Z = T(Y,Rfinal). In the general partially observed case, however, Y is only the regulator's observation or signal, while D is the actual disturbance-value that enters the Table of Outcomes.

Summary

The information-theoretic staged-selection formulation does not replace the Table of Outcomes, the outcome-to-essential-variable map, the acceptable region, the set-based episode, or the closure. It places a probability model over the staged regulatory event and reads the event as a query process: an episode is one path of a decision procedure for the regulatory relation Rt, each action unit instantiates a query from the family Qt, and the closure is the leaf.

Within a fixed knowledge state Mt, Knowledge To Be Discovered is an entropy stock and realized specific information is its signed stage-wise change. Realized contributions may be positive, zero, or negative; their sum along a path equals the initial realized KTD minus the terminal realized KTD, and their expectation gives the corresponding conditional mutual information. The relational Bridge Lemma links candidate narrowing to the existence of a class that is safe for every disturbance still possible.

For a relationally certainty-terminated binary protocol, the formulation yields a three-rung ladder:

KTDrel(y)≤DTμyQt(Rt∣y)≤𝔼[K∣Y=y].

The left rung is what must be known; the middle rung is what the regulator's questions allow; the right rung is how the regulator actually asks. The gap from Knowledge To Be Discovered to the expected protocol length splits into a coding gap below one bit, a query-family gap forced by the questions available, and a protocol gap that measures inefficient use of them. Without deterministic answers only the two ends of the ladder remain comparable, through Proposition Q1′.

In this limited formal sense, the Knowledge Discovery Process gives an “it from bit” reading of regulation: the realized response or response rule is the result of information-bearing selection from a larger space of possibilities[6], and the number of selections it took is bounded below by the information it had to obtain.

Learning (across episodes in a window)

The preceding sections described how a single episode discovers enough information to commit a response and produce a closure. The present section asks a different question: whether the discoveries made inside one episode are retained as stored regulatory structure for later comparable episodes.

Comparable episodes

A comparable task class defines the reference frame for comparison. A comparable episode is a realized closed episode whose Knowledge To Be Discovered can be evaluated inside that reference frame. Comparability is therefore not an intrinsic property of an episode alone. It is a relation between an episode and a declared task class.

Let τj denote a comparable task class. A comparable task class is not merely a collection of tasks that appear similar in ordinary language. It is a formal comparability frame for a recurring kind of regulatory problem. It fixes, or supplies explicit correspondences for, the disturbance-information variable Y , the required response-equivalence variable X , the response-equivalence abstraction g:R→X , and the modeling resolution at which Knowledge To Be Discovered is measured.

Thus, within a comparable task class, responses may differ at the concrete action level R , but they are compared only after being projected into the same response-equivalence class space X τ . Likewise, disturbances may differ at the raw observational level, but they must be represented through the same disturbance-information variable Y , or through an explicitly declared correspondence to it.

Comparability of task class therefore requires sameness of semantic structure: the same kind of disturbance information, the same kind of required response class, the same response-equivalence abstraction, and the same measurement resolution. It does not require the empirical probability distribution over X or Y to be identical across episodes. It means same regulatory measurement frame.

A closed episode εt is comparable under τj if the episode admits an interpretation in that task-class frame such that the following three quantities are well-defined over the same X and Y :

Htstart := H Mt (X|Y)
Htterm := H Mt ( X | Y , Ut )
Ht+1start := H Mt+1 (X|Y) .

Two closed episodes εt and εs are comparable episodes when there exists a task class τj such that both episodes are comparable under that same task class:

εt ∼ εs ⇔ ∃ τj such that εt ∈ Ετj and εs ∈ Ετj .

Here Ετj is the class of closed episodes interpretable under task class τj . The episodes may have different realized disturbances, different staged-selection paths, different committed responses, and different closure outcomes. They are comparable because the same response-selection uncertainty is being measured at the same abstraction level.

This means that all episodes assigned to τ j ask the same type of regulatory question: given a disturbance-information state, which response-equivalence class is required to keep the relevant outcome within the intended goal condition? In this sense, the same kind of regulatory problem is being solved.

For exact comparison, the following comparability conditions must hold:

  • the disturbance-information variable Y has the same semantic meaning across the compared episodes;
  • the required response-equivalence variable X has the same semantic meaning across the compared episodes;
  • the response-equivalence map g is fixed, or its changes are mediated by an explicit equivalence between response classes;
  • the partition of X is fixed;
  • the partition of Y is fixed;
  • the regulatory relation Rt is the same across the compared episodes; a change of the relation is goal revision, not learning;
  • the query family Qt is fixed, or its changes are declared, as an inclusion Qt⊆Qt+1 or an explicit correspondence between queries, in the same way as changes of the response-equivalence map;
  • the marginal entropy of the required response-equivalence variable is held fixed when changes in stored coupling and changes in KTD are read as exact duals.

Thus comparable episodes are not episodes that look identical operationally. They are episodes whose Knowledge To Be Discovered is measured against the same unresolved response-selection problem.

In plain terms: two episodes do not need the same disturbance, response, outcome, duration, or internal number of stages. They need to belong to the same regulatory problem: "Given this kind of disturbance information, which kind of response-equivalence class must be fixed?"

In the relational form, Htstart and Htterm are read as relational Knowledge To Be Discovered under Mt, each minimized over admissible selectors. The minimizing selector may differ between episodes, because it depends on the knowledge state. Comparisons of Knowledge To Be Discovered across episodes remain valid; the exact duality with stored coupling, ΔtI=−ΔtH(X∣Y), is read with the selector held fixed.

Learning as stored transformation

Within an episode, staged selection reduces Knowledge To Be Discovered by conditioning on the available disturbance information and the subsequent selection signals. Across episodes, learning occurs only if some of that reduction is incorporated into the regulator's stored law of action, so that the next comparable episode begins with stronger disturbance-response coupling and less residual Knowledge To Be Discovered.

Thus, in this section, learning is the across-episode transformation:

Mt → Mt+1

where Mt denotes the regulator's stored structural coupling at the start of episode t . In Ashby's terms, this is the regulator's learned law of action: the stored functional or probabilistic structure by which disturbances are mapped to responses. It is not the Table of Outcomes itself.

The fixed environmental relation remains:

T : D × R → Z

and the outcome-to-essential-variable map remains:

φ : Z → E

Learning does not alter T or φ in the fixed-table case. It alters the regulator's stored coupling Mt , and therefore alters the response law induced at the start of later episodes.

From staged discovery to stored learning

For episode t , let Mt be fixed at the start of the episode. It induces the regulator's epistemic probability law over the required response-equivalence class:

Pt (X|Y) := P (X|Y;Mt) .

Here X denotes the required response-equivalence class at the regulatory granularity being analyzed. In the unique-target case, this is the same variable previously denoted Xt* . Thus X is not merely actual response variation. It is the target response-class variable whose value must be fixed for successful regulation under the adopted criterion.

The initial Knowledge To Be Discovered at the start of episode t is:

Ht (X|Y) := HPt (X|Y) .

During the episode, the regulator observes an information-bearing sequence:

Ut := ( Ut,1 , … , Ut,kt ) .

This is the within-episode evidence stream generated by the staged selection process. It may include tests, observations, feedback signals, constraint checks, partial executions, or other action-unit results. The corresponding within-episode posterior Knowledge To Be Discovered is:

Ht (X|Y,Ut) .

The within-episode discovery is the reduction:

Ht (X|Y) − Ht (X|Y,Ut) .

This reduction is not yet learning. It is only discovery inside the current episode. Learning occurs only if the regulator stores and reuses the resulting information by updating its law of action for later comparable episodes.

After the episode closes, the regulator may update its stored coupling using the evidence stream, the class it actually fixed, the final concrete response, the produced closure, and the valuation of that closure. Let the closure-valuation signal be:

vt := 1 { φ (Zt) ∈ ηt } .

Then a general update form is:

Mt+1 := Update ( Mt , Yt , Ut , Xt,kt ^ , Rfinal,t , Zt , vt ) .

This expression does not assume a particular memory mechanism. The update may overwrite, patch, strengthen, weaken, version, or reorganize the stored coupling. The formal question is not how storage is implemented, but whether the next induced response law has less residual Knowledge To Be Discovered for the same task class.

Across-episode comparison of stored coupling

In the across-episode analysis, the task class must be comparable. The stored coupling may change, but the response classes whose uncertainty is being measured must not silently change meaning.

Thus, for each episode t , the stored coupling Mt induces:

Pt (X|Y) , Ht (X|Y) , It (X;Y) .

Here Ht (X|Y) is the residual lack of requisite knowledge at the start of episode t . It measures how much uncertainty remains, after the available disturbance information is known, about which required response-equivalence class must be fixed.

The mutual information: It (X;Y) is the stored requisite coupling at episode start: the amount of response-selection structure already aligned with the disturbance distinctions relevant to the task. It is an information-theoretic measure of the coupling induced by Mt .

The comparability of episodes requires that both episodes be evaluated over the same task-class variables, alphabets, and abstraction maps. This makes the quantities I(X;Y) and H(X|Y) semantically comparable across episodes. However, semantic comparability alone does not require the marginal response-class variety H(X) to remain constant, because two episodes can use the same response-class alphabet X τ while having different probability distributions over response classes. For example, both episodes may use the same response classes: X τ = { x 1 , x 2 , x 3 , x 4 } , but in one episode the required responses are evenly distributed, while in another one response class dominates. Same alphabet, different entropy.

For the narrower purpose of reading changes in stored requisite coupling ΔtI and changes in lack of requisite knowledge H (X|Y) as exact duals, impose the stronger constant marginal response-variety assumption:

Ht (X) = Ht+1 (X) .

The assumption means that the marginal variety of required response classes has not changed between the two episodes.

Under this assumption, the identity H(X) = I(X;Y) + H(X|Y) implies that any increase in stored requisite coupling is exactly matched by an equal decrease in lack of requisite knowledge.

Therefore, under the constant marginal response-variety assumption, an increase in stored requisite coupling is exactly equivalent to an equal decrease in residual lack of requisite knowledge:

ΔtI = − Δt H (X|Y) .

If the constant marginal response-variety assumption is relaxed, the two changes need not coincide. Part of the change in mutual information may then come from drift in the marginal entropy Ht(X) , not only from reduced conditional ignorance. In what follows, the constant marginal response-variety assumption is maintained.

Learning axiom

Learning Axiom (Structural Knowledge Accumulation). A regulator learns, in the structural-coupling sense, when the within-episode evidence stream is incorporated into an updated stored law of action such that the next comparable episode begins with less residual Knowledge To Be Discovered.

Formally, learning from episode t to episode t+1 requires:

Ht+1 (X|Y) < Ht (X|Y) .

Under the fixed marginal entropy assumption, this is equivalently:

It+1 (X;Y) > It (X;Y) .

In words: after the update, the regulator begins the next comparable episode with stronger stored disturbance-response coupling and therefore less residual uncertainty about which response-equivalence class must be fixed.

This axiom defines positive learning at the chosen regulatory granularity. If the update leaves Ht(X|Y) unchanged, then the episode has not improved stored requisite knowledge for that task class. If the update increases the conditional entropy, the regulator has degraded its stored coupling.

This learning criterion is still epistemic. It says that the regulator has reduced uncertainty about the required response-equivalence class. Regulatory success also requires correctness: the class actually fixed by the regulator must be acceptable for the disturbance information and must produce an acceptable closure through the Table of Outcomes.

Xk ^ ∈ Xacc,t (Y)

and, after concrete response commitment:

φ ( T ( D , Rfinal ) ) ∈ ηt .

Thus a regulator may learn in the epistemic sense and still fail if it stores or applies the wrong mapping. Conversely, a regulator may produce a successful closure once without learning, if the episode's discovery is not retained in Mt+1 .

In the query reading, the Learning Axiom describes the first of three channels through which a regulator can reduce what its episodes cost, called L2a, knowledge: the stored law of action sharpens the posteriors μy, and Knowledge To Be Discovered falls. The query family and the protocol need not change.

Falling Knowledge To Be Discovered lowers the floor of the ladder, but it does not by itself lower every rung above it. By Proposition Q1, averaged over observations, the unrestricted complexity satisfies

DTt+1−DTt<1−(KTDrel,t−KTDrel,t+1),

where DTt is the average over observations of DTμy(Rt∣y) under Mt. A fall of more than one bit therefore forces the expected number of queries down. A smaller fall need not: if, for example, a two-valued posterior support with probabilities (12,12) becomes a three-valued one with probabilities (0.8,0.1,0.1), each value requiring a different class, Knowledge To Be Discovered falls from 1 to about 0.92 bits while the unrestricted complexity rises from 1 to 1.2 queries.

Learning which distinctions to observe. Suppose the disturbance information is a vector of observation components, Yt = ( Yt,1 ,…, Yt,m ) , and for B⊆{1,…,m} write Yt,B for the corresponding sub-vector. In rough set theory a reduct is a minimal set of attributes that preserves what the full set determines. The entropy-based version [70] asks for a minimal B with

HMt ( Xt* ∣ Yt,B ) = HMt ( Xt* ∣ Yt ) ,

and an approximate reduct relaxes the equality to an allowance ε≥0 on the left-hand side. A reduct identifies the disturbance distinctions the regulator actually needs: the components outside B carry no response-relevant information beyond Yt,B under Mt . Preserving the conditional entropy also preserves the positive region of the decision-table reading, but not conversely, because the positive region is a support-level quantity. This points to a second kind of improvement in the stored law of action. The Learning Axiom concerns retained reduction of H(X∣Y) . Discovering a reduct leaves H(X∣Y) unchanged and is therefore not learning in the sense of that axiom; what it reduces is the observation needed to sustain the same coupling. The two can combine: an update Mt→Mt+1 may lower H(X∣Y) and, in doing so, change which subsets of components are reducts. Like Knowledge To Be Discovered itself, reducts are relative to the knowledge state.

Reducts have a query-level counterpart. A query reduct is a minimal sub-family Q′⊆Qt with DTμyQ′(Rt∣y)=DTμyQt(Rt∣y) for every y: the questions the regulator actually needs, given what it already observes. An entropy reduct removes standing observations that carry no response-relevant information; a query reduct removes questions that optimal procedures can do without. Neither changes Knowledge To Be Discovered.

Three adaptation channels

The ladder of Theorem Q has three rungs, and a regulator can improve at each of them separately. The feedback-loop reading in the accounting section labels the across-episode loop L2; it splits into three channels.

  • L2a, knowledge. The knowledge state changes, Mt→Mt+1, and Knowledge To Be Discovered falls. This is learning in the sense of the Learning Axiom.
  • L2b, questions. The query family grows, Qt⊆Qt+1, and the query-family gap falls, with Knowledge To Be Discovered unchanged. The regulator acquires a new sensor, test, or form of inquiry. This is structural adaptation of the regulator short of any change to the Table of Outcomes.
  • L2c, protocol. The decision procedure improves, with the knowledge state and the query family unchanged, and the protocol gap falls. The regulator asks the questions it already had in a better order, or stops asking those it does not need.

Proposition Q5 — Monotonicity of the channels. With the regulatory relation fixed:

  1. If Q⊆Q′, then DTμyQ′(Rt∣y)≤DTμyQ(Rt∣y) for every y, while KTDrel(y) and DTμy(Rt∣y) are unchanged. Hence ψ(y) does not increase.
  2. If the knowledge state and the query family are fixed, a change of protocol changes only 𝔼[K∣Y=y], and hence only g(y).
  3. A change of knowledge state can move DTμyQt(Rt∣y) in either direction for a given y; the Learning Axiom constrains only the average of Knowledge To Be Discovered, and through it, by the bound above, the average unrestricted complexity.

Proof. (1) Every procedure over Q is a procedure over Q′, and Knowledge To Be Discovered and the unrestricted complexity do not depend on the query family. (2) The first two rungs depend only on Mt, Rt, and Qt. (3) The example after the Learning Axiom. □

Example. The robot of the appendix chooses among four moves with posterior (12,14,18,18). Knowledge To Be Discovered and the unrestricted complexity are both 1.75. The engineer who always proposes a protocol of two questions has a protocol gap of 0.25 when any binary question is available; the remedy is L2c, a better order of questions. If the sensors can only answer “vertical or horizontal?” and then “which of the two?”, the same two-question protocol is optimal, and the same 0.25 is a query-family gap; the remedy is L2b, a new sensor. The observed count is identical in the two cases. What differs is the diagnosis, and it can only be made by declaring the query family.

Strong learning: posterior becomes prior

The learning axiom above is weak. It requires only that the stored coupling improve. A stronger assumption is obtained when the regulator stores and reuses, without loss, the uncertainty reduction achieved inside episode t .

Strong Learning Assumption (Posterior-Becomes-Prior Rule). For a stable task class, the posterior uncertainty achieved by the end of episode t becomes the prior stored uncertainty at the start of the next comparable episode:

Ht+1 (X|Y) := Ht (X|Y,Ut) .

In words, the discoveries made during episode t are not merely used once. They are incorporated into the stored structure that shapes the regulator's next prior response law.

Under the dual knowledge view, and under the invariant marginal entropy assumption, the same rule can be written as:

It+1 (X;Y) = Ht (X) − Ht (X|Y,Ut) .

This rule is not a consequence of conditioning alone. Conditioning happens inside the current episode. Learning requires retention: the posterior structure must be written into the regulator's stored law of action.

The Posterior-Becomes-Prior Rule therefore requires at least the following conditions:

  • the same task class or disturbance semantics across episodes;
  • the same response-equivalence map gt : 𝓡 → 𝓧t, or an explicit correspondence identifying the response classes across episodes;
  • successful retention of within-episode discoveries;
  • reuse of the retained structure in later episodes;
  • no intervening forgetting, context drift, or criterion change that invalidates the comparison.

Corollary (Complete Adaptation under Strong Learning). Under repeated successful strong learning, the stored requisite coupling approaches its task-class ceiling:

It (X;Y) → H (X)

and the residual lack of requisite knowledge approaches zero:

Ht (X|Y) → 0 .

In that limit, no further within-episode discovery is required to determine the response-equivalence class for the task class under study. H t ( X | Y ) = 0 This is complete adaptation at the chosen regulatory granularity.

In query terms, complete adaptation is the point at which episodes need no queries. By Proposition Q0 and the single-leaf remark of the Law of Requisite Variety section, KTDrel(y)=0 exactly when DTμyQ(Rt∣y)=0, for every query family. The approach to that point is less smooth than the approach of Knowledge To Be Discovered to zero. As long as some disturbance with positive posterior probability needs a different class from the others, a certainty-terminated procedure must ask at least one question, however small that probability is. Knowledge To Be Discovered can therefore tend to zero while the expected number of queries stays at one or more. Complete adaptation in query terms requires the support condition Xacc,t(y)≠∅, not only vanishing entropy.

Learning, goal revision, and behavioral revision

Learning must be distinguished from goal revision and behavioral revision. They modify different parts of the regulatory model.

Goal revision changes the acceptable essential-variable region:

ηt1 → ηt2

and therefore changes the acceptable outcome-set:

Oacc,t1 → Oacc,t2 .

Behavioral revision changes what the regulator does. At the rule level, this means a change in the response rule:

ρt1 → ρt2 .

Learning changes the stored law of action:

Mt → Mt+1 .

These changes may be related, but they are not identical. A criterion can change without learning. A behavior can change without learning if the regulator merely switches among already available rules. A regulator can learn without immediately producing a different outcome if the newly stored structure is not yet exercised in a later episode.

This distinction is important for the fixed-table case. Outcome-values are not deleted from T . They may lose or gain acceptability because ηt changes. But learning is not the change in acceptability itself. Learning is the change in stored coupling that improves later response selection under the task class being compared.

The adaptation channels add two further kinds of change to this list. Acquiring questions changes the query family, Qt→Qt+1; improving the protocol changes the decision procedure, ϑt→ϑt+1. Neither is learning in the sense of the Learning Axiom, because neither lowers Knowledge To Be Discovered. Neither is goal revision, because the regulatory relation is unchanged. A change of procedure is also distinct from behavioral revision: the response rule may stay the same while the questions asked before applying it change. All four kinds of change can lower the cost of later episodes, and the ladder of Theorem Q says which rung each one acts on.

5. Knowledge Discovery Accounting

This section keeps two ledgers. The operational ledger counts closures: the leaves the regulator's episodes reached, and whether what was committed there survived. The knowledge ledger records bits of Knowledge To Be Discovered at those closures. Queries sit between the two. They are counted where action units are observable, and their expected number is bounded below by the bits through the ladder of Theorem Q. Each knowledge row therefore also records the query rung of the ladder, and the across-episode loop splits into the three adaptation channels introduced with learning.

Operational ledger

Let the fixed regulatory frame be the Table of Outcomes:

T : D × R → Z

with outcome-to-essential-variable map

φ : Z → E

At time t , the acceptable essential-variable region is

η t ⊆ E

This induces the acceptable outcome-set:

O acc, t := φ − 1 [ η t ] = { z ∈ Z : φ ( z ) ∈ η t }

Closure events

A closure event at time u is an act in which the regulator selects a response ru ∈ R against an observed disturbance du ∈ D , producing the realized outcome-value:

z u := T ( du , ru ) .

When the closure event is generated by the response rule prevailing at time u, we have ru=ρu(du). The ledger can also be read more generally as recording realized response selections, even when no total response rule is reconstructed.

The closure event is recorded as:

c u = ( u , du , ru , zu ) .

It is an acceptable closure when produced iff:

zu ∈ O acc, u

Equivalently:

φ ( zu ) ∈ η u .

Thus an acceptable closure is not merely an outcome-value in the table. It is a realized action whose produced outcome was acceptable under the criterion prevailing at the time of production.

Window

Let the observation window be:

I = ( ta , tb ] .

The window contains the closure events whose event times fall inside the interval. Using a half-open interval avoids double-counting events at boundaries when consecutive windows are used.

Define the accepted closure history inside the window:

C I + := ( cu : ta < u ≤ tb , zu ∈ O acc, u ) .

This is an event history, not a set of outcome-values. Therefore it preserves multiplicity. If two different closure events produce the same outcome-value,

zu = zv , u ≠ v ,

they still count as two closure-fixing acts.

Gross counted closures

The gross counted closures in the window are:

S gross ( I ) := | C I + | .

The gross closure count Sgross tallies the acts cl ( εt (d) ) , not the values zt .

This counts all successful-at-production closure-fixing acts in the window. It does not count failed attempts. A failed attempt still consumes capacity: in the execution channel of section 6 it falls into the non-closure count, since Q(I)= N(I)− Sgross(I) , and it is a 0 of the program of “The record as a two-instruction program”. If an action produced an outcome that was not acceptable under the criterion prevailing at the time of production, zu ∉ O acc, u , then it does not enter C I + , because the ledger is tracking cases of successful regulation that may later cease to count under revised success criteria.

Historically realized outcome-set

The historically realized outcome-set induced by the accepted closure history is only the support of the trace:

O I r , + := { zu : cu ∈ C I + } ⊆ Z .

Because this is a set, it collapses duplicates. Thus:

| O I r , + | ≤ S gross ( I ) .

The inequality may be strict when multiple acceptable closure events produce the same outcome-value. Therefore O I r , + answers: which acceptable outcome-values appeared? By contrast, S gross ( I ) answers: how many acceptable closure-fixing acts were expended?

Continuously surviving counted closures through the window

The operational section evaluates originally successful closures over the criterion history of the window, not merely against the final criterion. A closure event cu ∈ C I + was successful when produced. It counts as a surviving closure at the end of the window only if its produced outcome-value was never invalidated by any acceptability criterion that applied after its production time.

Thus a closure event cu survives the window iff:

∀ t ∈ [ u , tb ] , zu ∈ O acc, t .

Define the surviving counted closures in the window as:

S surv ( I ) := # { cu ∈ C I + : ∀ t ∈ [ u , tb ] , zu ∈ O acc, t } .

Equivalently:

S surv ( I ) = ∑ cu ∈ C I + 1 { ∀ t ∈ [ u , tb ] , zu ∈ O acc, t } .

This is a survival-through-the-window count. It asks how many closures were successful when produced and were never invalidated by any acceptability criterion applied from their production time through the end of the window. It is therefore a continuous-survival count, not merely a final-criterion count.

Invalidation load

Define history invalidation load in the window as the number of originally successful closures that were later invalidated by at least one acceptability criterion before the end of the window:

W ( I ) := # { cu ∈ C I + : ∃ t ∈ [ u , tb ] , zu ∉ O acc, t } .

Equivalently:

W ( I ) = ∑ cu ∈ C I + 1 { ∃ t ∈ [ u , tb ] , zu ∉ O acc, t } .

Thus W ( I ) counts closure-fixing acts that were successful when produced but failed to survive the subsequent criterion history of the window. It does not count the corrective work that replaces them.

Therefore:

S gross ( I ) = S surv ( I ) + W ( I ) .

Hence:

S surv ( I ) = S gross ( I ) − W ( I ) .

Interpretation

The ledger separates four levels:

  • possible outcome-values in Z ;
  • the time-indexed criterion of acceptability O acc, t ;
  • realized closure events in the temporal trace;
  • closure events that remained continuously accepted through the rest of the window.

Ashby's table T remains fixed. Outcome-values are not deleted from T . What changes over time is the criterion by which fixed outcome-values are judged acceptable:

O acc, t := φ − 1 [ η t ] .

This distinction matters. A criterion revision is a change in the acceptable region, and therefore in the acceptable outcome-set:

O acc, t1 ≠ O acc, t2 .

A behavioral revision, by contrast, is a change in what the regulator actually does. At the rule level, this means a change from one response rule to another:

ρ t1 ≠ ρ t2

and, at the induced-outcome level, it means a change in the rule-induced outcome map or in its image:

y t1 [ D ] ≠ y t2 [ D ] .

The operational ledger in this section tracks criterion invalidation of already successful closures. It does not, by itself, prove that the regulator has changed its behavior. A closure can therefore be successful when produced and later cease to count because the goal criterion changes:

zu ∈ O acc, u but zu ∉ O acc, tb .

That is the operational meaning of W ( I ) in this ledger. More generally, W ( I ) counts closures that were successful when produced but were invalidated at least once before the end of the window, even if they later became acceptable again.

W ( I ) measures invalidated successful closure, not completed corrective rework. Corrective rework is observed only when a later closure event, response selection, or response-rule revision replaces or repairs the invalidated closure.

Corrective rework would require an additional behavioral event: the regulator must select a new response, produce a new outcome-value, or revise the response rule so that the revised behavior again satisfies the current acceptable outcome-set. That would be tracked by comparing the produced closure history or the response rules before and after the criterion revision, not merely by comparing O acc, u with O acc, tb .

Therefore, this ledger does not model deletion of outcome-values from T , and it does not yet model all behavioral replacement work. It models the narrower case in which a previously successful closure is invalidated by a later criterion of success. Behavioral revision is a separate question: it begins only when the regulator changes what it produces in response to disturbances.

Operational ledger for goal-model revision

The ledger above tracks the case in which a single time-indexed acceptable set O acc,t serves both as the criterion under which closures are produced and as the criterion against which they are later evaluated. A closure can be invalidated only by external goal revision, i.e. by a change in ηt itself.

A parallel case arises when the regulator does not know the acceptable region perfectly. The regulator acts under a believed acceptable region, while survival is judged against an authoritative acceptable region. In that case, invalidation is driven not by external goal revision but by revision of the regulator's model of the goal.

In this case, there are two acceptability structures:

  • the authoritative acceptable outcome-set, which remains fixed during the window;
  • the regulator's believed acceptable outcome-set, which may change as knowledge is discovered.

Authoritative and believed acceptability

Let the authoritative acceptable essential-variable region be:

η* ⊆ E .

The authoritative acceptable outcome-set is its pullback along φ :

O* := φ −1 [ η* ] = { z ∈ Z : φ (z) ∈ η* } .

The authoritative set O* does not change during the knowledge-discovery process. What changes is the regulator's believed model of the acceptable region.

At time t , let the regulator's believed acceptable essential-variable region be:

η ^ t ⊆ E

This induces the regulator's believed acceptable outcome-set:

O ^ t := φ −1 [ η ^ t ] ⊆ Z .

Thus, in goal-model revision, the operative distinction is not:

O acc, t1 ≠ O acc, t2

but rather:

O ^ t1 ≠ O ^ t2 while O* remains fixed.

This is why the process is called goal-model revision, not external goal revision. The goal did not change. The regulator's model of the goal changed.

The believed acceptable set may be incorrect: in general,

O ^ t ≠ O* .

The believed set may include outcome-values that the authoritative set excludes, exclude outcome-values that the authoritative set includes, or both.

Provisional closure events

A closure event at time u is still recorded as:

cu = ( u , du , ru , zu )

But in a goal-model-revision process, the closure is not yet judged by the authoritative set. It is provisionally accepted when produced iff the produced outcome-value belongs to the regulator's believed acceptable set at that time:

zu ∈ O ^ u .

Equivalently:

φ ( zu ) ∈ η ^ u .

A provisionally accepted closure is not necessarily an authoritatively acceptable closure. It is a closure that the regulator's model of the goal admits at the time of production.

Thus provisional acceptability is model-relative. It records what the regulator counted as successful under its current believed goal model.

Provisional closure history

Let the observation window be:

I = ( ta , tb ] .

The provisional closure history in the window is:

C ^ I + := ( cu : ta < u ≤ tb , zu ∈ O ^ u ) .

This is an event history, not a set of outcome-values. It preserves multiplicity. If the regulator produces several provisional closures for the same target position, all of those closure-fixing acts remain visible in the ledger.

Gross provisional closures

The gross provisional closures in the window are:

S gross ^ (I) := | C ^ I + | .

This counts all closure-fixing acts that were provisionally accepted under the regulator's believed model at the time of production.It does not yet say whether those closures were authoritative successes.

Net surviving closures

A provisionally accepted closure survives authoritative checking iff its produced outcome-value belongs to the fixed authoritative acceptable outcome-set:

zu ∈ O* .

Equivalently:

φ ( zu ) ∈ η* .

Define the authoritative net surviving closures in the window as:

S net (I) := # { cu ∈ C ^ I + : zu ∈ O* } .

This counts closure-fixing acts that were provisionally accepted when produced and were authoritatively acceptable.

Unlike the external goal-revision ledger, this survival test does not require a time-indexed criterion history after production. The authoritative acceptable set is fixed. The change is epistemic: the regulator later discovers whether a provisional closure was inside or outside the authoritative set all along.

Goal-model invalidation load

A provisionally accepted closure is invalidated by goal-model revision iff it was accepted under the believed model when produced but is outside the fixed authoritative acceptable outcome-set:

zu ∈ O ^ u but zu ∉ O* .

Define the goal-model invalidation load in the window as:

W model (I) := # { cu ∈ C ^ I + : zu ∉ O* } .

This counts closure-fixing acts that were successful relative to the regulator's believed model at production time, but failed the authoritative check. Wmodel (I) measures invalidated provisional closures, not completed corrective rework. Corrective rework is observed only when a later closure event or response-rule revision replaces or repairs the invalidated closure.

Ledger identity

Every provisional closure either survives authoritative checking or is invalidated by it. Therefore:

S gross ^ (I) = S net (I) + W model (I) .

Hence:

S net ( I ) = S gross ^ ( I ) − W model ( I ) .

This identity is not an entropy identity. It is an operational accounting identity: provisional closure work equals authoritative surviving closure work plus model-invalidation load.

Model-revision events

If the model updates themselves need to be represented explicitly, introduce a model-revision event:

mv = ( v , Mv− , Mv , O ^ v− , O ^ v , qv ) .

Here Mv− is the model before revision, Mv is the model after revision, and qv is the checking evidence, observation, comparison, or authoritative feedback that triggered the revision.

The model-revision event induces believed loss and gain sets:

L ^ v := O ^ v− ∖ O ^ v .
G ^ v := O ^ v ∖ O ^ v− .

A prior provisional closure cu is invalidated by the model-revision event mv if the closure occurred before the model revision and its outcome-value is among the values removed from the believed acceptable set:

u < v and zu ∈ L ^ v .

In the special case where the revision is caused by authoritative checking, the removed believed values are precisely those discovered not to belong to the authoritative acceptable set:

zu ∈ L ^ v ⇒ zu ∉ O* .

This makes the knowledge-discovery structure explicit: the regulator acts under a believed goal model, receives evidence, revises the model, and thereby invalidates some earlier provisional closures.

Corrective replacement

Goal-model invalidation does not, by itself, prove that corrective replacement has occurred. Corrective replacement requires a later closure event, response selection, or response-rule revision that repairs the invalidated closure.

The repair relation is additional structure. The invalidation ledger can count model-invalidation load without it. The repair relation is needed only when the modeler wants to identify which later closure repaired which earlier provisional closure.

Relation to the criterion-revision ledger

The criterion-revision ledger and the goal-model-revision ledger are two instances of the same accounting structure, separated by which acceptable set plays the role of production criterion and which plays the role of evaluation criterion.

In the criterion-revision ledger, both roles are played by the same time-indexed acceptable set O acc,t . A closure is acceptable when produced iff zu ∈ O acc,u , and it survives iff it remains in O acc,t for every t ∈ [ u , tb ] . Invalidation is then driven by external goal revision.

In the goal-model-revision ledger, the production criterion is the believed acceptable set O ^ u , while the evaluation criterion is the authoritative acceptable set O* . Invalidation is then driven by revision of the regulator's model of the goal, not by any change in the authoritative goal itself.

The two ledgers coincide in the degenerate case in which the regulator's believed model agrees at all times with the authoritative criterion, that is, O ^ t = O* for all t in the window, and the authoritative criterion is itself constant. In that degenerate case, Wmodel (I) = 0 and Snet (I) = S gross ^ (I) .

In general, the two ledgers answer different questions. W ( I ) measures invalidation by criterion change. Wmodel ( I ) measures invalidation by model mismatch under a fixed authoritative criterion. Neither measures corrective rework: corrective rework is a separate behavioral event tracked by changes in the response rule or in subsequent closure events.

The example in the next section instantiates the goal-model-revision ledger for a typing task in which the authoritative goal is fixed throughout, and the invalidation load Wmodel ( I ) records the discovery process by which the typist's believed model O ^ t is brought into agreement with the authoritative criterion O* .

Interpretation

This ledger separates three things that should not be collapsed:

  • provisional closure: the regulator produced an outcome accepted by its believed goal model;
  • model invalidation: later checking showed that some provisional closures were outside the authoritative acceptable set;
  • corrective replacement: later behavior repaired the invalidated closure.

Thus the goal-model-revision ledger does not say that the authoritative goal changed. It says that the regulator discovered that its believed model of the goal was wrong or incomplete. The authoritative acceptable set remained fixed, while the believed acceptable set was revised.

That is the operational form of a human knowledge-discovery process: provisional closure under an imperfect model, discovery of mismatch against the authoritative goal, invalidation of some earlier closures, and possible corrective replacement.

In query terms, the believed acceptable set answers a question the regulator puts to itself before it commits: is this outcome acceptable? Where O^u and O* disagree, that answer is wrong. A provisional closure that is later invalidated is a leaf whose output was accepted on a wrong answer and later contradicted by an authoritative one, so Wmodel(I) counts the wrong acceptability answers that were acted on.

The theory of searching with errors, which grew out of the Rényi–Ulam game, studies how many additional queries are needed to reach a correct answer when some answers may be false. Its lower bounds would bound the query cost that goal-model error imposes, where the present ledger only records the invalidations it produces. That link is left as a pointer; nothing below depends on it[73].

Operational counting of action units as Effective non-net-progress load

A counted atomic unit is either a non-closure action unit q t , i inside episode t, or a closure unit κ t of episode t. The episode protocol Q t := ( q t , 1 , … , q t , k t ) contains only non-closure action units. The closure unit is the terminal commitment event and is counted separately.

For a window of closed episodes I , the gross action-unit count is:

Q (I) := ∑ t∈I | Qt |

Observable invalidation load enters through the operational ledger. Some gross closure events are counted when produced but later fail to survive the window's evaluation criterion. In the external criterion-revision ledger, this invalidation load is W(I) . In the goal-model-revision ledger, the corresponding invalidation load is Wmodel (I) .

Definition (Effective non-net-progress load). If the measurement window records observed invalidation load or lost non-closure discrimination as Wobs(I) , then the effective non-net-progress load in window I is:

Qeff (I) := Q (I) + Wobs (I) .

This is an operational ledger quantity. It counts effective discriminating work at the observed action-unit level. It is the physical non-closure count Q(I) augmented by a one-unit non-progress debit for each invalidated gross closure. By construction Qeff(I) ≥ Q(I) , with equality if and only if Wobs(I) =0 ; whenever invalidation occurs, Qeff(I) strictly exceeds the number of non-closure units actually present on the channel.

Combining the definition with Snet = Sgross − Wobs :

Qeff(I) + Snet(I) = ( Q+Wobs ) + ( Sgross−Wobs ) = Q+Sgross

In the physical partition, each invalidated closure sits on the closure side, inside Sgross(I) . In the effective partition, that same closure has been removed from the surviving-closure side — it is not in Snet(I) — and re-booked as a non-progress debit inside Qeff(I) . No new units are introduced; the invalidated closures are simply re-attributed from “closure” to “load.” Equivalently, relative to the physical channel each invalidated closure is counted twice across the two partitions: once as the gross-closure unit it physically is, and once as the discrimination-equivalent debit it is charged to. This double appearance across partitions — not within either one — is precisely why Qeff is a defined effective load and not a action unit count.

In the query reading, every non-closure action unit instantiates a query from the regulator's family, with action units that expose nothing counted as constant queries. The gross action-unit count Q(I) is therefore the number of queries asked in the window, and |Qt| is the realized path length of episode t. The queries of an episode whose closure is later invalidated stay in Q(I); what the effective load adds is one query-equivalent for the closure itself, which reached a leaf but bought no surviving result. The debit of one per invalidated closure is a bookkeeping convention, not a measurement of how many questions the invalidation wasted. Read this way, Qeff(I) is the query load charged to the surviving closures of the window. Equivalently, Qeff(I) is the number of non-output instructions in the effective program of section 6.6.

Knowledge ledger

The operational ledger records realized closure events. The knowledge ledger attaches a bit-valued epistemic accounting row to each closure event. It answers a different question: after a closure and its associated update, how much Knowledge To Be Discovered remains for the next comparable episode?

The operational ledger counts closure-fixing acts. The knowledge ledger counts changes in residual Knowledge To Be Discovered. Therefore the two ledgers are linked by the same closure history, but they do not measure the same quantity.

Closure-indexed knowledge ledger

Let the operational closure event at episode index t be:

ct = ( t , dt , rt , zt ) .

Let τt denote the comparable task class under which the episode is evaluated. The task class fixes the meaning of the disturbance information Y , the required response-equivalence variable X , and the response-equivalence map used for comparison across episodes.

The closure-indexed knowledge ledger row is:

kt:=(ct,τt,Mt,Mt+1,Qt,Qt+1,Htstart,Htstart,real,Htterm,Ht+1start,DTtstart,DTtstart,real,|Qt|,DTt+1start,Btdisc,GtK,LtK).

This row records the epistemic state before the episode, the terminal posterior uncertainty reached inside the episode, and the stored uncertainty with which the next comparable episode begins. It also records the query rung of the ladder at both ends of the episode and the realized number of action units between them. The query family is part of the row because the query rung depends on it, and because a change of query family between comparable episodes must be declared. The entropy fields are read relationally: each is Knowledge To Be Discovered minimized over admissible selectors, and it is the conditional entropy of the required class in the unique-target case.

KTD states at closure boundaries

The expected starting Knowledge To Be Discovered for an episode t governed by knowledge state M t , averaged ex ante over the possible observation states Y = y , is:

Htstart := H Mt ( X | Y ) .

This is the residual lack of requisite knowledge induced by the stored law of action Mt before the episode's internal staged selection has occurred.

The realized counterpart is the realized row-level Knowledge To Be Discovered HMt (X∣Y=yt) , evaluated at the disturbance information actually observed for closure t,

Htstart,real := HMt (X∣Y=yt) .

Here X is the required response-equivalence class and Y=yt is the realized disturbance information available to the regulator at the start of episode t.

The two quantities are linked. For a fixed knowledge state Mt, the expected conditional entropy is the Y-average of the realized posterior entropies:

HMt (X∣Y) = ∑y PMt (Y=y) HMt (X∣Y=y) .

Let the episode's information-bearing evidence stream be:

Ut := ( Ut,1 , … , Ut,kt ) .

The expected terminal Knowledge To Be Discovered inside the episode is:

Htterm := H Mt ( X | Y , Ut ) .

It measures the ensemble posterior uncertainty. It does not yet say how much of that discovery has been retained.

The realized posterior uncertainty after the episode's actual evidence stream has been observed is:

Htterm,real := H Mt ( X ∣ Y = y t ​ , U t ​ = u t ​ ) .

This is the actual uncertainty realized after the episode. This quantity is evaluated under the knowledge state that governed the episode.

The stored Knowledge To Be Discovered at the start of the next comparable episode is:

Ht+1start := H Mt+1 ( X | Y ) .

This is the key knowledge-ledger balance after closure and update. It measures how much Knowledge To Be Discovered remains in the stored law of action for the next comparable episode.

Query states at closure boundaries

The query rung is recorded in the same two forms as Knowledge To Be Discovered. The expected starting query complexity is the least expected number of queries over Qt that the knowledge state Mt allows, averaged over observations:

DTtstart:=∑yPMt(Y=y)DTμyQt(Rt∣y).

Its realized counterpart is evaluated at the observation actually made at the start of episode t:

DTtstart,real:=DTμytQt(Rt∣yt).

The next comparable episode begins with DTt+1start, the same quantity under Mt+1 and Qt+1. Like Knowledge To Be Discovered, these are latent: they are fixed by the knowledge state and the query family, not observed. The realized path length |Qt| is the only field of the row that can be counted directly where action units are observable.

Under the hypotheses of Theorem Q, every row satisfies

Htstart,real≤DTtstart,real≤𝔼[|Qt|∣Y=yt],

with the expectation taken under Mt. The realized count |Qt| itself may fall below DTtstart,real on a lucky path; the ladder bounds its expectation, not each realization.

Within-episode discovery and retained learning

The expected within-episode discovery achieved by the staged selection process under M t is:

B t disc := H t start - H t term = I M t ( X ; U t | Y ) ≥ 0 .

This is the expected reduction in Knowledge To Be Discovered produced by the episode's information-bearing evidence stream, averaged over the possible realized observation and evidence states under M t . Equivalently, it is the conditional mutual information between the required response-equivalence class X and the episode evidence U t , given the starting disturbance information Y . Therefore the expected within-episode discovery is non-negative.

The corresponding quantity for the actual realized episode is:

B t disc,real := H t start,real - H t term,real .

This is the realized reduction in Knowledge To Be Discovered along the particular conditioning path actually traversed by episode t . Using the stage-wise event-specific information quantities defined for the realized conditioning path, it is equivalently:

B t disc,real = ∑ i = 1 k t σ t , i ent .

Unlike the expected quantity B t disc , the realized quantity B t disc,real need not be non-negative. A particular realized evidence path may increase uncertainty even though evidence reduces uncertainty on average. The non-negativity applies to the expectation, not to every realized episode.

Both quantities describe discovery inside the episode. Neither, by itself, establishes that the discovered information has been retained in the regulator's stored law of action. Stored learning is measured instead by comparing the expected starting Knowledge To Be Discovered of the current episode with that of the next comparable episode.

The retained KTD reduction is:

G t K := max ( H t start - H t + 1 start , 0 ) .

This is the retained knowledge gain recorded by the ledger. It counts the number of bits by which the next comparable episode begins with less expected Knowledge To Be Discovered. The comparison uses the expected starting quantities rather than their realized counterparts so that a change in stored knowledge is not confounded with differences between the particular disturbance states encountered in successive episodes.

The KTD increase is:

L t K := max ( H t + 1 start - H t start , 0 ) .

This is the stored knowledge loss recorded by the ledger. It counts the number of bits by which the next comparable episode begins with more expected Knowledge To Be Discovered. That may occur through forgetting, model drift, criterion misalignment, incorrect generalization, or degradation of stored coupling.

Knowledge ledger stock-flow identity

The knowledge ledger has the following exact stock-flow identity:

Ht+1start = Htstart - GtK + LtK .

This identity says that the next episode's initial Knowledge To Be Discovered equals the current episode's initial Knowledge To Be Discovered, minus retained knowledge gain, plus knowledge loss.

Equivalently, in stored-coupling form under the comparability assumption:

It+1start = Itstart + GtK - LtK .

The KTD form and the stored-coupling form are dual readings of the same update when the task class, disturbance-information semantics, response-equivalence map, and marginal entropy of the required response-equivalence class are comparable across episodes.

Stock-flow identity for the query rung

The query rung has a stock-flow identity of the same form. With the retained query gain and the query loss

GtDT:=max(DTtstart−DTt+1start,0),LtDT:=max(DTt+1start−DTtstart,0),

the identity is

DTt+1start=DTtstart−GtDT+LtDT.

Unlike the Knowledge To Be Discovered stock, this one can change through two channels. Write DTstart(M,Q) for the expected starting query complexity under knowledge state M and query family Q. The change splits exactly into a knowledge part and a question part:

DTt+1start−DTtstart=Δta+Δtb,
Δta:=DTstart(Mt+1,Qt)−DTstart(Mt,Qt),Δtb:=DTstart(Mt+1,Qt+1)−DTstart(Mt+1,Qt).

Δta is the effect of L2a with the questions held fixed, and Δtb the effect of L2b under the new knowledge state. When the query family only grows, Qt⊆Qt+1, Proposition Q5 gives Δtb≤0, and because the unrestricted complexity does not depend on the query family, Δtb is also the change in the average query-family gap under Mt+1. The split is exact, but the attribution follows a convention: knowledge is credited first. Crediting questions first gives a different, equally exact split.

Protocol improvement, L2c, does not move this stock, because the decision-tree complexity is already the minimum over procedures. It moves the expected length of the procedure actually used. Write K‾t for the expected number of action units of episode t under Mt, averaged over observations, and gtstart:=K‾t−DTtstart≥0 for the expected protocol gap. Then

K‾t+1−K‾t=Δta+Δtb+(gt+1start−gtstart),

and the last term is attributed to L2c. It also absorbs any change in how well an unchanged procedure suits a changed knowledge state or query family, so it is a protocol term in the accounting sense, not proof that the procedure itself was revised.

Relation to the operational ledger

The operational ledger and the knowledge ledger are joined by the closure event ct . The same closure can be read operationally as a produced outcome and epistemically as a boundary at which Knowledge To Be Discovered is measured.

Ledger Recorded unit Primary measure Question answered
Operational ledger Closure event Count of closure-fixing acts What was produced, preserved, or invalidated?
Knowledge ledger Closure-indexed KTD state Bits of residual Knowledge To Be Discovered Did the next comparable episode begin with less KTD?
Query rung of the knowledge ledger Closure-indexed query state Expected number of queries, and realized action units per closure Could the next comparable episode close with fewer questions, and was the excess due to knowledge, questions, or protocol?

The operational ledger identity:

Sgross (I) = Ssurv (I) + W (I)

is an accounting identity over closure counts. The knowledge ledger identity:

Ht+1start = Htstart - GtK + LtK

is an accounting identity over bits of Knowledge To Be Discovered. The two identities should not be conflated.

An invalidated closure may trigger learning, but invalidation is not itself learning. A corrective closure may show changed behavior, but changed behavior is not itself stored learning. Stored learning is demonstrated only when the next comparable episode begins with less residual Knowledge To Be Discovered:

Ht+1start < Htstart .

Queries sit between the two ledgers without merging them. Action units are counts, like closures; the query rung is an expectation over a model, like Knowledge To Be Discovered. Theorem Q relates the rung to the bits below it; nothing relates the rung by identity to closure counts. A fall in the query rung shows that the next comparable episode can be closed with fewer questions, whether through less ignorance or better questions. A fall in Knowledge To Be Discovered alone does not guarantee such a fall, as the example after the Learning Axiom shows.

Interpretation

The closure-indexed knowledge ledger separates four quantities that are often confused:

  • Within-episode discovery: the reduction from Htstart to Htterm inside the episode.
  • Retained learning: the reduction from Htstart to Ht+1start across episodes.
  • Knowledge loss: any increase in the next episode's starting Knowledge To Be Discovered.
  • Operational invalidation: a closure-count event in the operational ledger, not automatically a bit-valued knowledge loss.

Thus, a closure can be operationally successful but epistemically weak if it does not reduce future Knowledge To Be Discovered. A closure can involve rework and still produce positive learning if the next comparable episode begins with stronger stored coupling. Conversely, a closure can be accepted and still produce no learning if the discovered information is not retained.

The knowledge ledger therefore gives the operational ledger its epistemic counterpart: the operational ledger records what the regulator produced; the knowledge ledger records how much ignorance the regulator still carries forward.

Window-level accounting measures

So far, we have presented how the operational and knowledge ledgers are populated at the level of closure events and closure-indexed knowledge rows over an episode. For measurement purposes, those rows can also be aggregated over an observation window. Window-level accounting measures do not introduce new regulatory dynamics. They summarize the operational and epistemic traces already recorded by the ledgers.

The central measurement question is: over a given window and task class, how much residual Knowledge To Be Discovered did episodes begin with, on average? This section defines that quantity as an average of per-episode residual ignorances. It is not the entropy of a pooled variable, and it is not a new information-theoretic identity.

Measurement window and eligible knowledge rows

Let the observation window be:

I = ( ta , tb ] .

As in the operational ledger, the half-open interval avoids double-counting boundary events when consecutive windows are used.

The window contains closure-indexed knowledge-ledger rows whose closure times fall inside the interval. The knowledge ledger attaches a knowledge row ku to each closure event cu.

For a comparable task class τj , define the eligible knowledge-row history in the window as:

K I , τj := ( ku : ta < u ≤ tb , τu = τj ) .

This is an event history, not a set of unique task types. It preserves multiplicity: if several closure events in the same task class occur inside the window, each contributes one row.

Let the number of eligible rows be:

n I , τj := | K I , τj | .

If n I , τj = 0 , the task-class-specific window measures below are undefined for that window.

Average starting Knowledge To Be Discovered

For each eligible row ku , define the pre-episode residual ignorance as the row's starting Knowledge To Be Discovered:

Hpre (u) := Hustart = H Mu ( X | Y ) .

The task-class-specific average Knowledge To Be Discovered in the window is:

KTD avg τj (I) := 1 n I , τj ⋅ ∑ ku ∈ K I , τj Hpre (u) .

This is an average of per-episode residual ignorances. It answers: among the comparable episodes that closed inside the window, how much response-selection uncertainty did the regulator still carry at episode start, on average?

The same statistic can be evaluated over all task classes by omitting the task-class restriction:

K I := ( ku : ta < u ≤ tb ) , KTD avg all (I) := 1 | KI | ⋅ ∑ ku ∈ KI Hpre (u) .

The all-task-class version is useful as an overall operational dashboard measure. The task-class-specific version is usually more interpretable, because Knowledge To Be Discovered is meaningful only relative to a declared comparability frame.

Companion window knowledge measures

The same eligible knowledge-row history can be used to compute companion averages. The average terminal Knowledge To Be Discovered is:

KTD term-avg τj (I) := 1 n I , τj ⋅ ∑ ku ∈ K I , τj Huterm .

The average within-episode discovery is:

B avg disc, τj (I) := 1 n I , τj ⋅ ∑ ku ∈ K I , τj Budisc .

The average retained knowledge gain is:

G avg K , τj (I) := 1 n I , τj ⋅ ∑ ku ∈ K I , τj GuK .

The average knowledge loss is:

L avg K , τj (I) := 1 n I , τj ⋅ ∑ ku ∈ K I , τj LuK .

Finally, the average retained KTD reduction is:

R avg K , τj (I) := G avg K , τj (I) − L avg K , τj (I) .

This quantity is positive when the average ledger row records net retained reduction in future Knowledge To Be Discovered. It is negative when knowledge loss dominates retained gain.

The query rung and its gaps are averaged in the same way. The average starting query complexity is:

DTavgτj(I):=1nI,τj⋅∑ku∈KI,τjDTustart.

At the realized observation of each row, write δu, ψu, and guQ for the coding, query-family, and protocol gaps of Proposition Q2 evaluated at yu under Mu and Qu. Their window averages δavgτj(I), ψavgτj(I), and gavgQ,τj(I) are defined by the same formula as above, with DTustart replaced by the row's gap. The search excess of Lemma 1′ below uses a protocol gap gu measured against unrestricted questions; it equals ψu+guQ.

Relation to operational window measures

The operational ledger and the knowledge ledger are joined by the closure-event index. The operational ledger classifies closure events by their operational status: produced, surviving, invalidated, provisionally accepted, or authoritatively accepted. The knowledge ledger attaches bit-valued epistemic quantities to those same closure events.

This join supports measurement queries that neither ledger answers in isolation. The operational ledger alone can say which closures survived or were invalidated. The knowledge ledger alone can say how much within-episode discovery, retained knowledge gain, or knowledge loss was recorded for a closure-indexed row. Only the joined ledger can ask whether different operational classes of closures have different epistemic profiles.

For example, in the goal-model-revision ledger, the provisionally accepted closure history is:

C ^ I + := ( cu : ta < u ≤ tb , zu ∈ O ^ u ) .

The authoritative surviving subset is:

C I,net := ( cu ∈ C ^ I + : zu ∈ O* ) .

The model-invalidated subset is:

C I,model-loss := ( cu ∈ C ^ I + : zu ∉ O* ) .

Their corresponding closure counts are:

Snet (I) := | C I,net | , Wmodel (I) := | C I,model-loss | .

The two subsets partition the provisional closure history:

S gross ^ (I) = Snet (I) + Wmodel (I) .

Now join these operational subsets with the knowledge-ledger rows on the shared closure-event index. Let the knowledge rows corresponding to authoritative surviving closures be:

K I,net := ( ku : cu ∈ C I,net ) .

Likewise, let the knowledge rows corresponding to model-invalidated closures be:

K I,model-loss := ( ku : cu ∈ C I,model-loss ) .

The join does not create a new epistemic quantity. It selects knowledge-ledger rows according to an operational event class. Once that selection has been made, the same row-level knowledge quantities can be summarized conditionally over the selected closure population.

For authoritative surviving closures, the ex-ante average starting Knowledge To Be Discovered is:

KTD avg start,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net H u start .

(1)

Equation (1) averages, across the knowledge rows associated with net surviving closures, the ex-ante starting KTD H Mu ( X ∣ Y ) . For each row, this conditional entropy already averages over the possible values of Y under the episode-start model Mu .

The realized counterpart uses the actual disturbance state yu encountered by each surviving closure. The realized average starting Knowledge To Be Discovered is:

KTD avg start-real,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net H u start,real .

(2)

Equation (2) averages, over the same net surviving closure population, the realized row-level starting KTD H Mu ( X ∣ Y = yu ) . Thus equations (1) and (2) differ only in which row-level starting KTD quantity is aggregated: equation (1) uses the ex-ante conditional entropy, whereas equation (2) uses the entropy conditional on the disturbance state actually realized in that episode. The relation between these two window averages, and the conditions under which the realized average can represent the expected quantity, are developed later in the quantification layer.

The same joined subset can be used to compute the average terminal Knowledge To Be Discovered for net surviving closures:

KTD avg term,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net H u term .

The average within-episode KTD reduction for net surviving closures is:

ΔKTD avg within,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net ( H u start − H u term ) .

By the knowledge-ledger definition of within-episode discovery,

B u disc = H u start − H u term ,

so the same quantity can equivalently be written:

B avg disc,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net B u disc .

Equivalently,

ΔKTD avg within,net (I) = KTD avg start,net (I) − KTD avg term,net (I) .

For closures invalidated by goal-model revision, the corresponding average within-episode discovery is:

B avg disc,model-loss (I) := 1 Wmodel (I) ⋅ ∑ ku ∈ K I,model-loss B u disc .

The surviving and model-invalidated averages can therefore be compared. They answer the joined-ledger question: do closures that remain valid under the authoritative goal model and closures that are later invalidated differ in the amount of within-episode discovery associated with them? This comparison is unavailable from either ledger in isolation.

The same join can be applied to other knowledge-ledger quantities. For example, average retained knowledge gain for surviving closures is:

G avg K,net (I) := 1 Snet (I) ⋅ ∑ ku ∈ K I,net G u K .

The corresponding average retained knowledge gain for model-invalidated closures is:

G avg K,model-loss (I) := 1 Wmodel (I) ⋅ ∑ ku ∈ K I,model-loss G u K .

The same convention applies to all such conditional averages: if the cardinality of the selected operational event class is zero, the corresponding joined-ledger average is undefined for that window.

The ledgers can therefore be joined, conditioned, compared, and reported together. What should be avoided is treating closure counts and KTD bits as if they belonged to one conservation equation. The operational identity partitions closure counts. The knowledge-ledger identities track bit-valued epistemic quantities and their changes. The joined measures condition those bit-valued quantities on operational event classes. They are valid measurement queries, but they do not collapse the operational and knowledge ledgers into one common unit.

The same join applies to the query rung. For authoritative surviving closures, the realized starting query complexity and the realized path length average to:

DTavgstart-real,net(I):=1Snet(I)⋅∑ku∈KI,netDTustart,real,Q‾net(I):=1Snet(I)⋅∑ku∈KI,net|Qu|.

Under the hypotheses of Theorem Q for every row, averaging the per-row ladder gives

KTDavgstart-real,net(I)≤DTavgstart-real,net(I)≤1Snet(I)⋅∑ku∈KI,net𝔼u[Ku].

The right-hand side is the model-expected path length; its counted counterpart is Q‾net(I), which differs from it by sampling. Turning that counted average into an estimate of the left-hand side is the task of the quantification section.

Average KTD is not pooled entropy

The window statistic KTD avg τj (I) is not the entropy of a single pooled task variable. In general:

KTD avg τj (I) ≠ H ( XI | YI )

The left-hand side averages the conditional entropies attached to individual closure-indexed rows. The right-hand side would require constructing a new pooled probability model over window-level variables. Those are different operations.

For the same reason, the exact per-row duality between change in stored coupling and change in residual KTD does not automatically become an aggregate identity for window averages. In general:

Δ KTD avg τj (I) ≠ − Δ I avg τj (I)

Such a dual reading is valid only under stronger comparability conditions: the same task-class frame, the same response-equivalence abstraction, the same modeling resolution, and a fixed marginal entropy of the required response-equivalence variable. Even then, the identity applies cleanly at the row or matched-row level. Sliding-window averages add sampling, weighting, and window-composition effects.

The same caution applies to the query rung. DTavgτj(I) is an average of per-row decision-tree complexities, not the decision-tree complexity of a pooled problem, and the channel split Δa+Δb holds row by row, not automatically for window averages. What does survive averaging is every inequality of the ladder, because averages preserve inequalities. In particular KTDavgτj(I)≤DTavgτj(I) whenever the ladder holds row by row, and averaging the bracket of Proposition Q1 keeps the window average of the unrestricted complexity within one query of the window average of Knowledge To Be Discovered.

Sliding-window series

Moving the window over time produces a measurement series. For a sequence of window endpoints s0 < s1 < s2 < … , define:

Im := ( sm−1 , sm ] .

Then the task-class-specific KTD trend is the sequence:

KTD avg τj (I1) , KTD avg τj (I2) , KTD avg τj (I3) , …

A downward trend means that comparable episodes are beginning with less residual Knowledge To Be Discovered on average. An upward trend means that comparable episodes are beginning with more residual Knowledge To Be Discovered on average. A flat trend means that the regulator's stored coupling is not measurably changing at the chosen task-class granularity, or that gains and losses are offsetting within the window.

Summary

Window-level accounting turns the closure-indexed ledgers into practical measurement series. The operational ledger supplies counts of produced, surviving, and invalidated closures. The knowledge ledger supplies bit-valued measures of residual ignorance, within-episode discovery, retained gain, and knowledge loss.

The key practical statistic is:

KTD avg τj (I)

is average starting residual ignorance over comparable ledger rows in the window.

It is a useful summary statistic, not a replacement for the per-row knowledge-ledger identity. It should be read as an average burden of discovery carried into episodes, not as the entropy of a single pooled variable and not as the direct knowledge analogue of Sgross = Ssurv + W . The knowledge analogue remains the per-task-class stock-flow identity:

Hu+1start = Hustart − GuK + LuK ,

applied row by row and then summarized over the window.

Feedback-loop reading of revisions and the operational ledger

The preceding sections distinguish closure, acceptability, behavioral revision, learning, goal revision, goal-model revision, and ledger accounting. These constructions can now be read uniformly as feedback-loop-induced changes. The key point is that feedback does not name one kind of change. It names a loop in which a produced outcome is evaluated and the resulting signal is returned to some part of the regulatory system. What kind of revision occurs depends on what the feedback signal changes.

In the present formulation, the operational ledger is not itself the feedback loop. It is the memory trace that makes feedback across episodes observable. The feedback loop is the process by which closure events are recorded, evaluated, and then used to revise a response rule, a goal criterion, a believed goal model, or the regulator's stored law of action.

This is the cybernetic reading of regulation as a system of nested loops, where each loop is identified by its error signal, its revision target, and its characteristic time scale[7][57]. Ashby's two-loop scheme for adaptive machines is the historical antecedent: a fast inner loop selects responses within a fixed structure, and a slow outer loop changes the structure when essential variables leave the acceptable region[1][7].

The present formulation distinguishes three loops, because the formal model separates within-episode selection, across-episode learning of the response coupling, and revision of the goal or the regulator's model of the goal.

Feedback as a loop, not a revision type

At the first order, the regulator selects a response for a disturbance. A disturbance-value d is paired with a response-value r , and the Table of Outcomes produces the closure outcome:

z = T ( d , r ) .

The closure is then evaluated through the outcome-to-essential-variable map and the acceptable region prevailing at that time:

φ (z) ∈ ηt or φ (z) ∉ ηt .

A feedback signal is generated when this evaluation is returned to the regulator. But the feedback signal becomes a revision only when it changes some component of the regulatory system. If it changes the active response rule, the result is behavioral revision. If it changes the stored law of action, the result is learning. If it changes the acceptable region, the result is criterion revision. If it changes the regulator's believed model of the acceptable region, the result is goal-model revision.

Thus invalidation is not itself learning, and it is not itself corrective rework. Invalidation is a feedback signal. Learning, behavioral revision, criterion revision, and model revision are different possible uses of that signal.

The regulatory state

Throughout this section, the fixed environmental frame is still the Table of Outcomes:

T : D × R → Z

with outcome-to-essential-variable map:

φ : Z → E .

In the fixed-table case, these remain fixed. The revisable regulatory state at episode index t can be written as:

Θt:=(Mt,Qt,ϑt,ρt,η^t,ηt).

Here Mt is the stored law of action, ρt is the active response rule, η ^ t is the regulator's believed acceptable essential-variable region, and ηt is the authoritative acceptable region prevailing at time t .

The acceptable outcome-set induced by the authoritative region is:

Oacc,t := φ−1 [ηt] .

When the regulator acts under an imperfect model of the goal, its believed acceptable outcome-set is:

O ^ t := φ−1 [ η ^ t ] .

The two components added to the regulatory state are the query family Qt, the questions the regulator can ask, and the decision procedure ϑt, the way it asks them. Both are revisable in the fixed-table case: neither changes T, D, R, Z, E, or φ.

The authoritative and believed acceptable regions induce two regulatory relations: the authoritative relation Rt, built from ηt, and the believed relation R^t, built in the same way from η^t. The regulator's queries solve the believed relation; its closures are judged against the authoritative one.

The three loops

Each loop is identified by its error signal, by the component of the regulatory state it revises, and by the time scale on which it operates. All three loops are negative-feedback loops in the cybernetic sense: each acts to reduce a measured discrepancy between an observed quantity and a reference[7].

Loop L1 - selection within an episode

L1 is the inner regulatory loop. Its reference is the acceptable outcome-set Oacc,t , or its believed counterpart O ^ t when the regulator acts under an imperfect model.

Its error signals are the within-episode selection signals Ui generated by action units qi . Its output is the staged narrowing of the candidate response-set:

∅ ≠ Ck (d) ⊆ … ⊆ C1 (d) ⊆ C0 (d) ⊆ R .

The loop terminates when the regulator commits a response rt and produces the closure:

zt = T ( dt , rt ) .

L1 operates within one episode. It does not, by itself, alter the stored law of action, the believed goal model, or the authoritative acceptable region. This is Ashby's first feedback loop, the error-controlled regulator[1][7].

In query terms, L1 runs the decision procedure ϑt over Qt on one disturbance. Its error signals are query answers, its stopping condition, in the ideal case of Theorem Q, is relational certainty termination under the believed relation, and its cost is the realized path length |Qt|. L1 changes none of the three rungs of the ladder; it only traverses one path.

Loop L2 - learning across episodes

L2 is an outer loop whose reference is reduction in residual Knowledge To Be Discovered for a comparable task class. Its error signal includes the closure-valuation:

vt := 1 { φ (Zt) ∈ ηt } ,

together with the within-episode evidence stream Ut , the response class actually fixed, the final concrete response, and the produced closure. Its output is the update:

Mt → Mt+1 .

When successful by the learning axiom, L2 reduces the residual lack of requisite knowledge at the start of the next comparable episode:

Ht+1 (X|Y) < Ht (X|Y) .

L2 operates across episodes within a window of comparable task instances. It corresponds to adaptive learning of a pattern of behavior appropriate for the environment[1][7].

L2 has three channels, distinguished by the component of the regulatory state they revise and by the gap of the ladder they reduce:

  • L2a, knowledge, revises Mt→Mt+1. Its reference is the reduction of Knowledge To Be Discovered, as stated above; it is the channel the Learning Axiom describes.
  • L2b, questions, revises Qt→Qt+1. Its error signal is a persistent query-family gap: episodes that cannot close safely with the questions available, or that need more of them than the missing knowledge requires. Its reference is a smaller query-family gap.
  • L2c, protocol, revises ϑt→ϑt+1. Its error signal is a protocol gap: more action units per closure than the available questions require. Its reference is a smaller protocol gap.

All three act between episodes and change the prior structure of the next one. None of them changes the status of closures already produced.

Loop L3 - revision of the goal or the goal model

L3 is a higher-order outer loop whose reference is alignment of the criterion of success used by the regulator with the criterion against which closures are ultimately judged. It has two forms, corresponding to the two ledger cases developed above.

In the external goal-revision form, the trigger is an exogenous change in the authoritative acceptable region:

ηt1 → ηt2 .

This change propagates into the acceptable outcome-set through the pullback along φ , yielding deletion, addition, and substitution operations on Oacc,t .

In the goal-model-revision form, the authoritative region remains fixed. The feedback signal is checking evidence that registers a mismatch between the regulator's believed acceptable set and the authoritative set:

O ^ v− ≠ O* .

The output is a model-revision event that revises the believed acceptable region:

η ^ v− → η ^ v .

In both forms, L3 does not by itself produce behavior. It revises the criterion against which behavior is judged. Behavioral consequences arise only when L1 is re-engaged under the revised criterion, possibly using a different response rule.

In query terms, L3 revises the relation the queries solve. In the external form, a change of ηt changes Rt; in the goal-model form, revision of η^t moves the believed relation R^t toward the authoritative one. Either way the problem changes, so every rung of the ladder can change, Knowledge To Be Discovered included, and episodes before and after the revision are no longer comparable unless the change is declared. An L3 event can also make an adequate query family inadequate: questions that separated the disturbances well enough for the old relation may leave a class of disturbances with no common acceptable response under the new one (Proposition Q4). L3 can therefore create work for L2b.

Mapping revision targets to feedback loops

All revision types can be represented as feedback-induced changes, provided the loop level and revision target are made explicit. The type of feedback-induced revision is determined by which component of the regulatory state is changed.

Construction or revision type Loop Feedback signal / trigger Revision target Cybernetic interpretation
Staged selection trace L1 Selection signals Ui Candidate response-set Ci(d) The regulator narrows possible responses within one episode.
Closure L1 terminal output Commitment of rt Realized outcome-value zt The episode closes when a committed response produces an outcome.
Behavioral revision L1 re-engaged Prior closure evaluation or invalidation Response rule ρt → ρt+1 Feedback changes what response rule the regulator uses.
Learning L2a Closure-valuation, evidence stream, and retained episode information Stored law of action Mt → Mt+1 Feedback changes the stored coupling used in later comparable episodes.
Learning success criterion L2a Across-episode reduction of conditional entropy Residual Knowledge To Be Discovered The next comparable episode starts with less response-class uncertainty.
Query acquisition L2b Persistent query-family gap: episodes that cannot close safely with the available questions, or need more of them than the missing knowledge requires Query family Qt→Qt+1 Feedback adds questions the regulator can ask; the query-family gap falls while Knowledge To Be Discovered is unchanged.
Protocol improvement L2c Protocol gap: more action units per closure than the available questions require Decision procedure ϑt→ϑt+1 Feedback changes which questions are asked and in what order; the protocol gap falls while Knowledge To Be Discovered and the query family are unchanged.
Criterion revision L3 external goal-revision form Exogenous change in the authoritative acceptable region ηt → ηt+1 Feedback or external change revises the criterion of success.
Acceptability deletion, addition, and weak substitution L3 external goal-revision form Change in ηt Pullback acceptable outcome-set Oacc,t Fixed outcome-values gain or lose acceptability; they are not deleted from T .
Strict substitution L3 external goal-revision form Criterion revision plus replacement pairing supplied by the model Replacement correspondence λ12 The model specifies which lost acceptable value is replaced by which gained value.
Goal-model revision L3 goal-model form Checking evidence against the authoritative acceptable set Believed acceptable region η ^ t → η ^ t+1 Feedback changes the regulator's model of the goal, not the authoritative goal itself.
Model-revision event L3 goal-model form Authoritative checking or mismatch evidence Model revision event mv The believed loss and gain sets are induced by revising the goal model.
Corrective replacement L1 re-engaged after L3 Invalidation of an earlier closure Repair relation A later closure is treated as repairing an earlier invalidated closure.
Structural adaptation Higher-order adaptation beyond the fixed-table case Persistent failure or insufficiency of the existing regulatory frame T , D , R , Z , E , φ may change Feedback changes the system structure or the available regulatory repertoire.

The present fixed-table formulation allows behavioral revision, criterion revision, goal-model revision, and learning while holding T , D , R , Z , E , and φ fixed. Structural adaptation is a stronger case and should be modeled separately.

Query acquisition sits between learning and structural adaptation. It changes the regulator's repertoire of questions, which is part of the regulator, while the disturbance set, the responses, the Table of Outcomes, and the essential variables stay fixed. It therefore belongs to the fixed-table formulation, although it is structural from the regulator's point of view.

The operational ledger as feedback memory

The operational ledger records the closure history produced by L1. It preserves closure events after their production, allowing the system to evaluate not only whether a closure was acceptable when produced, but also whether it continues to survive later criteria, later model revisions, or later authoritative checking.

Let the operational ledger up to time t be the time-ordered history of recorded closure events:

Lt := ( cu : u ≤ t ) , cu = ( u , du , ru , zu ) .

The ledger answers the accounting question: which closures were produced, which survived, and which were invalidated? The feedback-loop formulation answers the question: which part of the regulator changed because closure evaluations were fed back into the system?

A feedback signal is generated when the ledger is evaluated against some current or authoritative criterion. In the external criterion-revision case, a prior closure may be invalidated because:

zu ∈ O acc,u but zu ∉ O acc,t , u < t .

In the goal-model-revision case, a prior provisional closure may be invalidated because:

zu ∈ O ^ u but zu ∉ O* .

These facts are ledger-visible feedback signals. They become revisions only when they are used to change some component of the regulatory state. Thus the ledger supplies feedback memory, but it does not collapse invalidation, learning, and repair into the same event.

The operational ledger as cross-loop bookkeeping

Under this reading, the operational ledger is bookkeeping over L1's closure history after L2 and L3 events inside the window have had their effects. Each ledger quantity has a loop interpretation.

The gross counted closures Sgross (I) and the gross provisional closures S gross ^ (I) count L1 outputs admitted under the criterion or believed criterion prevailing at production time. They are work-counts of closure-fixing acts.

The invalidation load W (I) counts L1 closures that L3, in its external goal-revision form, later invalidated by changing ηt somewhere in the window.

The goal-model invalidation load Wmodel (I) counts L1 closures that L3 (goal-model form) later invalidated by revising η^ t against the fixed authoritative η* . Both are L3-attributable invalidations of prior L1 work.

The surviving counted closures Ssurv (I) and the net surviving closures Snet (I) count L1 closures that were not invalidated by the relevant L3 process within the window.

The two ledger identities therefore have direct loop readings:

Sgross (I) = Ssurv (I) + W (I)
S gross ^ (I) = Snet (I) + Wmodel (I)

The first says that total accepted L1 output equals L1 output that continuously survived plus L1 output later invalidated by criterion revision. The second says that total provisional L1 output equals authoritatively surviving L1 output plus L1 output invalidated by goal-model mismatch.

Neither identity is an entropy identity. Both are operational accounting identities. They count closure-fixing acts, not bits of Knowledge To Be Discovered.

The ledger therefore answers a question that none of the three loops answer in isolation: over a window, how much of L1's produced closure history was preserved, and how much was retroactively invalidated by later criterion or model evaluation?

Invalidation, repair, and corrective rework

The ledger can show that a closure was invalidated, but invalidation is not the same as corrective rework. Invalidation is a feedback signal. Corrective rework requires a later behavioral event that repairs, replaces, or compensates for the invalidated closure. Thus a feedback-induced corrective replacement requires additional structure beyond the invalidation count.

The full feedback sequence is therefore:

closure → ledger record → evaluation → feedback signal → revision → possible corrective closure.

This is the cybernetic role of the operational ledger: it makes closure history available for later feedback, but it does not collapse invalidation, learning, and repair into the same event.

Time-scale separation and loop interaction

The three loops are separated by characteristic time scales, in the cybernetic tradition of nested adaptive control[1][7][57]. L1 operates within a single episode and terminates with a closure. L2 operates across consecutive comparable episodes within a window. L3 operates on an episodic-to-rare time scale, depending on whether the trigger is an exogenous goal change or a discovered model mismatch.

Because L3 acts outside the immediate L1 closure event, an L3 event inside a window can retroactively rewrite the acceptability-status of L1 closures already produced. This is why the operational ledger must track not only gross production count but also survival count under the criterion history or authoritative checking structure of the window.

Because L2 acts between L1 episodes, an L2 update changes the prior structure of the next L1 episode. It does not, by itself, change the status of past closures. Therefore W (I) and Wmodel (I) are L3-attributable invalidation loads, not L2 learning measures.

Loop interaction is still possible. An L3 invalidation may become an error signal for L2: a closure that ceased to count under a revised criterion provides evidence that the stored coupling Mt was aligned with the old criterion or the old believed model, not with the revised one.

Conversely, repeated L2 success may reduce the need for L3 in the goal-model-revision case by bringing the regulator's believed model into alignment with the authoritative criterion. This cross-loop coupling matches Ashby's observation that the adaptive loops of an ultrastable machine interact, with the slow loop reorganizing the structure on which the fast loop operates[1][7].

The present formulation models these loops as negative-feedback loops on their respective error signals. Positive-feedback dynamics - such as exploratory expansion of the candidate set, variety-amplifying changes to the stored law of action, or deliberate broadening of the believed acceptable region - would require explicit additional structure and are not part of the fixed-table formulation developed here.

The three channels of L2 typically run on different time scales. Protocol improvement can occur from one episode to the next, as a regulator reorders its questions. Knowledge updates accumulate over the episodes of a window. Query acquisition is usually slower, because a new question often requires a new instrument, test, or access to information. The channels also interact: a sharper knowledge state can make some questions unnecessary, so that L2a enables L2c, and a new question can change which procedure is optimal, so that L2b calls for L2c.

Summary

Read as a feedback-loop system, the formal model has three loops. L1 selects responses within an episode and closes it. L2 updates the stored law of action across comparable episodes and is evaluated by the learning axiom. L3 revises the criterion of success, either exogenously through external goal revision or endogenously through goal-model mismatch discovery.

The operational ledger is the cross-loop bookkeeping that records how L1's closure history was preserved or invalidated within an observation window. It is not itself the feedback loop. It is the memory trace that allows later evaluation, feedback, revision, and possible repair.

This reading does not introduce new formal objects. It names the cybernetic role of each construction already defined above and exhibits the regulatory model as a nested loop hierarchy in the sense of Ashby's adaptive machine and its successors[1][7][57].

The query reading adds one distinction to this hierarchy. A regulator can fail Ashby's requirement not only by lacking knowledge, which shows as Knowledge To Be Discovered, but also by lacking questions, which shows as a large or infinite query-family gap. A better protocol, L2c, cannot repair the second failure. It is repaired by acquiring questions, L2b, or by learning that the disturbances the present questions cannot separate do not in fact occur, which is L2a acting on the support of the knowledge state.

6.Quantifying the Knowledge Discovery Process

We estimate a non-physical latent information quantity from physical counted units, with bridge error handling the gap between counted physical action units and ideal information-reduction units.

From latent ignorance to operational measurement

The preceding sections defined the Knowledge Discovery Process in two related ways. Inside a regulatory episode, it is the staged reduction of Knowledge To Be Discovered. Across comparable episodes, it is the retained reduction of Knowledge To Be Discovered in the regulator's stored law of action. Those definitions are information-theoretic. They refer to quantities such as H(X|Y) , where X is the required response-equivalence class and Y is the disturbance information available to the regulator.

These latent quantities are not directly observable under a strict black-box measurement regime. The observer does not see the regulator's internal probability model, posterior distribution, candidate set, search path, binary questions, or elementary narrowing operations. In particular, the observer does not directly observe Q(I) as a count of internal knowledge-seeking actions. The primitive observation is only the external closure ledger.

For a completely black-box observer, the observable data in an observation window I are closure events, timestamps, and invalidation events. That is, the observer can record which closure commitments occurred, when they occurred, and which previously recorded closures were later invalidated. If the ledger supports it, an invalidation may also identify the closure that it invalidates. No assumption is made that the observer can see the cognitive or physical actions that occurred between closures.

In the query reading, what the observer sees are the leaves of the regulator's decision procedures: the closures, when they were committed, and which of them were later invalidated. The record of leaves and capacity is read in 6.6 as a two-instruction program. The queries asked on the way to each leaf are not seen. The residual that the operational layer computes is therefore an inferred query count: the number of action units the closures must have cost, given the capacity of the window. By the ladder of Theorem Q, its expectation is the top rung, the expected length of the regulator's actual protocol, and Knowledge To Be Discovered is the floor beneath it. The rest of this section states when the inferred count reaches down to that floor, and with which errors.

Latent window-level estimand

The operational construction targets the realized joined-ledger statistic KTD a v g s t a r t − r e a l , n e t ( I ) defined above, rather than the ex-ante statistic KTD a v g s t a r t , n e t ( I ) . The reason is that a realized window contains the actual disturbance states y u encountered by the surviving closure population, not an ensemble average over disturbance states that did not occur.

Consequently, when the knowledge state is stable across the window and the realized observations yu among the net surviving closure population are representative of the relevant M-induced marginal distribution on Y, the row average of Hustart,real estimates the common expected quantity HM (X∣Y) . Under Assumption E below, this estimate is consistent in the large-window limit.

Ergodic reconciliation of realized and expected KTD

Equations (1) and (2) differ only in whether the per-row conditional entropy is the ensemble quantity averaged over the disturbance law or the realized quantity at the observed disturbance-value. Their coincidence is therefore not automatic: it is an ergodicity condition on the disturbance process restricted to the comparable task class.

Assume the knowledge state is stable across the window, Mu = M for all u in the window. Define the empirical disturbance distribution induced by the net surviving closure population:

P^ I,net (y) := 1 Snet(I) ⋅ ∑ cu ∈ CI,net 1 { yu = y } .

Under stability of M, the realized latent average of equation (2) is exactly the empirical P^ I,net -average of the realized posterior entropies:

KTD avg start-real,net (I) = ∑ y P^ I,net (y) HM (X∣Y=y) ,

whereas the expected quantity of equation (1) is the same functional evaluated at the model-induced marginal PM (Y) :

HM (X∣Y) = ∑ y PM (Y=y) HM (X∣Y=y) .

Subtracting, the realized–expected gap is a reweighting error driven entirely by the mismatch between the empirical and model-induced disturbance distributions:

KTD avg start-real,net (I) − HM (X∣Y) = ∑ y P^ I,net (y) − PM (Y=y) HM (X∣Y=y) .

Assumption E — Net-survivor disturbance ergodicity. Under a stable knowledge state M u = M and within the comparable task class, the disturbance sequence indexed by the net surviving closure population is stationary and ergodic with invariant marginal P M ( Y ) . Consequently, its single-trajectory empirical distribution converges to the model-induced marginal:

P^ I,net ⟹ Snet(I)→∞ PM .

Under Assumption E the reweighting error vanishes in the large-window limit, and the realized average of equation (2) is a consistent estimator of the expected average of equation (1):

KTD avg start-real,net (I) ⟶ Snet(I)→∞ HM (X∣Y) = KTD avg start,net (I) .

Nothing in this reconciliation uses the form of the per-row quantity beyond its dependence on y under a fixed M. It therefore applies unchanged to the relational form, with HM(X∣Y=y) read as KTDrel(y), and to the query rung of the knowledge ledger: under a stable knowledge state and a stable query family, the net-survivor average of DTustart,real differs from DTstart by the same reweighting error, with DTμyQ(R∣y) in place of the posterior entropy, and converges to it under Assumption E.

Absent Assumption E, two cases should be distinguished. If the knowledge state remains stable but the realized disturbance mix over the surviving population is not representative of P M , the realized and expected averages differ by exactly the reweighting error above. If the knowledge state itself drifts, M u ≠ M , the fixed- M reweighting decomposition no longer applies: the difference may reflect both the realized disturbance mix and changes in the row-specific knowledge states. In either case, the operational construction below targets the realized average KTD avg start-real,net (I) , not the expected ex-ante average of equation (1).

This realized estimand is latent: under black-box observation we do not see the internal stages of an episode, we do not see the regulator's knowledge state Mu, and we do not see the per-stage selection signals Ui. What we do see is the external closure/timestamp/invalidation trace, from which action-unit quantities are either directly counted where available or inferred under the later unit convention.

The information-theoretic form of the latent quantity is already fixed upstream. For each row, realized starting Knowledge To Be Discovered is the posterior entropy H M u ( X ∣ Y = y u ) . Lemma 0 fixes the information discovered along a subsequent realized conditioning path as differences of Shannon entropies, preserving exact stage additivity and telescoping. What remains open is whether the row-level posterior entropy has a countable correlate. Nothing established so far connects that quantity in bits to a number of observable execution units. That is the task of the next layer.

Ideal binary-question-depth layer

The remaining uncertainty in a closure is latent: it concerns the response-equivalence class that would have to be selected for the realized disturbance, but that class is not directly observed as a visible work item, question, or decision step. Lemma 0, established for the realized conditioning path, has shown that this uncertainty is measured by a posterior entropy; what it has not supplied is a countable correlate of that entropy. To make the latent quantity operational, we need a bridge from entropy to ideal inquiry effort.

Lemma 1 - One-bit coding bracket. For each realized episode u, let the posterior distribution over required response-equivalence classes after the realized disturbance information is observed be:

P u , y u ( x ) := P M u ( X = x ∣ Y = y u ) .

Let q u ⋆ ( y u ) denote the minimum expected number of binary yes/no questions required to identify X under this posterior distribution. Equivalently, q u ⋆ ( y u ) is the expected depth of an optimal binary decision tree for P u , y u . Assume the posterior support of X under Y = y u is finite, or countable with finite entropy, so that the standard binary source-coding bracket applies, as per the finiteness condition stated in the Resolution-relativity of Knowledge To Be Discovered section.

In this lemma, a question means one binary distinction, ideally a yes/no distinction, used to reduce uncertainty about the required response-equivalence class. Question depth is the number of such binary distinctions needed along a particular inquiry path. Expected question depth is the average number of binary distinctions required under a probability distribution over possible response classes. Thus, question depth is literally the number of questions only in the ideal binary-question model

Then the optimal expected binary-question depth is bracketed by the posterior entropy within one bit:

H M u ( X ∣ Y = y u ) ≤ q u ⋆ ( y u ) < H M u ( X ∣ Y = y u ) + 1.

Here, the per-row posterior entropy HMu(X∣Y=yu) is exactly H u start,real evaluated at the realized yu, the posterior entropy whose functional form is fixed by Lemma 0.

In the special case where the posterior distribution is uniform over 2m equiprobable response classes, the bracket is tight: H M u ( X ∣ Y = y u ) = m . . For example, if a coin is hidden in one of eight equally likely boxes, then ( H ( X ) = log 2 ⁡ 8 = 3 ) , and the optimal binary-question strategy identifies the box in exactly three questions.

Averaged over the net surviving closure population C I,net in window I , the corresponding ideal-depth benchmark is:

KTD avg start-real,net (I) ≤ E¯ I,net [ q⋆ ] < KTD avg start-real,net (I) + 1

(3)

where E¯ I,net [ q⋆ ] denotes the average ideal binary question depth over net surviving closures in window I.

Equation (3) is the latent-to-ideal bridge. It says that the average optimal binary-question depth over net surviving closures brackets the latent KTD avg start-real,net (I) realized average Knowledge To Be Discovered within the standard sub-one-bit source-coding gap. It is purely information-theoretic. It does not yet involve any observable execution counts, and it does not yet say that the observed execution trace followed an optimal binary-questioning procedure. That stronger reading enters only through the ideality assumptions stated below.

Proof. The result follows from the standard source-coding interpretation of entropy: an optimal binary decision or coding procedure has expected length at least the entropy and less than entropy plus one bit. Averaging this per-row bracket over the net surviving closure population preserves the inequalities. □

Relational form of Lemma 1. Lemma 1 is the unique-target case of Proposition Q1. The optimal depth qu⋆(yu) is the unrestricted distributional decision-tree complexity DTμyu(Ru∣yu) of the row's regulatory relation, and when several classes are acceptable for the same disturbance, Proposition Q1 gives the same one-bit bracket with relational Knowledge To Be Discovered in place of the posterior entropy:

KTDrel(yu)≤DTμyu(Ru∣yu)<KTDrel(yu)+1.

Equation (3) holds in the same relational reading, with qu⋆(yu) read as the left-hand complexity. In the rest of this section, H(p0,u) and Hustart,real are read as KTDrel(yu) where the goal is set-valued; in the unique-target case the two readings coincide.

The scope of Lemma 1 should be read narrowly. It supplies a countable correlate, in ideal binary distinctions, of a quantity whose form was already fixed by Lemma 0. It does not license reading the posterior entropy as a question count outside the ideal binary-question model, and it does not underwrite additivity. The two lemmas are complementary and the division of labour matters: the source-coding gap does not compose across stages. Bracketing k stages of an episode separately accumulates up to k bits of slack, whereas the additive stage accounting underlying Lemma 0 remains exact however finely the episode is decomposed. Additivity of the knowledge ledger therefore rests on Lemma 0 alone; Lemma 1 is invoked once, at the level of the realized per-row posterior entropy.

An adaptive yes/no search strategy that identifies X is the same object as a binary prefix code for X: the sequence of answers along a path is the codeword, and stopping at identification is the prefix property [Cover & Thomas §5.7; MacKay Ex. 5.24]. Lemma 1 concerns the optimal inquiry. The regulator's realized conditioning path need not be optimal, and the quantity the operational layer will eventually count is the length of that realized path, not the optimal depth. What is needed is therefore a bracket on the expected realized length.

Definition (search excess). For realized episode u, let Ku denote the number of stages of the realized conditioning path of the episode, a random variable under Mu given Y=yu, and write 𝔼u[⋅] for expectation under that conditioning, taken over the path. Its realized value is written ku. The search excess of the episode is

εu := 𝔼u [Ku] − Hustart,real .

The definition is algebraic for any stage alphabet. Its interpretation as a non-negative search excess is specific to the binary case, where one conditioning stage can convey at most one bit: the quantity subtracts a bit-entropy from a stage count, and the two are commensurable only at m=2 .

Lemma 1′ — Search excess bracket. Let the episode satisfy P1-P3 with m=2 and terminate at certainty about the required response-equivalence class, as B2 requires of a surviving closure. Then:

(i) Nonnegativity. The search excess is nonnegative,

Hustart,real ≤ 𝔼u [Ku] , εu ≥ 0 .

No optimality, determinism, or even-splitting of the protocol is assumed. In particular the realized count ku is not bounded below by Hustart,real; a lucky early termination may resolve three bits of starting uncertainty in one binary stage. What the starting stock bounds is the expected length of the path.

(ii) Decomposition for a deterministic tree. Suppose in addition that each stage signal is a deterministic function of X given the prefix, so that the protocol is a binary decision tree for X and the path is determined by X. Define the coding gap

δu := qu⋆ (yu) − Hustart,real ∈ [0,1)

and the per-episode tree redundancy (gap)

gu := 𝔼u [Ku] − qu⋆ (yu) ≥ 0 .

Then the excess splits exactly into the two, and is bounded by the redundancy plus one bit:

εu = δu + gu < 1 + gu .

The second inequality is definitional once gu is introduced; its content lies entirely in whatever independent bound is placed on gu, which is the role of B1(i). Determinism is required here and only here: without it the realized path depends on signal noise as well as on X, the protocol is not a decision tree for X, gu may be negative, and only part (i) remains available.

(iii) Sharper coding gap. In the setting of (ii), let pmax be the largest posterior probability of a response-equivalence class under Pu,yu. Then the Gallager bound on Huffman redundancy applies to the optimal tree: δu ≤ pmax + 0.086 when pmax<1/2, and δu ≤ pmax otherwise [Gallager 1978; MacKay Ex. 5.28].

(iv) Tightness. εu=0 exactly when the posterior is dyadic and the protocol splits posterior mass evenly at every node. This is the even-split regime: it is the tightness case of a bracket, not an assumption imposed on units anywhere in what follows.

Proof. (i) By P3 the stage count is a stopping time, so no terminating realized signal string is a proper prefix of another. Under P1-P2 with m=2 , Corollary 0′ gives IMu ( X ; U≤K ∣ Y=yu ) ≤ 𝔼u [Ku] . Termination at certainty makes the left-hand side equal to the starting stock, by the telescoping identity of Lemma 0 and the Bridge Lemma. (ii) The identity is the definition of the two gaps; δu∈[0,1) by Lemma 1, gu≤r by B1(i), and gu≥0 holds because under determinism the protocol is one of the trees over which qu⋆ minimizes. (iii) and (iv) are the standard Huffman facts under the twenty-questions correspondence: an adaptive yes/no search that identifies X is a binary prefix code for X, the answer sequence along a path being the codeword and identification being the prefix condition [Cover & Thomas §5.7; MacKay Ex. 5.24]. □

Relational and restricted form of Lemma 1′. Part (i) holds without a unique target. If the episode is relationally certainty-terminated, Proposition Q1′ with m=2 gives KTDrel(yu)≤𝔼u[Ku], again without determinism of the stage signals.

Part (ii) compares the realized protocol with the optimal unrestricted tree. When the regulator can ask only the questions in a family Qu, its protocol gap against that optimum splits into what the family forces and what the regulator wastes:

gu=ψu+guQ,ψu:=DTμyuQu(Ru∣yu)−qu⋆(yu),guQ:=𝔼u[Ku]−DTμyuQu(Ru∣yu).

If each stage signal is a deterministic function of the disturbance, Proposition Q2 gives the three-term form of the search excess,

εu=δu+ψu+guQ,δu∈[0,1),ψu≥0,guQ≥0,

with all three terms read relationally. ψu=0 when the family is unrestricted, and (ii) is then recovered unchanged. Part (iv) acquires a condition on the family: an even split at every node requires the family to contain a question that splits the current posterior evenly. A dyadic posterior and an optimal use of the available questions are not enough when the questions are restricted.

Averaged over the net surviving closure population CI,net in window I, write εred(I) for the average search excess. Part (i) then gives the realized-path counterpart of equation (3):

KTD avg start-real,net ( I ) ≤ 1 S net ( I ) ∑ u ∈ C I , net 𝔼 u [ K u ] = KTD avg start-real,net ( I ) + ε red ( I ) ,

(3′)

with εred (I) ≥ 0 without B1 or determinism, under the binary hypothesis of Lemma 1′(i), and εred (I) < 1 + r under part (ii) together with B1(i). Equation (3) brackets the latent average by the depth of the inquiry the regulator could have made; equation (3′) brackets it by the expected length of the inquiry it did make. Only the second has a countable correlate in the execution trace, and it is the one the operational layer will estimate.

Two stages that return the regulator to its prior, in the sense of Corollary 0, are two wasted questions: they enter εu and nothing else. Null stages and uneven stages are treated the same way; none of them requires a separate charge or convention. The bracket is stated once per episode, for the whole realized path. As already noted, a bracket of this kind does not compose across stages, whereas the additive stage accounting of Lemma 0 does.

Lemma 1″ - Kraft decomposition of the search excess. In the setting of Lemma 1′(ii), the protocol is a binary decision tree for X under Pu,yu . For each class x∈supp(p0,u) let ℓu(x) be the depth of the leaf at which x is identified, so that 𝔼u[Ku] = ∑x p0,u(x) ℓu(x) . Because the stage count is a stopping time (P3), the answer strings are prefix-free, and the Kraft sum satisfies

vu := ∑x∈supp(p0,u) 2-ℓu(x) ≤ 1 .

and the probability distribution over response-equivalence classes is given by

p~u(x) := 2-ℓu(x) vu .

Then the search excess decomposes exactly as

εu = δu + gu = D ( p0,u ‖ p~u ) + log2 1vu ,

(3″)

and both terms are nonnegative. Moreover:

(a) Kraft slack charges null stages. vu=1 if and only if no stage of the tree is epistemically null, that is, if at every internal node both answers carry positive posterior mass. Equivalently, the tree is a full binary tree whose leaves are exactly the posterior support. A stage whose answer is already determined by the prefix leaves one branch empty and makes vu<1 .

(b) Divergence charges uneven splits. D(p0,u‖p~u)=0 if and only if p0,u=p~u . Together with (a), this recovers the even-split tightness case of Lemma 1′(iv).

(c) Bounded path range. If no stage is null and the posterior support has Mu≥2 classes, every leaf depth lies in [1,Mu-1] . The realized stage count then ranges over an interval of width at most Mu-2 . With null stages admitted, no such bound follows from Mu alone, and the range must be declared. Throughout, Mu denotes posterior support size and is distinct from the stage alphabet size m of P1.

(d) Wrong-model cost. Suppose the required class is in fact distributed as pu∗ while the tree is run and its expectation evaluated under p0,u . Then

𝔼pu∗ [Ku] - 𝔼u [Ku] = ∑x ( pu∗(x) - p0,u(x) ) ℓu(x) , which is at most | ⋅ | ≤ TV ( pu∗ , p0,u ) ⋅ ( ℓu,max - ℓu,min ) .

The identity (3″) also holds with pu∗ in place of p0,u . The Shannon-code case is the wrong-code theorem [Cover & Thomas, Thm 5.4.3].

Proof. Using the definitions of vu and p~u,

∑x p(x)ℓ(x) - H(p) = ∑x p(x) log2 p(x) 2-ℓ(x) = ∑x p(x) log2 p(x) cr~(x) = D(p‖r~) + log2 1c ,

[Cover & Thomas, proof of Thm 5.3.1; MacKay eq. 5.23]. For (a), a full binary tree with leaf set equal to the support satisfies Kraft with equality, and every empty branch removes its share of the sum. For (c), a full binary tree with M u leaves has depths between 1 and M u - 1 . For (d), write both expectations as sums over the same leaf depths. A difference of expectations of a function whose range has width w is at most w times the total-variation distance. □

Lemma 1″ refines the preceding remark. Null stages and uneven stages enter the search excess additively but through different terms: null stages through the Kraft slack, uneven stages through the divergence.

Lemma 1″ holds for every deterministic tree, over any query family, but it measures the coding, query-family, and protocol gaps jointly. The two terms do not separate the query-family gap from the protocol gap. One direction is clean: an optimal procedure over any family contains no null stage, because removing a null stage leaves a valid and shorter procedure, so a nonzero Kraft slack always signals a protocol gap, guQ>0. The divergence term is mixed. A restricted family can force uneven splits, and those enter the divergence exactly as uneven splits chosen by an inefficient regulator do. In the robot example with restricted sensors, the first question “vertical or horizontal?” splits the posterior (34,14), and the whole query-family gap of 0.25 appears as divergence. Only a declared query family distinguishes the two causes.

With the functional fixed by Lemma 0 and bracketed by Lemma 1, KTD avg start-real,net (I) may be read as the average amount of knowledge that still had to be discovered per net surviving closure, expressed in bits, bracketed below by the ideal binary-question depth of equation (3) and by the expected depth of the regulator's own inquiry in equation (3'). Both brackets are latent. What remains is to connect the second to observable execution counts..

Execution channel

We have already quantified the amount of knowledge to be discovered in terms of the average ideal binary-question depth over net surviving closures. In order to quantify regulator's efficiency, we now need to quantify the time available.

Durations do not enter the information ledger. The amount of response-relevant selection is invariant to whether the selection trace is executed quickly or slowly. However, durations matter when 1) we need to quantify the time available, and 2) the number of action units Qeff is inferred from elapsed time rather than directly observed as a sequence of counted action units.

We introduce a generic action unit type alphabet and tile the window I into ordered generic action units a. A generic action unit is one of three kinds: a non-closure action unit Q, a closure unit S, or an fedback unit F. The first two have already been introduced. Fedback units are capacity that goes neither into discriminating the next response nor into committing one - they are physical unfolding, feedback latency, waiting, or idle observation.

Every episode terminates in exactly one closure. The closure is the final response commitment after which the episode ends. The response has been selected and released into the world where the table of outcomes T produces the result z. This act is binary in form: at each moment the episode is either not yet closed or closed by commitment. It is not characterized by its information content, which is generally the residual selection entropy at the moment of commitment. What individuates the closure as a single countable act is that it is a discrete, time-bounded commitment, with an onset at the moment of commitment and completion when zu is produced. The closure duration is the commitment duration, not the post-closure physical execution or waiting interval. The closure time defines a generic action unit, while the post-closure physical execution or waiting interval is feedback time: feedback latency, idle observation, or physical unfolding.

We define the execution channel as a physical trace through which knowledge-discovery work becomes observable. The channel capacity is measured in counted generic action units, not in elapsed time.

The central object in this channel is the total number of counted generic action units N(I) consumed in window I. Then elapsed time supplies a physical trace, but it does not by itself determine how many counted non-closure units, closure units, or fedback units the trace contains. The same stretch of behavior admits more or fewer units depending on the grain at which it is segmented, just as the entropy of a behavior depends on how finely its actions are distinguished. This number becomes a fixed count only after a unit convention has been chosen.

A duration-to-count rule is needed to map time into counted channel capacity. A unit convention must therefore be declared before N(I) is fixed. Before that convention is fixed, the question “how many units occurred in interval I ?” is underdetermined.

Therefore, the execution channel will be read in two layers, and the order matters. The first layer is unit-convention constitutive: it fixes what counts as one generic action unit, and thereby fixes both the channel capacity N(I) and the partition of that capacity into closure, non-closure, and fedback units. The second layer is accounting: once units are defined, N(I) is a fixed tally, and every accounting identity operates on that fixed counted-unit ledger, independently of how the units are spread in time.

The accounting layer is duration-free: the identities count units, not time. The count itself, however, is not duration-free, because the choice of unit convention — in particular the time-normalized convention below — can make a unit's identity depend on elapsed time. Duration therefore does not enter the ledger; it enters the convention that constitutes the ledger.

Unit-convention layer

The unit-convention layer is constitutive. It declares what counts as one generic action unit. This layer is logically prior to every execution identity, because it constitutes both the total count N ( I ) and the type partition of units into closures, non-closures, and fedback.

Formally, let W ( t ) denote the observed work trace over interval I = [ t a , t b ] . A unit convention U maps this trace into a finite sequence of generic action units and a type map:

U : W ( t ) | I ↦ ( a 1 , a 2 , … , a n ; κ )

where a j is the j th generic action unit and κ assigns each unit to exactly one execution type:

κ ( a j ) ∈ { closure , non-closure , fedback }

The convention therefore constitutes the counted sequence:

N U ( I ) = | A U ( I ) | = n

Time-normalized unit convention Fix a reference length Δ > 0 and tile the time axis into consecutive intervals of length Δ , with the window endpoints I = [ t a , t b ] on the grid. Each Δ -interval inside I is one generic action unit, so that N(I) = |I| / Δ .

The type map κ assigns each interval to exactly one execution type: the interval containing a commitment onset is a closure unit; an interval containing pre-commitment selection activity for an open episode is a non-closure unit attributed to that episode; every other interval is a fedback unit. The count N ( I ) is dimensionless; elapsed time enters only through the choice of Δ, which the next subsection fixes.

The unit length is the closure-act duration

Under a time-normalized convention, the counted capacity N(I) is fixed only after the unit length Δ is fixed. The question is therefore not how many units occurred in the window, but how long one unit lasts. We fix Δ by anchoring it to the closure act, and then refine that anchor by the unit-length procedure. Throughout, the unit length is held constant across all closures in the window.

Let the duration of one closure commitment act be δcl . This is the natural unit-anchor for the channel, just as the keystroke is the unit-anchor for a typing trace: it is the task-defined act that closes one unit of regulatory work.

Set the unit length equal to the duration of one closure commitment act:

Δ := δcl .

Under this convention a closure occupies exactly one Δ by construction. This is not an assumption about closures. It is the choice of clock: the channel is measured in units of one commitment act. By the constant-unit-length assumption, δcl is taken constant across all closures in the window, up to noise, so that Δ is a single value for the window and the count and the elapsed time of the channel are related by:

N(I) = |I| Δ .
Non-closure units inherit the same length

A non-closure action unit is then defined as one Δ of pre-commitment selection activity, at the same temporal grain as the closure unit. Because both a closure unit and a non-closure action unit occupy exactly one Δ , the closure tally and the non-closure tally are denominated in the same unit and add directly in N(I) .

This commensurability is a statement about duration only. It licenses the count, and is independent of how many bits either act carries. In particular, the invalidation re-attribution Q(I) + Wobs(I) adds a non-closure count to a closure count, which is dimensionally honest precisely because each summand is a tally of one- Δ acts. The information content of a closure — generally its residual selection entropy at the moment of commitment, not a fixed one bit — is bounded separately by Proposition 0; any mismatch between the number of Δ- intervals and the number of stages is carried by the unit-inference term ε Δ ( I ) defined below.

The unit-length procedure fixes the closure length

The closure length δcl is not a single number. For any physical task it is recovered from experiment as a distribution p(δcl) over observed commitment durations, with a mean, a spread, and possibly a tail. The unit-length procedure has two jobs: to certify that this spread may be treated as noise, and then to collapse the distribution to the single constant value used as Δ .

Certification. The spread of p(δcl) may be treated as noise — and the constant-unit-length assumption thereby justified — only if no response-relevant discrimination is carried across that spread.

Collapse. Once the spread is certified as noise, the distribution is collapsed to a single representative value taken as the channel unit:

Δ := S [ p(δcl) ] ,

where S is a summary statistic of the observed duration distribution. Only the mean, Δ = E[δcl] , makes the count and the elapsed channel time agree in expectation, so that N(I) = |I| / Δ tracks total elapsed time.

The unit length fixes how many action units the ledger attributes to an episode. That number need not equal the number of stages of the episode's realized conditioning path, and the difference is not an information-theoretic quantity: by D1 the event-specific information functional depends only on the belief pair, never on elapsed time, so a mismatch between intervals and stages is a counting discrepancy and is carried by its own term.

Definition (unit-inference error). For each net surviving closure u ∈ Ω net ( I ) , let k u be the number of stages of its realized conditioning path, counted as in A5, and let n u be the number of non-closure action units the ledger attributes to u, so that Q net ( I ) = ∑ u n u . The unit-inference error of the window is the per-net-closure average

ε Δ ( I ) := 1 S net ( I ) ∑ u ∈ Ω net ( I ) ( n u - k u ) .

Two regimes. Where action units are counted directly, n u = k u and ε Δ ( I ) = 0 identically. Where the capacity is inferred from elapsed time, N ( I ) = | I | / Δ , the quantity n u is the number of Δ -intervals attributed to u other than its closure act, and ε Δ measures how far the interval count departs from the stage count.

Condition CΔ - at most one stage per interval. If no Δ -interval attributed to a net surviving closure contains two or more stages of that closure's conditioning path, then nu≥ku for every u and hence εΔ(I)≥0. A single stage spread over several intervals is admissible and only increases εΔ; what CΔ excludes is two stages inside one interval, which would make the ledger under-count the path. CΔ is what the conditional-increment check of the unit-length procedure certifies.

In query terms, one Δ-interval is one query slot, and condition CΔ says that a slot holds at most one query. A single query may occupy several slots, which only adds to εΔ; what CΔ excludes is two questions inside one slot, which would make the ledger count fewer queries than were asked.

The unit length therefore does two jobs, not one. It fixes the capacity of the window, N(I)=|I|/Δ, and it fixes the resolution at which queries are inferred, through the interval counts nu compared with the stage counts ku in εΔ(I). The order of construction is: calibrate Δ, choose the window I, fix N(I), and only then observe the closures. The closure counts Sgross(I), Wobs(I), and Snet(I) are outputs observed inside a capacity fixed in advance, so the estimator of equation (5) has a constant numerator and a random denominator. This is the setting of Corollary 2. The choice of summary statistic also matters: only Δ=𝔼[δcl] makes N(I) equal in expectation to the number of closure-acts' worth of time in the window. A median-based Δ gives a robust but biased capacity, and the bias passes multiplicatively into K^+1=N(I)/Snet(I).

Accounting layer

The accounting layer is combinatorial. Once a unit convention has been fixed, the execution sequence is no longer underdetermined. The interval I contains a fixed tally of constituted generic action units, and each unit belongs to exactly one execution type. In this layer, durations no longer appear. Duration has already done its work upstream by helping constitute the unit convention.

Dropping the convention subscript when the convention is clear, let:

A ( I ) = C ( I ) ∪˙ Q ( I ) ∪˙ F ( I )

where C ( I ) is the set of closure units, Q ( I ) is the set of non-closure units, and F ( I ) is the set of fedback units. The symbol ∪˙ denotes disjoint union.

Let N(I) denote the fixed generic action unit capacity of window I : the total number of counted generic action units consumed in the window. It is held fixed throughout, and the question is only how that fixed capacity is partitioned.

The corresponding counts are:

N ( I ) = | A ( I ) |
S gross ( I ) = | C ( I ) |
Q ( I ) = | Q ( I ) |
F ( I ) = | F ( I ) |

Therefore the execution-channel partition identity is:

N ( I ) = S gross ( I ) + Q ( I ) + F ( I )

This identity is duration-free. It is not a claim that all physical actions take the same wall-clock time. It is a claim that, under the chosen unit convention, each constituted generic action unit contributes one counted unit to the execution ledger.

The effective non-net-progress load remains the physical non-closure count augmented by the invalidation debit:

Qeff(I) := Q(I) + Wobs(I) .

The inflation of Qeff above Q is therefore exactly the observable invalidation load Wobs , and nothing else. Fedback is never folded into Qeff : a setup or idle unit is not a failed discrimination attempt, and charging it as one would misattribute non-progress.

Re-booking the invalidated closures from the closure side to the load side is a pure re-attribution, so the following identity holds in every window, with or without fedback:

Qeff(I) + Snet(I) = Q(I) + Sgross(I) .

The full capacity, however, may exceed this common value by the fedback it contains. The general capacity partition has three buckets — effective non-net-progress load, net surviving closures, and fedback:

N(I) = Qeff(I) + Snet(I) + F(I) .

Equivalently, using the re-attribution identity above:

N(I) = Q(I) + Sgross(I) + F(I) .

The fedback bucket is therefore the residual capacity left after discrimination work and closures are accounted for:

F(I) := N(I) − Qeff(I) − Snet(I) = N(I) − Q(I) − Sgross(I) .

Fedback units exist, occupy the generic action unit capacity, and consume time, but they carry no response-relevant discrimination and produce no closure.

These identities are tallies and hold for any unit convention and any assignment of durations tj . One further reading, however, is not free of the convention. Reading the ratio Qeff(I) / Snet(I) as an effective depth per closure, and summing Qinv+Wobs in a common load, additionally charges one closure unit at the same channel cost as one non-closure unit. Under the count partition this equal-cost convention is automatic; under any time- or effort-weighted reading it is an assumption and it is exactly the closure-unit-duration choice fixed in the unit-convention layer above.

Operational assumptions

The operational construction uses a fixed observation window:

I = ( ta , tb ] .

The following assumptions define the operational measurement model. They are not epistemological claims about how the regulator thinks. They are coding assumptions about how the externally visible execution trace is counted.

Assumption A1 - Fixed counted channel capacity. The observation window I has a fixed counted generic action capacity N(I) . This is the total number of counted generic action units available and consumed in the window.

Assumption A2 - Immediate fedback. The observation window I contains only non-closure action units and closure units, with no fedback units: F(I)=0 . Thus, every counted generic action unit is classified as exactly one of two types: a non-closure action unit or a closure unit. No counted unit belongs to both classes.

Assumption A3 - Net survival. The net surviving closure count equals the number of gross closures that remain accepted under the relevant window-level evaluation rule. Gross closures that fail to survive consumed capacity but do not contribute to net accepted closure output.

Assumption A4 - Closure counting. Each gross closure event consumes one closure unit when produced. A gross closure may later survive as accepted or be invalidated by the relevant evaluation criterion.

Assumption A5 — Binary queries. Every response-relevant action unit of every closure instantiates a query from the regulator's family Qt whose answer alphabet has at most two values, and the realized conditioning path satisfies P1–P3 with m=2. Equivalently, each stage signal alphabet has size at most two. In the direct-counting regime, one counted residual unit is one conditioning stage. In the time-normalized regime, counted residual units are Δ-intervals and need not coincide one-for-one with conditioning stages. Condition CΔ requires that each such interval contain at most one conditioning stage.

A5 is a statement about the signal alphabet and the conditioning protocol, not about how much information any stage discovers. It does not assert that a stage halves a candidate set, removes one bit, or removes anything at all. By Proposition 0, a binary stage conveys at most one bit in expectation and may realize any signed event-specific amount. The binary restriction is what makes one counted stage commensurable with one bit;.

Under A1-A5, the operational identities below are exact. They do not require access to the regulator's internal knowledge state Mt , to posterior probabilities, or to the internal order of staged selections inside an episode.

Operational action-unit layer

The ideal binary-question-depth layer concerns the minimum expected number of binary distinctions required under an ideal questioning scheme. The operational layer concerns what was actually counted in the execution channel. Under A1-A4, the execution stream is partitioned into counted non-closure action units and gross closure units.

Proposition 1 — Channel partition identity. Under A1–A2, the channel capacity N(I) is exhaustively partitioned into non-closure action units and gross closure units. With the capacity N(I) held fixed and the gross closures Sgross(I) counted, the non-closure count is therefore not a free quantity: it is forced to be the residual capacity left after the closures are counted:,

Q(I) = N(I) - Sgross(I) .

This is a pure accounting identity: on a single fixed-capacity channel, every counted unit is either a non-closure action unit or a gross closure unit, and Q(I) counts the former by tallying action units, with no recoding.

Proof. By A2, every counted unit is either a non-closure action unit or a closure unit, F ( I ) = 0 . By A4, gross closures are the closure units. Equivalently the fedback count vanishes, F ( I ) := N ( I ) − Q eff ( I ) − S net ( I ) = 0 , so the fixed capacity is exhausted by discrimination work and closures: N ( I ) = Q eff ( I ) + S net ( I ) = Q ( I ) + S gross ( I ) . Therefore the non-closure count equals total counted capacity minus gross closure count.

The black-box constraint matters here: under this observation model we can attribute each atomic unit to either the closure stream or the non-closure stream, but we cannot see the internal staged structure inside an episode.

The execution channel makes Q and Sgross observable; the operational ledger then makes W and Snet observable as well.

Observable invalidation load and net surviving closures

For the quantification below, the relevant observable invalidation load Wobs(I) is the gross closures in window I that fail to survive as net accepted closures.

By A3:

Snet(I) = Sgross(I) - Wobs(I) .

This identity is operational, not entropic. It partitions gross closure work into net surviving closure work and observable invalidation load.

The counted channel capacity N(I) is held fixed throughout. Under A1–A2 and A4, the physical unit count of the channel admits exactly one physical partition into generic action units, namely Proposition 1:

N(I) = Q(I) + Sgross(I) (physical partition) .

In this partition each of the Wobs(I) invalidated closures is counted once, as a consumed gross-closure unit inside Sgross(I) . The invalidated closures are real units; they occupy capacity on the channel.

The effective non-net-progress load in window I is:

Qeff(I) := Q(I) + Wobs(I) .

Qeff(I) is not a count of channel units. It is the physical non-closure count Q(I) augmented by a one-unit non-progress debit for each invalidated gross closure. By construction Qeff(I) ≥ Q(I) , with equality if and only if Wobs(I) =0 ; whenever invalidation occurs, Qeff(I) strictly exceeds the number of non-closure units actually present on the channel.

Combining the definition with Snet = Sgross − Wobs :

Qeff(I) + Snet(I) = ( Q+Wobs ) + ( Sgross−Wobs ) = Q+Sgross = N(I) .

So the same fixed capacity N(I) admits a second, effective partition:

N(I) = Qeff(I) + Snet(I) (effective progress partition) .

The two partitions of N(I) differ only in the bookkeeping of the invalidated closures, and N(I) itself is unchanged.

Proposition 1′ - Survival and boundary partition of the effective load. Four load quantities enter the partition.

  • Qnet(I) and Qinv(I) are the total non-closure units of the net surviving and of the invalidated closures of the window.
  • Qbd-(I) counts the non-closure units of the leading episode that lie before ta.
  • Qbd+(I) counts the non-closure units inside I that belong to the episode still open at tb.

Then

Qeff = Qnet + Qinv + Wobs + Qbd+ - Qbd- .

At most one episode straddles t a ; call it the leading episode, and let Q bd − ( I ) be its non-closure units lying before t a , with Q bd − ( I ) := 0 when no episode straddles t a . The leading episode's units are counted in full - pre-window units included - in whichever of Q net − ( I ) or Q inv − ( I ) its closure belongs to, and Q bd − ( I ) then removes those pre-window units from the window total. The identity below is therefore insensitive to whether the leading episode survives: if it does, Qnet (I)= ∑ u∈Ωnet(I) nu as in the definition of the unit-inference error; if it does not, the same full count is carried by Q inv ( I ) while Ω net ( I ) omits the episode entirely.

Counting in full is what keeps nu and ku commensurable for a leading episode that survives. Truncating its units at t a would leave nu short of Qbd⁻, Q bd − ( I ) ⁻, making εΔ(I) negative even under CΔ.

The invalidation re-attribution per net closure,

εinv(I) := Qinv(I)+Wobs(I) Snet(I) ≥ Wobs(I) Snet(I) ≥ 0 ,

is nonnegative with an observable lower bound. The boundary re-attribution per net closure,

εbd(I) := Qbd+(I)-Qbd-(I) Snet(I) , - Qbd- Snet ≤ εbd ≤ Qbd+ Snet ,

is signed, and both of its bounds are observable from closure timestamps. The loads are the units between the last closure before ta and ta, and between the last closure in I and tb. If n¯ bounds the units attributed to any single episode, both loads lie in [0,n¯] . For consecutive windows in which no episode spans a whole window, Qbd-(Im+1) = Qbd+(Im) , so the boundary loads telescope across a sliding-window series. Under the strict black-box regime, Qinv is not separately observed; only the lower bound on εinv is.

Proof. Under A2 and A4, every non-closure unit in I belongs to exactly one of three episode kinds: an episode whose closure is net surviving, one whose closure is invalidated, or the episode open at tb. Counting the leading episode in full adds Qbd- units lying outside I, which are subtracted. The definition Qeff=Q+Wobs completes the identity. The bounds on the boundary loads hold because the open episode's in-window units, and the leading episode's pre-window units, are each at most that episode's own unit count. □

Survival of the leading episode moves capacity between the two re-attributions. When it is invalidated, Q inv ( I ) carries its pre-window units and − Q bd − ( I ) credits them back, so ε inv ( I ) is larger and ε bd ( I ) smaller by Q bd − ( I ) / S net ( I ) than in the surviving case. Their sum is unchanged, as are the bounds above, since the lower bound on ε inv ( I ) uses only Q inv ( I ) ≥ 0 and ε bd ( I ) is defined from the positional quantities Q bd + ( I ) and Q bd − ( I ) alone. What changes is the reading: with the leading episode invalidated, neither term separately counts only capacity inside I, though Q net + Q inv + W obs + Q bd + − Q bd − still does.

The record as a two-instruction program

The effective partition of the execution channel has a direct reading as a program. Under A1–A4 every counted unit is either a net surviving closure or one of the units charged to the effective load, and nothing else. This subsection states that reading, fixes the single meaning of “one-bit” used in the rest of the article, separates the program from the answers the regulator receives while running it, and says how this program differs from a program in algorithmic information theory.

The program

Definition (program). Write the N(I) counted units of window I in time order, as a 1 at each unit that is a net surviving closure and as a 0 at every other unit. The resulting binary string is the program of the window. A 1 is an output instruction: it emits an acceptable closure that survives the window. A 0 is a non-output instruction: a step of finding out what to emit next, or a unit that the effective partition charges as load. An invalidated closure keeps its position in the string and is written as a 0, which is exactly the re-attribution of the effective partition.

By the effective partition,

N(I) = Snet(I)⏟number of 1s + Qeff(I)⏟number of 0s .

Lifting outputs to response-equivalence classes lets their content drop out: the program records that an acceptable closure was emitted, not which one. What the program does not settle is how its count relates to bits of Knowledge To Be Discovered. That is the bridge of Theorem 2, and it depends on what a 0 holds.

The meaning of “one-bit”

Each instruction is one bit of program text, so the program of window I is N(I) bits long. This is the only sense in which the estimator of equation (5) is called one-bit: it counts bits of program, one per counted unit. It does not assert that any unit removes one bit of uncertainty.

Nor is a bit of program a bit of description in the sense of algorithmic information theory. The program is a record of what the regulator did, not a compressed description, and it can itself be compressible: a run of lookups followed by a run of outputs has a description much shorter than its length. Recorded only as acceptable closures, as the program records them, the output is Snet(I) repetitions of one symbol, which needs only about log2 Snet(I) bits to describe. The length of the program measures the effort spent producing the output, not the information the output contains.

A bit of program and a bit of information are related only through the conditions of Theorem 2. Under A5, a 0 that belongs to a conditioning path carries one binary query. Under condition CΔ, no counted unit carries more than one. When units are counted directly, one 0 is one query. By Proposition 0, a binary query conveys at most one bit in expectation. Under these conditions, each program bit spent on a non-output instruction accounts for at most one bit of Knowledge To Be Discovered in expectation, which is what the upper bounds of Theorem 2 use.

The two kinds of bit coincide only in the ideal regime of Theorem 2(iv): direct counting, binary queries, and every query splitting the posterior evenly. When a query has an answer alphabet of size m, one program bit accounts for at most log2m bits, by the wider-alphabet remark after Theorem 2. When condition CΔ fails, a program bit may hide several queries, and no bound follows.

For a human regulator, a program bit can be bounded without counting what it holds. Under assumption H1, introduced with the ideality assumptions below, the non-closure time of an episode accounts for at most ch=ChΔ bits per interval, on average, of information about that episode's required class, whatever the size of its answer alphabets and however many queries its intervals hide. A human's program bit is then bounded by the human's information throughput rather than by the form of the query (Corollary H).

Folding: four kinds of unit into two instructions

The ledger knows four kinds of unit, so the unfolded program has four instructions: emit (a closure that survives), find out (a step of discrimination), wait (fedback: idle observation, feedback latency, or physical execution), and retract (a closure later invalidated). The two-instruction program folds the last three into 0. Assumption A2 folds every wait into a 0, by booking any unit that is not a closure as a non-closure unit. The effective partition folds every retraction into a 0.

Folding keeps the count exact. What it changes is the bridge to bits. A folded wait is a unit that discovers nothing; it enters the redundancy or the unit-inference error, loosening the upper bound of Theorem 2 without invalidating it. A folded retraction is treated in the last part of this subsection. Section 9 states, for each application, which kinds of unit it folds; the appendix to the realized typing trace lists them side by side.

The program and the answers

A program records which instructions were executed, not what they returned. When a non-output instruction is a query, its answer comes from outside the program: from the environment, a sensor, a script, or the regulator's memory. The answers are not part of the program. Two binary records of the same window must therefore be kept apart.

  • The program defined above. Its length is N(I) , and its symbols mean output and non-output.
  • The answer transcript: the realized signal values uu,1 … uu,ku of the conditioning paths, concatenated over episodes. Its length is the total number of stages, and its symbols are answer values.

Under direct counting, with no waits and no retractions, the number of 0s in the program equals the number of stages, up to the boundary loads of Proposition 1′. The two records then have the same length of discrimination, but not the same content. The information obtained is bounded by the answer transcript, through Corollary 0′, and not by the program: a single lookup that returns a chunk of letters is one 0 carrying many bits. The typing traces in this article write the program; the robot example in the appendix writes the answer transcript.

When the regulator is human, the answers are bounded a second way. The answer transcript may record a chunk of letters per lookup, but under H1 the information the lookup delivers about the letter in whose episode it occurs is at most ch bits per non-closure interval, on average. What it registers about later letters is not Knowledge To Be Discovered of this episode: it is episode-start information of the later episodes and lowers their starting Knowledge To Be Discovered. The bound concerns what the typist registers about the required class, not what the script displays.

Remark — This program and program size in algorithmic information theory. The word program is used here in a narrower sense than in algorithmic information theory. There a program is a self-delimiting description for a fixed universal machine, and the program-size complexity of an output is the length of the shortest description that produces it and halts [81]. Here the machine is one specific, non-universal regulator, the program is the one it actually ran, and nothing is minimised. The program is straight-line, with no loops, so its length is also the number of steps the regulator executed. Its length is therefore a step count, closer to a running time than to a program size, and it is not self-delimiting.

The two senses share one feature: an oracle's answers are not part of the program. They part on the queries. In algorithmic information theory a program relative to an oracle may consult it as often as it likes at no cost in size. Here every consultation is a non-output instruction, and every one is counted. It is this departure, not the shared feature, that makes the length of the program a query complexity: it counts how often the regulator had to go outside itself. Kolmogorov complexity enters the article elsewhere: at the level of whole answer transcripts, where it meets the entropy rate (Brudno's theorem, in the robot example of the appendix), and in the lower-bound methods for query complexity that rest on Kolmogorov arguments [78].

Failed and retracted outputs

An output can fail in two ways, and the ledger books them differently. A failure at production is a closure act whose outcome is unacceptable when it is committed: a missed shot, a wrong test answer. It is not a gross closure, and since Q(I)= N(I)− Sgross(I) , it is a non-closure unit and a 0. A retraction is a closure accepted when produced and invalidated later: a provisionally accepted letter rejected on checking. It is a gross closure, counted in Wobs(I) , and the effective partition writes it as a 0.

Proposition 1′ and Theorem 2 assign the units of a retracted closure to its own episode, and Corollary 3(iii) credits the estimator with the observed invalidation fraction Wobs/Snet . That is right when the failed output tells the regulator nothing about the closures that survive, as under criterion revision.

Remark — When a failed or retracted output is a query. Often the failed output is a trial for the same target as a surviving closure: a wrong letter followed by the right one for the same position, a miss followed by a make, a wrong answer followed by a corrected one. The rejection then answers “not this” for that target, and it is a stage of the surviving closure's conditioning path. Three consequences follow.

  1. The failed output belongs in the stage count ku of the surviving episode, not in a separate episode. For a retraction, the invalidation term εinv shrinks by the same amount.
  2. For a retraction, the credit Wobs/Snet in Corollary 3(iii) is not available for that closure. A failure at production was never in Wobs, so nothing changes there.
  3. The upper bound of Theorem 2 remains, with the average terminal residual of the graded-uncertainty remark added, because each trial was committed before the regulator could be sure of it. When the rejection answers only “wrong”, the stage is binary and A5 holds for it.

The two kinds of failure differ only in when the failure is recorded: at production for a miss, at checking for a retracted letter. The goal-model typing trace in the appendix is the worked case of retraction; basketball and intelligence testing in section 9 are the cases of failure at production.

The non-output count of the surviving closures

The non-output count Qeff(I)= N(I)− Snet(I) is inferred, not observed. Under A5, each non-output instruction of a surviving episode is one stage of that closure's conditioning path or, under time-normalized counting, one Δ-interval containing at most one such stage. By Proposition 1′ and A5, the non-output count attributed to the net surviving population is the sum of their realized stage counts,

Qnet (I) = ∑ u∈ Ωnet(I) ku + Snet (I) ⋅ εΔ (I) ,

where the unit-inference term εΔ(I) is identically zero when units are counted directly and equals the per-net-closure excess of Δ-intervals over stages when they are inferred from elapsed time. In source-coding terms, the non-output count is not a code length but a realized number of code symbols, and the entropy bound on prefix codes (Corollary 0′) is what makes it an upper bound on starting Knowledge To Be Discovered in expectation. The even-split regime is the case in which that bound is tight (Lemma 1′).

Operational one-bit effective-depth estimator.

Under A5 each inferred residual unit of Qeff(I)=N(I)-Snet(I) is one stage of some closure's realized conditioning path, or, under time-normalized counting, one Δ-interval containing at most one such stage. No bit-equivalence is assigned to it: by Proposition 0 a binary unit conveys at most one bit in expectation and may realize any signed amount, and the residual is a count of units rather than a charge.

Definition (Fixed-gross no-invalidation baseline). The baseline is a contrast operator on observables, not a causal counterfactual. Hold fixed the observed channel capacity N(I) and the observed gross closure count Sgross(I) fixed, and set the observable invalidation load to zero, Wobs(I) :=0 , so that Snet(I) = Sgross(I) . The baseline question depth per closure is then the average number of non-closure atomic units per gross closure:

KTD^ base 1bit (I) := Q(I) Sgross(I) = N(I)-Sgross(I) Sgross(I) − 1 := N(I) Sgross(I) - 1 .

(4)

The baseline reads the physical partition of the channel, which counts the invalidated closures as closures. Because it is defined by fixing observables and zeroing the invalidation load, equation (4) is exact as an operational count by construction. It is not a counterfactual estimate of what would have happened absent invalidation; it is the value of the same operational ratio evaluated at Wobs=0 . This removes any need to defend the baseline as a causal alternative, and it supplies the fixed-gross reference against which the effect of observable invalidation load is measured below.

Definition (Fixed-net estimator). The effective depth means the estimated average number of counted action units per surviving closure. The operational one-bit estimator for average effective depth per net surviving closure is:

KTD^ eff net (I) := Qeff(I) Snet(I) = N(I)-Snet(I) Snet(I) = N(I) Snet(I) - 1 .

(5)

The net estimator reads the effective partition of the channel, which counts the invalidated closures as load. By construction, it is also exact as an operational count. Equation (5) is exact as an operational estimator of average effective one-bit action-unit depth. For the estimator to be defined, the relevant window must satisfy: Sgross(I) > 0 , Snet(I) > 0 , 0 ≤ Wobs(I) < Sgross(I) , and N(I) ≥ Sgross(I) .

Both partitions of the channel divide the same fixed N(I) ; they differ only in whether the invalidated closures are counted as closures (baseline) or as load (net estimator).

In program terms, KTD^effnet(I) is the number of non-output instructions per output instruction in the program of the subsection “The record as a two-instruction program”: each counted non-output unit is one 0, and each invalidated closure is written as one 0. “One-bit” is the program bit of that subsection: one instruction, one bit of program text. It is not one bit removed. Under A5 and CΔ a 0 carries at most one binary query, and a binary query removes at most one bit in expectation (Proposition 0) and may remove any signed amount on a realized path. Under direct counting, with no invalidations and no stage cut by the window boundary, each 0 is one binary query of a surviving episode, and the estimator equals the average number of queries per surviving closure; in general the two differ by the error terms εinv+εΔ+εbd of Theorem 2. The estimator counts instructions; Theorem 2 below says when, and with which errors, that count bounds the bits.

Invalidation monotonicity theorem

The increase in effective depth due to observable invalidation load is defined as the difference between the net operational estimator and the no-observable-invalidation baseline of equation (4):

ΔKTD^ W 1bit (I) := KTD^ eff net (I) - KTD^ base 1bit (I) .

Substituting equations (4) and (5) gives:

ΔKTD^ W 1bit (I) = N (I) ( 1 Snet(I) - 1 Sgross(I) ) .

Using Snet(I) = Sgross(I) - Wobs(I) , this becomes:

ΔKTD^ W 1bit (I) = N (I) Wobs(I) Sgross(I) ( Sgross(I) - Wobs(I) ) .

Equivalently:

ΔKTD^ W 1bit (I) = N (I) Wobs(I) Sgross(I) Snet(I) = N (I) Wobs(I) Snet(I) + Wobs(I) Snet(I)

Theorem 1 - Invalidation monotonicity. Fix N(I) and Sgross(I) with N(I) ≥ Sgross(I) > 0 . Regard the invalidation-induced increase in effective depth as a function of the observable invalidation load W= Wobs(I) on the domain 0 ≤ W < Sgross(I) . Then on this domain the quantity ΔKTD^ W 1bit (I) is (i) nonnegative, and equals zero if and only if W=0 ; and (ii) strictly increasing in W .

Proof. Write G= Sgross(I) and c= N(I)/G >0 , so that ΔKTD^ W 1bit (I) = c WG-W . At W=0 the expression is zero, and it is positive for W>0 , which yields (i). Differentiating g ( W ) = W / ( G - W ) gives g′(W) = G/ (G-W)2 >0 , which yields (ii).

Corollary 1 - Increasing marginal invalidation cost. For fixed N(I) and Sgross(I) , the invalidation-induced increase in effective depth is strictly convex in W .. Thus each additional invalidated closure has a larger marginal effect than the previous one.

Proof. The second derivative g′′(W) = 2G/ (G-W)3 >0 on 0≤W<G is positive. Hence the invalidation burden grows faster than linearly as observable invalidation load increases.

This corollary is the operational amplification result. Invalidation does not merely subtract completed work. It reduces the denominator of surviving closures, so the same fixed channel capacity is spread over fewer accepted closures. The effective burden per surviving closure therefore rises nonlinearly with the invalidation fraction.

Theorem 1 is purely operational relative to A1-A4; it is a statement about the closed-form behavior of the accounting estimator alone. The convexity claim sharpens the earlier monotonicity remark - not only does each invalidated closure raise the effective depth, but successive invalidations raise it by strictly increasing increments, because the surviving denominator G-W shrinks as W grows.

The interpretation is narrow and operational. Observable invalidation load does not directly measure internal entropy. It increases the effective one-bit question depth because the same fixed channel capacity is now amortized over fewer surviving closures. Some closure-fixing acts consumed capacity but did not remain accepted at the end of the window. This is a selection-equivalent debit, not a direct count of completed corrective rework.

Invalidation fraction form

The same result can be expressed in terms of the observable invalidation fraction:

w(I) := Wobs(I) Sgross(I) .

Since Snet = Sgross (1-w) , equation (5) becomes:

KTD^ eff net (I) + 1 = KTD^ base 1bit (I) + 1 1-w(I) .

Therefore:

ΔKTD^ W 1bit (I) = ( KTD^ base 1bit (I) + 1 ) w(I) 1-w(I) .

This form exposes the amplification mechanism. As the invalidation fraction approaches one, the effective burden per surviving closure grows without bound. Operationally, this means that a process may consume large amounts of channel capacity while producing very few surviving closures.

Ideality assumptions

The operational estimator becomes an estimator of the latent Knowledge To Be Discovered only under explicit bridging assumptions. We collect them here so that every conditional claim below can name exactly which assumptions it requires. The accounting identities of the preceding subsections, and Proposition 1, hold unconditionally and do not depend on (B1)–(B5).

B1(i) Bounded upper search gap. For the two-sided envelope, assume that the window-level expected search gap satisfies εred (I) ≤ 1 + r for a declared tolerance r≥0 . Under the sharp certainty condition B2 below, together with the deterministic-tree premise of Lemma 1′, a sufficient per-closure condition is gu ≤ r , because then εred = 1 Snet ∑ u ( gu + δu ) < 1 + r . If terminal certainty is relaxed as described below, the same upper bound may still be imposed as a declared tolerance, but it is no longer supplied by the Huffman interpretation of Lemma 1′.

By Lemma 1″, the deterministic-tree gap is δu+gu = D(p0,u‖p~u) + log2(1/vu) . An equivalent per-closure form of B1(i), stated without separating the latent δu from gu, is therefore

D(p0,u‖p~u) + log2 1vu < 1 + r .

It bounds split unevenness and null-stage slack jointly, and it implies εred(I)<1+r directly.

B1(i-a) Query-family adequacy. For every net surviving row, the query-family gap satisfies ψu≤rQ for a declared tolerance rQ≥0. In particular the family can close the row safely, ψu<∞. When the family is unrestricted, rQ=0.

B1(i-b) Protocol near-optimality. For every net surviving row, the protocol gap relative to the available questions satisfies guQ≤rP for a declared tolerance rP≥0.

With deterministic stage signals, B1(i-a) and B1(i-b) together imply the per-row condition gu≤r of B1(i), with

r=rQ+rP,

because gu=ψu+guQ. The split adds no assumption. It records which part of the tolerance concerns the regulator's questions and which concerns its use of them, and so which adaptation channel, L2b or L2c, would narrow it.

B1(ii) Bounded unit-inference error. Either conditioning stages are counted directly, so that εΔ (I) = 0 , or the window satisfies condition CΔ together with the declared tolerance 0 ≤ εΔ (I) ≤ cΔ .

B2 - Certainty-terminated protocol with complete stage accounting. For every episode generated by the measured protocol class, conditional on its disturbance Y=yu , the protocol terminates only when no response-relevant uncertainty remains at the regulatory granularity represented by X . Equivalently,

𝔼u [ H ( pKu ) ] = 0 .

Because entropy is nonnegative, this is equivalent to H ( pKu ) = 0 almost surely under Mu . By the Bridge Lemma, an equivalent set-level statement is that the terminal epistemic response-candidate set projects to one response-equivalence class:

| gu [ C u,Ku R,epi ] | = 1

B2 is a design-time property of the protocol class, not a criterion used to filter realized closures after the fact. The episode-closing commitment performs no additional uncounted response-relevant discrimination: every conditioning stage required by the protocol before terminal certainty is represented in the counted conditioning path. Thus A5 establishes the semantic validity of the conditioning stages to be represented by the counted units, while B2 supplies the reverse coverage condition for each response-relevant conditioning stage to be counted in the path. Under time-normalized counting, CΔ additionally ensures that one inferred Δ-interval contains at most one stage, yielding nu ≥ ku and therefore εΔ (I) ≥ 0 .

B2 in relational form. When several response classes may be acceptable for the same disturbance, B2 is read as relational certainty termination: on every path, the class the protocol fixes is acceptable for every disturbance in the posterior support of that path. By the relational Bridge Lemma, this means that terminal relational Knowledge To Be Discovered is zero on every path and the protocol fixes a class that is safe for all disturbances still possible. Uncertainty among classes that are equally acceptable is not required to be resolved. In the unique-target case this is the singleton condition above. B2 in this form can fail for a reason the unique-target reading hides: if the regulator's query family is inadequate for a row in the sense of Proposition Q4, no protocol over that family can satisfy B2 for the row.

B3 Invalidation units are counted units. Units spent on gross closures that fail to survive, together with the closure unit itself, are counted capacity: each is a 0 of the program of “The record as a two-instruction program”. They enter the net estimator through εinv (I) ≥ 0 (Proposition 1′). When an invalidated closure is a trial for the same target as a surviving closure, its units are instead stages of that closure's conditioning path and enter ku; see “Failed and retracted outputs”. No amount of information is asserted for them.

B4 - Population matching. The counted closure population and the knowledge-ledger row population coincide for the window, so that the operational and latent quantities refer to the same closures. This matching is by row identity and does not select closures according to whether their realized traces appear certainty-terminated. Terminal certainty, when assumed, is supplied ex ante by B2 as a property of the protocol class.

B5 - Sequential path calibration. Index all gross episodes that intersect the window by order of start, j=1,2,… . The index includes the leading episode that began before ta, and the sequence continues past tb with the regulator's subsequent episodes. Let Fj-1 be the sigma-field generated by three sets of quantities: all realized quantities of episodes 1,…,j-1 (disturbance information, knowledge states, action units, signals, stage counts ki, attributed unit counts ni, closure times); the pre-window boundary load Qbd-(I) ; and the start state (yj,Mj) of episode j. Write ξj := kj - 𝔼j[Kj] . B5 requires

𝔼 [ ξj ∣ Fj-1 ] = 0 and aj ≤ ξj ≤ aj + d ,

for some Fj-1 -measurable aj and a declared constant d≥0. Because the conditional mean is zero, aj≤0≤aj+d , and therefore |ξj|≤d . Under deterministic trees without null stages, Lemma 1″(c) gives d≤M-2 , with M:=maxuMu the largest posterior support size, distinct from the stage alphabet size m.

B5 is a sequential calibration condition that conditions only on the past, not on the realized population of the window. This distinction matters because the capacity N(I) is a constant while the closure counts are random. By Proposition 1′, the completed episodes exhaust the capacity up to the boundary loads, so the size of the completed population is determined by the path lengths. In the direct-counting, invalidation-free case with no boundary load, ζ(I) is a function of Snet(I) and the row models alone. Centering the path fluctuation conditionally on the realized population would then force ζ(I)=0 almost surely. Under B5, Snet(I) remains random and the window end becomes a stopping time (Corollary 2).

H1 — Human intake capacity (human regulators only). This assumption is used only when the regulator is a human being, and only on the upper side of the bracket. It does not replace A5 and does not change what a stage or a query is. A lookup may still return an answer from a large alphabet, and A5 still fails for it. What H1 bounds is not the alphabet of an answer but the amount of information about the required response that a human regulator can take in per unit of time.

Let Ch denote the human information throughput: a declared constant, in bits per unit time. For episode u, with episode-start information Y=yu and unique required class Xu, let Tu be the non-closure time attributed to the episode, and let U≤Ku be the complete stopped transcript of what the regulator registers during that time, including any informative timing or stopping. H1 has two clauses.

  • (H1-a) Capacity. For every starting state with finite expected non-closure time,
    IMu (Xu;U≤Ku∣Y=yu) ≤Ch𝔼u[Tu∣Y=yu].
    The bound concerns information about the required class, not about the whole disturbance. The class is a function of the disturbance, so this is weaker than bounding I(D;U≤Ku∣Y=yu), and any task argued to satisfy the stronger form satisfies this one. In this form H1 says nothing about what a lookup registers about later closures; that information enters their episode-start information. The conditional requirement for each starting state is part of H1; an unconditional empirical average does not establish it.
  • (H1-b) Commitment on registered information. All information about Xu that the regulator obtains after the episode starts arrives through U≤Ku during the attributed non-closure time; none is taken in during the episode's own closure act. Information available when the episode starts, including what was registered during earlier closure acts, as when a typist reads ahead, is episode-start information, not intake of this episode.

Time attribution. Under the time-normalized convention the attributed time is Tu=ΔNunc, where Nunc is the random number of non-closure Δ-intervals attributed to episode u under Mu, and nu is its realized value. Query stages keep their meaning: an interval may contain several queries, part of one, or none. H1 therefore enters every result only through the per-interval capacity

ch:=Ch⋅Δbits per interval.

Intervals in which the regulator does nothing response-relevant, idle time, may be attributed to the episode they precede. They raise Tu without adding information, so they loosen every bound below and do not violate H1-a. Information taken in during earlier closure acts is not a violation either: it is booked as episode-start information of the episodes it concerns (see the subsection after Proposition H‴).

A lookup may register information about several later closures, as a look at a script shows the letters that follow. Its intervals belong to the episode in which it occurs. What it registers about later closures is part of their episode-start information; the subsection after Proposition H‴ sets out how this is booked.

The value of Ch. Ch is treated as a constant to be revised as research changes. Current estimates differ according to the bottleneck they measure: about 3 to 4 bits per second for information processed through cognitive control [32], and about 10 bits per second for human behavioral throughput, typing included [58]. Choosing which estimate applies is part of declaring H1 for a task: the lookups of that task must be argued to pass through the bottleneck whose capacity is declared. The cognitive-control estimate was obtained in a perceptual decision-making task, and its transfer to typing is an assumption, not a finding of [32].

No result below depends on the numerical value of Ch. A revised estimate changes ch and leaves every ledger count unchanged. Δ is not re-declared to follow Ch: it stays anchored on the duration of a closure act, as the unit convention requires, and it is fixed before a trace is read. A coefficient ch≠1 is carried explicitly rather than absorbed into the unit. For the value Δ=0.288 seconds used in the human applications of section 9:

Declared Ch (bits/s) ch at Δ=0.288 s (bits per interval) Source of the estimate
30.864Cognitive control, lower estimate [32]
3.471.000The slot rate declared for software development
41.152Cognitive control, upper estimate [32]
102.880Human behavioral throughput [58]

Relation between the operational estimator and latent KTD

For each net surviving closure u ∈ Ωnet (I) , let ku be the realized number of conditioning stages and let Ku be the corresponding path-length random variable under the fixed episode model Mu . Define

ζ (I) = 1 Snet ∑ u ∈ Ωnet (I) ( ku − 𝔼u [ Ku ] ) ,
ϵred (I) = 1 Snet ∑ u ∈ Ωnet (I) ( 𝔼u [ Ku ] − H ( p 0,u ) ) .

When the goal is set-valued, H(p0,u) is read as KTDrel(yu) and B2 in its relational form. The theorem then holds as stated except where its proof uses the deterministic-tree lemmas; the remark following the theorem gives those parts in relational form.

Theorem 2 (fixed-window decomposition and non-probabilistic envelope). Let Snet(I)>0 , and write K^ := KTD^effnet(I) , K∗ := KTDavgstart-real,net(I) . All window quantities are evaluated at I, and the argument (I) is suppressed below. Each part states the hypotheses it uses; they are cumulative in the order given.

(i) Exact realized decomposition. Assume A2-A4 and B4. Then for every fixed observation window I,

K^ - K∗ = ζ + εred + εinv + εΔ + εbd .

This is a realized fixed-window identity between defined quantities. It invokes no bound on any error term: no alphabet size, no protocol optimality, no terminal-certainty condition, and no claim about the informational content of a counted unit. In particular, because εΔ is defined as the residual average of ( n u − k u ) , the identity holds whatever the relation between counted units and conditioning stages.

(ii) Certainty-terminated bracket. Assume in addition A5 and B2.. Then Corollary 0′, applied row by row under the corresponding Mu, gives H(p0,u) ≤ 𝔼u[Ku] , hence εred≥0 and

K∗ ≤ K^ - ζ - εinv - εΔ - εbd .

If, in addition, every row satisfies the deterministic-tree hypothesis of Lemma 1′, then Lemma 1′(ii) gives, for each u∈Ωnet(I) , 𝔼u[Ku] - H(p0,u) = δu+gu with 0≤δu<1 and gu≥0 . Two cases then arise, according to whether the search gap is bounded row by row or only at the level of the window.

(ii-a) Per-row search-gap bound. Assume the per-row condition gu≤r for every u∈Ωnet(I) , which is the sufficient condition offered under B1(i). Then 0≤εred<1+r , and therefore

K^ -ζ -εinv -εΔ -εbd -1-r < K∗ ≤ K^ -ζ -εinv -εΔ -εbd .

(ii-b) Window-level search-gap bound. Assume instead only B1(i) in its stated window-level form, εred(I) ≤1+r . The same substitution then gives the bracket with the left inequality weakened to non-strict:

K^ -ζ -εinv -εΔ -εbd -1-r ≤ K∗ ≤ K^ -ζ -εinv -εΔ -εbd .

The conclusion of (ii-a) may also be reached from the Kraft decomposition of Lemma 1″, without separating δu from gu: if D(p0,u‖p~u) + log2 1vu <1+r for every row, then εred(I)<1+r and the strict bracket of (ii-a) holds. This is an alternative sufficient condition, not a restatement of gu≤r : the Kraft form bounds the two components jointly, so it is implied by the per-row condition together with δu<1 only when the resulting sum stays below 1+r, and it does not imply the per-row condition.

(iii) Count-level envelope retaining the realized path deviation. Assume the hypotheses of the two-sided bracket in part (ii), condition CΔ, and B1(ii), so that 0≤εΔ≤cΔ ; A5 is what gives εΔ a sign and a bound. Let ε¯inv be an observed or declared upper bound on the invalidation re-attribution. Using the bounds of Proposition 1′ on εinv and εbd, the latent average satisfies the non-probabilistic, not observable fixed-window envelope

K^ -ζ -ε¯inv -cΔ - Qbd+Snet -1-r < K∗ ≤ K^ -ζ - WobsSnet + Qbd-Snet .

The term ζ remains in both endpoints. The fixed-window accounting identity alone supplies no probabilistic bound on its realized magnitude; Corollary 2 supplies a high-probability envelope under B5.

(iv) Even-split regime. Assume (ii) and the deterministic-tree hypothesis of Lemma 1′(ii), so that each net surviving row's conditioning path is a binary decision tree for X with leaf depths ℓu . Suppose further that for every such row no stage is epistemically null and every stage splits the current posterior evenly - in the notation of Lemma 1″, vu=1 and p0,u = p~u . Then ℓu(x) = - log2 p0,u (x) , each p0,u is dyadic, and Lemma 1″(a)-(b) give εu=0 for every row, hence εred(I)=0 . Part (i) then reduces to

K^ - K∗ = ζ +εinv +εΔ +εbd .

If action units are counted directly, then εΔ=0 . If, in addition, no gross closure is invalidated in the window, then

K^ - K∗ = ζ + εbd .

No convergence of ζ to zero is asserted by this theorem.

Remark - determinism is not an additional restriction. Under B2, P1-P3 and the binary stage alphabets fixed by A5, Corollary 0′ gives, for every net surviving row,

H (p0,u) = IMu ( X; U≤Ku ∣ Y=yu ) ≤ H ( U≤Ku ∣ Y=yu ) ≤ 𝔼u [Ku] .

Since εu≥0 by Lemma 1′(i), the condition εred(I)=0 forces εu=0 for every net surviving row, and therefore equality throughout the display. Equality on the left requires H ( U≤Ku ∣ X, Y=yu ) =0 , that is, the realized answer string is a deterministic function of X given the disturbance information: this is exactly the deterministic-tree hypothesis of Lemma 1′(ii). Equality on the right requires the realized strings to satisfy P(s) = 2-ℓ(s) , which in the notation of Lemma 1″ is vu=1 together with p0,u = p~u . The hypotheses of part (iv) are therefore necessary as well as sufficient for its conclusion, so assuming determinism restricts nothing at the equality point: any regime in which the even-split conclusion holds satisfies the hypothesis automatically.

Proof. By equation (5) and Proposition 1′, K^ = ( Qnet+Qinv+Wobs +Qbd+-Qbd- ) / Snet . By the display in Reading the inferred residual, Qnet = ∑uku + SnetεΔ . Hence

K^ = 1Snet ∑u∈Ωnet ku +εinv +εΔ +εbd .

Adding and subtracting 1Snet ∑ 𝔼u[Ku] and using B4 together with K∗ = 1Snet ∑ H(p0,u) gives part (i) directly from the definitions of ζ and εred.

For part (ii), under B2 every net surviving closure terminates at certainty and, under A5, its realized conditioning path is binary. Therefore Lemma 1′(i) gives H(p0,u) ≤ 𝔼u[Ku] ; averaging over the matched population gives εred≥0 .

For (ii-a), the deterministic-tree hypothesis and Lemma 1′(ii) give 𝔼u[Ku] - H(p0,u) = δu+gu with 0≤δu<1 and gu≥0 ; the per-row hypothesis gu≤r then gives δu+gu <1+r for every row, and averaging over the matched population of Snet(I)>0 rows preserves the strict inequality, so 0≤εred<1+r . For (ii-b) the window-level hypothesis is the upper bound εred≤1+r itself, and the lower bound εred≥0 was established above from B2. Substituting the respective bounds into part (i) gives the two brackets.

For part (iii), condition CΔ and B1(ii) give 0≤εΔ≤cΔ . Proposition 1′ gives Wobs/Snet ≤εinv≤ ε¯inv and -Qbd-/Snet ≤εbd≤ Qbd+/Snet . Combining these with part (ii) yields part (iii).

For part (iv), vu=1 makes the Kraft-slack term log2 (1/vu) vanish and p0,u = p~u makes the divergence term vanish, so Lemma 1″ gives εu = δu+gu =0 for every net surviving row, hence εred(I)=0 . The leaf-depth identity follows from p~u(x) = 2-ℓu(x) / vu with vu=1 , so each p0,u is dyadic and each tree is optimal for it. Substitution into part (i) gives the displays. □

Remark — Wider stage alphabets. The binary restriction in A5 is needed for the unit-coefficient upper bound in Theorem 2. A limited one-sided extension survives when alphabet width is the only departure from that assumption. Suppose the accounting and attribution hypotheses used in Theorem 2(i) continue to hold (in particular A1-A2 and A4), population matching B4 continues to hold, P2-P3 and the terminal-certainty condition B2 remain valid, and each conditioning stage now has a signal alphabet of size at most m≥2 . Define L := log2 m .

Corollary 0′ then gives, for each surviving closure,

H u start,real ≤ L 𝔼u [ Ku ] .

Averaging over the same net surviving closure population and using the accounting identity used in Theorem 2(i) gives the one-sided bound

K∗ ≤ L ( K^ − ζ − εinv − εΔ − εbd ) , L = log2 m .

Wider branching therefore does not destroy the one-sided upper bound; it weakens it by the maximum information capacity L of one stage, and at m=2 equation (8) reduces to the upper side of part (ii). The derivation uses only Corollary 0′ and the accounting identity; it does not use Huffman optimality, Kraft slack, deterministic trees, fullness, or Lemmas 1, 1′ or 1″. The present theorem makes no corresponding two-sided claim for m>2 , because its lower endpoint uses the binary coding lemmas; obtaining one requires m-ary coding assumptions not adopted here.

Two consequences for Corollary 2. First, the factor L sits outside the whole bracket, so on the high-probability event the upper side of equation (7) becomes

K∗ ≤ L ( K^ - WobsSnet + Qbd-Snet + Zα ) ,

so the path fluctuation, the invalidation correction and the boundary load are amplified along with the estimator; multiplying the estimator term alone understates the bound. Second, the concentration argument itself is unaffected, since it uses only the declared increment range d and never the alphabet width. The support-size justification d≤M-2 does not carry over, however, because Lemma 1″(c) is binary; within the scope of this remark d must be declared rather than read off Lemma 1″(c).

Human time-capacity bound. The results of this part bypass the comparison between counted intervals and stages instead of redefining stages. Query stages, A5, condition CΔ, the unit-inference error εΔ and the path fluctuation ζ keep their meaning. The human bound compares information with time directly, and the fluctuation it needs is a fluctuation of interval counts.

Lemma H — Capacity-limited path (human regulator). Assume P2, P3, B2 and H1 for episode u. Then

H(p0,u) ≤ch𝔼u[Nunc].

(H)

P1, A5 and CΔ are not used. The per-interval capacity ch takes the place that log2m takes in Corollary 0′, but per counted interval, not per stage.

Proof. Under B2 the path terminates at certainty, so the telescoping identity of Lemma 0 and the Bridge Lemma give H(p0,u)=IMu(X;U≤Ku∣Y=yu), as in the proof of Lemma 1′(i). H1-b guarantees that this transcript carries everything the episode takes in about its class after it starts, and H1-a bounds it by Ch𝔼u[Tu]=ChΔ𝔼u[Nunc]. □

Relational form. If B2 is read relationally, the bound holds when H1-a is strengthened to the disturbance, I(D;U≤Ku∣Y=y)≤Ch𝔼[T∣Y=y], and the class the protocol fixes is a function of the transcript and has no entropy given the disturbance. Then KTDrel(yu)≤H(X^K∣Y=yu)=I(X^K;D∣Y=yu)≤I(U≤Ku;D∣Y=yu)≤ch𝔼u[Nunc]. The first inequality is the majorization step of Proposition Q1′, the equality uses that the fixed class has no entropy given the disturbance, and the next step is data processing. The weaker form of H1-a stated for the required class does not support the relational reading, because there is then no unique class to bound.

Corollary H — Human-channel upper bound. Assume A2–A4 and B4, with Snet>0, and the hypotheses of Lemma H for every net surviving row. Define the interval-count fluctuation

ζΔ(I):=1Snet∑u∈Ωnet(I)(nu−𝔼u[Nunc]).

Then, with the window argument suppressed,

K∗≤ch(K^−ζΔ−εinv−εbd) ≤ch(K^−ζΔ−WobsSnet+Qbd−Snet).

(H′)

Proof. By Proposition 1′, Qnet=∑u∈Ωnet(I)nu, Snetεinv=Qinv+Wobs and Snetεbd=Qbd+−Qbd−, so 1Snet∑u∈Ωnet(I)𝔼u[Nunc]=K^−εinv−εbd−ζΔ. Averaging (H) over the net surviving rows gives the first inequality; ch>0 preserves it. The second uses εinv≥Wobs/Snet and εbd≥−Qbd−/Snet. □

Corollary H uses the ledger identity, not the decomposition of Theorem 2(i), and so needs neither CΔ nor a bound on εΔ. The fluctuation ζΔ coincides with the path fluctuation ζ of Theorem 2 when every counted interval holds exactly one stage, realized and modelled, which is CΔ with equality.

B5Δ — Sequential calibration of interval counts. Index the episodes of the window in time order, j=1,2,…, with the filtration (ℱj) of B5, and assume 𝔼[nj∣ℱj−1]=𝔼Mj[Njnc]=:μj. This is B5 with interval counts in place of stage counts. Neither implies the other unless counted intervals and stages coincide.

Proposition H‴ — Observable high-probability human upper bound. Assume the hypotheses of Corollary H, B5Δ, and the stopping construction of Corollary 2 with interval counts in place of stage counts. Let Jmax be a deterministic bound on the stopping index JN. The bound Jmax=N+1 always holds; if every episode spends at least nmin≥0 non-closure intervals, then Jmax=⌊(N+Qbd−)/(1+nmin)⌋+1. Declare one of the following:

  • (a) Range. Each nj lies in an ℱj−1-predictable interval of width d; set Λα:=dJmaxln(1/α)/2.
  • (b) Average second moment. With qj:=𝔼[nj2∣ℱj−1], the completed episodes of the window satisfy ∑j<JNqj≤(JN−1)v¯ for a declared v¯ in intervals squared; set Λα:=2Jmaxv¯ln(1/α). The declaration concerns the window's average, not each episode: most episodes may spend no non-closure interval at all and a few a long one.

Then for every α∈(0,1), with probability at least 1−α,

K∗≤ch(K^−WobsSnet+Qbd−Snet+ΛαSnet).

(H″)

Every term is observed or declared. Realized-count variant. Under A2 and A4 the number of completed episodes is JN−1=Sgross, which is observed. Λα may be computed with Sgross in place of Jmax, at the price of replacing ln(1/α) by ln(G/α) and multiplying by (2+1/2)/2≈1.061, where G is the size of a dyadic grid of exponents; G=20 covers every count up to 238≈2.7⋅1011. The variant pays when Jmax is much larger than the number of closures, as it is when most episodes spend no non-closure interval.

Proof. Let ξj:=nj−μj and Mn:=∑j≤nξj, a martingale by B5Δ. Only its lower tail is needed. Set sj2:=d2/4 under (a) and sj2:=qj under (b); both are ℱj−1-measurable. Under (a), Hoeffding's lemma gives 𝔼[e−λξj∣ℱj−1]≤eλ2sj2/2. Under (b), the counts are nonnegative and e−y≤1−y+y2/2 for y≥0, so 𝔼[e−λnj∣ℱj−1]≤1−λμj+λ2qj/2≤e−λμj+λ2qj/2, which is the same bound. Hence exp(−λMn−λ22∑j≤nsj2) is a nonnegative supermartingale, and Ville's inequality gives, with probability at least 1−α and simultaneously for all n, −Mn≤λ2∑j≤nsj2+ln(1/α)/λ. Take n=JN−1. The sum is then at most (JN−1)s2≤Jmaxs2, with s2=d2/4 under (a) and s2=v¯ under (b). With λ=2ln(1/α)/(Jmaxs2), −MJN−1≤s2Jmaxln(1/α)=Λα. As in the proof of Corollary 2, SnetζΔ=MJN−1−∑invξj, the second sum running over the invalidated episodes. Since μj≥0, ξj≤nj, and the invalidated non-closure intervals sum to Qinv; hence −ζΔ−εinv≤(−MJN−1−Wobs)/Snet. Substituting into the first inequality of (H′), with εbd≥−Qbd−/Snet, gives (H″). For the realized-count variant, run the same argument at level α/G for each λ of the grid λg=λ12−(g−1), g=1,…,G, with λ1=2ln(G/α)/s2. All G bounds hold together with probability at least 1−α, and the optimal exponent for the observed count Sgross lies within a factor 2 of a grid point, which costs at most the stated factor. □

Remark — Reading Corollary H and Proposition H‴.

  • What the one-sided form gains. The upper side of the bracket needs only the lower tail of the martingale, so one tail suffices and ln(1/α) replaces ln(2/α). The invalidated episodes enter with a sign that cancels against Qinv, so there is no dWobs term. The counts are nonnegative, so version (b) needs no range at all; this matters when non-closure periods are long and heavy-tailed, as pauses in software development are, where any honest range d is large. The same argument sharpens the upper side of Corollary 2 (display (7′)).
  • Relation to A5. Corollary H has the form of the wider-alphabet remark with log2m replaced by ch. A5 is untouched: it remains the hypothesis of the binary results, and H1 is a second, independent route to the upper side that is open only for human regulators. When every interval is also exactly one binary stage, both routes apply, ζΔ=ζ, and the smaller factor, min{1,ch}, may be used.
  • Upper side only. H1 gives no lower bound on K∗. The capacity slack ρcap:=ch1Snet∑u∈Ωnet(I)𝔼u[Nunc]−K∗≥0 is not the search excess εred, and H1 supplies no Huffman or Kraft bound on it. The lower side of Theorem 2, the ideal-regime consistency of Corollary 4(iii), and the middle rungs of the query ladder all rest on binary decision trees, and none of them is available for a lookup with a large answer alphabet.
  • Expectation, not each interval. Like Proposition 0, H1 bounds expected intake. A single realized interval may deliver more than ch bits.
  • Terminal residual. If B2 is relaxed, Lemma H becomes H(p0,u)≤ch𝔼u[Nunc]+𝔼u[H(pKu)], and the window average of the last term, the terminal residual of the remark that follows, is added to the right sides of (H′) and (H″).
  • Dependence on the declared constant. The bounds depend on Ch only through the factor ch. A revised estimate rescales them and leaves every ledger count, Λα included, unchanged.
  • Empirical content. H1-a, in the form stated for the required class, bounds only what a look registers about the class of the episode in which the look occurs. What it registers about later closures is not Knowledge To Be Discovered of this episode and is not bounded by H1; it is episode-start information of the later episodes (see the subsection below). It is not retained gain GtK either, which compares the expected starting Knowledge To Be Discovered of comparable episodes and records a change in the stored law of action. If H1-a is strengthened to the disturbance, H1 also constrains the record: the total intake of a look, including what it registers about later closures, is then at most Ch times its expected duration, so a short look followed by a long run of outputs is consistent with H1 only if the knowledge state already made the run nearly certain, the look disambiguating rather than transmitting. Under the weaker form no such constraint follows.

Lookups that register information about later closures

A lookup may register information about several later closures, as a look at a script shows the letters that follow the one being sought. The ledger books this without any new construction.

  • Time. The lookup's non-closure intervals belong to the episode in which it occurs and are counted once (A2). Lemma H for that episode bounds its within-episode discovery, Btdisc=Htstart−Htterm=I(X;Ut∣Y), which concerns that episode's required class only.
  • Episode-start information. What the lookup registered about a later closure is part of that closure's episode-start information Y, in the same way as the realized predecessor move is part of the robot's Yn on correlated terrain. The later episode's starting Knowledge To Be Discovered is evaluated given it and is often zero. An episode that starts certain needs no non-closure interval, and Lemma H holds for it trivially. If what was registered has been forgotten by then, the later episode starts with the uncertainty actually present, and Lemma H charges its own non-closure intervals.
  • Consequence. Nothing is counted twice, no allocation of a lookup's time across closures is needed, and Corollary H and Proposition H‴ bound the same latent quantity K∗ that Theorem 2 brackets.
  • Information taken in while committing. The same booking applies to information registered during an earlier closure act, as when a typist reads ahead while typing. It is episode-start information of the closures it concerns, so it lowers their starting Knowledge To Be Discovered; it does not invalidate the bound. H1-b excludes only intake about an episode's own class during that episode's closure act, which would mean that the closure was not committed on what had been registered.
  • What K∗ does not count. Information that a closure's episode starts with is, by definition, not Knowledge To Be Discovered of that closure. The more of a task's information arrives before the episodes that use it, through lookups serving several closures or reading ahead, the smaller K∗ is relative to everything the regulator registered. The total information a typist takes in from a script, averaged per letter, is a different quantity; H1 in the form stated for the required class does not bound it.

Remark (graded terminal uncertainty). If B2 is relaxed so that some protocol paths may terminate with residual uncertainty, define the terminal residual as

H term res ( I ) := 1Snet ∑u∈Ωnet(I) 𝔼u [H(pKu)] .

Corollary 0′ then gives ε red ≥ - H term res ( I ) . Note, that H term res ¯ is a lower bound on ε red . Also, H term res ¯ is genuinely distinct from KTD avg term-real,net - the latter would average H ( p k u ) at the realized path, while H term res ¯ averages E u [ H ( p K u ) ] over the path taken by the realized protocol. Thus, if H term res ( I ) ≤ H term res ¯ , the exact decomposition implies the one-sided bound

K ∗ ≤ K ^ - ζ - ε inv - ε Δ - ε bd + H term res ¯ .

No corresponding Huffman upper bound on εred follows merely from H term res ( I ) > 0 , because the realized protocol no longer necessarily identifies X. If, for the relaxed regime, one separately declares - H term res ¯ ≤ ε red ≤ 1 + r , then

K ^ - ζ - ε inv - ε Δ - ε bd - 1 - r ≤ K ∗ ≤ K ^ - ζ - ε inv - ε Δ - ε bd + H term res ¯ .

In the sharp certainty-terminated case H term res ¯ = 0 , Lemma 1′ and B1(i) recover the coding interpretation and the strict bound ε red < 1 + r used in Theorem 2(ii).

Remark — Theorem 2 in relational and query-family form. Write K*:=1Snet∑uKTDrel(yu) and define εred against it. Then:

  1. Part (i) is unchanged; it is an identity between defined quantities.
  2. The upper side of part (ii) holds under A5 and relational B2 without determinism: Proposition Q1′ gives KTDrel(yu)≤𝔼u[Ku] row by row, so εred≥0. The wider-alphabet remark carries over in the same way, with the factor log2m.
  3. For the lower side, let every stage signal be a deterministic function of the disturbance, and assume B1(i-a) and B1(i-b) row by row. Proposition Q2 gives 𝔼u[Ku]−KTDrel(yu)=δu+ψu+guQ<1+rQ+rP, so part (ii-a) holds with the bracket
K^−ζ−εinv−εΔ−εbd−1−rQ−rP<K*≤K^−ζ−εinv−εΔ−εbd.
  1. Part (iv) requires all three gaps to vanish on every net surviving row: the class distribution of a minimizing selector is dyadic, δu=0; the family contains the even-split questions of an optimal procedure, ψu=0; and the regulator uses such a procedure, guQ=0. The deterministic-tree lemmas then apply to the class the protocol outputs, which must be a minimizing selector's class.

Parts (iii), Corollary 2, and the remarks on boundary bias and miscalibration use only the identity of part (i), the bounds on the error terms, and the increment range d; they hold in the relational form without change, with r replaced by rQ+rP wherever it appears.

The ladder of Theorem Q places these terms. The upper side of Theorem 2 rests only on the floor of the ladder, which binds every binary protocol whatever its query family; that is why it is untouched. The lower side spans the whole ladder, and its tolerance is the sum of the two gaps above the coding gap.

Concentration of the path fluctuation at fixed capacity

The capacity N(I) is a constant of the window, while the number of closures it contains is random. The completed population is therefore a renewal count: it is the number of episode cycles, each of length nj+1 , that fit into the fixed capacity. With the episode index and filtration of B5, define the stopping index as

JN := inf { j≥1 : ∑i≤j (ni+1) - Qbd-(I) > N(I) } .

The index JN is a stopping time with respect to (Fj) , because ni∈Fj for i≤j and Qbd-∈F0 . Under A2 and A4, the episodes j<JN are exactly those whose closures lie in I, so Sgross(I)=JN-1 , and episode JN is the one open at tb. Every cycle contributes at least one unit, and the first contributes n1+1-Qbd-≥1 ; hence JN≤N+1 , and in the clean case Snet=JN-1 .

Corollary 2 - High-probability envelope at fixed capacity. Assume A2, A4, B4 and B5, with N=N(I) constant and Snet(I)>0 . For every α∈(0,1) , with probability at least 1-α,

|ζ(I)| ≤ Zα(I) := d ( (N+1)ln(2/α) 2 + Wobs(I) ) Snet(I) .

(6)

The bound Zα depends only on the constant N, the observed Wobs and Snet, and the declared d. Substituted into Theorem 2(iii) while assuming A1 and A5, it gives, on the same event,

K^ -ε¯inv -cΔ - Qbd+Snet -1-r -Zα < K∗ ≤ K^ - WobsSnet + Qbd-Snet +Zα ,

(7)

where every term is observed or declared. The upper side needs only A1-A5, B2, B4, B5 and CΔ; the lower side additionally needs B1.

Under the wider-alphabet remark following Theorem 2, if alphabet width is the only departure from the binary A5 hypothesis, the corresponding high-probability upper side is

K∗ ≤ log2 m ( K^ − Wobs Snet + Qbd− Snet + Zα ) .

The factor log2m multiplies the entire upper endpoint. Thus the path-fluctuation allowance Zα , the boundary contribution, and the invalidation correction are amplified together with the estimator term. The binary equation (7) is recovered at m=2 .

Rate. Since ∑j<JN (nj+1) = N-Qbd++Qbd- ≥ N-n¯ , we have deterministically Sgross(I) ≥ (N-n¯) / (n¯+1) . With Wobs=0 , equation (6) therefore gives

Zα ≤ d (n¯+1) (N+1)ln(2/α)/2 N-n¯ = O ( dn¯ N-1/2 ) .

The path fluctuation thus shrinks with the constant capacity, not with the random closure count. Under direct counting with full trees, n¯≤m-1 and d≤m-2 , so the constant is at most quadratic in the largest posterior support size. A Freedman-type inequality with a conditional variance bound on Kj in place of the range d tightens it.

Proof. Let Mn := ∑j≤n ξj . By B5, (Mn) is a martingale with respect to (Fn), with increments in predictable intervals of width d. The same holds for the stopped martingale Mn∧JN . Hoeffding's lemma makes exp(sMn∧JN-s2nd2/8) a supermartingale for every real s. Doob's maximal inequality and a union bound over both tails then give

Pr ( maxn≤N+1 | Mn∧JN | ≥ λ ) ≤ 2 exp ( - 2λ2 (N+1)d2 ) .

Choose λ=d(N+1)ln(2/α)/2 . Since JN-1≤N , the value MJN-1 is among the terms of the maximum. The net population consists of the completed episodes minus the Wobs invalidated ones, so

Snet ζ = MJN-1 - ∑j<JN,invalidated ξj ,

and |ξj|≤d bounds the second term by dWobs . Dividing by Snet gives (6). Equation (7) follows by substituting (6) into Theorem 2(iii). The high-probability event concerns the maximum over n only, so dividing by the observed Snet leaves it unchanged. □

The invalidated contribution is bounded deterministically. Consequently, Corollary 2 does not require survival to be independent of path length, and survival may be decided at or after tb by any later criterion. The price of admitting survivor selection is the term dWobs/Snet .

Remark — One-sided upper envelope. The upper side of (7) uses only the lower tail of Mn. Under the hypotheses of the upper side of (7), with probability at least 1−α,

K∗≤K^−WobsSnet+Qbd−Snet+ dJmaxln(1/α)/2Snet,

(7′)

where Jmax is any deterministic bound on JN: N+1 always, and ⌊(N+Qbd−)/(1+kmin)⌋+1 when every episode spends at least kmin counted non-closure units. The same bound on JN may replace N+1 in (6). Proof. Hoeffding's lemma for the lower tail and Ville's inequality give −MJN−1≤dJmaxln(1/α)/2, as in the proof of Proposition H‴(a). For an invalidated episode ξj≤kj≤nj under CΔ, so the invalidated sum is at most Qinv and −ζ−εinv≤(−MJN−1−Wobs)/Snet. Substitute into the upper side of Theorem 2(ii), using εΔ≥0 under CΔ and εbd≥−Qbd−/Snet. □ Compared with the upper side of (7), the dWobs term is gone, one tail replaces two, and the episode count replaces the unit count. Under the wider-alphabet remark the whole right side of (7′) is multiplied by log2m, as in (7).

Remark (boundary bias). Because JN is bounded, optional stopping gives 𝔼[MJN]=0 , hence

𝔼 [ MJN-1 ] = - 𝔼 [ ξJN ] , | 𝔼 [ ξJN ] | ≤ d .

The episode that crosses tb i.e. overshoots the window, tends to be a long one, so the completed average is biased downward. This is the inspection paradox. In typical regimes 𝔼[ξJN]>0 , so the completed paths are on average shorter than their model expectation. This systematic component (the bias) of ζ is of order d/Snet and is already contained in Zα.

Remark (miscalibrated knowledge state). B5 fails when the required class is drawn from pj∗ rather than from the model posterior p0,j. For deterministic trees, Lemma 1″(d) identifies the resulting drift:

𝔼 [ ξj ∣ Fj-1 ] = βj := ∑x ( pj∗(x) - p0,j(x) ) ℓj(x) , |βj| ≤ d TV ( pj∗ , p0,j ) .

Applying Corollary 2 to the centered increments ξj-βj , whose ranges have the same width d, gives, with probability at least 1-α,

|ζ(I)| ≤ Zα(I) + d TV¯ ,

where TV¯ bounds the total-variation miscalibration of the net surviving rows. This is the ledger form of acting on a model that does not track the environment: it contributes a systematic term that does not shrink with N.

Remark — Miscalibration that averages out. The floor dTV¯ is a worst case. Suppose the drift depends only on the episode-start information, βj=β(Yj), that (Yj) is a Markov chain with transition operator P and Yj+1∈ℱj, and that the Poisson equation g−Pg=β has a bounded solution, as it does when β averages to zero under the stationary law of an ergodic finite chain. Then

∑j≤nβ(Yj) =∑j≤n[g(Yj+1)−Pg(Yj)] +g(Y1)−g(Yn+1),

a martingale with increments in intervals of width osc(g):=maxg−ming plus a boundary term of at most osc(g). Splitting α equally between this martingale and the calibrated one of the increments ξj−βj, the one-sided envelope (7′) holds with its last term replaced by

(d+osc(g))Jmaxln(2/α)/2+osc(g)Snet,

which shrinks with N. Miscalibration of this kind is invisible in the window total and visible only when stage counts are tabulated by episode-start state; it still biases each row, and it still leaves εred measured against the regulator's own model. For a chain that repeats its state with probability a and otherwise redraws from its stationary law, g=β/(1−a). The robot on correlated terrain in the appendix is an instance.

The net effective estimator is exact and comes from the ledger. The path fluctuation ζ is the difference between what the surviving population actually spent and what it was expected to spend given its disturbance states. It is the price of estimating an expectation from one window. Because the capacity N is fixed and the closure count is a renewal count inside it, ζ concentrates at rate N-1/2 under sequential calibration (Corollary 2). The redundancy εred is the excess of the regulator's search over the entropy of what it had to discover. It is nonnegative by the entropy bound on prefix codes, and by Lemma 1″ it splits exactly into a divergence charge for uneven splits and a Kraft-slack charge for null stages. The invalidation term εinv is capacity attributed to closures that did not survive. The unit-inference term εΔ is the cost of counting time instead of stages. The boundary term εbd is capacity belonging to episodes that straddle the window edges; it telescopes across a window series. Theorem 1 remains the standalone decomposition of the invalidation effect. The non-probabilistic envelope is conditional on A5 (alphabet size and the P1-P3 protocol), B2 and B4, and its lower side additionally needs B1. The high-probability envelope (7) adds B5.

Applicability of Equation (5)

Combining the ledger identity with the counting of invalidation load as units (B3), equation (5) is the operational counterpart to the latent realized average starting Knowledge To Be Discovered for surviving closures. It is exact by accounting. Its reading as an estimator of KTDavgstart-real,net(I) is governed by Theorem 2, which grades the reading by which hypotheses hold.

Under A1-A5, B2, B4, B5 and condition CΔ, equation (7) holds with probability at least 1-α. On that event, equation (5), corrected by the observed invalidation fraction and the observed pre-window boundary load and widened by Zα, is an upper bound on the latent quantity. This direction is derived from the entropy bound on prefix codes, not assumed. Adding B1 makes the bracket two-sided, with width

1 + r + cΔ + ( ε¯inv - WobsSnet ) + Qbd++Qbd- Snet + 2 Zα .

Adding dyadic posteriors, optimal search and direct unit counting collapses the redundancy and unit-inference terms to zero. What remains is invalidation load, boundary load, and a path fluctuation that shrinks as N-1/2.

What degrades the reading is specific, and different failures cost different things. If B1 fails — the regulator searches inefficiently, splits unevenly, or spends units that discover nothing — only the lower side of the binary bracket is lost. The upper bound is unaffected because an inefficient binary search spends more stages than the entropy requires, not fewer. If the binary alphabet-width clause of A5 alone fails, the upper bound need not disappear. If every conditioning stage has at most m outcomes and the remaining conditioning, attribution, terminal-certainty, and population-matching assumptions continue to hold, the wider-alphabet remark after Theorem 2 preserves the one-sided upper bound with a factor log2m . The present theorem makes no corresponding two-sided claim in that regime. If CΔ fails, so that a single time-normalized interval may contain several conditioning stages, the ledger may under-count the path and the upper bound is lost with it; direct stage counting removes this particular risk. The wider-alphabet factor does not repair such under-counting. If B2 fails, so that a closure is recorded while selection is still open, equation (5) estimates the information actually discovered rather than the starting stock, and the difference is the terminal residual. A different failure occurs when counted non-closure units do not correspond to stages of the episode's conditioning path. Then the execution ledger is measuring something other than the path quantity to which the information bound applies, so no operational-to-latent bound follows from that count in either direction. Finally, if the choice of which stage to execute is itself informative about X beyond the stage signals already represented in the conditioning path, the failure is not merely one of alphabet width. It is a failure of P2: the stated conditioning path omits response-relevant information, so the conditioning-path identity itself no longer applies. In these structural cases equation (5) may remain a valid operational ratio, but its reading as an entropy estimator is no longer supplied by the derived bound. A5 and CΔ are protocol and measurement-design conditions that can be inspected at design time; P2 is a modelling commitment about whether the represented conditioning path contains all response-relevant selection information.

For a human regulator there is one further route. When A5 fails because a lookup returns an answer from a large alphabet, assumption H1 can take its place on the upper side. Corollary H then makes equation (5) an upper bound on the latent quantity with the factor ch=ChΔ, and Proposition H‴ makes that bound observable with high probability. Neither uses condition CΔ: several queries in one interval, or one query spread over several, do not affect them, because they compare information with time rather than with stage counts. Idle time attributed to non-closure periods loosens the bound without invalidating it. Information registered during earlier closure acts, as when a typist reads ahead, or by one lookup about several later closures, does not invalidate it either: it is episode-start information of the closures it concerns and lowers their starting Knowledge To Be Discovered. The bound is lost when a closure is committed on information that was not registered before its closure act (H1-b, B2), and it is as good as the declared human information throughput Ch and as the argument that the task's lookups pass through the bottleneck it measures.

Failed and retracted outputs do not by themselves degrade the reading. A failure at production, such as a missed shot or a wrong test answer, is a non-closure unit and enters Q(I); a retraction enters Wobs(I) . When either is a trial for the same target as a surviving closure, it is a stage of that closure's conditioning path, so the mismatch case above does not apply to it. The remark “When a failed or retracted output is a query” in “Failed and retracted outputs” gives the consequences: the credit Wobs/Snet is not available for a retraction, and the upper bound holds with the average terminal residual of the graded-uncertainty remark added.

A further failure concerns the regulator's questions rather than its protocol. If the query family is inadequate for some rows, in the sense of Proposition Q4, then no protocol over that family can close those rows safely, and B2 fails by construction rather than by inefficiency. Only the graded-uncertainty bound remains, shifted by the terminal residual. If the family is adequate but restricted, B2 can hold while the lower side of the bracket widens by the query-family gap; the tolerance rQ then names that width. Neither condition is repaired by a better protocol. Acquiring questions, L2b, repairs both, and so can learning that the disturbances the present questions cannot separate do not in fact occur.

If B5 fails because the knowledge state is miscalibrated, the envelope does not collapse, but it acquires a systematic width. The path fluctuation carries a drift of at most dTV¯ per net closure, which does not shrink with the capacity of the window. Enlarging N cannot remove it; only revising the knowledge state can. In the terms of the feedback-loop reading, that revision is an L2a event: a change of knowledge state, not of questions or protocol.

7. Knowledge-Discovery Efficiency (KEDE)

Now we generalize the Knowledge-Discovery Efficiency (KEDE), a scalar metric that quantifies how efficiently a system closes the gap between the variety demanded by its environment and the variety embodied in its prior knowledge[28]. KEDE is an acronym for KnowledgE Discovery Efficiency. It is pronounced [ki:d].

Efficiency has an operational and a latent meaning. Operationally, a regulator is more efficient the fewer non-output instructions it executes per output, in the program of the subsection “The record as a two-instruction program”. Latently, it is more efficient the less Knowledge To Be Discovered it faces per output. They are not the same quantity.

Theorem 2 and Corollary 2 give the bridge between the operational counting ratio and the latent Knowledge To Be Discovered quantity. Therefore, KEDE must be defined in two layers: a latent entropy-based metric and an operational net metric computed from observed delivery counts. The operational metric is not assumed to be identical to the latent metric. It is an estimator whose relationship to latent knowledge-discovery efficiency is controlled by two envelopes: the fixed-window, non-probabilistic envelope of Theorem 2, and the high-probability envelope at fixed capacity of Corollary 2. Corollaries 3 and 4 below carry each envelope over to KEDE.

Operational net KEDE

For an observation window I, let the number of net surviving closures be Snet(I) = Sgross(I) − Wobs(I) . For N(I) ≥ Sgross(I) ≥ Snet(I) >0 the effective one-bit net depth estimator of equation (5), the number of non-output instructions per output instruction in the program of the subsection “The record as a two-instruction program”, is

K^ := KTD^ eff net (I) = N Snet −1

In this section, arguments (I) are suppressed when every quantity refers to the same window. Define the operational net efficiency by the reciprocal transform:

KEDE^ op net (I) := 1 1+ K^ = Snet N = Sgross − Wobs N

This is an exact operational count ratio under the ledger definitions. It is exact given its inputs, but only one of them is always observed: the 1s, Snet, are counted, while the length of the program, N, is counted only under direct counting and is otherwise declared through the unit convention, in which case the 0s, Qeff=N−Snet, are inferred and the error of the declaration is carried by εΔ. It is the fraction of the program of the subsection “The record as a two-instruction program” that is output: Snet 1s among N instructions. A counted action unit is a unit of expended capacity, written as one bit of program text. The one-bit convention of that subsection does not assert that a program bit conveys one bit of information. The bridge from program bits to bits of Knowledge To Be Discovered is Proposition 0, under A5 and CΔ, and Theorem 2 gives its errors. In particular, the displayed ratio does not by itself identify the latent conditional entropy.

Latent KEDE

The latent target is the average starting Knowledge To Be Discovered over the same net surviving closure population as the operational ledger. Using the realized row entropies defined earlier, write

K* := KTD avg start-real,net (I) = 1 Snet ∑ u∈ Ωnet (I) Hustart,real

The latent efficiency for that window is

KEDE* (I) := 1 1+ K*

In the single-task notation, this corresponds to the conceptual entropy definition:

KEDE = 1 1+ H(X|Y) .

KEDE is high when little knowledge remains to be discovered, and low when much knowledge remains to be discovered.

The realized window average need not equal this model-wide expectation: it averages the disturbance values and knowledge states actually represented among surviving closures. Both efficiencies lie in (0,1] when their respective depths are finite. Zero is a limiting value.

Factorisation of KEDE along the query ladder

Latent KEDE measures the knowledge a regulator lacked. Operational KEDE measures the capacity it spent. The ladder of Theorem Q lies between them, and each step down the ladder contributes one factor. Over the net surviving rows of the window, write

D¯:=1Snet∑uDTμyu(Ru∣yu),D¯Q:=1Snet∑uDTμyuQu(Ru∣yu),K¯:=1Snet∑u𝔼u[Ku],

for the averages of the unrestricted complexity, the restricted complexity, and the model-expected path length. The second is the window statistic DTavgstart-real,net(I) of the accounting section. Define the efficiencies these rungs would have if each were the regulator's depth:

KEDEQ:=11+D¯Q,KEDEexp:=11+K¯.

KEDEQ is the best efficiency attainable with the regulator's current knowledge and current questions. KEDEexp is the efficiency its actual protocol attains in expectation.

Proposition Q6 — KEDE factorisation. Assume A2–A4 and B4, with Snet>0. Then

KEDE ^ = KEDE * ⋅ ω cod ⋅ ω Q ⋅ ω P ⋅ ω M ,

with the coding, question, and protocol efficiencies and the measurement factor

ωcod:=1+K*1+D¯,ωQ:=1+D¯1+D¯Q,ωP:=1+D¯Q1+K¯,ωM:=1+K¯1+K^.

If every net surviving row satisfies the hypotheses of Theorem Q, then:

  1. 12<ωcod≤1, with equality on the right when the class distributions of minimizing selectors are dyadic;
  2. 0<ωQ≤1, with equality when every row's query family contains the questions of an optimal unrestricted procedure;
  3. 0<ωP≤1, with equality when the regulator uses an optimal procedure over its questions;
  4. ωM is determined by the measurement terms of Theorem 2: K^−K¯=ζ+εinv+εΔ+εbd. It is not confined to (0,1], because the path fluctuation and the boundary term are signed.

Proof. The product telescopes, and its two ends are KEDE*=1/(1+K*) and KEDE^=1/(1+K^). Under the hypotheses of Theorem Q, each row satisfies KTDrel(yu)≤DTμyu≤DTμyuQu≤𝔼u[Ku], and averaging preserves the order, so the first three factors are at most one. The bound D¯<K*+1 is the average of Proposition Q1, and gives ωcod>(1+K*)/(2+K*)≥12. The equality cases are those of Proposition Q2. For (4), Theorem 2(i) gives K^−K*=Γ, and εred=K¯−K* by definition. □

The first three factors are latent and diagnostic: none is observed. Their value lies in naming why the operational efficiency is low, and so which change would raise it. The coding efficiency cannot fall below one half and is not a target for improvement. A low question efficiency means the regulator lacks questions; it is raised by acquiring them, L2b. A low protocol efficiency means it wastes the questions it has; it is raised by asking them better, L2c. Low latent KEDE itself means it lacks knowledge, which only L2a raises. Without deterministic answers the middle rungs are undefined, and only the product ωcodωQωP=(1+K*)/(1+K¯) remains, which is at most one by Proposition Q1′.

Example. In the robot example, Knowledge To Be Discovered is 1.75 bits and KEDE*=1/2.75≈0.364. A regulator that always asks two questions has KEDEexp=1/3≈0.333. With unrestricted sensors the loss is all protocol, ωP=2.75/3≈0.917 and ωQ=1; with sensors that can only ask “vertical or horizontal?” it is all questions, ωQ≈0.917 and ωP=1. The efficiency is the same; the remedy is not.

When several response classes are acceptable for the same disturbance, each row entropy Hustart,real is read as relational Knowledge To Be Discovered, KTDrel(yu), and the conceptual definition becomes KEDE=1/(1+KTDrel). Imposing a unique target where the problem does not require one can only lower latent KEDE (Proposition Q0), by charging the regulator for distinctions among classes that were equally acceptable.

The exact fixed-window bridge

Let f(x):= 1/(1+x) . It is continuous and strictly decreasing on (−1,∞) and maps [0,∞) onto (0,1] . As in Theorem 2, write K^:= KTD^ eff net (I) and K∗:= KTD avg start-real,net (I) , so that KEDE^ = f (K^) and KEDE∗ = f(K∗) . Because f is decreasing, an upper bound on K∗ becomes a lower bound on KEDE∗ , and conversely.

The total gap of Theorem 2(i) has five components:

Γ (I) := ζ + εred + εinv + εΔ + εbd = K^ − K*

Under the accounting and population-matching hypotheses of Theorem 2(i), this is an identity for every fixed window. It does not require an entropy bound or a probability statement. The path fluctuation and boundary terms may be signed, so the identity alone does not order operational and latent KEDE.

By Proposition Q2, under deterministic stage signals the redundancy component splits further, εred=δ¯+ψ¯+g¯Q, into the window averages of the coding, query-family, and protocol gaps. The total gap then has seven named components, and the first three are the additive counterparts of the coding, question, and protocol efficiencies of Proposition Q6.

All window quantities are evaluated at I, and the argument is suppressed where no confusion arises.

Corollary 3 — Fixed-window KEDE envelope. Assume A1–A4 and B4, with Snet>0 and 0≤Wobs <Sgross . Each part states the hypotheses it adds; they are cumulative in the order given. No part of this corollary is probabilistic, and the path fluctuation ζ is kept explicit throughout.

(i) Exact form. The latent efficiency is an exact function of the operational estimator and the gap:

KEDE∗ = 1 1+ K^ −Γ .

(C1)

This is an identity, not a bound. The denominator equals 1+K∗≥1 , so KEDE∗≤1 . What the remaining parts supply is knowledge of the sign and size of the five components of Γ.

(ii) Exact multiplicative identity. The signed efficiency gap is exactly:

KEDE∗ − KEDE^ = Γ KEDE^ KEDE∗ , | KEDE∗ − KEDE^ | ≤ |Γ| KEDE^ .

(C2)

The operational efficiency understates the latent efficiency exactly when Γ>0 , and the absolute error is proportional to the operational efficiency itself.

(iii) Lower bound on latent efficiency. Assume in addition A5, B2 and condition CΔ . Then

KEDE∗ ≥ 1 1+ K^ −ζ − Wobs Snet + Qbd− Snet = Snet N −Wobs +Qbd− −Snetζ .

(C3)

B1 is not used. A regulator that searches inefficiently, splits posterior mass unevenly, runs null stages, or is counted in time units longer than one stage inflates K^ and never deflates it, so none of these can falsify (C3).

(iv) Two-sided envelope. Assume in addition B1, with 0≤εΔ ≤cΔ , and let ε¯inv be an upper bound on the invalidation re-attribution, observed where the ledger resolves which units belong to invalidated closures and declared otherwise. Define

Dup := K^ −ζ − ε¯inv −cΔ − Qbd+ Snet −r .

If Dup>0 , then, together with (C3),

KEDE∗ < min { 1 , 1 Dup } .

(C4)

The strict inequality holds under the per-row condition gu≤r of Theorem 2(ii-a). Under only the window-level form of B1(i), Theorem 2(ii-b), the inequality in (C4) is non-strict. If Dup≤0 , the only upper bound available is KEDE∗≤1 . The two sides differ in what they cost: (C3) uses only the observed invalidation fraction, the observed pre-window boundary load and proved signs, whereas the declared tolerances r, cΔ and ε¯inv enter only (C4).

Proof. (i) Theorem 2(i), which needs A2–A4 and B4, gives K∗= K^−Γ . Because K∗ is an average of posterior entropies it is non-negative, so applying f to both sides gives (C1).

(ii) The difference of reciprocals is

1 1+K∗ − 1 1+K^ = K^−K∗ (1+K∗) (1+K^) ,

which is (C2) after substituting K^−K∗ =Γ . The absolute bound follows from KEDE∗≤1 .

(iii) Under A5 and B2, Theorem 2(ii) gives εred≥0 . Proposition 1′ gives εinv ≥ Wobs/Snet and εbd ≥ −Qbd−/Snet , and condition CΔ gives εΔ≥0 . Hence K∗ ≤ K^−ζ −Wobs/Snet +Qbd−/Snet . The right-hand side is at least K∗≥0 , so it lies in the domain of f, and applying the decreasing f gives the first form of (C3). The second form uses 1+K^ =N/Snet , which holds under A1–A2.

(iv) Theorem 2(ii-a) gives εred<1+r , and Theorem 2(ii-b) gives the same bound non-strictly. B1(ii) gives εΔ≤cΔ , and Proposition 1′ gives εbd ≤ Qbd+/Snet . Substituting into K∗= K^−Γ gives K∗ > Dup−1 . If Dup>0 , both sides lie in (−1,∞) , and applying f with f(x−1)=1/x gives (C4). □

Corollary 4 — High-probability KEDE envelope at fixed capacity. Assume the hypotheses of Corollary 3(iii) together with B5, and fix α∈(0,1) . Let

Zα := d ( (N+1)ln(2/α) 2 + Wobs ) Snet

be the margin of Corollary 2. There is an event of probability at least 1−α on which all of the following hold simultaneously.

(i) Lower bound.

KEDE∗ ≥ 1 1+ K^ − Wobs Snet + Qbd− Snet + Zα = Snet N + (d−1) Wobs + Qbd− + d (N+1)ln(2/α) 2 .

(C5)

Every quantity in the second form is observed except the declared range d of B5. B1 is not needed.

(ii) Upper bound. Assume in addition the hypotheses of Corollary 3(iv), and define

Dupα := K^ − ε¯inv −cΔ − Qbd+ Snet −r −Zα .

If Dupα>0 , then

KEDE∗ < min { 1 , 1 Dupα } ,

(C6)

with the strict/non-strict distinction inherited from Corollary 3(iv). Equations (C5) and (C6) together are equation (7) pushed through f.

(iii) Ideal regime and consistency. Suppose action units are counted directly, so that εΔ=0 . Suppose every net surviving row satisfies the even-split hypotheses of Theorem 2(iv): a deterministic tree with vu=1 and p0,u = p~u , so that εred=0 . Suppose also that no gross closure is invalidated in the window, so that εinv=0 . Let n¯ bound the units attributed to any single episode, with N>n¯ . Then Γ=ζ+εbd and

| KEDE∗ − KEDE^ | ≤ ( Zα + n¯ Snet ) KEDE^ ≤ (n¯+1) ( d (N+1)ln(2/α) 2 + n¯ ) N−n¯ = O ( d n¯ N−1/2 ) .

(C7)

Hence, with d and n¯ bounded, KEDE^ ⟶ KEDE∗ in probability as the capacity N⟶∞ . Consistency is indexed by the fixed capacity of the window, not by the random closure count; the operational efficiency is consistent for the latent efficiency in the ideal regime, rather than merely bracketing it.

(iv) Miscalibrated knowledge state. If B5 fails because the required class is drawn from pj∗ rather than from the model posterior, then (C5)–(C7) hold with Zα replaced by Zα + d TV¯ . The added term does not shrink with N, so in the ideal regime the efficiency error is floored at d TV¯ KEDE^ and consistency is lost. Only revising the knowledge state, an L2a event, removes it.

(v) Wider stage alphabets. If alphabet width is the only departure from A5, with every stage alphabet of size at most m≥2 , and L:=log2m , then (C5) becomes

KEDE∗ ≥ 1 1+ L ( K^ − Wobs Snet + Qbd− Snet + Zα ) ,

and no upper bound on KEDE∗ other than 1 is claimed. The factor L multiplies the whole bracket, including the fluctuation, invalidation and boundary terms. Within this variant d must be declared, because the support-size bound of Lemma 1″(c) is binary.

(v′) Human regulators. If A5 fails because a human regulator's non-closure time may deliver answers from a large alphabet, and the hypotheses of Corollary H hold, then

KEDE∗≥11+ch(K^−ζΔ−WobsSnet+Qbd−Snet),

and if in addition the hypotheses of Proposition H‴ hold, then with probability at least 1−α

KEDE∗≥11+ch(K^−WobsSnet+Qbd−Snet+ΛαSnet),ch=ChΔ.

No upper bound on KEDE∗ other than 1 is claimed from H1. When ch<1, these lower bounds are higher than the binary ones. Condition CΔ and a range d are not needed: Corollary H compares information with time and never with stage counts, and the second-moment version of Proposition H‴ uses no range. Proof. Push (H′) and (H″) through the decreasing transform f, as in (i). □

(vi) Graded terminal uncertainty. If B2 is relaxed and Htermres ≤ Htermres ¯ , then (C5) holds with Htermres ¯ added inside the denominator bracket, next to Zα . (C6) is then available only if εred≤1+r is separately declared, and then only non-strictly.

Proof. Corollary 2 gives an event of probability at least 1−α on which |ζ|≤Zα . The event does not depend on which bound is read, so all parts hold on it simultaneously.

For (i), the argument of f in (C3) contains −ζ≤Zα . Enlarging the argument decreases f, which gives the first form of (C5). The second form substitutes 1+K^ =N/Snet and the definition of Zα .

For (ii), on the event, Dup ≥ Dupα , so K∗ > Dupα−1 , and the argument of Corollary 3(iv) applies.

For (iii), Theorem 2(iv) gives Γ=ζ+εbd . Proposition 1′ gives |εbd| ≤ n¯/Snet , since both boundary loads lie in [0,n¯] . The first inequality of (C7) then follows from (C2). The second uses the deterministic bound Snet = Sgross ≥ (N−n¯) / (n¯+1) from the rate paragraph of Corollary 2, with Wobs=0 , together with KEDE^≤1 .

Part (iv) is the miscalibration remark after Corollary 2. Part (v) is the wider-alphabet remark after Theorem 2, in its high-probability form. Part (vi) is the graded terminal-uncertainty remark, which gives εred ≥ − Htermres ¯ in place of εred≥0 . Each is then pushed through f as in (i). □

The tolerance r in (C4), (C6) and the definitions of Dup and Dupα is the sum rQ+rP of the query-family and protocol tolerances of B1(i-a) and B1(i-b). The upper bounds on latent KEDE therefore tighten in two separable ways: by better questions, which lower rQ, and by better use of them, which lowers rP. The lower bounds (C3) and (C5) use neither tolerance and are unaffected by either.

Assumption ladder for KEDE

The claims that operational KEDE supports about latent KEDE are graded by the hypotheses that hold. Each row adds to the row above it.

Level Assumptions added What holds for latent KEDE Unobserved or declared quantities
1. Accounting A1–A4, B4 (C1), (C2): exact identity; the sign of the efficiency gap is the sign of Γ All five components of Γ
2. Certainty-terminated binary protocol A5, B2, CΔ (C3): fixed-window lower bound; B1 not needed ζ
3a. Query-family adequacy B1(i-a): the query family can close every row safely, and its gap is at most a declared tolerance Nothing new on its own; it fixes the query-family part of the width of (C4) ζ; declared rQ
3b. Bounded protocol and unit error B1(i-b), B1(ii) (per-row (ii-a) or window-level (ii-b)); ε¯inv (C4): two-sided fixed-window envelope with r=rQ+rP; strict under (ii-a) ζ; declared rQ, rP, cΔ, ε¯inv
4. Sequential calibration B5 (C5), (C6): simultaneous bounds with probability at least 1−α None unobserved; declared d (and r, cΔ, ε¯inv for the upper side)
5. Ideal regime Direct counting, even-split deterministic trees, Wobs=0 (C7): consistency at rate O(dn¯N−1/2) Declared d, n¯

Departures from the ladder cost different things:

Departure Effect on the KEDE envelope
B1(i-b) fails (inefficient search, uneven splits, null stages) Upper bounds (C4), (C6) are lost; lower bounds (C3), (C5) are unaffected. Remedy: a better protocol, L2c.
B1(i-a) fails: query family adequate but restricted beyond the declared tolerance Upper bounds (C4), (C6) are lost; lower bounds (C3), (C5) are unaffected. A better protocol does not help. Remedy: more questions, L2b.
Inadequate query family (Proposition Q4) No protocol over the family can close the affected rows safely, so B2 fails by construction. Only the graded-uncertainty lower bound remains, shifted by the terminal residual.
B5 fails by miscalibration (C5), (C6) widen by dTV¯; the error in (C7) acquires a floor that does not shrink with N.
Binary stage alphabets replaced by m-ary ones Lower bound only, with the whole bracket scaled by log2m; no two-sided claim.
A5 fails for a human regulator (lookups with large answer alphabets), H1 holds Lower bounds (C3), (C5) hold with the bracket scaled by ch=ChΔ, with ζΔ in place of ζ in (C3) and Λα/Snet in place of the fluctuation allowance in (C5), through Corollary H, Proposition H‴ and Corollary 4(v′). Condition CΔ is not needed, nor, in the second-moment version, a range d. Upper bounds on KEDE∗ are lost. The number depends on the declared human information throughput Ch.
B2 relaxed to graded terminal uncertainty Lower bound shifted by Htermres¯; upper bound only under a separately declared search-gap bound.
CΔ fails (several stages per Δ-interval) The ledger may under-count the path, so the lower bounds are lost; B1(ii), and with it the upper bounds, presuppose CΔ. Direct stage counting removes this risk.
P2 fails, or counted units are not conditioning stages No operational-to-latent bound in either direction. KEDE^ remains a valid operational ratio but no longer estimates KEDE∗.

Interpretation

The operational KEDE^(I)=Snet/N is computed exactly from the ledger. Corollary 3 shows that it differs from the latent efficiency KEDE*(I) by exactly Γ times the product of the two efficiencies (C2). Proposition Q6 reads the same gap as a product of factors, one for each step from what the regulator had to know to what it was seen to spend.

A low operational KEDE can therefore have five different causes, and they call for different remedies:

  • Missing knowledge. Latent KEDE is low because much remained to be discovered. Only a better knowledge state, L2a, raises it.
  • Coding slack. Whole questions cannot ask for fractions of a bit. This costs less than one query per closure, keeps the coding efficiency above one half, and vanishes for dyadic posteriors. It is not a target for improvement.
  • Missing questions. The question efficiency is low because the regulator's queries cannot split its uncertainty well, or cannot resolve it at all. Only acquiring questions, L2b, raises it.
  • Wasted questions. The protocol efficiency is low because the regulator asks its questions in a poor order, asks some that tell it nothing, or keeps asking after it could have closed. A better protocol, L2c, raises it.
  • Measurement and invalidation. The measurement factor collects the invalidation load, the unit-inference error, the boundary load, and the path fluctuation. Invalidation is reduced by aligning the regulator's goal model with the authoritative goal, L3; the other three are properties of the measurement, and the path fluctuation shrinks with the capacity of the window.

The invalidation, redundancy, and unit-inference parts of the gap all push in the same direction: they make KEDE^ understate the latent efficiency. Within a fixed window, and up to the path fluctuation and boundary load, operational KEDE is therefore a conservative reading of latent KEDE, and the corrected lower bound (C3) exceeds it whenever invalidations occur. At the high-probability level of (C5), the survivor-selection allowance dWobs/Snet inside Zα offsets that invalidation credit, so the certified lower bound sits below KEDE^ by the price of estimating from one window.

Only the first and last causes are visible in the ledger without further declarations: latent KEDE through the bounds of Corollaries 3 and 4, and invalidation through the observed invalidation fraction. Separating missing questions from wasted ones requires declaring the regulator's query family. The observed count does not distinguish them, as the robot example shows, and neither does the Kraft decomposition of Lemma 1″.

Operational KEDE reports the fraction of counted capacity associated with net surviving closures. Latent KEDE reports a bounded transform of the average starting knowledge gap for those closures. Theorem 2 identifies the conditions under which the former can bound or approximate the latter. Comparisons across task classes or organizations therefore require comparable definitions of capacity, closures, and surviving work, together with an assessment of the relevant envelope terms, and, where the factorisation is used, comparable query families.

KEDE evaluates the efficiency of discovering the required response, not merely whether the final outcome was successful. Two systems may achieve the same outcome, but one may arrive there through a short, knowledge-rich path while the other requires many unsuccessful attempts. The factorisation adds a second distinction: two systems with the same KEDE may be limited by different things, one by what it does not know, another by what it cannot ask, a third by how it asks. KEDE therefore measures not just performance, but the efficiency with which environmental complexity is converted into actionable knowledge, and it indicates which part of that conversion is failing.

8. Knowledge Discovery Rate

The counting ledger and its efficiency transform are duration-free: they tally counted units, not time. Elapsed time entered the construction only once, upstream, where the unit convention fixed what counts as one generic action unit. This section reattaches elapsed time to the ledger and obtains rates. The result is the operational counterpart of Shannon's source-rate identity, with one standing qualification: the rates defined below are rates of instructions of the program of the subsection “The record as a two-instruction program”, not of bits. As in the operational one-bit estimator and in the KEDE section, a counted unit is one bit of program text and is not assigned any amount of information. The connection between instruction rates and bit-valued rates is supplied only by Theorem 2 and its corollaries, and is stated explicitly below.

Knowledge Discovery Rate is the time-normalized operational discovery load. Its factorization has the same dimensional form as source rate = symbol rate * information per symbol, but the per-closure factor is an operational estimator of latent KTD, not Shannon entropy rate itself.

Notation. Rates are written with ν. The letters r and R are not used for rates: r already denotes a committed response-value rt and the search-gap tolerance of B1, while R, Rfinal and the superscript R denote responses. The elapsed duration of the window I is written |I| and is measured on the same clock as the closure-act duration δcl. It is a duration, not a count.

The unit convention

Under the time-normalized convention, the unit length is the duration of one commitment act, Δ:=δcl , and non-closure action units inherit the same length. Every counted unit, whether a read or a commitment, then occupies one Δ and is one instruction of the program, and the channel runs at the constant rate:

νact := N(I) |I| = 1Δ instructions per unit time.

A discovery episode that needs K reads consumes K+1 counted units: K non-closure action units and one closure unit, that is, K non-output instructions and one output instruction. The closure is not free. It is the act that commits the move, and the ledger charges it at the same channel cost as a read.

Earlier, Δ:=δcl is part of the constitutive unit convention: it determines what counts as one action unit in the time-normalized regime. The accounting identities then operate on those units. The rate conversion itself is simply division by the elapsed duration |I| . On the other side, Δ determines the numerator's unit construction when using time-normalized counting; |I| converts the resulting accumulated count into a per-time quantity. This distinction also lets the rate construction work cleanly under direct stage counting, where no Δ-based count inference is needed.

Shannon's source-rate identity

For a stationary source emitting symbols V1,V2,… , the entropy rate is the average uncertainty per emitted symbol:

H∞ := lim n→∞ 1n H ( V1,…,Vn ) (bits per symbol).

If the source emits rs symbols per unit time, its information rate is

Rs = rs ⋅ H∞ (bits per unit time).

The subscript s marks Shannon's quantities, which are used only in this subsection. The two factors carry different units: H∞ is per symbol and rs is per unit time; only their product is a rate in bits per unit time. The entropy rate is an asymptotic, distributional quantity, and recovering it from a single trajectory requires stationarity and ergodicity. The ledger quantities below are finite-window counts on a single trajectory, so the correspondence developed in this section is a correspondence of units and of roles in a factorization, not an identification of objects.

Window rates

Definition (Window rates). For an observation window I=(ta,tb] with |I|=tb−ta>0 , define the action rate, in instructions per unit time:

νact(I) := N(I)|I|

the closure rate, in output instructions per unit time:

νcl(I) := Snet(I)|I|

the effective load rate, the rate at which counted discrimination load accrues:

νdisc(I) := Qeff(I)|I|

Its unit is non-output instructions per unit time.

The physical discrimination rate:

νdiscphys(I) := Q(I)|I|

Its unit is non-output instructions other than invalidated closures, per unit time.

All four are measured in counted units per unit time, and every counted unit is one instruction of the program of the subsection “The record as a two-instruction program”. νcl is the regulator's throughput of net surviving closures. νdisc is the rate of effective non-net-progress load: it includes the one-unit invalidation debit of every invalidated closure, so it counts capacity that did not produce surviving closures. νdiscphys drops that debit and counts only non-closure units physically present on the channel. None of the four is a rate of bits; bits per unit time enter only through the latent rate below. In this section, arguments (I) are suppressed when every quantity refers to the same window.

Under A2 the window contains no fedback units. The feedback interval (tcommit,toutcome] of every closure therefore contains no counted unit: the outcome is realized within the closure unit itself. Each closure is placed in the window by a single time, and the window membership of closures, which every rate below depends on, is unambiguous.

Proposition 2 — Rate partition and factorization. Assume A1–A4, with Snet>0 and 0≤Wobs<Sgross . Then:

(i) Rate partition. The action rate splits into closure throughput and effective load rate:

νact = νcl + νdisc .

(R1)

(ii) Factorization. Both rates factor through the operational net efficiency and through the effective one-bit net depth estimator of equation (5), K^ := KTD^effnet (I) :

νcl = νact ⋅ KEDE^opnet , νdisc = νact ⋅ ( 1 − KEDE^opnet ) = νcl ⋅ K^ .

(R2)

(iii) Effective versus physical. The effective load rate exceeds the physical discrimination rate by exactly the invalidation rate:

νdisc − νdiscphys = Wobs |I| ≥ 0 ,

with equality if and only if Wobs=0 .

Proof. Under A1–A4 the fixed capacity admits the effective progress partition N=Qeff+Snet (Proposition 1 and the re-attribution identity). Dividing by |I| gives (R1). Since KEDE^opnet = Snet/N , the first two equalities of (R2) follow from (R1). Equation (5) gives K^ = Qeff/Snet , so νcl K^ = (Snet/|I|) (Qeff/Snet) = νdisc . Part (iii) is Qeff−Q=Wobs divided by |I|. □

Remark — Two choices of symbol, one numerator. Equation (R2) contains two factorizations of the same window quantity Qeff/|I| , differing only in which unit is treated as the emitted symbol. If the symbol is the counted action unit, the symbol rate is νact and the per-symbol factor is the load fraction Qeff/N = 1− KEDE^opnet ∈[0,1) . This is a fraction of capacity, bounded by one by construction, not an entropy-rate analogue. If the symbol is the net surviving closure, the symbol rate is νcl and the per-symbol factor is the effective depth per closure K^. Only this second factor occupies the entropy-rate slot of Shannon's identity, because only it is an estimator, under Theorem 2, of a per-symbol uncertainty. In terms of the program, the first choice reads it instruction by instruction, and the second reads it episode by episode, each output together with the 0s charged to it. The two factorizations use the same numerator and the same clock; they are not two different rates.

Remark — Throughput versus load. Equation (R1) separates the pace of the channel, νact, from how that pace is spent. At fixed pace, raising efficiency moves rate from load to closures: as KEDE^opnet→1 , νdisc→0 and νcl→νact ; as KEDE^opnet→0 , the whole channel is spent on discrimination and invalidation and νdisc→νact . The closure rate is therefore the performance quantity. The load rate is not a performance score: a high νdisc means that much of the channel is spent selecting responses or charged to closures that did not survive.

Remark — Time-normalized convention. Under the time-normalized unit convention, N=|I|/Δ , so νact=1/Δ in every window. Equation (R2) then reduces to νcl = KEDE^opnet /Δ and νdisc = (1− KEDE^opnet )/Δ : the rates add only the clock to KEDE. Pace is a separate observable only when action units are counted directly, so that N and |I| are measured independently.

Remark — Fedback. A2 is load-bearing in this section, because elapsed time is exactly where fedback units appear: waiting, feedback latency, and physical unfolding consume time without discriminating or committing. If A2 is dropped, the capacity partition has three buckets, N=Qeff+Snet+F , and (R1) becomes

νact = νcl + νdisc + νfb , νfb := F(I)|I| .

The identity νdisc = νcl ⋅ Qeff/Snet survives, because it is definitional, but Qeff/Snet no longer equals N/Snet−1 , and Snet/N no longer equals 1/(1+Qeff/Snet) . The times tcommit and toutcome also separate, so a convention must fix which of them places a closure in the window. Placing it at tcommit, where the closure unit is anchored and charged, keeps νcl commensurate with νact; acceptability is still evaluated at toutcome, as in the operational ledger. The results below assume A2 and do not use this extension.

Latent discovery rate

Definition (Latent rates). With K∗ := KTDavgstart-real,net (I) as in Theorem 2, define the latent discovery rate and the latent realized discovery rate of the window:

νdisc∗ := νcl ⋅ K∗ = 1|I| ∑ u∈Ωnet(I) Hustart,real , νdiscreal := 1|I| ∑ u∈Ωnet(I) Budisc,real .

Both are measured in bits per unit time. νdisc∗ is the rate at which starting Knowledge To Be Discovered of surviving closures enters the window. νdiscreal is the rate at which it is actually removed along the realized conditioning paths, using the within-episode discovery Budisc,real = Hustart,real − Huterm,real of the knowledge ledger. They differ by the rate of terminal residual uncertainty:

νdisc∗ − νdiscreal = 1|I| ∑ u∈Ωnet(I) Huterm,real ≥ 0 .

Under B2 every surviving row terminates at certainty, every terminal entropy vanishes, and the two latent rates coincide. The name discovery rate for the ν disc ∗ is therefore licensed only under B2. Without B2 the latent discovery rate overstates realized discovery by the terminal residual, as in the graded-uncertainty remark after Theorem 2.

Corollary 5 — Rate envelope. Each part states the hypotheses it adds; they are cumulative in the order given. Write Γ = ζ+ εred+ εinv+ εΔ+ εbd for the total gap of Theorem 2(i).

(i) Exact rate gap. Assume A1–A4 and B4, with Snet>0 . Then, for every fixed window,

νdisc − νdisc∗ = νcl ⋅ Γ .

(R3)

This is an identity, not a bound. It uses no entropy bound and no probability statement, and the path-fluctuation and boundary components of Γ may be signed.

(ii) Physical rate as ceiling. Assume in addition A5, B2 and condition CΔ. Then νdisc∗ = νdiscreal and

νdisc∗ ≤ νdiscphys + Qbd− − Snet ζ |I| .

(R4)

B1 is not used. Inefficient search, uneven splits, null stages, and time units longer than one stage all inflate the counted rate and never deflate it, so none of them can falsify (R4).

(iii) Two-sided envelope. Assume in addition B1, with 0≤εΔ≤cΔ , and let ε¯inv bound the invalidation re-attribution as in Corollary 3(iv). Then

νdisc∗ > νdisc − νcl ( ζ + ε¯inv + cΔ + 1 + r ) − Qbd+ |I| ,

strictly under the per-row condition of Theorem 2(ii-a) and non-strictly under the window-level condition of Theorem 2(ii-b). When the right-hand side is not positive, only νdisc∗≥0 is available.

(iv) High-probability envelope at fixed capacity. Assume in addition B5 and fix α∈(0,1) . On an event of probability at least 1−α , the upper bound of (ii) and the lower bound of (iii) hold simultaneously with the path fluctuation replaced by observed and declared quantities:

νdisc − νcl ( ε¯inv + cΔ + 1 + r ) − Qbd+ + Mα |I| < νdisc∗ ≤ νdiscphys + Qbd− + Mα |I| ,

(R5)

where

Mα := Snet Zα = d ( (N+1)ln(2/α) 2 + Wobs )

is the path-fluctuation margin of Corollary 2 expressed in counted units. The upper side of (R5) needs only the hypotheses of (ii) and B5; the lower side additionally needs B1. Every quantity in the upper side is observed except the declared range d of B5.

(v) Ideal regime and consistency. Suppose action units are counted directly, every net surviving row satisfies the even-split hypotheses of Theorem 2(iv), no gross closure is invalidated in the window, and n¯ bounds the units attributed to any single episode. Then, on the same event,

| νdisc − νdisc∗ | νact ≤ d (N+1)ln(2/α) 2 + n¯ N = O ( d N−1/2 ) .

(R6)

Relative to the pace of the channel, the counted load rate is therefore consistent for the latent discovery rate as the capacity N→∞ . As for KEDE, consistency is indexed by the fixed capacity of the window, not by the random closure count.

Proof. (i) Theorem 2(i) gives K^ − K∗ = Γ . Multiplying by νcl>0 and using (R2) gives (R3). (ii) Under B2 every terminal realized entropy of a surviving row is zero, so the two latent rates coincide. Corollary 3(iii) gives K∗ ≤ K^ −ζ −Wobs/Snet +Qbd−/Snet . Multiplying by νcl, using νcl/Snet =1/|I| and Proposition 2(iii) gives (R4). (iii) The proof of Corollary 3(iv) gives K∗ > K^ −ζ −ε¯inv −cΔ −Qbd+/Snet −r−1 ; multiplying by νcl gives the stated bound. (iv) Corollary 2 gives an event of probability at least 1−α on which |ζ|≤Zα ; on it, νcl|ζ| ≤ Mα/|I| , which substituted into (ii) and (iii) gives (R5). (v) Theorem 2(iv) gives Γ=ζ+εbd , and Proposition 1′ gives |εbd| ≤ n¯/Snet . By (R3), |νdisc−νdisc∗| ≤ (Mα+n¯) /|I| with Wobs=0 ; dividing by νact=N/|I| gives (R6). □

Remark — Why the ceiling is the physical rate. The upper bound (R4) is stated in terms of νdiscphys, not νdisc. The invalidation debit re-books an invalidated closure as load, but it never represents discrimination about the required response-equivalence class of a surviving closure, so the effective rate overstates the latent rate by at least Wobs/|I| up to path fluctuation and boundary load. Units spent inside invalidated episodes, which remain in Q, are not separately observable under the black-box regime and remain in the ceiling.

Remark — Departures. If B5 fails because the knowledge state is miscalibrated, (R5) holds with Zα replaced by Zα+d TV¯ , and the relative error in (R6) acquires the floor d TV¯ KEDE^opnet , which does not shrink with N. If alphabet width is the only departure from A5, with every stage alphabet of size at most m and L=log2m , only the upper side survives, scaled as a whole:

νdisc∗ ≤ L ( νdiscphys + Qbd− + Mα |I| ) ,

with d declared rather than read off Lemma 1″(c). If B2 is relaxed, the upper side holds with the terminal-residual bound νcl Htermres ¯ added, and it bounds the latent discovery rate, not the realized discovery rate.

If the regulator is human and H1 replaces A5 on the upper side, the same scaling applies with ch in place of L, and the one-sided allowance of Proposition H‴ may replace Mα: with probability at least 1−α,

νdisc∗≤ch(νdiscphys+Qbd−+Λα|I|).

Since νdiscphys≤νact=1/Δ, the leading term is at most Ch: up to boundary load and path fluctuation, a human regulator cannot acquire Knowledge To Be Discovered faster than its declared information throughput, whatever the alphabet of its lookups. The allowance grows only as the square root of the number of episodes, so its contribution to the rate vanishes as the window lengthens.

Remark — The rate ladder. The ladder of Theorem Q has a rate form. With the window averages of Proposition Q6, define the query-limited rate and the protocol-expected rate:

νQ:=νcl⋅D¯Q,νexp:=νcl⋅K¯.

νQ is the rate at which queries would accrue if every surviving closure were reached by an optimal procedure over the regulator's questions; νexp is the rate at which its actual protocol accrues them in expectation. Under the hypotheses of Theorem Q for every net surviving row,

νdisc*≤νcl⋅D¯≤νQ≤νexp,νdisc−νexp=νcl⋅(ζ+εinv+εΔ+εbd).

The fraction of the expected query rate that discovers knowledge factorises as KEDE does:

νdisc*νexp=K*D¯⋅D¯D¯Q⋅D¯QK¯,

whenever the denominators are positive. Each factor is at most one; they are the coding, question, and protocol parts of the rate in linear rather than reciprocal form. The left-hand side is latent, like the factors.

Discovery rate is not a learning rate

The latent rates of this section measure discovery inside episodes. The knowledge ledger separates within-episode discovery Budisc from retained gain GuK, and the same separation applies to rates. At fixed closure throughput, successful L2a learning lowers the starting Knowledge To Be Discovered of later comparable episodes and therefore lowers νdisc∗ in later windows. A falling discovery rate at constant νcl is the rate signature of learning, not of lost capacity. A sliding-window series of νdisc∗ for a fixed task class carries the same information as the KTD trend of the window-level accounting section, scaled by the closure rate of each window.

The other two adaptation channels move the rates in the opposite direction. At fixed pace νact, acquiring questions, L2b, or improving the protocol, L2c, lowers the number of action units per closure, raises the closure rate, and so raises the latent discovery rate, with Knowledge To Be Discovered per closure unchanged. A regulator can therefore increase its discovery rate without learning anything in the sense of the Learning Axiom. Learning, L2a, has two effects at fixed pace: it lowers the knowledge each closure must discover and raises the number of closures, so its net effect on the discovery rate has no fixed sign. Its signature is in the knowledge per closure, not in the rate.

Relation to Shannon's identity

Equation (R2) and the definition of νdisc∗ give the dictionary

rs ⟷ νcl , H∞ ⟷ K∗ (estimated by K^ ) , Rs ⟷ νdisc∗ (estimated by νdisc ) .

What distinguishes the discovery rate from an information rate in Shannon's sense is the variable whose uncertainty is counted. K∗ is uncertainty about the required response-equivalence class X, the variable whose resolution regulation needs. Activity that does not discriminate among response classes contributes nothing to νdisc*, even though it may contribute counted units; in (R3) it appears as Kraft slack inside εred or as unit-inference error εΔ. Activity that does discriminate, but with questions poorer than the problem allows, appears in εred as the query-family gap. Three qualifications attach to the dictionary.

Finite window versus limit object. K∗ is a finite-window average of realized posterior entropies, each under its own row model Mu. Under a stable knowledge state and Assumption E, K∗ → HM (X∣Y) as Snet→∞ , so that νdisc∗ ≈ νcl ⋅ HM (X∣Y) in large windows. HM(X∣Y) is a per-closure conditional entropy. It coincides with an entropy rate of the required-class process only when successive episodes are conditionally independent given their disturbance information under M. A stable knowledge state also excludes L2 updates inside the window, so the strict Shannon reading is available only for windows in which the regulator is not learning.

Bits per counted unit hold only in expectation. In the even-split regime of Theorem 2(iv), every binary stage satisfies Proposition 0 with equality: given any prefix, its signal carries exactly one bit in expectation, and 𝔼u[Ku] = Hustart,real . Realized stages still do not remove one bit each. With posterior (12,14,14) and the even-split question “is it class 1?”, the answer yes removes 1.5 bits in one stage, while no removes 0.5 bits and leaves a second stage that removes one bit. The expected path length is 1.5 stages, equal to the 1.5 bits of starting uncertainty, but no single stage carries exactly one bit. Reading νdisc in bits per unit time is therefore an expectation-level reading, licensed by (R6) and not by any per-unit property.

Outside the ideal regime the counted rate is a bound, not an estimate. When search is inefficient, splits are uneven, units are inferred from time, or closures are invalidated, νdisc overstates νdisc∗ by νcl ( εred+ εΔ+ εinv ) , up to the signed path-fluctuation and boundary terms of (R3). The certified operational statement is then (R4) or (R5): the physical discrimination rate is a ceiling on the latent discovery rate.

Summary

Reattaching elapsed time to the ledger splits the pace of the channel into closure throughput and effective load, νact = νcl + νdisc , with KEDE as the split fraction. The closure rate is the performance quantity; the load rate is a counted rate of units, not of bits. Its latent counterpart, the rate at which starting Knowledge To Be Discovered of surviving closures is resolved, differs from it by exactly νcl⋅Γ . Under a certainty-terminated binary protocol the physical discrimination rate is a ceiling on the latent rate; under bounded search the bracket is two-sided; under sequential calibration both sides hold with high probability from observed and declared quantities; and in the even-split regime the counted rate is consistent for the latent rate at relative rate O(dN−1/2) . The factorization νdisc∗ = νcl ⋅ K∗ has the form of Shannon's source-rate identity; it becomes that identity in the strict sense only for a stable knowledge state, conditionally independent episodes, and an expectation-level reading of counted units.

Along the query ladder, both KEDE and the discovery rate factorise into a coding part, a question part, and a protocol part, followed by a measurement part. The factorisation is latent and diagnostic. It does not sharpen any envelope; it says which of the regulator's knowledge, questions, or protocol is limiting its efficiency, and therefore which adaptation channel would raise it.

Applications

The knowledge-centric perspective builds on Ashby's Law of Requisite Variety by emphasizing that successful outcomes depend not only on a system's range of possible responses, but also on its ability to select the right response for each disturbance. This requires internal “system knowledge” that maps disturbances to appropriate actions. As Francis Heylighen proposed in his “Law of Requisite Knowledge,” effective regulation demands more than variety - it demands informed selection[29]. This knowledge-centric lens provides a foundation for analyzing how systems - biological, technical, or organizational - achieve control not just through options, but through understanding. The model we present operationalizes this perspective by counting what the record shows and bounding the informational requirements a system must satisfy to achieve its observed level of regulatory performance.

In what follows, we apply this perspective to typing, software development, a published learning curve, intelligence testing, basketball, manual assembly, and industrial assembly lines. Each application reads its record as the program of the subsection “The record as a two-instruction program”: a 1 is an output instruction, which emits an acceptable closure, and a 0 is a non-output instruction, such as a lookup of a script, a pause to recall, a slot spent thinking, or an attempt that did not produce an acceptable outcome. Lifting the outputs to acceptability classes lets the particular content drop out, which is what response-equivalence classes are for.

In every application the effective depth of equation (5), N/Snet−1, is the number of non-output instructions per output instruction, and KEDE is the fraction of the program that is output. Both are exact given the declared length N; only the 1s are always observed. The applications are ordered by how their 0s are observed: one by one, inferred from time, counted as failed attempts, or inferred from a rated capacity. Each then states which conditions of the bridge to bits it meets: one 0 holds at most one binary discrimination (A5 and condition CΔ), every discrimination leaves a 0 (B2), and an output is emitted only once it is certain; where outputs are committed under uncertainty, the graded-uncertainty remark after Theorem 2 adds the terminal residual to the bound.

Where the regulator is a human and a 0 can deliver more than a binary answer, as a look at a script does, A5 fails but assumption H1 can take its place on the upper side. The count then bounds Knowledge To Be Discovered with the factor ch=ChΔ, where Ch is the declared human information throughput: in expectation by Corollary H, and with probability 1−α, from observed and declared quantities only, by Proposition H‴. What a look registers about later closures is episode-start information of those closures. Only the two applications in which a human types use this route; the robot of the appendix satisfies A5 and uses Theorem 2 and Corollary 2 directly.

Declaring the units of the program

Applying the model requires three declarations: what a 1 is, the closure; what one unit of capacity is, which fixes the length of the program; and what a 0 is, the non-output instruction. The first two fix the ratio N/Snet, which is exact in every case. A 0 is a unit of program text, not a query. A query is the discriminating operation that 0s carry out. When action units are counted directly, one 0 is one query. When capacity is inferred from time, a 0 is one slot: a query that takes longer than one slot spans several 0s, and condition CΔ forbids several queries in one slot; the excess of 0s over queries is the unit-inference error εΔ. The third declaration fixes how the ratio bridges to bits. A 0 bounds at most one bit of Knowledge To Be Discovered when it holds at most one binary discrimination (A5 and condition CΔ), and the bound covers the whole of Knowledge To Be Discovered when every discrimination leaves a 0 (B2). When an output is committed before it is certain to be acceptable, as a shot or a test answer is, the bound is shifted by the terminal residual uncertainty, by the graded-uncertainty remark after Theorem 2. The table summarizes the applications below.

For the two applications in which a human types, a third declaration is available: the human information throughput Ch of assumption H1. It bounds the bits a 0 can deliver by ch=ChΔ in expectation, even when the 0 holds a lookup that is not a binary question. It is a declared constant: a revised estimate changes the factor, and may change the declared Δ of these two applications, but not the counts.

Application What a 1 is What a 0 is Bridge to bits
Typing (observed one by one) A letter typed One query: one look at the script or one attempt to recall, one per interval Upper bound with factor ch under H1 and B2 (Corollary H); with probability 0.95, about 0.80 to 1.07 bits per letter for Ch between 3 and 4 bits/s (Proposition H‴). Without H1, only the wider-alphabet bound, which is weak because one lookup returns many letters
Software development (inferred from time) A slot with a typed symbol A 0.288 s slot without one Upper bound with factor ch under H1, B2 and B5Δ for non-typing slots, without CΔ and without excluding idle time; the high-probability form needs a declared average second moment of non-typing slots per symbol. Or unit-coefficient upper bound under CΔ, A5 and A2
Robot, appendix (observed one by one) A move A sensor reading from a declared family All three rungs of the ladder and every gap between them
Learning curve (inferred from time) A completed book A 0.46-month unit of capacity without a book Relative to the best block; CΔ fails at this grain, so the count does not bound bits
Intelligence testing (failed attempts) A correct answer A wrong answer: a step that did not yield an acceptable closure Upper bound shifted by the terminal residual, if every reasoning step leaves a 0
Basketball (failed attempts) A made shot A missed shot: a return to getting into a shooting position Upper bound shifted by the terminal residual, if every repositioning leaves a 0
Tightening screws (rated capacity) A screw tightened A machine-rated slot without a completed screw Upper bound, loosened because a human closure act spans several machine slots
Assembly line (rated capacity) A finished product A rated slot without one: stoppage, waiting, or defect Upper bound, loosened by stoppages and waiting; most discrimination was done at design time

N is a count of units, not a rate: it is the length of the program in the window, fixed by the second declaration. Under time-normalized counting the unit length Δ is fixed once, by the unit convention of section 6, and is not re-chosen to suit the observed rate of outputs; each application below states its Δ and where it comes from. A 0 is any non-output unit, whether a lookup, a pause, a wait, or a failed attempt, and not only a wrong outcome.

What each application folds. Section 6 defines the four kinds of unit (emit, find out, wait, retract) and how the two-instruction program folds them. The applications differ in what they fold:

  • the typing application and the robot fold nothing: every 0 is one lookup or one sensor reading;
  • the goal-model trace above folds retractions, but each retraction is a trial for a surviving letter, so nothing is loosened;
  • basketball and intelligence testing fold failures at production, each a trial for a later success, and omit the steps of finding out that end in success, which is a question of completeness, not of folding;
  • software development, tightening screws, and the assembly line fold waits, idle time and motor execution into 0s, which keeps the upper bound but loosens it;
  • the learning curve folds nothing new, but each of its 0s is a stretch of work long enough to hold many queries, a question of grain (CΔ), not of folding.

Typing the longest English word

Program reading. This is the application in which the 0s are observed one by one. The trace below is a program with two instructions, type (1) and lookup (0). Each letter typed is one closure, with a single acceptable symbol at each position, so the goal has a unique target. Each look at the script, and each pause to recall, is one execution of the lookup instruction, and each takes one interval. Where the scenario says that a lookup took two time intervals, it was two looks at the script, one to find the word and one to read the next letters, so it is written as 00. Every 0 is therefore one query. This telling observes the 0s directly, because the typist reports each look and each pause. The realized trace in the appendix retells the same trace as a black-box observer sees it, from the letters and the elapsed intervals alone, with the 0s inferred.

Let's use an example scenario to see Ashby's law applied to human cognition and knowledge work.

For that we'll have myself executing the task of typing on a keyboard the word “Honorificabilitudinitatibus”. It means “the state of being able to achieve honours” and is mentioned by Costard in Act V, Scene I of William Shakespeare's “Love's Labour's Lost”. With its 27 letters “Honorificabilitudinitatibus” is the longest word in the English language featuring only alternating consonants and vowels.

The way I will execute this task is to go to the "play text" or "script" of “Love's Labour's Lost”, look up the word and type it down. The manual part of the task is to type 27 letters. The knowledge part of the task is to know which are those 27 letters.

In order to track the knowledge discovery process I will put "1" for each time interval when I have a letter typed and "0" for each time interval when I don't know what letter to type.

I start by taking a good look at the word “Honorificabilitudinitatibus” in the script of “Love's Labours' Lost”. That takes me two time intervals. Then I type the first letters “H”, “o”, and “n”.I continue typing letter after letter: “o”, “r”. At this point I cannot recall the next letter. What should I do? I am missing information so I go and open up the script of “Love's Labours Lost” and I look up the word again. Now I know what the next letter to type is but acquiring that information took me one time interval. This time I have remembered more letters so I am able to type “i”,”f”,”i”,”c”,”a”,”b”,”i”. Then again I cannot continue because I have forgotten what were the next letters of the word, so I have to look it up again.in the script. That takes two more time intervals. Now I can continue my typing of “l”,”i”,”t”. At this point I stop again because I am not sure what were the next letters to type, so I have to think about it. That takes one time interval. I continue my typing with “u”,”d”,”i”. Then I stop again because I have again forgotten what were the next letters to type, so I have to look it up again in the script of “Love's Labours Lost”. That takes two more time intervals. Now I know what the next letter to type is so I can continue typing “n”,”i”.At this point I cannot recall the next letter. so I have to look it up again in the script. That takes two more time intervals. After I know what the next letter to type is I can continue typing “t”,”a”,”t”,”i”,”b”,”u”,”s”. Eventually I am done!

At the end of the exercise I have the word “Honorificabilitudinitatibus” typed and along with it a sequence of zeros and ones.



H o n o r


i f i c a b i



l i t


u d i



n i



t a t i b u s
0 0 1 1 1 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 1 1 1 1 1

In the table we have separated the manual work of typing from the knowledge work of thinking about what to type.

We made visible both the manual work and the knowledge discovery parts of a Knowledge Discovery process.

The first row of the table shows the knowledge I manually transformed into tangible outcome - in this case the longest English word. The second row of the table is the program I executed. There is a 0 for each time interval when I was missing information about what to type next, and a 1 for each time interval when I typed a letter. Each 0 is a query I needed to ask in order to acquire the missing information about what letter to type next. Each 1 is an output instruction, available because the letter was already known to me.

We know that there is knowledge applied when we see the tangible outcome of the process. We know there was knowledge discovered when we see there was at least one selection made.

In the exercise above we witnessed the discovery and transformation of invisible knowledge into visible tangible outcome.

KEDE calculation

We can calculate the KEDE for this sequence of outcomes.

KEDE=SN=2737=0.73

The effective depth of equation (5) is the number of non-typing intervals per typed letter, and since every 0 is one query, it is also the number of queries per letter:

KTD^neteff=NS−1=3727−1≈0.37.

Ten queries were asked for 27 letters. They come in six runs: an initial look of two queries, four lookups of one or two looks each, and one pause to recall. Each interval holds exactly one query and no query spans two intervals, so condition CΔ holds with equality and the unit-inference error is zero, εΔ=0. No letter was invalidated, so εinv=0.

The figure of 0.37 is not a number of bits under A5. A look at the script answers with a chunk of several letters, so its answer alphabet is very large, and A5 fails. The wider-alphabet remark after Theorem 2 still gives an upper bound, with each query weighted by the logarithm of its alphabet size, but with an alphabet this large the bound is weak. What the trace measures directly is the number of calls to the script per closure: in the terms of subsection 6.6, the 0s are calls to the script and the 1s are output symbols, and what each call returned belongs to the answer transcript, not to the program.

Because the typist is human, assumption H1 gives a usable bound in bits all the same. Declare an interval to be one keystroke duration, Δ=0.288 seconds, as in the software-development application. The declaration is made before the trace is read; the 37 units then occupy 10.656 seconds, 2.88 of them without a keystroke. Each look occurs in the episode of the letter it was made for, and its intervals are counted there. Whatever the script displays, those intervals deliver at most ch=ChΔ bits each about that letter, on average. What the look shows about the following letters is episode-start information of their episodes: they begin with little or nothing left to discover and spend no interval. The window is the whole task and no letter was invalidated, so Corollary H gives

K∗≤ch(1027−ζΔ)

bits per letter, where ζΔ is the realized excess of the letters' non-closure intervals over what the typist's knowledge state led one to expect. To remove it, declare B5Δ with a range of d=2 intervals per letter. Every episode uses at least the unit of its closure, so at most Jmax=N+1=38 episodes fit in the window, and Proposition H‴(a) gives, with probability at least 0.95, Λα/Snet=238ln20/2/27≈0.559 and

K∗≤ch(0.370+0.559)≈0.929ch.
Declared Ch (bits/s) ch Bound at ζΔ=0 (bits/letter) Bound with probability 0.95 (bits/letter) KEDE∗ at least, with probability 0.95
30.8640.3200.8030.555
3.471.0000.3700.9290.518
41.1520.4271.0700.483
102.8801.0672.6760.272

The allowance is large because the window holds only 27 letters; it shrinks as the inverse square root of the window for longer texts. The bound moves only with the declared Ch, and it is an upper bound only: H1 gives no lower bound, so no bracket and no rung of the query ladder follows from it.

The trace is stylized in one respect that H1 makes visible. A real look at a printed script takes longer than one keystroke, so under time-normalized counting it would span several intervals. That adds 0s and loosens the bound, but does not invalidate it. Under the stronger form of H1-a, stated for the disturbance, H1 also says what the looks did. The first look, two intervals long, then registered at most about 2ch bits in all, in expectation, yet five letters followed without another look. That is consistent only if the typist's knowledge state already made those five letters nearly certain once the look had identified the familiar prefix: the look disambiguated rather than transmitted. Under the weaker form used for the bound, H1 limits only what the look delivered about the first of the five letters.

The example also shows that query counts depend on the declared query family, while bits do not. With a single query “show the word” and a perfect memory, one call would serve all 27 letters. With only binary queries such as “is the next letter x?”, each forgotten letter would need about log226≈4.7 of them. The Knowledge To Be Discovered is the same in both cases; the query counts differ by more than an order of magnitude. In this trace the typist's limit is memory, not questions: each lookup could have shown the whole word, and the repeated lookups are the cost of forgetting it. That is a limit of the knowledge state, which only L2a removes.

The robot: a fully declared query family

The robot example of the appendix is the one case in which every element of the ladder is declared: the knowledge state is a posterior over moves, the query family is the set of sensor readings the robot can make, and each move is a closure. With posterior (12,14,18,18), Knowledge To Be Discovered is 1.75 bits. A protocol that always asks two questions spends 0.25 queries per move beyond it. Whether that excess is a protocol gap or a query-family gap depends only on which questions the sensors can answer, as shown after Theorem Q and in the KEDE factorisation. The robot is therefore the reference case against which the less fully declared applications below can be read.

Program reading. For one move, the program is the robot's reads followed by its commit. Under the protocol that always asks two questions every move is the program 001: two non-output instructions and one output. Under a protocol that asks one, two, or three questions according to the answers, a move is 01, 001, or 0001, and the average length is 1.75+1 instructions. The program does not record which move was made. The answers do: the 0s and 1s of the robot example in the appendix are answer values, such as the codeword 110, and form the answer transcript of subsection 6.6, not the program. Both have the same number of non-output symbols per move here, because every read is counted directly and is binary.

Measuring software development

In order to use the KEDE formula (8) in practice we need to know both S and N. We can count the actual number of symbols of source code contributed straight from the source code files. For N we want to use some naturally constrained value.

N is the maximum number of symbols that could be contributed for a time interval by a single human being.

In the below formula for N we want to use some naturally constrained value:

To achieve this, the following estimation is performed. We pick T = 8 hours of work because that is the standard length of a work day for a software developer.

To calculate the value of r we need to pick the symbol duration t.

The value of the symbol duration time t is determined by two natural constraints:

  1. the maximum typing speed of human beings
  2. the capacity of the cognitive control of the human brain

Typing speed has been subject to considerable research. One of the metrics used for analyzing typing speed is inter-key interval (IKI), which is the difference in timestamps between two keypress events. We see that IKI is defined equal to the symbol duration time t. Hence we can use the research of IKI to find the symbol duration time t. It was found that the average IKI is 0.238s [26]. There are many factors that affect IKI [6]. It was also found that proficient typing is dependent on the ability to view characters in advance of the one currently being typed. The median IKI was 0.101s for typing with unlimited preview and for typing with 8 characters visible to the right of the to-be-typed character but was 0.446s with only 1 character visible prior to each keystroke [7]. Another well-documented finding is that familiar, meaningful material is typed faster than unfamiliar, nonsense material[8]. Another finding that may account for some of the IKI variability is what may be called the “word initiation effect”. If words are stored in memory as integral units, one may expect the latency of the first keystroke in the word to reflect the time required to retrieve the word from memory[55].

Cognitive control, also known as executive function, is a higher-level cognitive process that involves the ability to control and manage other cognitive processes that permit selection and prioritization of information processing in different cognitive domains to reach the capacity-limited conscious mind. Cognitive control coordinates thoughts and actions under uncertainty. It's like the "conductor" of the cognitive processes, orchestrating and managing how they work together. Information theory has been applied to cognitive control by studying the capacity of cognitive control in terms of the amount of information that can be processed or manipulated at any given time. Researchers found that the capacity of cognitive control is approximately 3 to 4 bits per second[32][33], That means cognitive control as a higher-level function has a remarkably low capacity.

Based on the above research we get:

  1. Maximum typing speed of human beings to be r=1/t=1/0,238=4.2 symbols per second
  2. Capacity of the cognitive control of the human brain to be approximately 3 to 4 bits per second. A binary question conveys at most one bit in expectation, so this capacity allows at most 3 to 4 fully informative binary questions per second.
  3. Asking questions is an effortful task and humans cannot type at the same time, so a slot is spent either typing a symbol or finding out what to type. For one slot to hold at most one question, condition CΔ, the slot rate should not exceed the question rate.

Since 4.2 symbols per second is a higher rate than 3 to 4 questions per second, we need a slot rate between 3 and 4 per second.

In order to get a round value of maximum symbol rate N of 100 000 symbols per 8 hours of work we pick symbol duration time t to be 0.288 seconds. That is a bit larger than what the IKI research found but makes sense when we think of 8 hours of typing. Having t of 0.288 seconds makes a symbol rate r of 3.47 symbols per second. That is between 3 and 4 and matches the capacity of the cognitive control of the human brain.

The cognitive-control figure enters the rest of this section as the declared human information throughput Ch of assumption H1. With Ch=3.47 bits per second and Δ=0.288 seconds, the per-interval capacity is ch=ChΔ≈1 bit. If a later estimate of Ch differs, the factor ch changes with it. The slot stays anchored on the duration of one closure act, a keystroke, and is constrained by the inter-key interval findings above; it is not shortened to keep ch at 1, and a coefficient other than 1 is carried explicitly.

We define CPH as the maximum rate of characters that could be contributed per hour. Since r is 3.47 symbols per second we get CPH of 12 500 symbols per hour. We substitute T = h and r=CPH and the formula for N becomes:

N= h×CPH

where h is the number of working hours in a day and CPH is the maximum number of characters that could be contributed per hour. We define h to be eight hours and get N to be 100 000 symbols per eight hours of work.

Total working time consist of four components:

  • Time spent typing (coding)
  • Time spent figuring out WHAT to develop
  • Time spent figuring out HOW to code the WHAT
  • Time doing something else (NW)

The ideal system is one in which the time spent doing something else, TNW, is zero. The bound derived below does not need that assumption: slots spent doing something else are counted as 0s, which makes the bound looser but leaves it valid. Using the new formula for N, the effective depth of equation (5) becomes

K^ = h×CPH S − 1

Note, that since N is calculated per hour so S also needs to be counted in an hour.

We see that the more symbols of source code contributed during a time interval, the fewer of its slots were non-output, and the smaller the effective depth. We want to compare the performance of different software development processes in terms of the efficiency of their knowledge discovery processes. Hence we rearrange the formula to emphasize that.

Sh×CPH=11+K^

(8)

The right hand part is the KEDE we defined earlier. Thus, we define an instance of the metric KEDE - the general metric that we introduced earlier. This version of KEDE is for the case of knowledge workers that produce tangible outcome in the form of textual content:

KEDE=Sh×CPH

(9)

KEDE from (9) contains only quantities we can count or declare in practice: the symbols S are counted, and the capacity h×CPH is declared. KEDE also satisfies all properties we defined earlier. It has a maximum value of 1 and minimum value of 0; it equals 0 when K^ is infinite; it equals 1 when K^ is zero; it is anchored on a natural constraint - the maximum typing speed of a human being.

If we convert the KEDE formula into percentages then it becomes:

KEDE=Sh×CPH×100%

(10)

We can use KEDE to compare the knowledge discovery efficiency of software development organizations.

Program reading. Here the 0s are not observed; they are inferred from time. One unit is a slot of Δ=0.288 seconds, a slot with a typed symbol is a 1, and a slot without one is a 0. Each typed symbol closes an episode, and the slots without a symbol before it are that episode's non-closure intervals. A pause before a burst of typing therefore belongs to the episode of the burst's first symbol; what the developer settles during it about the rest of the burst is episode-start information of the later symbols, which then spend no slot. The reading of KEDE as a bound on Knowledge To Be Discovered rests on assumptions about the developer, none of which is A5 or condition CΔ; a slot may hold a question with a large answer, several questions, or none.

  • H1. The information about a symbol that the developer takes in during the slots of its episode is at most Ch times their duration, in expectation.
  • H1-b and B2. Each symbol is typed on what was registered before it is typed. Information taken in while typing earlier symbols, by reading ahead or by settling the next statement while finishing the current one, is episode-start information of the later symbols. It lowers their starting Knowledge To Be Discovered and does not invalidate the bound.

A2 is satisfied by convention: every slot without a typed symbol, idle or not, is attributed to the episode of the next symbol. Idle slots lengthen episodes and loosen the bound without invalidating it, because H1 bounds information per unit of time and idle time adds time without information. Under these assumptions Corollary H gives, in bits per surviving symbol,

K∗≤ch(K^−ζΔ−εinv−εbd), KEDE∗≥11+ch(K^−ζΔ−εinv−εbd).

K∗ is the Knowledge To Be Discovered left at the start of each symbol's episode. What the developer works out while typing is not part of it; the bound concerns what remained to be discovered in the slots without typing.

An observable bound. The fluctuation ζΔ is the realized excess of non-typing slots over what the developer's knowledge state led one to expect. Most symbols have none and a few have a long pause, so a declared range would be thousands of slots and the range-based allowance useless: with d=12500 slots, one working hour, it exceeds 200 bits per symbol for a working day. The second-moment version of Proposition H‴ needs instead a declared average v¯, over the window's symbols, of the conditional second moment of non-typing slots, in slots squared. The number of completed episodes is the observed Sgross, so the realized-count variant needs no log of bursts. As an illustration, declare v¯=9⋅103, as when one symbol in ten follows a pause whose second moment is 9⋅104, a root-mean-square pause of 300 slots or about 86 seconds. Suppose each working day of N=100000 slots ends with Snet=20000 surviving symbols, so K^=4 and KEDE^=0.2, with no retracted symbols and no boundary load. With probability at least 0.95, using the realized-count variant with G=20:

Window Λα/Snet Upper bound on K∗ (bits/symbol) at Ch = 3 / 3.47 / 4 bits/s KEDE∗ at least, at Ch = 3 / 3.47 / 4 bits/s
1 day2.465.58 / 6.46 / 7.450.152 / 0.134 / 0.118
20 days0.553.93 / 4.55 / 5.240.203 / 0.180 / 0.160

The allowance shrinks as the inverse square root of the window, so for monthly or longer windows the observable bound approaches 1/(1+ch(K^−Wobs/Snet+Qbd−/Snet)). The value v¯ is a declaration about the developer, like d: it can be checked against pause-length distributions in keystroke logs of comparable work, and a pause distribution without a finite second moment defeats it.

The developer's query family, the documentation, tests, colleagues, and searches available to them, is not declared. KEDE therefore cannot say whether a low value reflects missing knowledge, missing questions, or wasted ones. Comparing teams that work with the same tools holds the query family approximately fixed and makes the comparison one of knowledge and protocol. A human typing calibration also does not by itself quantify the complete knowledge work of a software organization: automatically generated, pasted or tool-inserted text needs its own closure and capacity convention.

Calculating Effective Depth from a Reported Learning Curve

Reported learning curves usually give time, cost, or speed as a function of accumulated practice. They do not directly report the internal selection trace of the actor. Nevertheless, if the repeated output can be interpreted as a sequence of closure acts, the learning curve can be re-expressed in Knowledge Discovery terms by defining the interval capacity from the best observed closure rate.

The demonstration below uses Ohlsson's data on Isaac Asimov's book-writing career[59]. The paper says Asimov wrote nearly 500 books over more than 40 years, treats one book as one practice trial, and groups the data into blocks of 100 books because individual books vary greatly in length and complexity. In that study, the completion of one book is treated as one completed knowledge-discovery episode. Therefore, for this reconstruction, one completed book is treated as one surviving closure.

Let Ii denote block i . Let S(Ii) be the number of surviving closures in the block, and let N(Ii) be the counted closure capacity of the same elapsed interval.

Under the time-normalized unit convention, counted capacity is defined by the elapsed duration of the interval divided by the selected unit length:

N ( I ) = | I | Δ

Learning-curve literature already treats the asymptote as the best possible performance limit[60]. At asymptotic performance, the actor still performs work, but no longer pays the earlier knowledge-discovery penalty. In many published learning curves, the asymptote is not directly known. Ohlsson's Asimov paper is exactly like this. Ohlsson states that, after mastery, the tail of the curve approximates a horizontal line, and the asymptote represents the best possible performance. For this reconstruction, the unit length Δ is not taken from an external physical limit. It is calibrated from the best observed block in the reported learning curve. In Asimov's data, the best observed closure rate occurs in the fourth block: 100 books in 46 months. Therefore:

Δ = 46 months 100 books = 0.46 months per book-closure

Equivalently, the empirical closure capacity rate is:

rmax = 100 46 ≈ 2.174 book-closures per month

The capacity of each block is then calculated as the number of book-closures that could have been produced in that block's elapsed time if the process had operated at the best observed closure rate:

N ( I i ) = | I i | Δ = | I i | ⋅ r max

From the effective one-bit net estimator in equation (5), the effective depth of each block, the non-output capacity per book, is then:

KTD ^ eff net ( Ii ) = N ( Ii ) S net ( Ii ) - 1 .

And the corresponding operational Knowledge Discovery Efficiency is:

KEDE ( I i ) = S ( I i ) N ( I i ) = 1 1 + KTD ^ eff net ( I i )

Calculation

Block Elapsed time Surviving closures S(I) Capacity N(I) KTD ^ eff net ( I i ) KEDE ( I i )
1 237 months 100 books 515.22 book-closures 4.152 0.194
2 113 months 100 books 245.65 book-closures 1.457 0.407
3 69 months 100 books 150.00 book-closures 0.500 0.667
4 46 months 100 books 100.00 book-closures 0.000 1.000
5 42 months 90 books 91.30 book-closures 0.014 0.986

Interpretation

This conversion reads the published learning curve as an effective-depth curve, KTD^effnet (Ii) against block i. The best observed block defines the empirical closure-unit length: Δ=0.46 months per book-closure. Earlier blocks are then interpreted relative to that later demonstrated capacity.

In the first block, the elapsed interval had an empirical capacity of approximately 515 book-closures, but only 100 surviving closures were produced. The effective depth is therefore 4.152. In operational terms, this means that for each surviving book-closure, the process consumed enough elapsed capacity for approximately 4.152 additional non-surviving or pre-closure units of discovery burden. By the third block, the same method gives KTD^effnet (I3) ≈0.500 . The process still contains substantial discovery burden, but much less than in the early career phase. By the fourth block, the observed process reaches the empirical capacity anchor, so KTD^effnet (I4) =0.000 by calibration.

Notice that the effective depth does not necessarily fall to zero in the final block. The final block produced 90 books in 42 months. At the best observed rate, that interval had capacity for approximately 91.30 book-closures. Therefore the final block has a small positive KTD^effnet (I5) ≈0.014 and KEDE≈0.986 . This is conceptually preferable to forcing the last observation to be the zero-depth point merely because it appears last in the sequence.

Methodological caveat

This is not a direct measurement of Asimov's internal knowledge discovery process. It is a reconstruction from reported learning-curve data. The calculation assumes that completed books are comparable enough to serve as closure units and that the best observed block is a reasonable empirical estimate of book-writing action capacity. Differences in book length, genre, research burden, publishing process, and external constraints may also affect the observed rate. Therefore, the result should be read as learning-curve-calibrated, not as a direct ledger measurement of Knowledge To Be Discovered (KTD).

The demonstration nevertheless shows how published learning curves can be translated into Knowledge Discovery terms. A learning curve becomes a visible trace of declining discovery burden: as knowledge accumulates, the number of capacity units consumed per surviving closure falls, and operational KEDE rises.

Program reading. Here a 1 is a completed book and a 0 is one empirical book-closure-equivalent of capacity, 0.46 months of work, that produced no book. The effective depths in the table are exact as program text: non-output units per book. Their bridge to bits is weak at this grain. One unit of 0.46 months holds very many discriminations, so condition CΔ fails, and the counts cannot be read as upper bounds on Knowledge To Be Discovered in bits. They measure the non-output capacity spent per book relative to the best observed block, which calibration sets to zero. A finer grain, such as working days, would be needed to bring the reading within the conditions of Theorem 2.

The curve also cannot say which adaptation channel produced the decline. A falling effective depth is what L2a produces, as knowledge of the craft accumulates; it is also what L2b produces, as better sources, tools, or collaborators give access to better questions, and what L2c produces, as working habits improve. A change in the mix of books would produce it too, which is a failure of comparability rather than any kind of adaptation. The learning curve records that the cost per closure fell, not why.

Testing Intelligence

Today all measure intelligence by the power of appropriate selection (of the right answers from the wrong). The tests thus use the same operation as is used in the theorem on requisite variety, and must therefore be subject to the same limitation. (D, of course, is here the set of possible questions, and R is the set of all possible answers). Thus what we understand as a man's “intelligence” is subject to the fundamental limitation: it cannot exceed his capacity as a transducer. (To be exact, “capacity” must here be defined on a per-second or a per-question basis, according to the type of test.)[3]

We can also use our model to the testing of human and AI intelligence. We infer this capacity from performance under variety - i.e., how many different problems a system or a person can solve correctly.

The dominant mathematical models for testing intelligence by the number of answered problems are benchmark datasets like MMLU, GSM8K, MATH, and FrontierMath. These models measure intelligence by the raw count or percentage of correctly solved problems, with more advanced benchmarks designed to minimize guessing and require deep reasoning.

From the knowledge-centric perspective:

  • The disturbances are the questions Q={q1,q2,q3,...qn}
  • The person gives responses R={r1,r2,r3,...rn}
  • The outcomes are E={e1,e2,e3,...en} ∈{0,1}
So: Intelligence is the capacity to consistently produce 1s in E, despite the variety in D.

Several mathematical models and benchmark datasets are used to evaluate intelligence - especially artificial intelligence (AI) - by measuring the number and complexity of math problems answered correctly. These models serve as standardized tests for both AI and, by analogy, human intelligence[52].

Massive Multitask Language Understanding (MMLU):

  • MMLU is a widely used benchmark that tests AI models on a broad range of subjects, including mathematics at various levels (high school, college, abstract algebra, formal logic).
  • The test is typically formatted as multiple-choice questions, and performance is measured by the percentage of correct answers out of the total number of questions
  • For example, advanced AI models have achieved up to 98% accuracy on math sections of MMLU, indicating high proficiency in standard math tasks but not necessarily deep reasoning

Grade School Math 8K (GSM8K)

  • GSM8K is a dataset of 8,500 high-quality, grade school-level word problems designed to test logical reasoning and basic arithmetic skills.
  • Evaluation is based on exact match accuracy: the number of problems answered exactly correctly divided by the total number attempted
  • This benchmark is used to assess step-by-step reasoning and the ability to handle linguistic diversity in problem statements.

MATH (Mathematics Competitions Dataset)

  • MATH consists of problems from high-level math competitions (e.g., AMC 10, AMC 12, AIME), focusing on advanced reasoning rather than rote computation.
  • Performance is measured by the percentage of correct answers, with human experts (e.g., IMO medalists) providing a reference for top-level performance
  • The dataset is challenging for both humans and AI, with LLMs typically scoring much lower than expert humans.

FrontierMath[53]

  • FrontierMath is a new benchmark featuring hundreds of original, expert-level math problems spanning major branches of modern mathematics.
  • Problems are designed to be "guessproof" and require genuine mathematical understanding, with automatic verification of answers
  • The benchmark is used to assess how well AI models can understand and solve complex mathematical problems, similar to human performance.

In human intelligence testing, Psychometric models such as IQ tests or psychometric approaches also use the number of correctly answered problems as a key metric. These tests are standardized, and the raw score (number of correct answers) is often converted into a scaled score or percentile.

As an example we will use the Exact Match metric as the evaluation method[52]. Given that each question in our benchmark dataset has a single correct answer and the model produces a response per query, Exact Match ensures a rigorous evaluation by comparing the extracted answer to the ground truth.

Let ŷi represent the extracted answer from the model's outcome for the ith question, and let yi be the corresponding ground truth answer. The Exact Match accuracy is computed as:

Exact Match (%) = ∑ i=1 N 𝟙 ( normalize ( ŷi ) = normalize ( yi ) ) N x 100

where:

  • N is the total number of evaluated questions.
  • 𝟙(⋅) is the indicator function, returning 1 if the extracted model response matches the ground truth after preprocessing, and 0 otherwise.
  • normalize(⋅) is a function that standardizes formatting, trims spaces, and normalizes numerical values.

The knowledge discovery efficiency of an LLM can be calculated as:

Exact Match accuracy=SN=KEDE
where S is the number of correct answers and N is the total number of evaluated questions..

Let's pick the case of the performance of GPT-4o on the MATH benchmark, which achieved a significantly lower accuracy of 64.88%, lagging behind its peer models[52]. The effective depth of equation (5) is then:

KTD^neteff=1KEDE^−1=10064.88−1≈0.541.

Program reading. The record of a test is a program in the same two-instruction language as the typing trace. A correct answer is an output instruction, an acceptable closure. A wrong answer is a non-output instruction: a step that did not yield an acceptable closure. Because acceptability is judged when the answer is produced, a wrong answer is a failed attempt, not a closure, and the ledger books it as a 0 rather than as an invalidation. The figure of 0.541 is therefore the number of non-output instructions per output instruction: about 0.54 failed steps for every correct answer.

In the terms of “Failed and retracted outputs”, a wrong answer is a failure at production. In a benchmark each item is answered once, so a wrong answer is not a trial for the same target as a later correct one, and it is not a stage of any correct answer's path. Its 0 is load charged to the correct answers. Such load can only add to the count, so it loosens the upper bound without breaking it. If the solver were allowed to retry an item, a wrong answer followed by a correct one would be the case of the remark “When a failed or retracted output is a query”, and the wrong answer would be a stage of the correct answer's path.

The bridge from this count to bits has two conditions specific to testing. First, an answer is committed before the solver can be certain it is right, which is why some answers are wrong. The graded-uncertainty remark after Theorem 2 then applies: the count bounds average Knowledge To Be Discovered from above once the average terminal residual uncertainty at the moment of answering is added to it. Second, the bound covers every discrimination only if every discrimination leaves a 0. In a benchmark, reasoning steps that end in a correct answer leave no trace, so the program records the steps that failed but not all the steps that were taken. Recording the solver's intermediate steps, or the time it spends per item, would complete the program.

Ashby's own remark at the head of this subsection points the same way. The test items are the disturbances, and the capacity he names is defined per second or per question: the program has to be counted at the grain of the solver's steps for its length to measure that capacity.

Basketball Game

We can also use this model to assess the performance of a basketball player.

  • Timeframe is a basketball game.
  • We observe N total shot attempts.
  • S of them are successful (shot made).
  • We record a binary outcome sequence
    E∈{0,1}N
  • The empirical success rate is:
    θ=S​N

Interpretation using Ashby's Law

The basketball shot is a regulation problem: the player must control their body and respond to the game environment to produce the desired outcome. The player is faced with a series of disturbances (D) in the form of different shots to make under different conditions. The player responds with a selection, drawn from their internal skills (regulatory variety R) in the form of different shooting techniques. Each shot is uncertain whether it will be successful. The outcome E is whether the shot is made (1) or missed (0).

Over N shots, the success rate

θ=S​N
reflects how often the player's internal variety is sufficient to absorb the variety in the environment - an operational measure of regulatory success.

In basketball and intelligence testing, S/N happens to look like a success rate. That is only because every non-closure unit there is a failed attempt, a consequence of how units are defined in those cases, not a general identity.

In this case, θ becomes a practical proxy for how often the regulator (player) has sufficient internal variety to absorb the disturbance presented by the game. However, it is important to note that this is a simplified model and does not account for all the complexities of basketball performance. For example, the player may have different success rates depending on the type of shot, the position on the court, or the level of defense. These factors can all affect the player's ability to regulate their performance and should be considered when interpreting the results. Thus θ is a useful heuristic for P(E=1), but the full picture includes the quality of the mapping from disturbances to responses, the regulatory relation, not just the quantity of successes.

Applying the Model

NBA keeps track of field goal attempts and makes for each player. The most field goal attempts by a player in a single NBA game is 63, achieved by Wilt Chamberlain during his legendary 100-point game against the New York Knicks on March 2, 1962 We take this as the natural constraint so N=63. We can also take the number of successful shots S=36, which is the most field goals made in a single game by a player[13].

We can calculate the KEDE for this sequence of outcomes.

KEDE=SN=3663=0.571

The effective depth of equation (5) is:

KTD^neteff=NS−1=6336−1=0.75.

The 36 made shots are output instructions and the 27 misses are non-output instructions, 0.75 of them per make.

We've turned a real-world sequence of basketball shots into a count of non-output instructions per made shot, measured in attempts rather than in time.

We can also use this model to assess the performance of a basketball team. In this case the success rate coincides with the field goal percentage (FG%) of the team which is the percentage proportion of made shots over total shots that a player or a team takes in games. There is a statistical distribution for NBA field goal percentage (FG%) [10]. Analysts and researchers often study the distribution of FG% across players or teams to understand scoring efficiency and trends[11]. The NBA record for the highest FG% in a single game by a team is 69.3%, set by the Los Angeles Clippers on March 13, 1998, when they made 61 of 88 shots[12].

For example, in the 2023-24 season, team FG% ranged from about 43.5% (lowest) to 50.6% (highest), with the league average typically falling in the mid-to-high 40% range[11]. If we take the average FG% of 45%, we can calculate the effective depth of equation (5):

KTD^effnet=1KEDE−1=1FG%−1=10.45−1≈1.22

That means that the average team executes about 1.22 non-output instructions, missed shots, for every made shot.

Program reading. The player, like the typist, executes a program with two instructions: shoot, and get into a shooting position. A made shot is an output, an acceptable closure. A miss is not a closure: it fails when it is produced, so the ledger books it as a failed attempt, a 0, after which the player must again work into a position from which to shoot. The effective depth of 0.75 is the number of those steps per made shot.

Two conditions govern the bridge from this count to bits. The first is that a shot is committed before the player can be certain it will go in. The graded-uncertainty remark after Theorem 2 then gives the upper bound with the average terminal residual uncertainty added. A small example shows why the residual matters. Suppose a player has two equally likely ways to release a shot, only one of which works, and learns from a miss which one it is. Knowledge To Be Discovered is one bit. The expected number of misses per make is one half. The count falls short by half a bit, and that half bit is exactly the average terminal residual: half the time the player scores with the first shot while still uncertain by a full bit. In this example the miss is a trial for the same target as the make that follows, so it is a stage of that make's path, the case of the remark “When a failed or retracted output is a query” in “Failed and retracted outputs”. The second condition is completeness. With attempt counts, a repositioning step leaves a 0 only when it ends in a miss; repositioning that ends in a made shot leaves none. Counting the game in time slots, as the typing trace does, would record every step and complete the program.

Tightening screws

We can apply our model to motor tasks such as furniture assembly. In this context, a natural unit of outcome - or “quantum” - is the tightening of a single screw.

Skilled workers engaged in manual assembly tasks can typically insert and tighten standard screws at a rate of 6'-12 screws per minute under optimal, repetitive conditions - such as those found in furniture construction or industrial assembly lines. In contrast, automated screw-tightening machines can achieve significantly higher rates, often between 30 and 60 screws per minute [34] More complex manual tasks, such as high-torque applications involving ratchets or Allen keys, typically reduce the rate to 2'-4 screws per minute due to the increased effort and precision required. In surgical or medical contexts, such as orthopedic screw insertion, accuracy and the avoidance of overtightening are paramount; here, rates often fall to 1'-2 screws per minute, or approximately one screw every 30'-60 seconds [46].

Context Typical Rate (screws/minute) Notes
Automated (machine) 30'-60 For comparison, not manual
Fast, repetitive tasks 6'-12 Assembly line, minimal torque required
High-torque/manual 2'-4 Metalwork, ratchets, Allen keys
Surgical/precision 1'-2 Orthopedic, high accuracy, low speed

The key observation is that rates decrease as torque, task complexity, or required precision increases. If we take the machine rate as the maximum possible outcome N and the observed human rate as S, we can count how many machine-rated slots the human operator spends per screw that complete none, the effective depth of equation (5).

KEDE=SN KTD^effnet=1KEDE−1=NS−1=6012−1=4

This implies that, for every screw, the human operator's program contains about four machine-rated slots without a completed screw.

The rate at which a person tightens screws depends on various factors, including:

  • Screw type and size
  • Material being fastened
  • Required torque
  • Tool used (screwdriver, ratchet, etc.)
  • Operator skill and fatigue
These constitute the disturbance variety D faced by the human operator. The operator, acting as a regulator, responds with selections from their internal repertoire of skills - the regulatory variety R.

This interpretation aligns with existing research, which suggests that task difficulty directly influences the amount of information a task imparts [47, 48]. When difficulty is appropriately matched to the individual's skill level, the task yields maximal informational value [49], and the time required reflects the interaction between task complexity and the individual's regulatory capacity [50].

Using our model, we transform a sequence of real-world actions in furniture assembly into a count of rated slots per screw tightened. By the program reading below, that count is an upper bound on the Knowledge To Be Discovered per screw, and a loose one.

Program reading. A 1 is a screw tightened and a 0 is a machine-rated slot without a completed screw. The ratio of four is exact as program text. Its bridge to bits is loosened by the choice of slot. The unit convention anchors the unit length on the duration of one closure act, and here the slot is the machine's closure time, which is shorter than the human's. A human screw-tightening therefore spans several slots, and the slots spent turning the screwdriver are booked as 0s although they belong to the output act, not to finding out what to emit. Under A2 the figure of four remains an upper bound on Knowledge To Be Discovered per screw, but a loose one, and the true value may be much smaller. Anchoring the slot on the operator's own best closure time, as in the learning-curve application, removes this slack.

Assembly Line

We can also use this model to assess the knowledge discovery efficiency of an assembly line.

The assembly line is a system that transforms raw materials into finished products. The assembly line has a set of disturbances (D) in the form of different raw materials, machines, and processes. The assembly line responds with a selection, drawn from its internal structure (R) in the form of different machines, processes, and workers.

From a knowledge-centric perspective most of the knowledge discovery happens in the design phase of the assembly line. This is the planning for design, fabrication and assembly. This activity has also been called design for manufacturing and assembly (DFM/A) or sometimes predictive engineering. It is essentially the selection of design features and options that promote cost-competitive manufacturing, assembly, and test practices[51]. Thus most of the disturbances D are already absorbed by the design of the assembly line. The workers therefore find most of the knowledge they need already built into the line and its operating procedures.

Assembly line efficiency (AE) is the ratio of the outcome to the maximum possible outcome, often expressed as a percentage.

The efficiency of the assembly line can be calculated as:

AE=SN=KEDE
where S is the actual outcome and N is the maximum possible outcome.

We can assume that an assembly line is designed to produce a certain number of successful products (S) with a maximum rate of N products per hour. So for example, a shoe manufacturer has an actual outcome of 100 shoes per day, and a maximum potential outcome of 120 shoes per day. Their production line efficiency would be 83%. Now, we can calculate the effective depth of equation (5), the capacity left unconverted per finished shoe:

KTD^neteff=1AE−1=120100−1=0.2

To optimize the AE, companies can apply DFA guidelines, such as minimizing the number and variety of parts, standardizing the fasteners and connectors, and simplifying the assembly sequence and orientation[51].

Interpreting the results involves a comprehensive analysis of the data to understand where and why inefficiencies occur. In general, the higher the AE, the better the design. On the other hand, AE close to 100% might indicate under-utilised capacity. It's essential to compare high efficiency with industry capacity standards to determine if an increase in production is feasible and beneficial.

If AE is consistently below industry benchmarks, this could highlight several potential issues:

  • Machinery: It may indicate that machines are outdated, malfunctioning, or not suitable for the required tasks.
  • Labour Skills: Low efficiency might be due to workforce training gaps.
  • Process Design: Sometimes, the workflow or layout of the production line itself causes inefficiencies.

Program reading. A 1 is a finished product and a 0 is a rated slot without one. The 0s are of three kinds: stoppages, waiting, and defective units, each a slot that did not yield an acceptable closure. A2 books all three as non-output instructions. Defects are steps of finding out what works; stoppages and waiting are mostly not, so they loosen the upper bound without invalidating it. The figure of 0.2 is therefore an upper bound on the Knowledge To Be Discovered per shoe that the running line must still find out.

The application illustrates a point made with rule-level staged selection. A well-designed line holds a response rule that already maps the expected disturbances to acceptable responses, so its programs contain few 0s. The discriminations were made when the line was designed, in the selection of the rule, and design for manufacturing and assembly is the name of that earlier search. A high assembly efficiency records that the search was done well; it does not record its cost.

Remark — Applicability in principle. Wherever a finite entropy bound limits the number of physically distinguishable states of a system, those states admit a finite binary encoding, so the worst-case number of binary queries needed to identify any one of them is finite. In holographic settings the bound is A/(4lp2ln2) binary discriminations for a region of boundary area A, with lp the Planck length. The binary query, the unit of the estimator, is therefore available in principle for any such system. This is a statement about applicability, not a way to fix the capacity of a measurement window.

Appendix

Example: A Two-Dimensional Parabola

Consider the parabola as an example of the distinction between a variable, the set of its possible values, the set of realized values, and the graph of a mapping.

Let D be the disturbance variable, and let its set of possible values be

D = R

Let Z be the outcome variable, with possible outcome-values

Z = R ≥ 0

and define the realized outcome map by

z ρ : D → Z
z ρ ( d ) = d 2

The parabola is then the graph

G = { ( d , z ) ∈ D × Z : z = d 2 }

Thus:

  • the x-axis contains disturbance-values d ∈ D
  • the y-axis contains outcome-values z ∈ Z
  • the parabola itself is the graph of the map z ρ (d)

Distinct disturbance-values may yield the same outcome-value:

z ρ ( - 2 ) = 4 , z ρ ( 2 ) = 4

The repetition belongs to the mapping, not to the set Z . The set of possible outcome-values still contains the value 4 only once.

Where are E and its essential-variable region η?

The essential-value space E appears only once one specifies how outcomes are evaluated. There are two natural cases:

Case 1: The y-axis already represents the essential values

If the outcome-values themselves are the essential values, then

E = Z

and the evaluation map is simply the identity:

φ : Z → E , φ ( z ) = z

In that case, E is represented directly by the y-axis and the essential-variable region is literally a subset of the y-axis. For example,

η = [ 0 , 4 ] ⊆ E

Geometrically, η is the allowed vertical segment on the y-axis.

Then the question of successful regulation becomes: z ρ [ D ] ⊆ η

If D = R then z ρ [ D ] = [ 0 , ∞ ] so success fails if η = [ 0 , 4 ], because outcomes larger than 4 occur. But if disturbances are restricted to D = [ - 2 , 2 ] then z ρ [ D ] = [ 0 , 4 ] so success holds since η = [ 0 , 4 ].

Case 2: E is an evaluative space distinct from the y-axis

If the y-axis shows fine-grained outcomes, while regulation cares only whether those outcomes are acceptable, then define

E = { acceptable , unacceptable }

with evaluation map

φ : Z → E
φ ( z ) = { acceptable if 0 ≤ z ≤ 4 unacceptable if z > 4 }

Then the y-axis is still Z but E is now a coarser evaluative variable and the essential-variable region is

η = { acceptable } ⊆ E

In this second case, E is not directly identical with the y-axis unless we add the extra evaluation layer. Rather, the y-axis is first mapped into an evaluative space by φ.

In one sentence, in the parabola example, the curve shows the mapping from disturbance-values to outcome-values; E appears only after we decide how those outcome-values are to be evaluated for survival or acceptability; and η is the acceptable part of that evaluative space.

The Parabola as Possibility, and Realization as a Subset of Possibility

This full graph G is the possible disturbance-outcome relation generated by the mapping.. Actuality appears only after a realized disturbance subset D r ⊆ D is specified.

Then the set of realized outcome-values is

Y = z ( D r ) ⊆ Z

and the realized part of the graph is

G r = { ( d , z ( d ) ) : d ∈ D r } ⊆ G

Thus, the x-axis contains possible disturbance-values, the y-axis contains possible outcome-values, the full parabola represents the possible disturbance–outcome relation, and only a subset of its points need be realized in fact.

For example, if the realized disturbances are

D r = { - 2 , 0 , 3 }

then the realized outcome-values are

Y = { 0 , 4 , 9 }

not because these are all possible outcome-values, but because these are the values generated by the disturbances that actually occurred.

Example of a Human Knowledge Discovery Process with Goal-Model Revision

We now apply the same set-based structure to a simple human knowledge-discovery task: typing the word “Honorificabilitudinitatibus”.

The authoritative target word is:

Honorificabilitudinitatibus

It has 27 letters. Let P={1,...,27} be the set of target-word positions, and let A be the set of possible typed symbols. Let wi denote the authoritative correct symbol at position i in the target word.

Outcome-value space

The outcome-value space records what symbol was typed for what intended target position. Thus we define:

Z = P × A

An outcome-value has the form:

z = ( p , a ) ∈ Z

where p is the intended position in the target word, and a is the typed symbol. For example, (1,H) means that the symbol H was typed for position 1.

Essential-variable space

In this example, both the outcome-value space and the essential-variable space are two-dimensional. The two dimensions are target-word position and symbol. Thus the outcome-value space has coordinates (p,a) , and the essential-variable space uses the same two goal-relevant dimensions: which symbol occupies which target-word position for purposes of judging success.

There are two equivalent ways to read this example.

Reduced correspondence. In the reduced case, the concrete outcome-value is already the essential-variable value:

E = Z = P × A
φ = id E

So, in the reduced case:

z = (p,a) = e

Bijective correspondence. In the bijective case, the outcome-value space and essential-variable space are conceptually distinct but have the same two-dimensional coordinate structure:

Z = P × A
E = PE × AE

where PE is the essential-variable coordinate for target position, and AE is the essential-variable coordinate for the symbol occupying that position. An essential-variable value may be written as e = (p,a)E , where e means: “the goal-relevant state in which symbol a occupies target position p .”

Thus every relevant outcome-value corresponds to exactly one essential-variable value, and every relevant essential-variable value corresponds to exactly one outcome-value.

The outcome-to-essential-variable map is:

φ ( (p,a) ) = (p,a)E

This map is bijective on the relevant modeled spaces. The concrete outcome (p,a) and the essential-variable value ep,a are not the same object, but they carry the same distinctions for this regulatory model.

Authoritative acceptable essential-variable region

The authoritative acceptable essential-variable region is the set of goal-relevant states corresponding to the correct position-symbol pairs in the target word:

η* := { ei,wi : i ∈ P } ⊆ E

For example:

e1,H ∈ η*

but:

e1,x ∉ η* and e1,q ∉ η*

The authoritative acceptable region does not change during the exercise. The target word remains the same. The Shakespeare text did not change. The required symbol at position i did not change.

Authoritative acceptable outcome-set

The authoritative acceptable outcome-set is the pullback of the authoritative acceptable essential-variable region along φ :

O* := φ −1 [ η* ]

In the bijective case, this gives:

O* = { (i,wi) : i ∈ P } ⊆ Z

In the reduced case, since φ = idE , the same condition reduces to:

O* = η*

Thus the example can be read either way. In the reduced reading, outcome-values already are essential-variable values. In the bijective reading, outcome-values and essential-variable values are conceptually distinct but informationally equivalent under φ .

Believed acceptable outcome-set

The typist does not initially know the authoritative acceptable outcome-set perfectly. At time t , the typist acts under a believed model of the acceptable essential-variable region:

η ^ t ⊆ E

This induces the typist's believed acceptable outcome-set:

O ^ t := φ −1 [ η ^ t ]

In the reduced representation:

O ^ t = η ^ t

From the typist's subjective perspective, an incorrect symbol may initially appear to belong to the acceptable outcome-set. Later checking against the authoritative word reveals that it does not. This is not external goal revision, because η* and O* remain fixed. What changes is the typist's believed model: η ^ t and therefore O ^ t .

That is goal-model revision, not goal revision.

Closure events

Each typed symbol is a closure event:

cu = ( u , du , ru , zu )

where:

  • u is the closure-event index;
  • du is the current demand to type a symbol for an intended target position;
  • ru is the selected keypress;
  • zu is the resulting typed outcome-value.

The produced outcome-value is:

zu = ( pu , au )

where pu is the intended target position and au is the typed symbol.

pu is different from the closure-event index. The closure-event index u records when the typing act occurred. The target-position index pu records which position the act attempted to satisfy. Therefore, u ≠ pu in general.

For example:

					z₁ = (1, x)   provisionally accepted, later invalidated
					z₂ = (1, q)   provisionally accepted, later invalidated
					z₃ = (1, H)   survives authoritative checking
					z₄ = (2, o)   survives authoritative checking
					z₅ = (3, n)   survives authoritative checking
						

This preserves the distinction between event order and target position. Events 1, 2, and 3 are three different closure events, but all three attempt to satisfy position 1.

Operational trace

The following trace contains 37 typed closure events. The letters marked with 1 are the closure events that survive authoritative checking. The letters marked with 0 are provisional closure events that are later invalidated.

Closure event 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
Target position pᵤ 1 1 1 2 3 4 5 6 6 7 8 9 10 11 12 13 13 13 14 15 16 16 17 18 19 19 19 20 21 21 21 22 23 24 25 26 27
Typed symbol aᵤ x q H o n o r e i f i c a b i o r l i t m u d i p v n i y k t a t i b u s
Operational Ledger status 0 0 1 1 1 1 1 0 1 1 1 1 1 1 1 0 0 1 1 1 0 1 1 1 0 0 1 1 0 0 1 1 1 1 1 1 1

Reading only the surviving closure events, the events marked 1, gives the final accepted output:

Honorificabilitudinitatibus

Reading the events marked 0 gives the provisional closures that were later rejected:

x, q, e, o, r, m, p, v, y, k

Provisional closure history

Let the observation window be:

I = ( ta , tb ]

A closure is provisionally accepted when produced if its outcome-value belongs to the typist's believed acceptable outcome-set at that time:

zu ∈ O ^ u

The provisional closure history in the window is:

C I +^ := ( cu : ta < u ≤ tb , zu ∈ O ^ u )

In this example, all 37 typed symbols were provisionally accepted when produced. Therefore:

S gross ^ (I) := | C I +^ | = 37

Authoritative survival and model invalidation

A provisional closure survives if its produced outcome-value belongs to the authoritative acceptable outcome-set:

zu ∈ O*

The net surviving closures are:

S net (I) := # { cu ∈ C I +^ : zu ∈ O* } = 27

A provisional closure is invalidated by goal-model revision if it was accepted under the believed model when produced, but later judged outside the authoritative acceptable outcome-set:

zu ∈ O ^ u but zu ∉ O*

The goal-model invalidation load is:

W model (I) := # { cu ∈ C I +^ : zu ∉ O* } = 10

Therefore the operational ledger identity is:

S gross ^ (I) = S net (I) + W model (I)
37 = 27 + 10

Interpretation

In the reduced case, the typed outcome-value already is the essential-variable value. In the bijective case, the typed outcome-value is a concrete outcome description, and the essential-variable value is the corresponding goal-relevant description.

In this example, the 10 invalidated events are actual typed outputs. Each one was a provisional closure produced under the typist's then-current belief i.e. their model of the goal. Later, after checking the authoritative word in the script, the typist discovered that those provisional closures did not belong to the authoritative acceptable outcome-set.

Quantity Value Meaning
S gross ^ (I) 37 All provisionally accepted typed closures.
Snet (I) 27 Typed closures that survived authoritative checking and form the final word.
Wmodel (I) 10 Typed closures invalidated after goal-model revision.
Qeff (I) 10 Effective load: no untyped intervals, Q(I)=0, plus the 10 invalidated closures rebooked as non-output instructions.
KTD^effnet (I) 10/27 ≈ 0.37 Estimated Knowledge To Be Discovered per surviving letter, equation (5). Its bound on Knowledge To Be Discovered is worked out below.
This operational ledger makes visible a kind of knowledge discovery process: acting from an incorrect believed model, then discovering that some completed closures must be invalidated and replaced.

In this example, the authoritative goal did not change. The word to be typed remained the same. What changed was the regulator's model of the goal. The process therefore involved goal-model revision, invalidation of provisional closures, and corrective replacement, not external goal revision.

Knowledge To Be Discovered perspective

In this example, the demand space is Dτ={demand to type position p:p∈P} , the response space is Rτ=A , because the concrete response is the selected keypress or typed symbol ru=au∈A . The closure outcome records that response together with the intended target position: zu=(pu,au)∈P×A . For knowledge accounting, we use the lifted response-equivalence space Xτ=P×A , where the response class is represented together with the target position, because the required response-class is not merely a symbol, but a symbol-for-a-position, and the closure records which symbol was typed for which intended position.

The task class τletter is: type the correct symbol for a target-word position in “Honorificabilitudinitatibus.” Under that task class, every closure event in the trace is one episode of the same regulatory problem: given a target position p, determine and emit the authoritative symbol required at that position. Thus all closure events in the example are comparable under the same task class τletter They share the same disturbance space P , the same concrete response space A , the same outcome space P×A , and the same lifted target response-class space Xτ=P×A .

This example makes explicit why Knowledge To Be Discovered should be defined over the authoritative target response-class, not over the actual response-class emitted by the regulator. The typist produces actual closures, but those closures are not automatically the closures required by the authoritative goal.

For a closure event u , the actual produced outcome-class is:

Xu = zu = ( pu , au )

where pu is the intended target position and au is the symbol actually typed.

But the authoritative target response-class for that same event is:

Xu* = ( pu , w pu )

where w pu is the authoritative symbol required at the intended target position.

Thus the central distinction is:

Xu = what the typist actually produced
Xu* = what the authoritative goal required

Knowledge To Be Discovered is the remaining uncertainty, given the typist's current information state, about the authoritative target response-class:

Knowledge To Be Discovered = H Mt ( Xt* | Yu )

In the generic time-indexed notation used elsewhere, this is:

Knowledge To Be Discovered = H Mt ( Xt* | Yt )

Here Yt represents the disturbance, observation, or demand presented to the regulator at time t . The typist's current knowledge, memory, belief, and goal-model state is represented separately by Mt . In this example, the demand may identify the target position, while the model Mt determines how much uncertainty remains about the authoritative symbol required at that position.

If the typist is attempting to respond to a disturbance on position 1, the authoritative target response-class is:

X* = (1,H)

But the actual produced response-classes were:

							X₁ = (1, x)   actual closure, later invalidated
							X₂ = (1, q)   actual closure, later invalidated
							X₃ = (1, H)   actual closure, survives authoritative checking
							

Therefore:

X1 ≠ X1*
X2 ≠ X2*
X3 = X3*

This is why the strong alignment assumption:

Xt = Xt*

does not hold during this knowledge-discovery process. The typist's actual closures are not yet reliably aligned with the authoritative target closures. Some actual closures are exploratory, mistaken, provisional, or based on an incorrect believed goal model.

Consequently, the entropy of the actual response-class:

H Mt ( Xt | Yt )

does not necessarily measure Knowledge To Be Discovered. It measures residual variation in what the regulator actually does. That variation may come from several sources: randomness, indecision, exploration, implementation variation, motor error, or even consistently wrong behavior. From an outside black-box perspective, these sources are not directly separable merely by observing the emitted closures.

By contrast:

H Mt ( Xt* | Yt )

measures uncertainty about the target response-class that would satisfy the authoritative goal. That is the knowledge the regulator must acquire, infer, retrieve, or otherwise make available in order to close the demand correctly.

Why the operational ledger is evidence of discovery, not the entropy itself

The operational ledger:

37 = 27 + 10

is not itself an information-theoretic entropy identity. It is an operational accounting identity:

provisional closures = surviving closures + invalidated closures

The 10 invalidated closures are not 10 bits of Knowledge To Be Discovered. They are observable consequences of acting before the relevant target knowledge was fully available. They show that the typist's believed model allowed closures that the authoritative goal later rejected.

Thus:

W model (I) = 10

measures model-invalidation load, not Knowledge To Be Discovered directly. It is a trace left by knowledge discovery in the closure history. The entropy measure remains:

H Mt ( Xt* | Yt )

because the unresolved question is not merely “what will the typist type?” but “what response-class is required for authoritative success, given what the typist currently knows?”

Role of goal-model revision

The authoritative goal does not change in this example. The word remains:

Honorificabilitudinitatibus

Therefore O* and η* remain fixed. What changes is the typist's believed model:

O ^ t , η ^ t

This is why the process is goal-model revision, not external goal revision. The typist discovers that the believed acceptable outcome-set was too permissive. After checking, some provisional closures are removed from the believed acceptable region, and replacement closures are produced.

From the Knowledge To Be Discovered perspective, checking the authoritative word changes the information state:

Yt → Yt+1

and therefore can reduce:

H Mt ( Xt* | Yt )

When the typist finally knows that position 1 requires H, the uncertainty about the authoritative response-class for that position collapses:

H Mt ( X * | Y = ( 1 , H ) is known ) = 0

At that point, no further knowledge must be discovered for that particular target-position closure. The remaining task is execution: producing the known required response.

This example supports the definition:

Knowledge To Be Discovered := H ( Xt* | Yt )

because the discovered knowledge is knowledge about the target response-class required for successful regulation. The target is not whatever the regulator happens to emit. The target is the authoritative closure that would place the outcome inside the acceptable region.

The actual response-class Xt can be wrong, exploratory, random, inconsistent, or provisionally accepted under a mistaken model. Therefore:

H ( Xt | Yt ) ≠ Knowledge To Be Discovered

in the general case.

Only under the strong alignment assumption:

Xt = Xt*

does the actual response-class entropy coincide with target response-class uncertainty:

H ( Xt | Yt ) = H ( Xt* | Yt )

This shows that the strong alignment assumption fails during knowledge discovery. The typist does not begin with perfect knowledge of the authoritative target closure. The typist emits closures, invalidates some of them, revises the believed goal model, and eventually aligns actual closures with authoritative target closures.

The example therefore serves as a reference case for why Knowledge To Be Discovered is H(Xt*|Yt): the uncertainty that matters is uncertainty about the authoritative target closure, not merely variation in the closures actually produced.

Relational reading and the program reading of the trace

This example is the unique-target case of the regulatory relation. Each target position has exactly one acceptable symbol, so every section of the relation is a singleton, Xacc(p)={(p,wp)}, the only admissible selector is the target map, and relational Knowledge To Be Discovered reduces to HMt(Xt*∣Yt) as used above. Had the task admitted variants, for example either of two accepted spellings, the relational definition would charge the typist only for what distinguishes acceptable spellings from unacceptable ones, not for the choice between the two.

Read as a program, this trace uses the same two instructions as the realized trace that follows and the typing application, with one difference: when a 0 becomes a 0. Here all 37 events were typed, and the ten zeros are letters retracted after checking. At execution they were outputs; they become non-output instructions only when the authoritative check rejects them. The effective partition of the ledger performs exactly that rebooking: each invalidated closure moves from the closure side to the load side. The effective program therefore has N=37 instructions, 27 outputs and 10 non-outputs, and N/Snet−1=10/27≈0.37 is again the number of non-output instructions per output.

Each run of zeros sits directly before the correct letter for the same position: x and q before H at position 1, e before i, o and r before l, m before u, p and v before n, y and k before t. The retracted letters are trials. Each rejection answers “not this letter” for the position that the surviving letter then fills, so the retractions are stages of the surviving closures' paths, not separate episodes. The checks answer only “wrong”: at positions 1, 13, 19 and 21 a second wrong letter followed the first, so a rejection did not reveal the right letter. Each stage therefore has a binary answer, and A5 holds here, as it does not for the lookups of the realized trace.

The ten retracted letters are the worked case of the remark “When a failed or retracted output is a query” in the subsection “The record as a two-instruction program” of section 6. Each retraction is a stage of the surviving letter's conditioning path, so its unit belongs in that letter's stage count; the invalidation credit Wobs/Snet =10/27 is not available; and the upper bound of Theorem 2 holds with the average terminal residual added. This is the same bridge as for missed shots in basketball, recorded at checking instead of at production.

Bounding Knowledge To Be Discovered from this trace

Theorem 2(i) splits the estimate exactly: K^−K∗ = ζ+εred+ εinv+εΔ+ εbd , where K^=10/27 and K∗ is the average Knowledge To Be Discovered per surviving letter. In this trace three of the five terms are zero.

Term Value here Why
εinv 0 Every retraction is a trial for the letter that survives at the same position, so its unit is counted in that letter's stage count ku, not in the invalidation term.
εΔ 0 Each typed event occupies one interval, and each retraction is one stage.
εbd 0 The window is the whole task, so no episode is cut by its boundary.
ζ realized, sign unknown The realized stage counts against their expected values. Over 27 letters it is not negligible; Corollary 2 bounds it with high probability.
εred ≥− H¯termres A5 holds: each check answers only “wrong”. B2 fails: each provisional letter was committed before the typist could be sure of it. The graded-uncertainty remark after Theorem 2 then gives this lower bound.

So K^=10/27 is exactly the average number of binary stages per surviving letter, and

K∗ ≤ K^ −ζ + H¯termres = 1027 −ζ + H¯termres bits per letter.

This is an upper bound on Knowledge To Be Discovered in bits: by Proposition 0 each retraction, a binary stage, removes at most one bit in expectation. The invalidation credit Wobs/Snet of Corollary 3(iii) is not taken, because the retractions are stages of the surviving letters. The realized trace that follows reaches the same estimate, 10/27, and stops one step earlier.

A Realized Knowledge Discovery Trace

This example is the black-box retelling of the typing application in section 9. There the typist reports each look and each pause, so the 0s are observed one by one. Here the same trace is read as an outside observer reads it, from the letters that land and the elapsed time, so the 0s are inferred. The example shows how such a trace can be used to bound the amount of knowledge discovered during a task. The purpose is not to estimate the entropy of English, nor the entropy of a word. The purpose is to make visible the knowledge-discovery burden experienced during one realized execution of one task, and what an outside observer can recover of it.

The Task

Consider the task of typing the word “Honorificabilitudinitatibus” on a keyboard. The word has 27 letters. The manual part of the task is to press the correct keys. The knowledge part of the task is to know which 27 letters must be typed, and in which order.

For the human time-capacity bound, take the normalized interval to be Δ=0.288 seconds, as in the typing application. This assigns 10.656 seconds to the 37-unit scenario and 2.88 seconds to its ten non-closure units. These durations are declared scenario values, not stopwatch measurements. Query stages and their alphabets keep their definitions.

In this example, the typist does not remember the whole word at once. Sometimes the next required letter is already available from memory. At other times, the typist has to stop, think, or look up the word again. We record this process with a binary trace:

  • 1 means the required next letter was already available for action.
  • 0 means the typist was missing information and spent the interval on one query: one look at the word in the script, or one attempt to recall it, before continuing. A lookup that took two intervals was two looks, one to find the word and one to read the next letters, so every 0 is one query.

Thus, the ones are the surviving closures of the process — the letters that actually land — and the zeros are the missing-information intervals that separate them. This 0/1 string is the ground truth of what happened inside the process. A strict black-box observer does not see it: the zeros are not directly observable as such. We display the full trace here only to make the underlying process concrete. What the observer actually records is which letters landed and how much clock time the window afforded. The missing-information intervals are recovered afterward as a residual, not counted off the trace.

The Trace and What Is Actually Observed

The ground-truth trace — shown here for illustration, not as observer-side data — can be summarized as follows. The Qi column records the missing-information intervals that in fact occurred in each subset; under the black-box regime these are inferred, not seen, and are listed here only so the arithmetic can be followed.

Segment Trace Queries \(Q_i\) Typed letters \(S_i\) Estimated KTD \(\widehat{KTD}_i = Q_i / S_i\) Total intervals \(N_i = Q_i + S_i\)
Initial lookup and first remembered chunk 00 11111 2 5 2/5 7
Second remembered chunk 0 1111111 1 7 1/7 8
Third remembered chunk 00 111 2 3 2/3 5
Fourth remembered chunk 0 111 1 3 1/3 4
Fifth remembered chunk 00 11 2 2 2/2 4
Final remembered chunk 00 1111111 2 7 2/7 9
Total Realized execution trace ∑ Q i = 10 ∑ S i = 27 K T D ^ = 10 / 27 ∑ N i = 37

Note: The observer's route to N is the clock, not this sum. The observer sees 37 clock ticks, and the 27 letters that actually land. The 10 missing-information intervals are inferred as a residual, not directly observed.

Each 0 is one query, so the Qi column counts queries: ten of them, in six runs, one run at the start of each subset. Each interval holds exactly one query and no query spans two intervals, so condition CΔ holds with equality and the unit-inference error is zero, εΔ=0. The residual that the observer infers from the clock is then the number of queries itself.

Two quantities are available to a black-box observer of this window.

First, the surviving closures. Twenty-seven letters landed, so S n e t = 27 . This is read directly from the closure ledger.

Second, the window capacity. Under the declared convention, one closure-act-equivalent unit is the time to commit one letter, of normalized duration Δ . The window's elapsed clock time affords 37 such units, so N = | I | Δ = 37. This count comes from the clock and the unit convention, not from any tally of missing-information intervals.

The residual is then inferred, not observed: Q e f f = N − S n e t = 37 − 27 = 10. The ten missing-information intervals are the closure-act capacity the window afforded but did not convert into surviving closures. They are recovered from N and S n e t ; the observer never counts them directly.

Aggregating Estimated KTD Across Subsets

The trace can also be analyzed by decomposing the task into six subsets. Each subset has its own number of inferred, not counted, questions Qi, its own number of typed symbols Si, and therefore its own local estimate of Knowledge To Be Discovered:

KTD^i = QiSi .

However, the estimate for the whole task is not obtained by simply summing the six KTD^i values, nor by taking their unweighted arithmetic average. The subsets have different sizes. A subset of seven typed symbols must contribute more to the aggregate value than a subset of two typed symbols.

Therefore, the aggregate is computed as a weighted average, where each local value KTD^i is weighted by the proportion of symbols in that subset:

KTD^effnet = ∑i=1nQi ∑i=1nSi = 1S ∑i=1n (Ni−Si) = 1S ∑i=1n Si (NiSi−1) = 1S ∑i=1n Si KTD^i .

Here:

  • n is the number of subsets;
  • Si is the number of typed symbols in subset i;
  • Qi is the number of questions in subset i;
  • KTD^i=Qi/Si is the estimated Knowledge To Be Discovered per letter in subset i;
  • Ni=Qi+Si is the total number of intervals in subset i;
  • S=∑i=1nSi is the total number of typed symbols.

Using the six subsets from the table:

KTD^effnet = ∑i=1n SiS QiSi = 527×25 + 727×17 + 327×23 + 327×13 + 227×22 + 727×27 = 1027 ≈ 0.37 .

So the aggregate estimate remains 0.37 per typed letter. The subset calculation gives the same result as the direct calculation, but it makes clear why the local KTD^i values must be weighted by subset size. Subset estimates cannot be added directly. The correct aggregate is the symbol-weighted average:

KTD^effnet = 1S ∑i=1n Si KTD^i .

Because every 0 is one query, each local estimate KTD^i=Qi/Si is the number of queries per letter in subset i, and the weighted aggregate, 10/27 ≈ 0.37, is the number of queries per letter for the whole task. Whether that number also bounds Knowledge To Be Discovered in bits is settled after the operational estimator below.

The Operational Estimator

The effective one-bit estimator of equation (5) over this interval is:

K T D ^ e f f 1 b i t , n e t ( I ) = N ( I ) S n e t ( I ) − 1.

Substituting the observed values gives:

K T D ^ e f f 1 b i t , n e t ( I ) = 37 27 − 1 = 10 27 ≈ 0.37 .

So, in this realized execution, the estimated Knowledge To Be Discovered is 0.37 per typed letter: 0.37 queries per letter, with the queries inferred from the clock rather than counted.

This number is exact as a ratio given its inputs: the 27 letters are counted, and the length of 37 comes from the clock and the unit convention. What it bounds is worked out next. It should not be confused with the Shannon entropy of English or the entropy of the word itself.
Bounding Knowledge To Be Discovered from this trace

The accounting of Theorem 2(i) runs as for the goal-model trace above, with K^=10/27 again:

Term Value here Why
εinv 0 No letter was retracted, Wobs=0.
εΔ 0 Each interval holds exactly one query and no query spans two, so CΔ holds with equality.
εbd 0 The window is the whole task.
ζ realized, sign unknown As for the goal-model trace.
εred no unit-coefficient bound under A5; under H1, K∗≤ch(K^−ζΔ) B2 holds: each letter is typed only after it has been read. A5 fails: a look at the script answers with a chunk of several letters, so the answer alphabet is large, and Theorem 2(ii) does not apply. The typist is human, so H1 can replace A5 on the upper side (Corollary H). It compares information with time directly; the stages keep their meaning.

So K^=10/27 is exactly the average number of queries per letter: the second rung of the query ladder, reached without error. Under A5 alone, the step from queries to bits is where this trace stops. Without A5 a single look can remove more than one bit, and the wider-alphabet remark after Theorem 2 restores a bound only by weighting each look with the logarithm of the number of answers it could return, which with chunks of letters is a weak bound.

The typist is human, so H1 offers a second route. Each look occurs in the episode of the letter it was made for, and its intervals are counted there. What it shows about the following letters is episode-start information of their episodes, which therefore begin nearly certain and spend no interval. With Δ=0.288 seconds, a look delivers at most ch=ChΔ bits per interval about its own letter in expectation, whatever chunk the script displays, and Corollary H gives

The two typing traces of this appendix spell the same word and give the same estimate, 10/27, with the same three error terms at zero. They differ in which assumption carries the step to bits. In the goal-model trace each stage is a check that answers only “wrong”, A5 holds, and the estimate bounds Knowledge To Be Discovered in bits once the terminal residual is added. In this trace each stage is a look that returns several letters, A5 fails, and the estimate bounds Knowledge To Be Discovered in bits only through the human channel, with the factor ch set by the declared human information throughput.

The trace as an oracle program

This trace is a program in the sense of the subsection “The record as a two-instruction program” of section 6. The machine that runs it is the typist, a specific, non-universal machine whose state is the knowledge state Mt, with oracle access to the script. Lookup (0) spends one interval refreshing the typist's buffer from the oracle or from memory. Type (1) emits the next acceptable closure from the buffer. Each 0 is one execution of lookup, one query, and each takes one interval, so the program's 0s, its queries, and its non-typing intervals are the same ten units.

A trace is a valid program only if it never executes type on an empty buffer. Validity therefore encodes the typist's memory capacity, and that capacity is what makes some lookups unavoidable. For the same machine and the same oracle, the shortest valid program contains the fewest lookups the typist's memory permits. Per output, that is the analogue of the middle rung of the ladder, the least number of queries over the typist's own query family. The difference between the actual program and the shortest valid one is the analogue of the protocol gap: lookups that a better chunking of the word would have avoided.

The program is 37 bits long, and each lookup returned a chunk of several letters, so the information obtained is not bounded by the program length; section 6 explains why the answers are not part of the program. Under H1, the Knowledge To Be Discovered of the 27 letters sums instead to at most ch times the expected number of 0s, ten up to the fluctuation. What the lookups showed about letters other than the one sought is episode-start information of those letters and is not part of that sum. As a stream, this trace is not self-delimiting unless S or N is known or an end marker is added; this matters only if prefix complexity is invoked.

Why This Corresponds to H ( X ∣ Y = y u )

The important point is that this calculation is based on the actual observed knowledge state of the typist during the task. We are not averaging over all possible memory states, all possible words, or all possible observations. We are looking at the realized case: this typist, this word, this sequence of remembered and forgotten chunks.

For that reason, the corresponding latent quantity is the realized row-level conditional uncertainty:

H u s t a r t := H M u ( X u ∣ Y u = y u ) .

Here, X u is the required response-equivalence class for closure event u , and Y u = y u is the actual knowledge state available at the start of that episode. The estimator uses the realized value y u , not the full distribution of possible values of Y u .

By contrast, H M u ( X u ∣ Y u ) would be an ex-ante quantity. It would require a probability distribution over possible knowledge states before the realized state is known. That is not what the trace records. The trace records what actually happened.

The operational ledger estimates realized Knowledge To Be Discovered: H ( X ∣ Y = y u )
The theoretical quantity H ( X ∣ Y ) belongs to an ex-ante model that averages over possible observation states.

The correspondence is a bound, not an identity, and it needs the conditions of Theorem 2 or of Corollary H. Under A5 it fails here: a lookup returns a chunk of letters, so its answer alphabet is large and the queries are not binary, and the wider-alphabet remark gives only a weak bound. Because the typist is human, assumption H1 supplies the bound instead: Knowledge To Be Discovered per letter is at most ch times the non-closure intervals per letter, up to the interval-count fluctuation, and at most ch(0.370+0.559) with probability 0.95. What a lookup showed about later letters is episode-start information of those letters, not Knowledge To Be Discovered. The 0.37 per letter is a count of intervals, and in this trace also of queries; multiplied by ch=ChΔ, it is an upper bound in bits, whose value moves only with the declared human information throughput.

Example: A Robot Discovering Its Next Move

Imagine a robot exploring the surface of a distant moon. At each step of its journey, the robot must select one of four possible moves:

  • move up,
  • move down,
  • move left, or
  • move right.
Each move has a fixed step size.

From a knowledge-centric perspective, the robot is discovering which move its goal calls for. The knowledge needed to select the move could, in principle, come from many places. It might come from the robot's sensors, from a map stored in its memory, from a rule learned during earlier exploration, or from a signal transmitted from Earth. The source is immaterial. The only quantity that matters is how much uncertainty must be eliminated before one move can be selected.

The robot discovers which move its goal calls for by following a protocol. The protocol proceeds through a sequence of stages. At each stage, the robot performs the same generic action unit. Executing that action unit exposes a realized signal Ui , which in this case is in the form of a binary symbol. The realized symbol takes the value zero or one and constitutes the information-bearing signal at that stage. In this example 0 and 1 are answer values, the answer transcript of subsection 6.6, not output and non-output program instructions. Every read is a non-output program instruction, whichever value it returns, and every committed move is an output instruction. By processing those realized signals stage by stage, the robot progressively eliminates moves that are no longer compatible with the observed signal path.

The protocol does not directly hand the robot a completed instruction. Instead, it structures a process through which the robot discovers which move is required.

At episode t , three objects must be kept distinct. The robot's knowledge state Mt contains its stored law of action and its model of the terrain. The episode-start information Yt is the response-relevant information actually available to the robot before the episode's internal staged discovery begins. The required response class Xt is the move that must be identified for successful regulation.

These roles are different. The knowledge state Mt determines the epistemic probability model by which the robot interprets what it observes. The episode-start variable Yt is realized conditioning information supplied to that model. Thus Yt is not absorbed into Mt . A stored model tells the robot how to use episode-specific information; it is not the same thing as that information.

That distinction lets two robots receive the same Y = y but have different KTD because their M t 's interpret that evidence differently. Conversely, one robot with the same M t can encounter different Y t 's in different episodes.

The Knowledge To Be Discovered at the start of the episode is therefore:

KTD t,0 := H Mt ( Xt ∣ Yt ) .

For one realized episode-start context Yt = yt , the corresponding uncertainty is H Mt ( Xt ∣ Yt = yt ) .

For readability, when one episode is being analyzed we suppress the episode index and write X , Y , and Ui , while retaining Mt when the epistemic model needs to be shown explicitly.

So the robot begins each episode with a definite amount of uncertainty about X , and the practical question is this:

What is the most efficient possible protocol the robot can follow to discover its next move - that is, to determine which response is required using the fewest protocol stages on average?

The resulting robot example would then demonstrate the entire knowledge-discovery architecture in one line: H M t ( X t ) = I M t ( X t ; Y t ) + H M t ( X t ∣ Y t ) . followed by staged discovery through U 1 , U 2 , … .

Action units and realized signals are two different things

The robot resolves the move by running a protocol Qt : a sequence of action units q1 , q2 , … .

An action unit is a discriminating operation. It specifies what the robot does to obtain the next piece of evidence. In this particular example, the action unit is simply:

Read the next binary symbol.

Executing that action unit exposes a realized signal Ui , which in this case is in the form of a binary symbol, whose realized value is either zero or one. That realized signal updates the robot's epistemic state and may rule out some of the moves still compatible with what the robot knows.

The distinction to hold onto is that the action unit qi and its realized signal Ui are not the same thing. The action unit is what the robot does; the realized signal is the evidence made available by performing that action.

There is another distinction that will matter later. A binary symbol is not, by definition, one bit of information. Zero and one are possible values of the signal. A bit, by contrast, is a unit used to measure information or uncertainty reduction. One protocol stage may expose one binary symbol without that stage necessarily contributing one full bit of expected information.

In this example, which action unit is performed next is fixed in advance by the protocol and the robot's knowledge state. That protocol structure, on its own, therefore tells nothing additional about the required move. The informational update is carried by the realized values of the signals.

That is why "receiving an instruction" and "discovering the next move" can be represented as the same process here. The uplink from Earth is simply one possible source of epistemic signals feeding the protocol. Whether a realized signal comes from Earth, from an onboard sensor, or from stored knowledge changes the source of the evidence, but not the staged account of discovery.

So discovering the move means running a procedure, allowing the realized signals to reduce the set of candidate moves, and stopping when the protocol reaches a terminal state in which the required response X has been identified.

Fix a realized episode-start context Y=y . Under the robot's current knowledge state Mt , the four moves are not equally likely to be the required one. The conditional distribution of the required response is:

PMt (X=up∣Y=y) = 12 ,
PMt (X=down∣Y=y) = 14 ,
PMt (X=left∣Y=y) = 18 ,
PMt (X=right∣Y=y) = 18 .

Thus, given this episode-start context, half of the time the required move is up, one quarter of the time it is down, and the two remaining moves each occur one eighth of the time. For this first worked case, successive required moves are independent across episodes; the previous required move supplies no additional response-relevant information beyond the episode-start information already represented by Y.

Now imagine three engineers who each propose a discovery protocol - one straightforward, one clever, and one very theoretical.

Each protocol can be judged by two quantities that sound alike but mean different things. The first is how many protocol stages the robot runs before the required move is identified. This is a count of physical action units, measured in stages or binary symbols read. The second is how much response-relevant uncertainty remains after the episode-start information is already known. This is Knowledge To Be Discovered: H Mt ( X ∣ Y=y ) , measured in bits.

These are not the same thing. One asks what the robot does; the other asks what it must still discover. The terrain and the episode-start information determine the conditional uncertainty. The protocol determines how many action units are expended to remove it.

The straightforward protocol

The first engineer suggests assigning a two-symbol binary code word to every possible move:

00 → up,
01 → down,
10 → left,
11 → right.

Under this protocol, each episode always contains exactly two stages.

At the first stage, the robot performs the action unit:

Read the next binary symbol.

The realized symbol is the information-bearing signal. Its value eliminates two of the four possible moves.

At the second stage, the robot performs one more action unit and reads one more binary symbol. The second realized signal eliminates all but one of the remaining candidates.

For example, suppose the robot reads:

U1 = 1

followed by:

U2 = 0 .

The realized binary signal conditioning path is therefore:

10.

According to the protocol, the unique move compatible with that path is left.

This approach is simple. The robot always reads two binary symbols, groups them into a two-symbol code word, and maps the resulting word to one move.

However, it ignores the fact that the four moves do not occur with equal probabilities. The move up occurs much more frequently than either left or right, yet all four moves are assigned conditioning paths of exactly two stages.

A more efficient discovery protocol

The second engineer proposes a variable-length prefix protocol:

0 → up,
10 → down,
110 → left,
111 → right.

Under this protocol, the robot still performs the same basic action unit at every stage:

Read the next binary symbol.

The action unit specifies the discriminating operation. The realized value of the binary symbol is the epistemic signal that updates the robot's candidate set.

The protocol may terminate after one, two, or three stages, depending on the realized signal path.

Suppose the first observed signal is:

U1 = 0 .

No other valid code word begins with zero. The robot can therefore conclude immediately that:

X = up .

Its initial set of candidate moves,

{ up, down, left, right } ,

has been reduced to:

{ up } .

The move has been discovered after one protocol stage.

Now suppose instead that:

U1 = 1 .

The move is not yet known. Three candidates remain:

{ down, left, right } .

The protocol therefore continues. The robot performs another action unit:

Read the next binary symbol.

If the next realized signal is:

U2 = 0 ,

the observed prefix is:

10.

Only down is compatible with that path, so the robot discovers:

X = down .

If, however, the second signal is:

U2 = 1 ,

the observed prefix is:

11.

Two possibilities remain:

{ left, right } .

The robot must therefore perform a third action unit and read one more binary symbol.

If:

U3 = 0 ,

the realized path is:

110,

and the robot discovers left.

If:

U3 = 1 ,

the realized path is:

111,

and the robot discovers right.

The protocol can therefore be represented as a staged candidate trace. For the realized signal path (1,1,0) , the epistemic candidate trace is:

{up, down, left, right} → {down, left, right} → {left, right} → {left} .

The action-unit sequence determines how the robot obtains the next piece of evidence. The realized signals determine which candidates survive. The mere act of reading another binary symbol does not reveal which move is correct. The information-bearing event is the realized value of that symbol.

In the notation of a staged knowledge-discovery process, the structure is:

qi ⇒ Ui ⇒ C i-1 X → CiX .

Here:

  • qi is the action unit "read the next binary symbol";
  • Ui is the realized signal, whose value is zero or one;
  • C i-1 X is the set of response classes still possible before the signal is observed; and
  • CiX is the updated candidate set after the signal is observed.

The protocol structures the discovery episode. The realized signals carry the information.

Knowing when the move is discovered

Allowing different moves to require different numbers of protocol stages creates an immediate difficulty. How does the robot know when it has observed enough binary symbols to identify the move? How does it know where one discovery episode ends and the next begins? How does the robot know it is finished? It stops the instant the live candidate set of response classes becomes a singleton i.e. the instant the realized signals have fixed one required response class.

In the formal terms used above, the robot conditions on realized signals until the terminal Knowledge To Be Discovered reaches zero, which in this unique-target example occurs exactly when the terminal candidate-class set contains one element.

The constraint that makes this unambiguous is prefix-freeness: no completed discrimination path is the beginning of another completed path, or equivalently, no code word in a set is a prefix of any other code word.

Consider the four code words again:

0, 10, 110, 111.

The answer lies in the structure of the code words. No code word is the prefix of another code word.

The one-symbol word 0 identifies up, and no longer code word begins with zero. Once up is identified at the first stage, the robot does not continue executing action units for that episode.

The two-symbol word 10 identifies down, and no longer code word begins with 10 .

The prefixes 1 and 11 are not complete code words. They represent intermediate epistemic states in which more than one move remains possible.

The robot therefore follows a simple rule:

Continue reading binary symbols until the realized signal prefix matches a complete code word.

At that point, the required move has been uniquely identified and the episode closes.

Suppose the signal stream begins with:

1.

The robot cannot yet commit to a move because down, left, and right all remain possible.

If the next binary symbol is zero, the prefix becomes:

10.

Only down is compatible with that prefix, so the robot closes the episode and selects down. The next unread binary symbol begins a new discovery episode.

Suppose that symbol is zero. Because 0 is already a complete code word, the robot immediately discovers up.

Now suppose the following binary signal path is:

110.

After the first 1, three candidates remain. After the second 1, two candidates remain. After the final 0, only left remains. The move is discovered, and the episode closes.

A code with this property is called a prefix-free code, or more commonly a prefix code. The prefix-free condition is what makes the staged conditioning path unambiguous.

If a fifth move were assigned the code word:

100,

the protocol would fail. After observing:

10,

the robot would not know whether it had completed the code word for down or had merely observed the first two binary symbols of the new code word 100 .

The response class would appear to be fixed and unfixed at the same time. By preventing one code word from being the prefix of another, the protocol ensures that every terminal signal path corresponds to one unambiguous closure.

Reading binary symbols until a complete code word forms is therefore the coding-theoretic description of the same event that, in the staged knowledge-discovery formalism, is represented by conditioning until only one response class remains possible.

Those proportions - one half, one quarter, one eighth, and one eighth - exactly match the prior probabilities of the four moves. That alignment between probability mass and path length is the essential information-theoretic structure behind the efficiency of the protocol.

The average number of discovery stages

The operational cost of the protocol can now be calculated as the expected number of protocol stages required to discover a move. In this example, every stage consists of one action unit and exposes one realized signal in the form of a binary symbol.

Up requires one stage and occurs with probability one half. Down requires two stages and occurs with probability one quarter. Left and right each require three stages and each occurs with probability one eighth.

Let K denote the number of protocol stages required to close an episode. Its expected value is:

E [K] = 12×1 + 14×2 + 18×3 + 18×3 . = 1.75 stages per discovered move.

Equivalently, because each stage reads exactly one binary symbol, the protocol consumes an average of 1.75 binary symbols per discovered move.

This is more efficient than the fixed-length protocol, which always requires two stages and therefore always reads two binary symbols.

The variable-length protocol sometimes spends more than two stages. Discovering left or right requires three action units. But those moves occur infrequently. The additional operational cost of their longer paths is more than offset by the one-stage path assigned to up, the most probable move.

At this point we have measured the length of the protocol, not the amount of information discovered. The quantity E[K] is measured in protocol stages. Whether those stages correspond to the same numerical amount of information is a separate question.

Knowledge To Be Discovered along the protocol

Let X denote the required move and let Y=y denote the response-relevant information already available when the episode begins. Before any internal discovery signal is observed, the robot's Knowledge To Be Discovered for this start context is:

KTD0 = H Mt ( X ∣ Y=y ) .

For the conditional distribution used in this example:

H Mt ( X ∣ Y=y ) = − 12 log2 (12) − 14 log2 (14) − 18 log2 (18) − 18 log2 (18) = 1.75 bits.

The unit is bits, because this quantity measures residual epistemic uncertainty, not protocol length.

As the protocol runs, the information-bearing signals U1 , U2 , and so on are added to the conditioning information. After i stages, the expected residual Knowledge To Be Discovered is:

KTDi := H Mt ( X ∣ Y , U≤i ) .

Along one realized conditioning path Y=y , U1=u1 , … , Ui=ui , the corresponding posterior uncertainty is:

H Mt ( X ∣ Y=y , U≤i = u≤i ) .

In this robot example the protocol terminates only when the required move has been uniquely identified. Therefore every completed realized path satisfies:

H Mt ( X ∣ Y=y , U≤K = u≤K ) = 0 .

The realized signal path has therefore removed all of the Knowledge To Be Discovered that remained after the episode-start information was known. For the present start context, that amount is H Mt (X∣Y=y) = 1.75 bits.

We can now compare two quantities that must remain conceptually distinct:

E [K∣Y=y] = 1.75 protocol stages,

while:

H Mt (X∣Y=y) = 1.75 bits.

The first quantity measures how many physical action units the protocol expends. The second measures how much uncertainty still had to be removed after the robot's episode-start information was already available. They collapse to the same numerical value here only because this conditional distribution is dyadic and the binary protocol is probability-matched to it.

The protocol as a binary discovery tree

The prefix code can be represented as a binary tree. Each internal node represents an epistemic state in which several moves remain possible. Each action unit instructs the robot to read another binary symbol.

Each outgoing branch represents one possible realized signal:

Ui = 0

or:

Ui = 1 .

Each leaf represents a terminal response class. The tree begins with all four candidate moves.

The first binary distinction separates up from all other moves:

0 → up,
1 → down, left, right.

The next distinction separates down from left and right:

10 → down,
11 → left, right.

The final distinction separates left from right:

110 → left,
111 → right.

The tree is therefore not merely a visual representation of an encoding. It is a staged knowledge-discovery protocol.

The robot begins at the root with unresolved Knowledge To Be Discovered. Each executed action unit exposes another binary signal. Each realized signal moves the robot along one branch of the tree and updates the posterior distribution over possible moves. The episode terminates when the path reaches a leaf and only one required response class remains possible.

Why this protocol is perfectly matched to the distribution

This is the tightness case of Lemma 1′ with gu=0 and zero coding gap.

The probability of each move is related directly to the length of its conditioning path.

For up:

P ( up ) = 12 = 2 -1 .

For down:

P ( down ) = 14 = 2 -2 .

For left and right:

P ( left ) = P ( right ) = 18 = 2 -3 .

Thus, for every move x ,

P ( x ) = 2 - ℓ(x) ,

where ℓ(x) is the number of protocol stages, equivalently the number of binary symbols in the code word, required to discover move x .

Equivalently:

ℓ(x) = - log2 ( P(x) ) .

A highly probable move receives a short conditioning path. A less probable move receives a longer path. This is not an arbitrary design choice. It is the central information-theoretic principle behind efficient coding. The protocol assigns shorter code words to less surprising outcomes and longer code words to more surprising outcomes.

The two engineers on the query ladder

The two protocols are two decision procedures for the same regulatory relation, each move being one required class. The robot's queries are binary symbols on an uplink that can encode any binary question about the move, so its query family is unrestricted. The ladder of Theorem Q then reads:

KTDrel(y)=1.75=DTμy(R∣y)=DTμyQ(R∣y)≤𝔼[K∣Y=y].

The posterior is dyadic, so the coding gap is zero; the family is unrestricted, so the query-family gap is zero. The second engineer's protocol is optimal, so its protocol gap is zero too, and every rung coincides. The first engineer's protocol has 𝔼[K∣Y=y]=2, and its whole excess is protocol gap:

δ=0,ψ=0,g=0.25.

In the factorisation of Proposition Q6 the first engineer's KEDE, 1/3, is latent KEDE, 1/2.75, times a protocol efficiency of 2.75/3≈0.917. The remedy is a better protocol, L2c i.e. the second engineer's protocol.

One binary stage is not necessarily one bit of information

This is Proposition 0: the bound is on the expected information of a stage, and the realized amount on a given branch may be more or less than one bit.

It is important not to infer from the binary nature of the protocol that every executed stage must contribute one bit of information.

At stage i , the robot executes one action unit and observes one binary signal Ui . The expected epistemic contribution of that stage is:

IMt ( X ; Ui ∣ Y=y , U<i = u<i ) ,

which is measured in bits.

For a general binary discrimination, that quantity may be strictly less than one bit. If, for example, the two possible realized signals at some stage occur with probabilities 0.9 and 0.1 , then one binary symbol is still observed, but the uncertainty of that binary outcome is only:

- 0.9 log2 (0.9) - 0.1 log2 (0.1) ≈ 0.469 bits.

So an action unit, a realized binary symbol, and a bit of information are three distinct concepts.

The present protocol is special because every internal node that the robot can reach divides the remaining probability mass exactly in half.

At the beginning of an episode, the first signal is zero precisely when the required move is up. Since:

P ( up ) = 12 ,

the first realized signal is zero with probability one half and one with probability one half.

Conditional on the first signal being one, the remaining candidates are down, left, and right. Within that remaining probability mass, down accounts for exactly one half:

P ( down ∣ U1 = 1 ) = 14 12 = 12 .

Therefore, conditional on reaching the second stage, its next realized binary signal is again zero or one with equal probability.

Conditional on the first two signals being 11 , the remaining candidates are left and right, and they are equally probable. The third signal is therefore once again zero or one with equal probability.

Thus, at every stage actually reached by the protocol, the next binary discrimination evenly divides the remaining probability mass. Conditional on reaching stage i , the realized binary signal has entropy:

HMt ( Ui ∣ Y=y , U<i = u<i ) = 1 bit for every prefix that reaches stage i .

Because the signal at each reached stage is also exactly the discrimination needed to determine which branch contains X , each executed stage contributes one bit of expected information about the required move.

This is why the numerical alignment found above is exact:

E [K∣Y=y] = 1.75 stages,

and:

HMt (X∣Y=y) = 1.75 bits.

On average, the protocol therefore expends one action unit for each bit of Knowledge To Be Discovered that it removes. But this statement is a consequence of the even-split structure of this particular protocol. It is not a general identity between action units and bits of information.

From Discovery Efficiency to Knowledge Discovery Rate

So far, nothing in the robot example has depended on time. We have counted how many protocol stages are required to discover a move and how much uncertainty those stages remove, but we have not asked how quickly either happens. Time enters through the unit convention of the execution channel (see the unit-convention layer), and the robot is the place to make that convention concrete.

The unit convention for the robot

Under the time-normalized convention, the unit length is the duration of one commitment act, Δ:=δcl , and non-closure action units inherit the same length. We take the robot's elementary action unit, read the next binary symbol, to last one Δ as well.

Closure rate

Here νcl(I) is the rate at which moves are committed. For the robot every committed move survives and stages are counted directly, so Wobs=0 , εΔ=0 , and over a long window K^(I) → E[K] . The robot then commits νact / (E[K]+1) moves per unit time.

Two discovery rates

The article defines the effective load rate as

νdisc(I) := Qeff(I) |I| = νcl(I) ⋅ K^(I) = νact ( 1− KEDE^(I) ) .

The quantity the robot example is really about is the latent discovery rate:

νdisc∗(I) := νcl(I) ⋅ K∗(I) , K∗(I) = KTD avg start-real,net (I) .

In the expectation reading, where the window is long enough for the path fluctuation and boundary terms of Theorem 2 to vanish, K∗(I) → HMt (X∣Y) and the latent rate is νcl ⋅ HMt (X∣Y) bits per unit time.

The two rates differ by exactly the number of bits each counted discrimination unit delivers. Define the effective discovery information per discrimination stage:

Hdisc(I) := K∗(I) K^(I) = 1 − Γ(I) K^(I) ,

where Γ(I) is the total gap of Theorem 2(i). Then:

νdisc∗(I) = νdisc(I) ⋅ Hdisc(I) = νact ⋅ ( 1− KEDE^(I) ) ⋅ Hdisc(I) .

The units make the factorization explicit:

unitstime ⋅ discrimination unitsunit ⋅ bitsdiscrimination unit = movestime ⋅ bitsmove = bitstime .

The quantities that enter are conceptually separate:

  • νact: counted units per unit time, how fast the channel runs. It is fixed by the unit convention.
  • 1−KEDE^ : the fraction of counted capacity spent on discrimination rather than commitment, the article's hact. It is a fraction, not an entropy rate.
  • Hdisc: bits actually removed per discrimination (discovery) stage, the conversion rate from load to knowledge. It is a property of the pairing of a protocol with a knowledge state, not of the binary stream alone.
  • νcl: moves discovered and committed per unit time.
  • HMt (X∣Y) : Knowledge To Be Discovered per move, the uncertainty left after the episode-start information is known. It coincides with the marginal entropy H(X) when Y is uninformative under Mt, and with the entropy rate H(X) for the calibrated first-order Markov terrain below.
  • E[K] : discrimination stages per move, a count, not bits.
  • νdisc: discrimination load in non-output instructions per unit time, a count, not bits.
  • νdisc∗: bits of Knowledge To Be Discovered actually removed per unit time.

The ceiling on Hdisc

The robot's stage alphabet is binary, so Corollary 0′ applies with m=2 for each realized start context:

HMt (X∣Y=y) ≤ E [K∣Y=y] .

Averaging over Y gives HMt (X∣Y) ≤ E[K] . The bound runs through the conditional entropy, not the marginal entropy H(X) : the robot has to discover only what its episode-start information has not already settled. In the robot example the gap Γ reduces to the search excess εred = E[K] − HMt (X∣Y) ≥0 , so in the expectation reading:

Hdisc = HMt (X∣Y) E[K] = 1 − εred E[K] ≤ 1 ,

with equality exactly in the even-split regime of Lemma 1′(iv): dyadic posteriors, probability-matched trees, and no null stages.

Grain dependence

E[K] , 1−KEDE^ , Hdisc and νdisc all depend on what the unit convention counts as one action unit. νcl and HMt (X∣Y) do not: re-segmenting the same physical episode into finer or coarser units changes neither the elapsed time per committed move nor the uncertainty removed. The latent rate νdisc∗ is therefore invariant under a change of grain, and it is the quantity to compare across protocols and across regulators. The comparison is meaningful only inside one declared comparable task class τj (same X, same Y, same response-equivalence map, same resolution), and only once Knowledge To Be Discovered has been evaluated against a common reference law, because KTD is relative to a knowledge state. The Markov terrain below shows why that last qualification matters.

Knowledge discovery rate for the straightforward protocol

The first engineer's protocol always reads two symbols, so E[K]=2 and each move consumes three counted units: two reads and one commitment.

νcl(1) = νact3 , KEDE^(1) = 13 , νdisc(1) = 23 νact .

Each move removes 1.75 bits, so:

Hdisc(1) = 1.752 = 0.875 , νdisc∗(1) = νact3 ⋅ 1.75 ≈ 0.583 νact .

The gap is Γ=εred=0.25 . In the notation of Lemma 1′(ii) all of it is tree redundancy: the posterior is dyadic, so the coding gap δu is zero, while the fixed-length tree is not optimal for it, so gu=0.25 . The latent efficiency KEDE∗ =1/2.75≈0.364 exceeds the operational 1/3 by exactly the amount Corollary 3(ii) requires:

KEDE∗ − KEDE^ = Γ KEDE^ KEDE∗ = 14 ⋅ 13 ⋅ 411 = 133 .

Knowledge discovery rate for the probability-matched protocol

The second engineer's protocol needs E[K]=1.75 reads per move, so each move consumes 2.75 counted units on average:

νcl(2) = νact2.75 , KEDE^(2) = 411 ≈ 0.364 , νdisc(2) = 711 νact ≈ 0.636 νact .
Hdisc(2) = 1.751.75 = 1 , νdisc∗(2) = νdisc(2) = 711 νact ≈ 0.636 νact .

Here Γ=0 : stages are counted directly, nothing is invalidated, and every reached split is even. This is the ideal regime of Corollary 4(iii), in which operational and latent efficiency coincide and the operational rate is the latent rate.

The two rates rank the protocols in opposite orders

StraightforwardProbability-matched
E[K] (reads per move)21.75
Counted units per move32.75
νcl/νact=KEDE^0.3330.364
KEDE∗0.3640.364
Hdisc (bits per read)0.8751
νdisc/νact (operational)0.6670.636
νdisc∗/νact (latent)0.5830.636

The operational rate is higher for the worse protocol. Its extra read is real channel load, and νdisc counts it as discrimination load, but it removes no Knowledge To Be Discovered. The latent rate and the closure rate rank the protocols correctly: at the same νact, the matched robot commits 3/2.75≈1.091 times as many moves per unit time and discovers knowledge 1.091 times as fast. νdisc measures how fast discrimination capacity is consumed; νdisc∗ measures how fast knowledge is discovered. They coincide exactly when Hdisc=1 , which is the rate form of Γ=0 .

For a fixed channel rate, Hdisc is the fraction of the one-bit-per-discrimination-stage ceiling that a protocol converts into actual uncertainty reduction, and νdisc∗ = νact (1−KEDE^) Hdisc is the knowledge it discovers per unit time.

The instantaneous-commitment limit

If the commitment act is treated as taking no time, a read rate νact gives νcl = νact / E[K] , and the latent rates become 0.875νact and νact. That is the limit δcl≪τread . It is not free under the convention Δ:=δcl : a read lasting many Δ occupies several Δ-intervals and enters the ledger through the unit-inference term εΔ≥0 (condition CΔ still holds, since no interval contains two stages). The ranking by νdisc∗ is the same under either convention. In the limit, νdisc equals νact for both protocols and cannot rank them at all.

Breaking the coincidence: a distribution that is not dyadic

The numerical agreement found above - E [K∣Y=y] = 1.75 stages and H Mt (X∣Y=y) = 1.75 bits - is easy to misread as an identity. It is not. It is a coincidence manufactured by two conditions that happened to hold together: the conditional distribution over required moves was dyadic, and the protocol was probability-matched to it. Remove either condition and the two quantities come apart.

To see this, suppose the robot has descended into a shallow crater. For the episode-start context Y=y , the robot's current model assigns:

PMt (X=up∣Y=y) = PMt (X=down∣Y=y) = 13 ,
PMt (X=left∣Y=y) = PMt (X=right∣Y=y) = 16 .

The required knowledge

The Knowledge To Be Discovered at the start of this episode is:

KTD0 = H Mt (X∣Y=y) = 2⋅ 13 log23 + 2⋅ 16 log26 ≈ 1.9183 bits.

A probability-matched protocol would ideally assign a required move x a conditioning path of length:

ℓ (x∣y) = − log2 PMt (X=x∣Y=y) .

A protocol cannot execute 1.585 action units. The robot reads a whole binary symbol or it does not read one at all. The ideal path length is not an integer, so no binary protocol can be probability-matched to this distribution.

The distribution is not dyadic: its probabilities are not powers of one half. The matching that made the previous protocol exact is therefore unavailable in principle, not merely undiscovered.

The best protocol available

The second engineer returns and constructs the shortest possible prefix protocol for the new distribution. This time the answer is a fixed-length one:

00 → up,
01 → down,
10 → left,
11 → right.

Every episode closes after exactly two stages, so:

E [K] = 2 stages per discovered move.

This is a genuinely uncomfortable result for the first engineer's rehabilitation and the second engineer's pride at once. The variable-length protocol that served the earlier distribution so well now buys nothing. One can verify that every alternative prefix assignment including the variable-length assignment with path lengths (1,2,3,3) also yields an expected stage count of at least two. The ordered assignment with lengths (1,2,3,3) does average two reads, as does the fixed-length code. But other assignments average more. Assigning lengths (3,3,1,2) to (up,down,left,right), for example, gives: E [ K ] = 2.5 . The straightforward protocol is optimal here, and it is still not efficient.

The two quantities separate

We may now place the two readings side by side:

E [K] = 2.0000 protocol stages,

while:

H Mt (X∣Y=y) ≈ 1.9183 bits.

The numbers no longer agree, and the inequality runs in the direction the general bound requires:

E [K∣Y=y] ≥ H Mt (X∣Y=y) .

The gap is the redundancy of the protocol:

E [K] − H(X) ≈ 0.0817 stages

This represents the unused information capacity. Its first stage is informative, not null.

Roughly eight hundredths of an action unit per episode is expended without removing any Knowledge To Be Discovered. The robot pays it because binary protocols are quantised and this distribution is not.

Where exactly the waste sits

The deficit is not distributed evenly across the protocol, but concentrated entirely in the first stage. The gap between expected stages and entropy is gu+δu, i.e. one row's contribution to εred

At the first stage, the realized signal U1 separates {up, down} from {left, right}. Those two branches do not carry equal probability mass:

P ( U1=0 ) = 23 , P ( U1=1 ) = 13 .

One whole binary symbol is read, but the uncertainty of that symbol is only:

H ( U 1 ) = − 23 log2 23 − 13 log2 13 ≈ 0.9183 bits.

The second stage, by contrast, is perfectly balanced. Conditional on U1=0, up and down remain and are equally probable; conditional on U1=1, left and right remain and are equally probable. Hence:

H( U2 ∣ U1 ) = 1 bit.

The two stages together deliver:

0.9183 + 1 ≈ 1.9183 = HMt (X∣Y=y) ,

which is exactly the required knowledge, as it must be: the protocol does close the episode with certainty. But it takes two action units to deliver 1.9183 bits, because the first action unit is an unbalanced discrimination and an unbalanced discrimination cannot carry a full bit.

This is the earlier warning made concrete. An action unit, a realized binary symbol, and a bit of information are three distinct things. In the dyadic example they were numerically interchangeable. Here the first stage exposes one binary symbol and contributes 0.9183 bits, and the ledger no longer balances stage by stage.

The efficiency of the protocol

For the dyadic distribution and its matched protocol, H disc = 1 i.e. one bit removed per action unit spent, which is why the two numbers collapsed into one. For the crater distribution:

H disc = 1.91832 ≈ 0.9591 bits per discrimination stage.

The ratio is bounded above by one and attains one only under the two conditions stated at the outset. It is a property of the pairing of a protocol with a distribution, not of either alone.

Can the gap be closed?

It can be narrowed, though never eliminated for this distribution. Suppose the robot defers commitment and discovers n consecutive required moves in a single extended episode, assigning code words to whole blocks of moves rather than to individual ones. Because the moves are drawn independently, the block has entropy nH(X), and the optimal block protocol satisfies:

H ( X ) ≤ E[Kn] n < H ( X ) + 1n .

The quantisation penalty is incurred once per block rather than once per move, so it is diluted as 1/n. The average stage count per discovered move can therefore be driven as close to 1.9183 as desired, and H disc as close to one as desired, but the robot must wait longer before it learns any move at all. Efficiency is bought with latency.

No protocol, of any block length or any cleverness, brings the expected number of action units per discovered move below H(X). That is what makes entropy the right measure of the knowledge the situation demands rather than merely a description of one protocol's behaviour.

On the query ladder, the crater isolates the coding gap. The query family is still unrestricted, so ψ=0; the fixed-length protocol is optimal, so g=0; and the whole excess is

δ=DTμy(R∣y)−KTDrel(y)=2−1.9183≈0.082.

The coding efficiency of Proposition Q6 is 2.9183/3≈0.973. No adaptation channel removes this gap within single-move episodes. Block coding narrows it only by changing the unit of closure from one move to a block of moves, which changes the regulatory problem, not the regulator's knowledge, questions, or protocol.

Restricted sensors: when the questions, not the protocol, are the limit

The two examples so far isolate the protocol gap and the coding gap. The third gap of the ladder, the query-family gap, needs a robot that cannot ask every question. Return to the dyadic posterior

PMt(X=up∣Y=y)=12,PMt(X=down∣Y=y)=14,PMt(X=left∣Y=y)=PMt(X=right∣Y=y)=18,

and replace the uplink by two sensors. One answers “vertical or horizontal?”: it separates {up,down} from {left,right}. The other answers “first or second of the pair?”: it separates {up,left} from {down,right}. The query family is Q={qV,qP}.

The best protocol over these sensors

Each sensor alone leaves two moves indiscernible, and the two moves in each pair require different responses. A certainty-terminated procedure must therefore use both sensors on every path, and every path has length two:

DTμyQ(R∣y)=2.

Together the two sensors separate all four moves, so by Proposition Q4 the family is adequate; it is merely restricted. The second engineer's protocol is not available, because its first question, “is it up?”, is not in the family. The first engineer's protocol, which reads the vertical-or-horizontal sensor first and the pair sensor second, is optimal.

The same numbers, a different diagnosis

The ladder now reads

KTDrel(y)=1.75=DTμy(R∣y)<DTμyQ(R∣y)=2=𝔼[K∣Y=y],

so the gaps are

δ=0,ψ=0.25,g=0.

The robot spends exactly what the first engineer's robot spent in the unrestricted case, two reads per move, and its effective depth, KEDE, and total gap of Theorem 2 are the same. But the first engineer is now optimal, and the 0.25 is not waste of the questions the robot has; it is the cost of a question it lacks. In the factorisation of Proposition Q6 the protocol efficiency is one, and the question efficiency is 2.75/3≈0.917.

Lemma 1″ does not see the difference. The realized tree is full, so its Kraft slack is zero, and the whole 0.25 sits in the divergence term: the first question splits the posterior (34,14), an uneven split that the family forces rather than one the regulator chose. The operational ledger does not see the difference either. Only the declared query family does.

The remedy

No improvement of protocol, L2c, can close this gap, and no improvement of knowledge, L2a, is needed: the robot already knows the posterior. Adding a third sensor that answers “is it up?” is an L2b event. With it the second engineer's protocol becomes available, DTμyQ′(R∣y)=1.75, and the query-family gap vanishes. In the crater, by contrast, the restricted sensors cost nothing: the optimal unrestricted protocol there is the fixed-length one, which these two sensors already implement, so ψ=0 in the crater even with the restricted family. Whether a restriction on questions matters depends on the knowledge state as well as on the questions.

When the terrain has memory: the entropy rate stops being a formality

In the example so far, the required moves were sampled independently. That assumption was convenient, but it quietly disables one of the quantities we will need later. The entropy rate of a process is defined as the limit of the normalised joint entropy,

H ( X ) = lim n→∞ 1n H ( X1, …, Xn ) ,

but when the moves are independent and identically distributed the joint entropy is simply n H ( X ) , the limit collapses to H(X) , and the definition does no work at all. The averaging over n is ceremonial. To make the entropy rate earn its place we need a source whose successive required responses are not independent.

Terrain with momentum

Suppose the moon's surface is not featureless. Slopes, ridges and crater walls persist across several steps, so the move that was required a moment ago is evidence about the move required now. Concretely, let the required move follow a stationary Markov chain in which, with probability α, the terrain simply continues and the previous move is required again; and with probability 1−α the terrain resets and a fresh move is drawn from the original distribution π:

P( Xn=j ∣ Xn−1 =i ) = α δij + (1−α) πj .

This construction has a property we want: whatever α we choose, the stationary distribution of the chain is exactly π itself. The long-run frequencies of the four moves are unchanged from before:

π = ( 12, 14, 18, 18 ) for (up, down, left, right).

The marginal uncertainty about a single required move considered without episode-start information is still:

H Mn (Xn) = 1.75 bits per move.

But the robot no longer begins episode n without relevant prior information. The previous required move is already known when the new episode begins. We therefore make that realized predecessor the episode-start information:

Yn := X n−1 .

This does not place the previous move inside the knowledge state Mn . The knowledge state contains the robot's learned transition law - its model of how one move predicts the next. The variable Yn contains the realized predecessor value supplied to that model in this episode.

The initial Knowledge To Be Discovered is consequently:

KTD n,0 = H Mn ( Xn ∣ Yn ) = H Mn ( Xn ∣ X n−1 ) .

Take α=12 . Listing each conditional distribution in the order (up, down, left, right), the four rows are:

P ( ⋅ ∣ up ) = ( 34, 18, 116, 116 ) ,
P ( ⋅ ∣ down ) = ( 14, 58, 116, 116 ) ,
P ( ⋅ ∣ left ) = ( 14, 18, 916, 116 ) ,
P ( ⋅ ∣ right ) = ( 14, 18, 116, 916 ) .

The entropy rate of the correlated source

For a stationary Markov chain the limit defining the entropy rate has a closed form. The joint entropy grows by exactly one conditional entropy per additional move, so the limit is

H ( X ) = H ( Xn ∣ Xn−1 ) = ∑ i πi H ( Xn ∣ Xn−1 =i ) .

The four conditional entropies are:

  • after up: 1.186 bits;
  • after down: 1.424 bits;
  • after left: 1.592 bits;
  • after right: 1.592 bits.

Weighting these by π gives:

H ( X ) = 12⋅1.186 + 14⋅1.424 + 14⋅1.592 ≈ 1.347 bits per move.

The entropy rate and Knowledge To Be Discovered

For a stationary first-order Markov source, the entropy rate is the conditional entropy of the next move given the previous move:

H (X) = H ( Xn ∣ X n−1 ) .

In this worked example the robot's knowledge state is taken to represent the same transition law used to define the source. Because Yn = Xn−1 , the entropy rate and the robot's average initial Knowledge To Be Discovered coincide:

H (X) = H Mn ( Xn ∣ Yn ) ≈ 1.347 bits per move.

The relationship to the marginal uncertainty is now exactly the knowledge decomposition:

H Mn (Xn) = I Mn ( Xn ; Yn ) + H Mn ( Xn ∣ Yn ) .

When we add the response-relevant knowledge already available at episode start to the newly discovered knowledge, we obtain the total marginal uncertainty:

1.75 ≈ 0.403 + 1.347 bits per move.

The approximately 0.403 bits are response-relevant knowledge already available at episode start through Yn , as interpreted by the model stored in Mn . They are not Knowledge To Be Discovered. The remaining approximately 1.347 bits are.

Which robot is wasteful depends on what it knows

The second engineer's prefix code is unchanged, and because the stationary marginal is unchanged, so is its operational cost: E[K]=1.75 stages per move. Whether that is waste is not a property of the terrain alone. Knowledge To Be Discovered is HMn (Xn∣Yn) , evaluated under the robot's knowledge state, and two robots on the same terrain can carry different knowledge states.

Robot A carries Miid: it models successive required moves as independent draws from π. It receives the same episode-start information as any other robot, the realized predecessor Yn=Xn−1 , but its stored model gives that information no use:

PMiid ( Xn=j ∣ Yn=i ) = πj for every i , IMiid (Xn;Yn) =0 , HMiid (Xn∣Yn) = 1.75 bits.

Robot B carries Mmk, which contains the transition law of the terrain. For Robot B the predecessor is informative, with the values computed above:

IMmk (Xn;Yn) ≈ 0.403 , HMmk (Xn∣Yn) ≈ 1.347 bits.

Relative to its own knowledge state, Robot A wastes nothing. Its single codebook is the probability-matched dyadic code for π, so E[K] =1.75= HMiid (Xn∣Yn) and its search excess εred, which Lemma 1′ measures against the governing knowledge state, is zero. The 0.403 bits are not redundancy of Robot A's protocol. They are the difference between two knowledge states.

The article reads that difference in two ways, one for each of the outer feedback loops.

Calibration. Measured against the terrain, Robot A's model is miscalibrated, so the sequential calibration condition B5 fails. Lemma 1″(d) gives the drift for each predecessor i: the expected stage count of Robot A's codebook under the true row p∗(⋅∣i) minus the 1.75 stages it expects under π:

βi = ∑x ( p∗(x∣i) − π(x) ) ℓ(x) .
Predecessor π Stages under true row Stages under π βi
up1/21.3751.75−0.375
down1/41.8751.75+0.125
left1/82.3751.75+0.625
right1/82.3751.75+0.625

Row by row the drift is large, but it averages to exactly zero: π is the stationary law of the predecessor, so ∑i πi p∗(x∣i) = π(x) and the drifts cancel. This is why Robot A's long-run count is still 1.75. The miscalibration is invisible in the average stage count and becomes visible only when stage counts are tabulated by predecessor.

Learning. Replacing Miid by Mmk is an L2a update, Mt→Mt+1 in the notation of the learning section. The stationary marginal π is unchanged, so the constant marginal response-variety assumption Ht(X) = Ht+1(X) =1.75 holds exactly, and the change in stored coupling is the exact dual of the change in Knowledge To Be Discovered. The knowledge-ledger row for the update records:

Htstart =1.75 , Ht+1start ≈1.347 , GtK ≈0.403 , LtK =0 .

This is the Learning Axiom in its cleanest form. The terrain did not change and the code did not change, yet the next comparable episode begins with 0.403 fewer bits to discover, because the stored law of action now interprets evidence the robot was already receiving.

We can make a source-relative comparison: against the terrain's floor H(X) rather than against Robot A's own Knowledge To Be Discovered and say that the memoryless protocol "overspends by 0.403 stages per move". It is a legitimate comparison, but a different one, and it splits into two parts:

1.75−1.347 ⏟ ≈0.403 = (1.75−1.46875) ⏟ 0.28125learning + (1.46875−1.347) ⏟ ≈0.1217εred .

The first term is recovered by learning the transition law and then using it, through the conditional codebooks below. The second is Robot B's own search excess, measured against Mmk: the conditional distributions are not dyadic, so even a perfectly calibrated robot cannot reach 1.347 stages per move one move at a time. The coincidence that made stages and bits collapse to one number has been broken not by changing the code, but by changing what the robot knows.

A protocol conditioned on the episode-start information

The repair is not to place the previous move inside the knowledge state. The knowledge state Mn stores the transition law that tells the robot how the previous move changes the distribution of the next one. The realized episode-start information Yn = X n−1 then selects, through that stored model, which discrimination tree is appropriate for the current episode. The action unit does not change. At every stage the robot still performs:

Read the next binary symbol.

What changes is which move sits at which leaf. Four codebooks are used, one for each realized value of Yn , each matched to the corresponding conditional distribution.

after up:     0 → up,  10 → down,  110 → left,  111 → right,
after down:  0 → down,  10 → up,  110 → left,  111 → right,
after left:   0 → left,  10 → up,  110 → down,  111 → right,
after right: 0 → right,  10 → up,  110 → down,  111 → left.

The tree shape is identical in all four cases; only the leaf labelling differs. The expected number of stages, conditional on each predecessor, is 1.375, 1.5, 1.625 and 1.625 respectively, giving:

E [ K ] = 12⋅1.375 + 14⋅1.5 + 14⋅1.625 = 1.46875 stages per discovered move.

The ordering is now the general one rather than the degenerate one:

H Mn ( Xn ∣ Yn ) ≈ 1.347 ≤ E [K] = 1.46875 < 1.75 = H Mn (Xn) .

Why the residual gap remains, and what closes it

The conditional protocol does not reach the bound. It leaves about 0.122 stages per move on the table, and the reason is the one identified earlier: the conditional distributions are no longer dyadic. A probability of 34 cannot be matched by any whole number of binary discriminations, so some reached stages divide the remaining probability mass unevenly and contribute strictly less than one bit.

On the query ladder the change from Robot A to Robot B is a pure L2a event. The query family is unchanged: both robots read the same binary symbols, which can encode any binary question. Each robot's protocol is optimal for its own knowledge state: Robot A's single codebook is the Huffman code for π, and each of Robot B's four conditional codebooks is the Huffman code for its conditional distribution. Both protocol gaps are therefore zero, and so are both query-family gaps. Learning lowered Knowledge To Be Discovered by about 0.403 bits and the optimal expected depth by 0.28125 stages; the remaining 0.1217 stages of Robot B are entirely coding gap, the price of non-dyadic conditional distributions. The tree shape of the protocol did not change, only its leaf labels, which is what an optimal procedure does when the knowledge state changes and the questions do not.

Consequence for the rate reading

This also fixes which quantity belongs in the latent knowledge discovery rate. With closure rate νcl,

νdisc∗ = νcl ⋅ HMn (Xn∣Yn) .

For the calibrated Robot B on the stationary first-order Markov terrain, HMmk (Xn∣Yn) = H(X) , so νdisc∗ = νcl ⋅ H(X) .

The latent rate is model-relative. Two robots on the same terrain can be compared by it only after their Knowledge To Be Discovered is evaluated against a common reference law. Under its own model, Robot A discovers 1.75 bits per move. Under the terrain's law, 0.403 of those bits per move were already available through Yn to any robot whose knowledge state could read them. Crediting Robot A with the marginal entropy therefore overstates its discovery, relative to the terrain, by:

νcl ⋅ I (Xn;Yn) bits per unit time,

with the mutual information evaluated under the terrain's law. Under the unit convention of the rate section, where one read and one commitment each occupy one Δ, the two robots compare as follows:

Robot A (Miid, one codebook) Robot B (Mmk, conditional codebooks)
E[K]1.751.46875
Counted units per move2.752.46875
νcl/νact=KEDE^0.3640.405
KTD under own model1.751.347
KEDE∗ under own model0.3640.426
Γ against own model00.1217
νdisc∗/νact, own model0.6360.546
νdisc∗/νact, terrain law0.4900.546

Judged by its own knowledge state, Robot A looks perfectly efficient, with Γ=0 and KEDE^ = KEDE∗ , while Robot B carries a positive gap. Yet Robot B commits about 11% more moves per unit time ( 2.75/2.46875≈1.114 ) and, judged by the terrain, discovers knowledge 11% faster. KEDE∗ measures efficiency relative to what the regulator knows. Learning shows up elsewhere: as a lower Knowledge To Be Discovered, a higher closure rate, and a higher operational efficiency KEDE^ (0.364 → 0.405).

The entropy rate therefore appears here for a specific reason: in this first-order Markov example, under a knowledge state calibrated to the terrain, it equals the average Knowledge To Be Discovered per episode after the robot conditions on its episode-start information. The conceptual definition of KTD remains HMt (X∣Y) .

The robot example is an ideal illustration of both KEDE and Theorem 2:

Case KTD ^ eff net ( I ) KTD avg start-real,net ( I ) Γ (source) KEDE ^ KEDE ∗
Matched dyadic 1.75 1.75 0 0.364 0.364
Fixed-length, dyadic 2 1.75 0.25 (g_u) 0.333 0.364
Restricted sensors, dyadic 2 1.75 0.25 (ψ_u) 0.333 0.364
Crater 2 1.918 0.082 (δ_u) 0.333 0.343
Markov, conditional books 1.469 1.347 0.122 (δ_u) 0.405 0.426

One finite window: the envelopes of Theorem 2 and Corollary 2

The table reports expected values. A measurement sees one window. Suppose Robot B's window affords N=2469 units and closes Snet=1000 moves, with no retracted move and window edges at episode boundaries, so K^=1.469 and KEDE^=0.405. Robot B meets every hypothesis of Theorem 2 and Corollary 2. Each stage reads one binary symbol (A5), each stage is one counted unit (CΔ with equality, so εΔ=0), every path ends at a leaf (B2), and its knowledge state Mmk is the terrain's law, so stage counts are sequentially calibrated (B5) once the predecessor move is in the filtration. Each conditional codebook is a Huffman code for its row, so the search gap is zero and B1 holds with r=0. Stage counts lie between 1 and 3, so d=2, and every move spends at least one read and one commitment, so at most Jmax=⌊2469/2⌋+1=1235 episodes fit in the window. With probability at least 0.95:

Envelope Allowance Bound on K∗ (bits/move) Bound on KEDE∗
Two-sided, (7) with N+1Zα=0.1350.334 to 1.6040.384 to 0.750
Two-sided, (7) with JmaxZα=0.0950.374 to 1.5640.390 to 0.728
One-sided, (7′)0.086at most 1.555at least 0.391
True value under Mmk1.3470.426

The lower end is K^−1−r−Zα. Its width is dominated by the coding term 1+r, which does not shrink with the window, not by the fluctuation, which does. The upper end is already within 0.21 bits of the truth after a thousand moves, and it is the end that a human regulator also has.

Robot A shows what miscalibration costs. Its window of the same 1000 moves affords N=2750 units, so K^=1.75, which is also its Knowledge To Be Discovered under its own model. Measured against the terrain, B5 fails: the drift table above gives βi between −0.375 and 0.625. The worst-case remark after Corollary 2 charges dTV¯=2⋅0.4375=0.875, the largest row distance being that after a left or a right move, and the one-sided upper end becomes 1.75+0.091+0.875≈2.72 bits per move: a floor that no window length removes. But the drifts average to zero under the stationary law, and the terrain is a chain that repeats its move with probability 1/2, so the remark on miscalibration that averages out applies with g=2β and osc(g)=2. With Jmax=1376, the allowance is (41376ln40/2+2)/1000≈0.20, so with probability 0.95 the upper end is 1.95 bits per move and KEDE∗≥0.339 under Robot A's own model, against the true 0.364. The miscalibration is invisible in the window total, as the drift table showed, and the upper envelope survives it; what it does not survive is a reading of the 1.75 bits as Knowledge To Be Discovered relative to the terrain, which the comparison of the two robots above already ruled out.

Is a still more efficient protocol possible?

Could some more sophisticated protocol process long sequences of moves together and reduce the average number of binary symbols required per discovered move even further?

This leads to the third engineer.

The third engineer does not begin by proposing another concrete code. Instead, they ask what properties a perfectly efficient binary discovery protocol must have.

Their key intuition is that a perfectly compressed binary signal stream should contain no exploitable predictability. If predictable structure remained in the stream, then redundancy would remain as well, and a better coding scheme could exploit that structure to reduce the average description length.

The correct lower bound for a protocol that is allowed to use the robot's episode-start information is therefore:

E [K] ≥ H Mn ( Xn ∣ Yn ) .

For the stationary first-order Markov terrain used here, and under the assumption that the robot's model matches the source transition law, this is the same numerical floor as the source entropy rate:

H Mn ( Xn ∣ Yn ) = H (X) ≈ 1.347 .

The earlier E[K] ≥ H Mt (X∣Y) reading was a special case in which the available episode-start information supplied no additional knowledge about the required move. Then:

H Mt (X∣Y) = H Mt (X) .

By contrast, a protocol that deliberately ignores Yn and uses one fixed codebook is restricted by the marginal distribution and therefore cannot exploit the approximately 0.403 bits of response-relevant information already available at episode start. The distinction is therefore not that the knowledge state suddenly contains the previous move. The model is in Mn ; the realized side information is Yn ; and Knowledge To Be Discovered is what remains after conditioning on it.

Both quantities are properties of the source i.e. the terrain. What distinguishes them is whether the dependence between successive responses is taken into account. The protocol determines only which of the two floors is reachable. Structure in the environment is, from the robot's perspective, discoverable knowledge it does not have to pay for twice.

The reason the inequality holds is just that conditioning reduces entropy. The difference between H(X) and H(X) isn't distribution vs protocol. It's marginal vs joint: one asks how uncertain a move is considered in isolation, the other how uncertain it is given everything that came before. The terrain generates the moves; the protocol only discovers them. Nothing the engineers do changes either H(X) or H(X) .

Where the protocol enters is one level down, in the bound E [ K ] ≥ H ( X ) . . The protocol determines E [ K ] ; the source determines the floor. No protocol design can lower a floor - E [ K ] is the protocol's, H ( X ) is the terrain's.

H(X)=1.75 is the floor for protocols that do not exploit Yn: protocols whose knowledge state lacks the transition law, such as Robot A's, and protocols whose knowledge state has it but which still run one fixed codebook. Neither can get below 1.75 stages per move, however cleverly its code is designed, because neither turns the predecessor into conditioning information. Only a protocol run under a knowledge state that contains the transition law, and that selects its codebook by the realized Yn, can approach 1.347. The previous move itself is never inside the knowledge state: it arrives in Yn each episode, and the knowledge state supplies the law that reads it. Both numbers are lower bounds; they bound different protocol classes. It is the marginal entropy that acquires the protocol-relative reading, not the entropy rate.

Every quantity so far has been a per-move figure, and every one of them has been attainable, at least in principle, by a robot that closes one episode, acts, and walks on. There is one more benchmark available, and it is the first that the robot cannot reach while remaining a robot. It answers a question we have not yet been able to ask:

Had we been allowed to encode the entire N-move trajectory jointly, how short could its description have been?

Blocking as commitment, blocking as description

The earlier appeals to block coding carried a cost we noted only in passing. If the robot defers commitment until n moves have been resolved together, it does not learn the first move until the last one has been drawn. Pushed to its limit — the whole traverse as one block — the robot would discover its entire route only after the route was over. That is a perfectly good compression scheme and a useless protocol for an agent that has to walk.

But two different things travel under the name block, and only one of them costs latency. A block may be the unit of commitment, meaning the robot resolves nothing until the block closes; or it may be the unit of description, meaning that we, afterwards, treat the sequence ( X1, …, XN ) as one draw from the source. The robot may encode each required move separately, closing an episode and acting at every step, and the second reading still applies in full. The asymptotic equipartition property is a theorem about the source, not about the protocol. It does not care whether anything was deferred.

What changes is the scale at which we describe what happened. After the robot has completed N episodes, the observer may regard the realized sequence

XN = ( X1 , X2 , … , XN )

as one completed trajectory. The episode remains the unit of commitment: the robot must determine and execute one move at a time. The completed trajectory may nevertheless be used as a unit of description: once N moves have occurred, we can ask how much information is required to distinguish that realized path from the other paths that could have occurred.

Retrospective blocking gives the accounting of a block code at no latency cost. It does not give the compression. That asymmetry is the whole of what follows.

The traverse as an answer transcript

Because the code is prefix-free, the concatenation of the N code words is itself a block code for the N-block — a particular one, and generally not a good one. The traverse leaves behind a single binary answer transcript U1 U2 … UL , the answer transcript of subsection 6.6 (the program of the same traverse is longer by one output per move), of random length

L = ∑ i=1 N Ki .

By the law of large numbers L/N → E[K] , so the transcript is close to N ⋅ E[K] symbols long, with fluctuation of order N. The trajectory it identifies, meanwhile, lives in a typical set of about 2 N H(X) members, out of 4N = 22N conceivable ones. Correlated terrain is a smaller world of possible journeys than its move frequencies alone would suggest.

Shannon-McMillan-Breiman: what one long realized path tells us

The definition above is an ensemble statement: it is written in terms of the entropy of distributions over possible trajectories. For the robot example we want something stronger. We observe one path actually traversed by the robot. Can the information density of that realized path itself be connected to the entropy rate?

For a finite-valued stationary ergodic source, the Shannon-McMillan-Breiman theorem (also known as the Asymptotic Equipartition Property), gives exactly that connection:

− 1N log2 P ( XN ) → N→∞ a.s. H ( X ) .

Here a.s. means almost surely. With probability one under the source law, the average information content of a sufficiently long realized trajectory converges to the entropy rate. This means the asymptotic information density (or negative log-likelihood per symbol) of an almost-surely realized sample path converges to the statistical entropy rate of the underlying stochastic process.

Using the chain-rule decomposition above, the same result may be read as

1N ∑ i=1 N [ − log2 P ( Xi ∣ Xi−1 ) ] → N→∞ a.s. H ( X ) .

This is the important bridge between the episode and the trajectory. Each move is still committed locally. Its informational contribution depends on the path that preceded it. Yet when those contributions are averaged along almost any sufficiently long path generated by a stationary ergodic terrain, they settle on a single source-level quantity: H ( X ) .

Remark — Brudno's theorem. The Shannon-McMillan-Breiman theorem speaks of the probability of the realized path. Brudno's theorem speaks of its algorithmic content. For a stationary ergodic source over a finite alphabet, the Kolmogorov complexity per symbol of almost every realized trajectory converges to the entropy rate [82]:

1NK(X1,…,XN)→H(X)almost surely.

The two theorems agree on the number and differ on what it describes. Shannon-McMillan-Breiman says that the realized path is about as improbable as a typical one; Brudno says that it is about as incompressible as a typical one, with no reference to the source distribution in the description. That second reading is what the test of “convicting the answer transcript” below relies on: an outside observer who compresses the robot's answer transcript without knowing the terrain can, for a long enough transcript, reach the entropy rate and no further. In the limit, the robot's transcript exceeds that length per move by exactly its excess of queries over the entropy rate. It is the transcript that compresses to the entropy rate; the program, which records only how many reads preceded each move, does not identify the trajectory at all.

The source-coding interpretation

Shannon's source-coding result now gives the operational meaning of that limit. A long stationary ergodic trajectory has a typical set containing, asymptotically, approximately

2 N H ( X )

relevant trajectories, each carrying probability of approximately

2 − N H ( X ) .

Thus a lossless description of long trajectories can asymptotically approach N H ( X ) bits, or H ( X ) bits per move.

This does not require the robot to postpone its moves until an N-move block has been collected. The source-coding theorem supplies an asymptotic descriptive benchmark. The robot's commitment process and the observer's descriptive scale are different things.

If the robot continues to use a fixed code matched only to the marginal probabilities, it may expend more action units than the correlated source actually requires. A correlation-aware protocol can instead exploit the conditional law P ( Xi ∣ Xi−1 ) at each episode. Ideally, the informational burden associated with the realized move is then

− log2 P ( Xi ∣ Xi−1 ) ,

and over a long stationary ergodic path the average of those burdens converges to H ( X ) .

What the block means in this example

We can therefore state the distinction precisely:

  • Episode as unit of commitment. The robot discovers and executes Xi before proceeding to the next episode.
  • History as informational context. The uncertainty associated with the next commitment is determined by P ( Xi ∣ Xi−1 ) , not merely by the marginal probability of that move.
  • Trajectory as unit of description. After N episodes have closed, the observer may treat XN as one stochastic object whose information content is − log2 P ( XN ) .

The Shannon-McMillan-Breiman theorem then connects the local and global views:

lim N→∞ [ − 1N log2 P ( XN ) ] = H ( X ) almost surely.

The robot commits locally, but the path is informative globally. The entropy rate is the asymptotic amount of new response-relevant uncertainty per commitment once the predictive structure of the terrain has been taken into account. It is therefore not merely an ensemble average. Under stationarity and ergodicity, it is also the information density approached by almost every sufficiently long trajectory the robot actually traverses.

The benchmark

According to the Shannon Source Coding Theorem, for a stationary ergodic source, the absolute minimum expected description length in binary digits (bits) L∗ (N) required to code or describe a sequence of length N from a stationary ergodic source X approaches the source entropy rate times the length of the sequence:

L∗ (N) ≈ N ⋅ H(X) bits,

This figure is achievable to within an additive constant, and unachievable below it. Both halves of that claim are worth spelling out, because the constant is where the last of the quantisation penalty hides.

The typical set contains at most

| A ε (N) | ≤ 2 N ( H(X) + ε )

trajectories. Order them however we like — lexicographically, say — and the description of a typical trajectory is simply its position in that ordering. Naming one item out of m requires ⌈ log2 m ⌉ binary symbols, so the index occupies

⌈ N ( H(X) + ε ) ⌉ < N ( H(X) + ε ) + 1 binary symbols.

The +1 is nothing but the rounding up. The quantity N ( H(X) + ε ) is in general not a whole number, and the robot cannot read a fractional symbol. It is the same quantisation penalty that cost the crater protocol 0.0817 stages on every single move — except that here it is paid once for the entire traverse rather than once per move, which is precisely the 1/N dilution promised earlier.

That +1 covers indexing within the typical set. A complete scheme also has to say which set the trajectory fell in, since atypical trajectories still occur with small probability and must remain decodable. The standard construction prefixes a flag bit: 0 followed by the typical-set index, or 1 followed by a plain N log 2 4 = 2 N description otherwise.

0 followed by the index within A ε (N) ,
1 followed by the trajectory written out in full.

Writing a trajectory out in full costs two symbols per move, since there are four moves and

log2 4 = 2 , giving 2N symbols for the fallback branch.

So, one symbol more is still needed. The length of the typical branch is:

ℓtyp < N ( H(X) + ε ) + 2

with the second symbol of being the flag.

The length of the atypical branch is:

ℓatyp = 2N + 1 ,

Since the atypical branch is taken with probability at most δ, which can be made as small as we please by taking N large enough, the expected description length is

E [ℓ] ≤ (1−δ) ( N ( H(X) + ε ) + 2 ) + δ ( 2N+1 ) ,

and dividing through by N gives the per-move cost

E [ℓ] N ≤ H(X) + ε + 2δ + 2N .

Every term after the first can be driven to zero: ε by choice, δ and 2/N by taking a longer traverse. The two rounding symbols are therefore an O(1) overhead against a description of some 1347 symbols, and they do not disturb the benchmark.

The other half is what makes L∗ (N) a benchmark rather than merely a construction. Suppose some scheme allots fewer than

2 N ( H(X) − ε ) distinct descriptions.

Whatever trajectories it can name, their total probability tends to zero as N→∞ , because typical trajectories each carry probability close to 2 − N H(X) and there are too few names to go round. It becomes virtually certain that the traverse that actually occurred has no description — which, for the robot, means an episode closing on the wrong response class. No code with fewer than 2 N ( H ( X ) − ε ) words has error probability bounded away from one. So N ⋅ H(X) is not the best description anyone has yet managed to find. It is a wall.

For a thousand-step traverse of the correlated terrain, the three quantities can be read side by side:

Quantity Thousand-step traverse What it measures
N⋅H(X) 1750 bits uncertainty if each move is judged in isolation
N⋅H(X) 1347 bits the trajectory's actual information content
N⋅E[K] 1469 binary reads what the conditional protocol actually spent

The first two are properties of the terrain, differing only in whether the dependence between successive moves is counted. The third belongs to the protocol. The memoryless protocol of the second engineer would have spent 1750 binary reads on the same traverse, and a naive two-reads-per-move protocol 2000 binary reads. The trajectory itself required 1347 bits.

Why the benchmark is out of reach

L∗(N) is the cost of describing a trajectory that is already known in full. Whoever computes it sees X1 through XN simultaneously. The robot never does: at the moment it must resolve X1, the later moves have not been drawn. The comparison is therefore between an agent operating under causality and an archivist operating without it, and the gap

N ⋅ E[K] − N ⋅ H(X) ≈ 122 symbols per thousand steps

is what the robot pays for acting in time, counted in protocol symbols in excess of the bits the trajectory contains. For the memoryless Robot A the same excess is about 403 symbols, and the difference between the two figures is worth stating plainly, because the two resources are not paid for in the same currency:

  • 0.28125 stages per move are recovered by learning and conditioning. Once the knowledge state contains the transition law, conditioning on the realized predecessor in Yn costs no latency. The learning is a one-time L2a update to the stored law of action; after it, the robot still closes one episode per move and acts immediately.
  • About 0.1217 stages per move remain, and are recoverable only by blocking, that is, only by deferring commitment.
Conditioning on the past is free once the law that reads it has been learned; deferral about the future is not. A robot that must keep walking can have all of the first and none of the second. The residual is the price of its being an agent rather than an observer.

Where the residual sits

The remaining waste is not spread evenly across the four code books. Comparing the stages each spends against the bits each is required to deliver:

Predecessor π E [K∣⋅] H(X∣⋅) Waste
up1/21.3751.1860.189
down1/41.51.4240.076
left1/81.6251.5920.033
right1/81.6251.5920.033

Weighting by π returns approximately the 0.1217 above, but note the concentration: of that total, 0.0945 — over three quarters — is the first read of the after-up book alone. That read asks whether the terrain is still doing what it was already doing, when the answer is three-quarters likely to be yes, and so delivers 0.8113 bits for one whole action unit. It is also the read the robot performs most often, since up is the most common predecessor.

Convicting the record

Convicting the answer transcript

The third engineer's criterion was that a perfectly efficient signal stream should contain no exploitable predictability. That criterion can only be tested retrospectively: a single episode is far too short to show structure. Hand the completed answer transcript to an outside observer who has never seen the terrain, and ask them to compress it.

Under the memoryless protocol they succeed easily. Under the conditional protocol the fingerprint is fainter, and it degrades in stages. Every code book emits exactly 0.9375 zeros per move, as answer values, against 1.46875 symbols, so the transcript has

P(0) = 0.93751.46875 ≈ 0.638 , H(U) ≈ 0.944 bits per symbol.

An observer exploiting nothing but that bias — understanding neither the protocol nor the terrain — would recode the transcript from 1.46875 down to about 1.387 stages per move. Real compression, and proof of real waste. But the floor is 1.347, and the last stretch is invisible to any such analysis: the protocol's effective yield

Hdisc = 1.3471.46875 ≈ 0.917

sits below the stream's own marginal entropy of 0.944, and the difference is precisely the structure an order-zero observer cannot see. To find it one must parse the transcript into episodes, track which code book was in force at each position, and only then notice that certain positions in certain books divide the remaining probability mass unevenly. The redundancy has become conditional on parse state. The better the protocol, the more of the protocol one must already understand in order to prove it imperfect.

Two caveats, and the point

The benchmark is asymptotic and the traverse is finite: L∗(N) = NH(X) + o(N) , with a correction of order logN for a Markov source, plus a one-time constant to name X0, since the conditional code books are state-dependent. At N=1000 these are small but not zero, so about 1347 bits is the right register. And the transcript's length is itself random, concentrating on N⋅ E[K] rather than equalling it.

The compressibility of the transcript is the proof of the robot's overspending, and it is only ever available after the fact. Which is the situation the robot is permanently in. It can be shown to have been inefficient; it could not, from inside any single episode, have known. Perfect efficiency is a property visible only from the end of the journey, and the robot is never there until the journey is over.

What if E changes?

The above definitions assume that the essential-variable space E and the outcome-to-essential-variable map φ are held fixed. Under that assumption, addition, deletion, and substitution are changes in acceptability-status caused by a change in the acceptable region η .

if E changes, then the logic changes conceptually. This is not merely a different acceptable region. It is a different evaluative model: different dimensions of what counts as goal-relevant.

That matters because Ashby treats the system as a set of variables selected for analysis; changing the selected variables changes the modeled system/criterion, not just the essential-variable region. Ashby's framework is explicitly functional and behavior-oriented rather than material-object-oriented, so the selected variables are part of the formal description[1]. Umpleby also emphasizes that, for Ashby, the “system” is a set of variables selected by an observer[57].

If the essential-variable space itself changes, then the change is no longer merely a revision of the acceptable region. It is a change in the evaluative model. In that more general case, one would write φt : Z → Et and ηt ⊆ Et . The acceptable outcome-set would still be comparable across time if the outcome-space Z remains fixed:

Oacc,t := φt−1 [ ηt ] ⊆ Z

In that case, deletion, addition, and substitution can still be defined as set differences inside Z , but their interpretation changes: they may result from a changed acceptable region, a changed essential-variable space, a changed mapping from outcomes to essential variables, or some combination of these.

If the outcome-space Z remains fixed, then we can still define:

O acc, t 1 := φ t 1 −1 [ η t 1 ] ⊆ Z O acc, t 2 := φ t 2 −1 [ η t 2 ] ⊆ Z

Now both acceptable outcome-sets are again subsets of the same Z , so our deletion/addition/substitution logic still works.

L 12 = O acc, t 1 ∖ O acc, t 2 G 12 = O acc, t 2 ∖ O acc, t 1 P 12 = O acc, t 1 ∩ O acc, t 2

But the interpretation is different.

The change may now be caused by:

  • a changed acceptable region η t
  • a changed essential-variable space E t
  • a changed outcome-to-essential-variable map φ t
  • or some combination of the above.

So we can still say: outcome-value z lost acceptability, but we should not say: z lost acceptability only because the goal region changed.

It may have lost acceptability because the system is now being evaluated through different essential variables.

Example

At t 1 , suppose success is evaluated only by temperature:

E t 1 = E temp

At t 2 , success is evaluated by temperature and blood sugar:

E t 2 = E temp × E glucose

An outcome z that was acceptable at t 1 may become unacceptable at t 2 , not because the acceptable temperature range changed, but because blood sugar is now included as a goal-relevant dimension.

So the outcome has been “deleted” from the acceptable outcome-set, but this deletion comes from a change in evaluation space, not merely a narrower acceptable region.

In summary, changing E does not destroy the deletion/addition/substitution logic if we compare pulled-back acceptable outcome-sets inside the same fixed Z . But it changes the meaning of those changes. They are no longer purely goal-region revisions; they may be changes in the evaluative model itself.

In query terms, a change of the essential-variable space is a change of the regulatory relation Rt, through the acceptable response-sets it induces. It is an L3 event of the evaluative kind. The queries the regulator could ask before may no longer separate the disturbances that now matter, so a change of E can make an adequate query family inadequate, and episodes before and after it are not comparable unless the change is declared.

What learning could also do (but we are explicitly excluding)

Not every form of learning improves regulation H(E|Y) in Ashby's sense. Other possibilities include:

  1. Expanding action variety without selectivity

    Learning might increase H ( X ) (more possible actions, tools, behaviors) without reducing H ( X | Y ) .

    • The system becomes more capable in principle
    • But still does not know which action to take
    • Regulation does not improve

    This violates Ashby's requirement that variety must be constrained, not merely expanded.

    This case should be kept apart from expanding the regulator's questions. Adding actions without selectivity raises the variety of responses and leaves the regulator no better at choosing among them. Adding queries, the L2b channel, leaves the responses and Knowledge To Be Discovered unchanged but lets the regulator find the right response with fewer questions, as the restricted-sensors robot shows. The first is excluded because it does not improve regulation. The second does improve it, by lowering the query-family gap, and it is included in the model as an adaptation channel, although it is not learning in the sense of the Learning Axiom.

  2. Improving buffering instead of knowledge

    Learning might increase buffering capacity q (delay, slack, tolerance), so disturbances are absorbed without better action selection.

    • Outcomes may improve
    • But I ( X : Y ) does not increase
    • Regulation improves without learning the mapping

    This is explicitly separated from knowledge in Ashby's extended formulation.

  3. Changing goals or success criteria

    Learning could redefine what counts as success E .

    • Apparent performance improves
    • But the structural coupling (mapping) is unchanged
    • Information-theoretically, nothing about H ( X | Y ) need change

    This is semantic drift, not cybernetic learning.

  4. One-off adaptation without structural retention

    The system may succeed through exploration Z without storing the result.

    • Regulation succeeds this time
    • Next encounter repeats the same uncertainty
    • No accumulation of I ( X : Y )

    This is regulation, not learning.

How to cite:

Bakardzhiev D.V. (2025) Knowledge Discovery Efficiency (KEDE) and Ashby's Law https://docs.kedehub.io/knowledge-centric-research/kede-ashbys-law.html

Works Cited

1. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal. 1948;27(3):379-423. doi:10.1002/j.1538-7305.1948.tb01338.x

2. Ashby, W.R. (1956). An Introduction to Cybernetics; Chapman & Hall,

3. Ashby, W. R. (2011). Variety, Constraint, And The Law Of Requisite Variety. 13, 18.

4. MacKay, D. M. (1950). Quanta! aspects of scientific information. Philosophical Magazine; 41, 289-311;

5. Cover, T. M. & Thomas, J. A. (2006). Elements of Information Theory, 2nd ed., Wiley. §5.2-5.5 (Kraft, entropy bound, source coding bracket), §5.7 (Huffman codes and twenty questions), §2.8 (data-processing inequality).

6. Wheeler, J. A. (1990). Information, physics, quantum: The search for links. In W. H. Zurek (Ed.), Complexity, entropy, and the physics of information (Vol. 8, pp. 3'-28). Taylor & Francis.

7. Yaneer Bar-Yam.(2004) Multiscale variety in complex systems. Complexity, 9(4):37{45,

8. Ashby, W.R. (1991). Requisite Variety and Its Implications for the Control of Complex Systems. In: Facets of Systems Science. International Federation for Systems Research International Series on Systems Science and Engineering, vol 7. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-0718-9_28

9. Shannon, C. E. Communication theory of secrecy systems. Bell System technical Journal, 28, 656-715, 1949

10. Kubatko J, Oliver D, Pelton K, et al. A starting point for analyzing basketball statistics. J Quant Anal Sports 2007; 3: 1'-22.

11. Sports Reference LLC. "NBA League Averages." Basketball-Reference.com - Basketball Statistics and History. https://www.basketball-reference.com/leagues/NBA_stats_totals.html.

12. Bucks post highest single-game field-goal percentage by any team in 21st century https://sports.yahoo.com/article/bucks-post-highest-single-game-040313061.html

13. https://www.statmuse.com/nba/ask/most-field-goals-made-record-in-a-game-nba-player

14. Lewis, G. J., & Stewart, N. (2003). The measurement of environmental performance: an application of Ashby's law. Systems Research and Behavioral Science, 20(1), 31'-52. https://doi.org/10.1002/sres.524

15. Norman, J., & Bar-Yam, Y. (2018). Special Operations Forces: A Global Immune System? In Springer Unifying Themes in Complex Systems IX (pp. 486'-498). Springer International Publishing. https://doi.org/10.1007/978-3-319-96661-8_50

16. Norman, J., & Bar-Yam, Y. (2019). Special Operations Forces as a Global Immune System. In Springer Evolution, Development and Complexity (pp. 367'-379). Springer International Publishing. https://doi.org/10.1007/978-3-030-00075-2_16

17. O'Grady, W., Morlidge, S., & Rouse, P. (2014). Management Control Systems: A Variety Engineering Perspective. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2351099

18. Love, T., & Cooper, T. (2007). Digital Eco-systems Pre-Design: Variety Analyses, System Viability and Tacit System Control Mechanisms. 2007 Inaugural IEEE-IES Digital EcoSystems and Technologies Conference, 452'-457. https://doi.org/10.1109/dest.2007.372013

19. Love, T., & Cooper, T. (2007). Complex built‐environment design: four extensions to Ashby. Kybernetes, 36(9/10), 1422'-1435. https://doi.org/10.1108/03684920710827391

20. Bushey, D. B., & Nissen, M. E. (1999). A Systematic Approach to Prioritizing Weapon System Requirements and Military Operations Through Requisite Variety. Defense Technical Information Center. https://doi.org/10.21236/ada371943

21. Jones, H. P. (2018). Evolutionary stakeholder discovery: requisite system sampling for co-creation.

22. Grimm, D. A. P., Gorman, J. C., Robinson, E., & Winner, J. (2022). Measuring Adaptive Team Coordination in an Enroute Care Training Scenario. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 50'-54. https://doi.org/10.1177/1071181322661074

23. Becker Bertoni, V., Abreu Saurin, T., & Sanson Fogliatto, F. (2022). Law of requisite variety in practice: Assessing the match between risk and actors' contribution to resilient performance. Safety Science, 155, 105895. https://doi.org/10.1016/j.ssci.2022.105895

24. Tworek, K., Walecka-Jankowska, K., & Zgrzywa-Ziemak, A. (2019). Towards organisational simplexity - a simple structure in a complex environment. Engineering Management in Production and Services, 11(4), 43'-53. https://doi.org/10.2478/emj-2019-0032

25. Chester, M. V., & Allenby, B. (2022). Infrastructure autopoiesis: requisite variety to engage complexity. Environmental Research: Infrastructure and Sustainability, 2(1), 012001. https://doi.org/10.1088/2634-4505/ac4b48

26. van der Hoek, M., Beerkens, M., & Groeneveld, S. (2021). Matching leadership to circumstances? A vignette study of leadership behavior adaptation in an ambiguous context. International Public Management Journal, 24(3), 394'-417. https://doi.org/10.1080/10967494.2021.1887017

27. Ulrik, S., & Isabella, A. (2023). Variety versus speed: how variety in competence within teams may affect performance in a dynamic decision-making task.

28. Bakardzhiev, D., Vitanov, N.K. (2025). KEDE (KnowledgE Discovery Efficiency): A Measure for Quantification of the Productivity of Knowledge Workers. In: Georgiev, I., Kostadinov, H., Lilkova, E. (eds) Advanced Computing in Industrial Mathematics. BGSIAM 2022. Studies in Computational Intelligence, vol 641. Springer, Cham. https://doi.org/10.1007/978-3-031-76786-9_3

29. Heylighen, F., & Joslyn, C. (2001). Cybernetics and Second Order Cybernetics. In R. A. Meyers (Ed.), Encyclopedia of Physical Science and Technology, Eighteen-Volume Set, Third Edition (pp. 155-170). Academia Press. http://pespmc1.vub.ac.be/Papers/Cybernetics-EPST.pdf

30. Schwaninger, M., & Ott, S. (2024). What is variety engineering and why do we need it? Systems Research and Behavioral Science, 41(2), 235'-246. https://doi.org/10.1002/sres.2964

31. AULIN‐AHMAVAARA, A.Y. (1979), "THE LAW OF REQUISITE HIERARCHY", Kybernetes, Vol. 8 No. 4, pp. 259-266. https://doi.org/10.1108/eb005528

32. Wu, T., Dufford, A. J., Mackie, M. A., Egan, L. J., & Fan, J. (2016). The Capacity of Cognitive Control Estimated from a Perceptual Decision Making Task. Scientific Reports, 6, 34025.

33. Abuhamdeh S (2020) Investigating the “Flow” Experience: Key Conceptual and OperationalIssues. Front. Psychol. 11:158.doi: 10.3389/fpsyg.2020.00158

34. Automatic Screw Tightening Machine and Its Hidden Features

35. Keating, C. B., Katina, P. F., Jaradat, R., Bradley, J. M., & Hodge, R. (2019). Framework for improving complex system performance. INCOSE International Symposium, 29(1), 1218-1232. https://doi.org/10.1002/j.2334-5837.2019.00664.x

36. S. Engell (1985). An information-theoretical approach to regulation.

37. K. Kijima, Y. Takahara, B. Nakano (1986). ALGEBRAIC FORMULATION OF RELATIONSHIP BETWEEN A GOAL SEEKING SYSTEM AND ITS ENVIRONMENT.

38. W. Kickert, J. Bertrand, J. Praagman (1978). Some Comments on Cybernetics and Control. IEEE Transactions on Systems, Man and Cybernetics.

39. S. Engell (1985). Information-theoretical bounds for regulation accuracy. IEEE Conference on Decision and Control.

40. Hui Zhang, Youxian Sun (2003). Bode integrals and laws of variety in linear control systems. Proceedings of the 2003 American Control Conference, 2003.

41. R. Conant (1969). The Information Transfer Required in Regulatory Processes. IEEE Transactions on Systems Science and Cybernetics.

42. S. Engell (1987). Analysis of Regulation Problems based on Real-Time Rate-Distortion Theory. American Control Conference.

43. Hui Zhang, Youxian Sun (2003). Information theoretic limit and bound of disturbance rejection in LTI systems: Shannon entropy and H/sub /spl infin// entropy. SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

44. N. C. Martins, M. Dahleh (2008). Feedback Control in the Presence of Noisy Channels: “Bode-Like” Fundamental Limitations of Performance. IEEE Transactions on Automatic Control.

45. Hui Zhang, Youxian Sun (2003). H/sub /spl infin// entropy and the law of requisite variety. 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

46. Tsuji, M., Crookshank, M., Olsen, M., Schemitsch, E. H., & Zdero, R. (2013). The biomechanical effect of artificial and human bone density on stopping and stripping torque during screw insertion. Journal of the mechanical behavior of biomedical materials, 22, 146'-156. https://doi.org/10.1016/j.jmbbm.2013.03.006

47. Akizuki, K., & Ohashi, Y. (2015). Measurement of functional task difficulty during motor learning: What level of difficulty corresponds to the optimal challenge point?. Human movement science, 43, 107'-117. https://doi.org/10.1016/j.humov.2015.07.007

48. Bootsma, J. M., Hortobágyi, T., Rothwell, J. C., & Caljouw, S. R. (2018). The Role of Task Difficulty in Learning a Visuomotor Skill. Medicine and science in sports and exercise, 50(9), 1842'-1849. https://doi.org/10.1249/MSS.0000000000001635

49. Akizuki, K., & Ohashi, Y. (2013). Changes in practice schedule and functional task difficulty: a study using the probe reaction time technique. Journal of physical therapy science, 25(7), 827'-831. https://doi.org/10.1589/jpts.25.827

50. Goldhammer, F.; Naumann, J.; Stelter, A.; Tóth, K.; Rölke, H.; Klieme, E.: The time on task effect in reading and problem solving is moderated by task difficulty and skill. Insights from a computer-based large-scale assessment - In: The Journal of educational psychology 106 (2014) 3, S. 608-626 - URN: urn:nbn:de:0111-pedocs-179679 - DOI: 10.25656/01:17967; 10.1037/a0034716

51. Boothroyd, G., and P. Dewhurst, "DESIGN FOR ASSEMBLY", Dept. of Mechanical Engineering, University of Massachusetts, Amherst, Massachusetts, 1983.

52. Jahin, A., Zidan, A. H., Bao, Y., Liang, S., Liu, T., & Zhang, W. (2025). Unveiling the mathematical reasoning in deepseek models: A comparative study of large language models. arXiv preprint arXiv:2503.10573.

53. FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI

54. Francis Heylighen, Cybernetic Principles of Aging and Rejuvenation: The Buffering- Challenging Strategy for Life Extension, Current Aging Science; Volume 7, Issue 1, Year 2014, . DOI: 10.2174/1874609807666140521095925

55. Ostry, D. J. (1980). Execution-time movement control. In G. E. Stelmach & J. Requin (Eds.), Tutorials in motor behavior (pp. 457-468). Amsterdam: North-Holland.

56. Siegenfeld, A. F., & Bar-Yam, Y. (2025). A Formal Definition of Scale-Dependent Complexity and the Multi-Scale Law of Requisite Variety. Entropy, 27(8), 835. https://doi.org/10.3390/e27080835

57. Umpleby, S. A. (2009). Ross Ashby's general theory of adaptive systems. International Journal of General Systems, 38(2), 231-238. https://doi.org/10.1080/03081070802601509

58. Zheng, J., & Meister, M. (2024). The unbearable slowness of being: Why do we live at 10 bits/s?. Neuron.

59. Ohlsson, S. (1992). The Learning Curve for Writing Books: Evidence from Professor Asimov. Psychological Science, 3(6), 380-382.

60. L. B.S. Raccoon. 1996. A learning curve primer for software engineers. SIGSOFT Softw. Eng. Notes 21, 1 (Jan 1 1996), 77-86. https://doi.org/10.1145/381790.381805

61. Deweese, M. R., & Meister, M. (1999). How to measure the information gained from one symbol. Network: Computation in Neural Systems, 10(4), 325-340. https://doi.org/10.1088/0954-898X_10_4_303

62. Gallager, R. G. (1978). Variations on a theme by Huffman. IEEE Trans. Inform. Theory 24(6), 668-674. (Redundancy bound p_{\max} + 0.086.)

63. MacKay, D. J. C. (2003). Information Theory, Inference, and Learning Algorithms, CUP, ch. 5 (Ex. 5.24, 5.28).

64. McMillan, B. (1956). Two inequalities implied by unique decipherability. IRE Trans. Inform. Theory 2(4), 115-116. (Optional; Cover & Thomas suffices.)

65. Azuma, K. (1967). Weighted sums of certain dependent random variables. Tôhoku Math. J. 19, 357-367; Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. JASA 58, 13-30. (For Theorem 2(iv).)

66. Metzler, R., Bar-Yam, Y., & Kardar, M. (2004). Information flow through a chaotic channel: prediction and postdiction at finite resolution. Physical review. E, Statistical, nonlinear, and soft matter physics, 70(2 Pt 2), 026205. https://doi.org/10.1103/PhysRevE.70.026205

67. Pawlak, Z. (1982). Rough sets. International Journal of Computer and Information Sciences, 11(5), 341-356.

68. Pawlak, Z. (1991). Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic Publishers, Dordrecht.

69. Kryszkiewicz, M. (1998). Rough set approach to incomplete information systems. Information Sciences, 112(1-4), 39-49.

70. Ślęzak, D. (2002). Approximate entropy reducts. Fundamenta Informaticae, 53(3-4), 365-390.

71. Liang, J., & Xu, Z. (2002). The algorithm on knowledge reduction in incomplete information systems. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(1), 95-103.

72. Li, R.-P., & Wang, D. (2004). A new uncertainty measure of knowledge in incomplete information systems. Proceedings of the Third International Conference on Machine Learning and Cybernetics, Shanghai, 1486-1490.

73. Traub, J.F., Werschulz, A.G. (2008). Information-based Complexity and Information-based Optimization . In: Floudas, C., Pardalos, P. (eds) Encyclopedia of Optimization. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-74759-0_281

74. H. Buhrman and R. de Wolf, "Complexity measures and decision tree complexity: a survey", Theoretical Computer Science 288 (2002) 21-43.

75. A. C.-C. Yao, "Probabilistic computations: toward a unified measure of complexity", Proc. 18th IEEE FOCS (1977) 222-227.

76. J. F. Traub, G. W. Wasilkowski and H. Woźniakowski, Information-Based Complexity, Academic Press, 1988.

77. A. Pelc, "Searching games with errors: fifty years of coping with liars", Theoretical Computer Science 270 (2002) 71-109.

78. SS. Laplante and F. Magniez, "Lower bounds for randomized and quantum query complexity using Kolmogorov arguments", Proc. 19th IEEE Conference on Computational Complexity (2004) 294-304

79 Laplante, S. (2006). Lower Bounds Using Kolmogorov Complexity. In: Beckmann, A., Berger, U., Löwe, B., Tucker, J.V. (eds) Logical Approaches to Computational Barriers. CiE 2006. Lecture Notes in Computer Science, vol 3988. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11780342_32

80. P. Grünwald and P. Vitányi, "Shannon information and Kolmogorov complexity" (2010).

81. G. J. Chaitin, "From Philosophy to Program Size: Key Ideas and Methods. Lecture Notes on Algorithmic Information Theory", Estonian Winter School in Computer Science, March 2003; https://arxiv.org/abs/math/0303352

82. A. A. Brudno, "Entropy and the complexity of the trajectories of a dynamical system", Trudy Moskovskogo Matematicheskogo Obshchestva 44 (1982); English translation in Transactions of the Moscow Mathematical Society (1983) 127-151.

Getting started