A projection of Norem's internal ledger and the Tier 1 falsification card, under the same discipline: dated, signed, anchored, append-only. New here? Start with the introduction.
Norem is built on one claim: that the reasoning moves of a discipline can be measured directly, rather than through proxies for them. If that is true, a test built from those moves is far harder to game, because teaching to it is teaching those moves. The claim can fail in several places, and each place has its own test here.
Tier 1 is the first time real learners meet the system. One topic from medieval English history, taught from published accounts by two historians (whether both historians' lessons ship in Tier 1 is an open question; see section 5), with a small cohort working through a sequence of instruments, every action recorded with the context it was taken in. Before any learner arrives, the material is checked by outside historians, the compiled instruments are checked for leaks, and parts of the apparatus are rebuilt blind by someone who has not seen the author's answers. Afterwards, historians read the recorded traces without being told what the system scored, and the predictions below are adjudicated against what happened.
Not that Norem teaches anything; Tier 1 cannot see learning. It can see whether an instrument measures the act it names, whether it stays silent about the answer, whether a historian can read a learner's trace, whether learners got there by reasoning, whether the failure types the system looks for are the discipline's own and actually occur, and whether the accounts were read fairly. Those are the entries in section 1 (signed: …; proposed: …; both counts computed from the mirror), written before contact, each with the observation that would fail it and the person who decides.
Before learners arrive, Norem writes down what would prove each part wrong, who decides, and what counts as evidence. Once the external anchor exists, each text is hashed and anchored outside Norem's control on the day it is signed, so any later rewrite would be detectable. Until then, every date here is Norem's own claim. Most texts are published in full. A few cells are sealed, because publishing them would tell a learner what a good answer looks like, or tell the second outside reader what the first one found; each sealed cell names the condition that opens it. Nothing on the ledger is edited; changes are dated appends. Mistakes, including small ones, go in the corrections register.
The version line at the top says which versions of the register and the rules every result was produced under. Section 4 (what went wrong) and section 3 (who checked, blind to what) are the parts hardest to fake; start there. Section 1 is the predictions. Section 9 explains the terms. Sealed means the hash is public now and the text comes later. Proposed means drafted and not yet binding. Fired means the system was wrong in the way the entry said it could be, after the check for firing for the wrong reason has been run, and the entry says what happens next.
The dates, once anchored, and the outside readers' returns, hashed on receipt and confirmable with the readers themselves, are the parts nobody can fabricate after the fact. Everything else, you can check.
F-n card entries, PR-n pre-registrations, M-n rulings, O-n open items, OQ-n open questions. Two are this ledger's own: CR-n corrections and OC-n outside checks. Exploratory analyses with no failing observation (H-n) will form a separate sub-register when they exist and are never counted among predictions.| what | when | kind |
|---|
No latencies. Appends, reveals and closure are each defined as an anchored block, so there is no interval between the event and its publication to bound.
Every falsification-card entry and every pre-registration as a row. Signed entries are published in full with their block: claim, failing observation, adjudicator, minimum n, class-6 test, needs. A few cells inside some entries are sealed until a gate (rule 6); the text column counts them. Proposed entries are hashed until signed. Class 4 (mechanical) entries are complete and open, and collapsed by default to keep the page readable. The note column is editorial.
| row | date | cohort | version line at the event | recorded |
|---|---|---|---|---|
| first contact opened | [generated] | [cohort id] | snapshot of the version line | pre-registrations whose anchors precede this row; proposed blocks marked unsigned at contact |
| cohort closed | [generated] | [cohort id] | snapshot of the version line | n reached; which G-C condition fired (n or eight-week backstop) |
| id | class | block | date | adjudicator | runs | status | text | touched by | blocked on | note (editorial) |
|---|
| id | date | status | text | card entry | note |
|---|
Dated, numbered, with what each superseded. The version line at the top of the page is the current state; a result anywhere on this ledger can be read back to the ruleset it was produced under. Ruling texts live in the internal ledger and are sealed; their hashes are published at the first public commit.
| id | date | what it did (editorial, until texts are open) | supersedes | text |
|---|
| block | date | standing | ruling | bytes | sha-256 |
|---|
The hashes above were computed on the consolidated file's heading ranges and stand in for the canonical signed blobs until canonicalisation.
Who read what, blind to what, under which frozen brief, paid how much. Verdicts as counts and rule outcomes. Names and words only with consent. Reviewer 1's verdicts are sealed until reviewer 2 has returned, because reviewer 2's independence is what the fidelity check depends on.
| id | role | brief | blind to | paid | status | verdicts | entries |
|---|
The dated corrections register. Each row names what was wrong, how it was found, and what followed. A correction that touches a pending reviewer's material has its content sealed and its existence and date open.
| id | date | what was wrong | found by | what followed | standing |
|---|
Separated from rulings. Each line the card is first evidence for carries its grade as written and the entry that tests it; texts are sealed with the entries. Below them, the rulings the card is waiting on, and the marks logged for the invariants' amendment record.
| line | title | grade as written | entry | in Tier 1 |
|---|
| id | question | bears on | opened |
|---|
| id | mark | standing |
|---|
The JSON this page renders from. Row counts, statuses and hashes above are read from it, not typed. ledger.json beside the prose; same ids, same rows.
Definitions as the card uses the terms, with the nearest standard term where one exists and where the card's use departs from it. This section is editorial: it has a changelog, not a signature, and no row is checked against it. Terms whose full definition would describe what a competent trace looks like are given in the sealed layer until first contact closes. Where a term is defined by an invariant line the card only cites, the source column names the line and the definition here is from usage.
| set | members |
|---|---|
| defect class | 1 fabricated or misread content · 2 missing producer (a pole with no source behind it) · 3 unfounded structure · 4 mechanical · 5 loaded selection (the apparatus carries the answer) · 6 analysis defects (the reading of the data is wrong). Classes 5 and 6 have no standing practice in the field and are described in advance. |
| adjudicator | R0 a second hand at compile time who has not seen the author's answer · R1 a historian on the fidelity of the reading, no learner · R2 a historian on the apparatus, no learner · R3 a historian or education researcher on the register's failure typology, no learner · R4c a historian reading a learner's trace · R4t transcripts across a cohort |
| runs | CMP compile-time check, no learner · SYN synthetic runs may pass or fail it · REV a reviewer, no learner · HUM human learners only. A split entry has a guard half (SYN or CMP) that runs first and an evidence half (HUM or REV) that only humans supply. |
| status | open · fired · passed · expired unfired · voided · conditional (fires only if a ruling admits it) · placeholder (needs a document not in hand) · dormant (ruled off; reopenable) · proposed (drafted, unsigned) · superseded (replaced by dated amendment) · sealed |
| grade | commitment (a design decision, not a claim) · supported · supported on paper · hypothesis · conjecture · speculation. A ⚑ marks a line under live doubt. |
| namespaces | F-n card entry · PR-n pre-registration · M-n ruling · O-n open item · OQ-n open question · S-n system invariant · A-n atom invariant · T-n working session · CR-n correction · OC-n outside check · H-n research ledger |
| gates |
| term | as the card uses it | nearest standard term · where it departs | source |
|---|---|---|---|
| act | A disciplinary reasoning move the system can measure. Each act has a record: its typing, a crux statement, and its characteristic failures. Two candidate acts are one act unless they can fail independently. | knowledge component (KLI); construct. Departs: an act is individuated by its failure modes, not by the content it covers. | S-1, S-2, F-3.4a |
| register | The closed, versioned list of acts (v0.5-rc.3). Admission runs through a gate; new acts enter by demand, never by addition at the card. | competency framework. Departs: closed alphabet, versioned, with a gate; not finalised before learner contact. | S-14, P-9, A-41 |
| lineage · U/S rivals | A family of related act candidates. The unified (U) reading holds the family as one act; the split (S) reading as several. Which is right is an empirical question the card tests. | no standard term. | S-1, S-7, F-3.4 |
| pole | One of the things an act relates. An act's crux is in holding its poles together, never in either pole alone. | relatum. Departs: poles may legitimately circulate inside the instrument; recovering them is not recovering the act. | A-8, A-18 |
| relation | The composition over poles that one atom measures. Two atoms of one act triangulate only on different relations. | no standard term. | A-11 |
| cut · geometry · arity | What an act asks the learner to separate. Geometries: relate two things; position along a tension; sort a set. The arity is the number of poles. For two-pole acts, whether the committed object sits on the edge between poles or on a pole (edge- or node-bearing). Geometry is an authored field of the act record and fixes the instrument family. | no standard term; nearest: response format. Departs: the format is derived from the act, not chosen for the item. | A-33, A-37, A-48 |
| atom (binding atom) | The unit that both instructs and measures one act on one relation. Four faces: ρ the compiled scene, μ the outcome index that bands a commit, κ* the expected placement, and τ. | task model; item family. Departs: instruction and measurement are one object, not two aligned ones. | A-1 (τ defined there; not expanded in the card) |
| item | A compiled instance of an atom on an arc: a stem, an instrument, an accepted range, an id that reads back to a register act. | item. Same. | F-4.2, O-44 |
| instrument · family · member | The structured means by which the learner commits. The family (selector, slider, placement, sort) is fixed by the act's geometry and gate-enforced; the member within a family may be the learner's choice. | response format. Departs: family is derived; member is the one thing left to the learner. | A-37, S-53 |
| commit · token | The learner's recorded action on an item. Every commit is stamped with instrument version, provenance, staging level and whether it was pulled or pushed. A pulled token also records what opened it. | response; process-data event. Departs: the stamps are part of the measurement, not metadata. | S-24, A-23, A-46, F-4.1 |
| placement · expected placement | Where a learner commits on an instrument. The expected placement is the discipline's, authored before contact; where learners are routed, it is the routed master's. | answer key. Departs: declared stable across the discipline or master-relative, and tested as such (F-5.2). | S-58, S-0.12 |
| band · band table | The outcome category μ assigns a commit: correct, or a typed failure. The band table declares, per act and before contact, whether its bands are binary, narrow or wide. A wide band means the act is delivered, never rescued with sub-criteria. | rubric level; scoring category. Departs: typed failures are identity-bearing and declared in advance; learner data may correct their frequencies but never mint one. | A-5, S-5, S-53, S-54 |
| failure type · commit-type · learner-type | A characteristic way an act fails. A commit-type is how the learner got there on this commit and feeds μ's band; a learner-type is a stable disposition, resolvable across atoms, and feeds the profile. | misconception; bug (Brown & Burton). Departs: whether a type belongs to the learner rather than the commit is itself under test (F-C.7). | S-2, F-4.4 |
| misconception prior | A pre-registered ranking, per site, of which failure bands are expected to fire most. Seeds μ; the learner corrects the frequencies. | bug library; prior. Departs: a ranking, signed before contact, falsified by observed order. | O-15, F-6.4 |
| declared resolution | The failure types an atom separates on its own. A band is read at the atom's resolution and no finer; types only a compiled combination of atoms separates are read across atoms. | no standard term; nearest: diagnostic resolution. | F-4.4 |
| open range · premature closure | A declared range within which learner weighting is not bandable as failure. Closing a question the sources leave open is a banded shape. | no standard term. | A-5 |
| measurement identity | The compiled atom announces which act it measures. Tested forward by whether a reader not told the act can derive it from the scene and commits. | construct identification. Departs: tested by blind derivation, not by expert agreement on a label. | A-1, F-6.1a |
| composition | Relating the poles: the act performed. Contrasted with construct-irrelevant processes a commit can be produced by instead: recall or recognition, comprehension substitution, apparatus-solving. | construct-irrelevant variance (Messick); cognitive validity (Smith 2017). Same sense. | S-0.3, F-6.6 |
| term | as the card uses it | nearest standard term · where it departs | source |
|---|---|---|---|
| master | A historian whose published work is extracted as an arc's terrain. Learners are routed to a master's position without being told whose it is. | exemplar; source text. Departs: the arc is master-relative by construction; the reveal shows both. | M-10 |
| arc | A compiled sequence of atoms over one topic on one master's terrain. Tier 1 runs one arc per master on the same substrate. | unit; lesson sequence. Departs: compiled, with a release partial order; never forks into a wrong-content branch. | A-14, A-21, A-46 |
| terrain · extraction · source line | The master's chapters factored into rows under the extraction protocol: a blind pass that lists what the work does, a neutral pass that factors each act against the register. A source line is one row, shipped to the learner as material and carrying a pole. | content analysis; corpus annotation. Departs: the register is applied blind to the other master, and a second extractor tests the labels. | S-37, F-3.9 |
| producer | The source behind a pole: a line or a prior commit that could have produced it. A pole with no producer is a class-2 defect. | evidence; warrant. | A-42–A-44, S-43 |
| grain | The level at which a pole or a claim is realised: document versus entry; bundle versus part. The same act read at a finer or coarser grain is not a different act. | granularity. Same. | S-36, S-49 |
| part · partition | A bundled historical subject is decomposed into parts, each with its own direction and timing. Learner claims are read per part. The sort of statements into parts is the partition sort. A part can be absent in a master, asymmetric (one direction supportable), or fused with another. | colligatory concept (Walsh). Departs: the partition is operational, fixed by an anchor table, and the sort is itself a candidate act (currently ruled not one). | S-16, S-18, F-5.6, M-36 |
| anchor table · anchor id | The ids that fix what each part is, by the master's own definition. Keys the reveal grids and the claim rows. | codebook. Same sense. | F-1.3, S-30 |
| scene (ρ) | What the learner sees at an item: the released lines, the staging level, the probe text. The scene is tested for leaning (F-5.4) separately from the answer. | stimulus. Departs: compiled from the release schedule; audited for leaks. | A-1 |
| staging · staging level | How much of a relation has been released to the learner before a commit, on a ladder of levels from unreleased through tension-flagged to relation-asserted. The rungs are themselves under test. | scaffolding level. Departs: staging is compiled release of content, not help; a commit's level is a stamp on it. | A-23, A-24, A-26 |
| release · release edge · dual schedule | A compiled event that raises staging: a line shown or a prior atom's commit. The dual schedule is the compiled order of releases and commits; a release reached any other way is a leak. | prerequisite graph. Departs: enforced by an embargo compiler, audited at runtime by the observer's sheet. | A-45, A-23, F-2.3 |
| salience threshold | Per relation, the staging level above which placements change shape. Nine such thresholds are conjectured; each is a hypothesis. | no standard term. G-C the shape they change to | A-30, F-C.1 |
| probe · probe inventory | A line the runtime voice speaks to prompt the learner without releasing content. The inventory is versioned and is itself an object under test. | prompt; hint. Departs: a probe may not raise staging; one that does is a leak. | K-6, S-19, F-5.4 |
| the Historian | The runtime voice that speaks to the learner. Its dynamic text is checked against the embargo; pedagogy it authors must be certified as encounter-authoring. | tutor agent. Departs: may not author content the schedule has not released. | A-4, F-2.3 |
| embargo · leak | Compile-time control over what a learner can have seen before a commit. A leak is a relation reaching a staging level by a path the schedule did not author, including by two individually legal dynamic lines composing across turns. | no standard term; nearest: information leakage. | A-15, A-17, B6 |
| reveal | After a commit, the learner sees where the masters placed, both of them, on a grid keyed by the anchor table. | feedback. Departs: shows plural positions, not one correct one. | S-13, S-30, C-1 |
| routing · stance menu | At level 1 the learner picks a stance from a menu; the stance maps to a master and the learner is scored against that master on master-relative items. The labels are tested for loading. | condition assignment. Departs: self-selected, screened, and the screen is itself a card entry. | M-10 arm 1, P-6, F-5.3 |
| delivered | Shipped to the learner as scene or material without being scored. Everything master-relative that cannot be scored neutrally is delivered; an act whose band is wide is delivered. | unscored; formative. Departs: delivery is a ruling with a reason, recorded. | S-56, M-43 |
| term | as the card uses it | nearest standard term · where it departs | source |
|---|---|---|---|
| frame | The learner's declared object: a problem, a form, and a revisable claim. The room builds around it. A frame failure can open a commit the schedule had not served. | problem statement plus thesis. Departs: the runtime's primary object, revised across the arc, not a prompt answered once. | S-10 |
| form · slot | A declared configuration over acts that a claim must fill. Slots are what the learner is probed to commit in. The slot set is derived from the discipline's genres and tested (F-3.6). | genre; task format. Departs: declares the claim's schema, leaves the claim open. | S-11, S-48 |
| claim · claim table · claim maintenance | The learner's structured claim: per part, a direction, a strength, an until and a scope. Claim maintenance checks it against the learner's own released commits and fires one of four failure kinds, among them contradiction, loss of support and unresolved obligation. | argument model; constraint check. Departs: coherence is checked against the learner's own commits, not against the master. | S-21, TA §3 |
| provenance · pulled · pushed | Whether a commit was opened by the learner's own frame failure (pulled) or served by the schedule (pushed). Stamped on every commit; the pulled count is reported, not designed. | learner-initiated vs system-initiated. Departs: pulled is opened by a detected failure, and the distinction is itself under test (F-3.8). | S-24, S-25, A-46 |
| first contact | A learner's first commit on an item. Only first contact is scored; a repeat, a commit after switching master, or a task that ships a line the arc ships, is not. | first attempt. Departs: later attempts go to the diary and are never scored. | S-33 |
| level bundle | A level is a bundle of compile-time parameters (what is pre-filled in the frame), not a point on a scale. | difficulty level. Departs: bundles are ordered by pre-commitment and the order is testable (F-C.3). | S-22, PR-LVL |
| trace | A learner's recorded commits with their stamps. Must be legible to a historian who was not watching (Crux C). | process data; log. Departs: legibility to an outside reader is a card entry, not an assumption. | A-40, F-6.1 |
| curve · profile | The per-act accumulation over a learner's commits (curve) and the learner-level record it feeds (profile). The inference rules of the profile have no home yet. | learning curve; learner model. Departs: nothing is inferred into the atom; cross-atom rules are frozen before contact. | D-1, A-2, F-4.4 |
| diary · ledger | Diary: investigation records, synthetic runs, think-aloud sessions. Ledger: scored first-contact commits and everything signed. Nothing moves from diary to ledger. | no standard pair. | S-33, amendment 2 |
| term | as the card uses it | nearest standard term · where it departs | source |
|---|---|---|---|
| falsification card · entry | The pre-contact list of claims Tier 1 is the first evidence for. Each entry names the observation that would fail it, its adjudicator, its minimum n, a test for firing for the wrong reason, and what it needs in hand. A claim with no failing observation is not on the card. | pre-registration. Departs: every entry carries its own alternative-explanation test, and the card's scope is the system's scope (S-62). | S-45, S-62 |
| class-6 test | On every entry, the observation that would show the entry fired for a reason other than the one claimed. Must be run before a firing counts. | alternative explanation; robustness check. Departs: pre-registered per entry. | F-6.3 |
| scope law | Tier 1's scope is read off the card: an entry on the card is a component Tier 1 must guarantee; what no entry observes is out of scope and not tracked. | no standard term. | S-62, M-9 |
| ladder · rung | The order in which evidence is sought. Rung 1 fidelity of the reading; 2 the apparatus; 3 the register's failure typology; 4 one learner closes the loop; 5 portability; 6 teaching versus selection. Tier 1 is rung 4. | validation hierarchy. | Verification Ladder |
| Crux A · Crux C | A: the reading of each master is fair to his pages; adjudicated by paid outside historians on a frozen brief. C: the trace is legible to a historian who was not watching. | content validity; interpretability. | F-1.1, A-40 |
| second hand | A person outside the topic who re-derives an authored object (bands, geometry, the act itself) from the record without seeing the author's answer. Agreement is evidence the object is in the record; disagreement is a repair or a defect. | blind second coder. Departs: re-derivation, not agreement rating. | F-6.5, F-5.7a, F-3.11 |
| trace reader | A historian who reads a learner's commits before seeing any band or expected placement and says what the learner did. | blind rater. Departs: order of exposure is a rule. | F-6.1 |
| synthetic learner | The author's model of a learner, run against the artifact. May find mechanical defects; may never move an instrument, band, threshold or placement; never scored; stored apart. | simulated student. Departs: never evidence for a human-adjudicated entry. | S-0.12, S-63, amendment 2 |
| concurrent verbal report | A subsample performs the arc while thinking aloud without explanatory prompts; commits are coded blind by two coders into composition or a construct-irrelevant process. | think-aloud protocol (Ericsson & Simon). Same. | F-6.6 |
| guard | An entry that must pass on the same commits before another may fire. A synthetic guard half shows an occasion exists; it is never evidence. | precondition. | F-6.1, F-3.5, C-8 |
| ceiling | What a document or entry could see when it was written: how many readers, whether any learner, whether the thing was in hand. | limitations. Departs: stated per entry, before the result. | v0.1 preamble |
| residue | Reasoning in a work that the acts and delivered components do not capture. Declared, counted, never scored. | construct under-representation. Departs: counted per work to detect an absorbing boundary. | S-3, F-3.10 |
| repair ladder | Repairs run from least to most identity-bearing: instrument member, then band declaration, then act record. After any repair the blind entry on the repaired object re-runs; a repair the blind re-derivation does not reproduce is a fit. | no standard term; nearest: protection against overfitting. | C-7 |
| commit–reveal | The hash of a signed text is published on the day of signature; the text is published when the row's gate is met. | cryptographic commitment. Same. | this ledger |
Rendered from the repository once it exists; typed in this draft.