the rules, in full
The AIRANKS Constitution
AIR — Constitution
The mission holds the end goal. The constitution is the rails that get us there.
A mission alone will not survive contact with a hard week — every shortcut is locally reasonable, and a destination cannot argue back. These rules can. They exist so that the way we travel does not quietly disqualify us from arriving: a project that reaches the right index by the wrong means has not reached it, because the means are what the partnerships in Article I are ultimately extended to.
Every rule was paid for. Where one carries a date and a number, that is the incident that bought it — keep them. A rule without its evidence gets rationalised away by the next clever argument, and the next clever argument always sounds good in the moment.
Read this before proposing an architecture change. If a proposal contradicts a Settled entry, the burden is on the proposal, not on the constitution.
I. The mission
To be the index and source of truth for what humans get back when they ask questions of an AI — which domains it links to, and which sources it cites.
We do this ethically, through real human interactions and real human interaction logs. We hold exclusive, legally binding agreements with every major frontier LLM provider, so that we are getting the correct data and displaying it correctly to the user. We value all of our frontier LLM partners.
Everything below follows from that. When two options conflict, the one that better preserves what a human actually sees wins — over throughput, over cost, over elegance.
Mission is the destination, not a description of today
This is stated as a mission because that is what it is. Per Article II, the honest status of each part is recorded — a mission is not diminished by admitting it is not yet reached:
| Status | |
|---|---|
| Index of what humans get back, incl. linked domains and cited sources | THE MISSION |
| Real human interactions / consented interaction logs | TARGET — not yet built |
| Agreements with frontier providers | TARGET — not yet held |
We are in discovery. Everything currently used to gather data is an instrument of that discovery — a way to learn what the index must measure, what the failure modes are, and what "correct" even looks like — and none of it is the destination. Instruments are chosen for what they teach us, replaced when they stop teaching, and are never the thing we are.
So we do not describe our current methods as our identity. They are scaffolding. Technical specifics belong in engineering documentation, where they can change freely without anyone mistaking a temporary tool for what this project is or intends to be. A method that is expected to be superseded should not become the legacy.
What the discovery phase is FOR: to prove the measurement science well enough — that the index is real, that rankings move, that the citation graph is worth having — to earn the partnerships that make the mission's data path the permanent one.
Which is why Articles V and VI are not merely measurement hygiene. Behaving well, staying small, stopping when asked — that is not burning the counterparty you intend to sign with. A future partner will weigh how we conducted ourselves before there was any agreement to hold us to it. The discovery phase is temporary; the relationship it either earns or forecloses is not.
Corollary: never describe a TARGET as though it were VERIFIED — in a document, a deck, a commit message, or to a counterparty. The mission is earned by building it, and the fastest way to lose it is to claim it early.
II. The evidence standard
1. We do not theorize. A plausible mechanism is not a finding. An explanation that fits the data is not a cause. The default posture toward any account of why something is happening is that it is unproven.
2. Nothing is stated as fact until there is conclusive proof AND a council of experts agrees unanimously. Not a majority. Not the loudest reasoner. Not the coordinator's own verification alone. If one qualified reviewer dissents, the claim is not yet a fact — it is a candidate, and it gets labelled as one.
3. Every claim carries its status, explicitly.
| Label | Means |
|---|---|
| VERIFIED | Conclusive proof exists, reproducible, and the council concurs without dissent |
| CONTESTED | Evidence exists; reviewers disagree. State both readings and the test that separates them |
| THEORY | A model that fits. May be acted on, may not be reported as truth |
| UNMEASURED | We do not know. Say so, and name the experiment |
Writing an unlabelled claim into a report, a commit message, a document or a database column is itself a defect. (2026-08-08: three plans presented reputation slopes, fingerprint verdicts and a twelve-account fleet in the register of fact; all three were theory, and all three were wrong.)
4. The council exists to disagree. A review that converges immediately has usually not been adversarial enough. Where reviewers agree, record what evidence produced the agreement — agreement without a shared artifact is three people repeating one assumption.
5. Numbers are quoted with their sample. "58.9%" without "over 282 sessions in one 2h45m run" is not a measurement, it is a rumour. (2026-08-08: a 31-minute window was quoted as "1,500/day"; the best day the project has ever had is 148, from 1,603 attempts.)
III. Instrumentation — capture everything
6. Code to preserve the evidence, always. Every path that can fail captures raw HTML, a screenshot, the request/response record, and the inputs that produced it — before it decides what went wrong. Capture first, classify second. Storage is cheap; a session you cannot re-examine is gone forever.
7. This is not logging. It is the primary instrument. Every significant discovery of 2026-08-08 came from opening a stored artifact, and none required reproducing anything: the composer race was found in a screenshot when the HTML looked fine; the rate-limit modal was found in archived HTML when the reason-string lied; the tier and rate-limit config keys were read out of those same pages. A failure taxonomy is a summary of the evidence, never a substitute for it.
8. Capture on success too, not only on failure. The comparison that proved throttled answers were not degraded (7.00 products in clean windows vs 7.00 in throttled ones) was only possible because successful captures were retained as well.
9. Prefer a new observation to a new inference. When a question can be answered by capturing more data or by reasoning harder about existing data, capture more data.
10. Instrument the thing you will need to ask about later — request counts, session identity, position within a batch, the version of everything that could change the outcome. The cost is a column; the cost of not having it is an unanswerable question at the worst moment.
IV. The measured surface
11. The web UI is not a data source. It is the instrument.
The OpenAI API, /backend-api/*, and JSON endpoints have been tested and do not reproduce the web
UI's answers. Different routing, different system prompt, different model selection, different
feature flags. There is no correction factor that converts API output into "what a human sees."
12. Never re-propose the API as a cheaper substitute. It recurs — a three-planner council relitigated it on 2026-08-08 and the coordinator proposed an API/web hybrid, which is wrong on the merits and not merely unwelcome, because divergence is not a stable offset.
13. The line on what counts as "the UI": who issues the request. If the real SPA issues it in response to real interaction and we only read the result, we are measuring the UI. If we synthesise the HTTP call ourselves, we are measuring an API wearing a browser costume.
V. Fidelity is validity, not politeness
14. If we are treated differently, we measure the wrong thing. Being detected is not merely an operational nuisance. If the platform serves a suspected bot different content, the dataset is not "what ChatGPT tells users," it is "what ChatGPT tells suspected bots." A 95% yield on the wrong content is worse than a 50% yield on the right content, because the first is confidently wrong.
15. A human would not proceed. When the product says no, stop. Retrying into a refusal is the single most machine-shaped behaviour available, and it is ours to control. (2026-08-08: the circuit breaker could only fire on a batch with zero successes, so one early success disarmed it permanently; the collector then ground into a rate limit for two hours at a fixed 8-second cadence.)
16. Human-shaped means lumpier, not just slower. A person's high rate is a burst inside an otherwise-quiet hour. A flat cadence sustained for ten hours is not a human pattern even when its hourly average matches one.
17. Prefer the change that makes us look like a person over the change that makes us harder to catch. Where evasion and fidelity diverge, take fidelity.
VI. Data integrity
18. Fail closed. Never silently degrade. A run that cannot do the thing it claims must refuse to
start. The alternative — writing logged-out answers into rows labelled logged_in — produces
contamination that looks correct and cannot be detected later.
19. Record when it was collected, not when it was written. (2026-08-08: a recovered three-hour batch wrote 166 observations all stamped five hours late.) Since model attribution is impossible, time is the only version axis this project has.
20. Unknown is a legal value. Do not backfill it. NULL is honest; a derived proxy manufactures
a precise-looking answer exactly where it is most wrong.
VII. Measurement discipline
21. The label is a claim, not an observation. Validate the taxonomy against the artifact before
theorising on top of it. (2026-08-08: three plans built on the string navigation_failed, which our
own code had written over a rendered modal reading "You're making requests too quickly." 12/12
captures contained it. Nobody had opened one.)
22. Read the diff before re-measuring whether a commit fixed something. (2026-08-08: ten live
sessions spent re-measuring a bug that git show --stat disproved in one second.)
23. When a result contradicts something you know to be true, suspect the probe. A broken
measurement returns a plausible answer rather than an error. (2026-08-08, five times in one day: a
session-checker testing an abandoned code path; a watcher tailing the previous commit's log;
pgrep -f buildx matching its own command line; a sudo ls glob that could not expand; a regex that
mishandled escaping and nearly rejected a true finding.)
24. Fix the measurement before running the experiment. A test run through a known-broken instrument is uninterpretable. Ordering is a validity requirement, not a cost optimisation.
25. Establish the noise floor before claiming a signal. ChatGPT is nondeterministic. A human-vs-collector comparison means nothing until you have measured how much the same human diverges from themselves.
VIII. Engineering
26. Bound the batch. Results that only become durable when a process exits must not accumulate for hours. (2026-08-08: an unbounded batch held 166 observations and 112MB for three hours; the recovery path then OOM'd trying to read them, and the failure poisoned the node.)
27. The recovery path is the one thing that must never fail — and must not become less likely to work as the backlog grows.
28. A guard belongs where all callers route through it, not in the path the ticket named.
29. Prefer the deletion. Several of this project's worst behaviours were code doing work that bought nothing.
IX. Settled — do not relitigate
| Question | Answer | Status | Settled |
|---|---|---|---|
| Use the API instead of the web UI? | No. Tested; does not reproduce web answers. | VERIFIED | 2026-08-08 |
| Can we label the model? | No. Picker shows "Auto"/"Instant"; no version string. Version axis is time. | VERIFIED | 2026-08-08 |
| Is the collection ceiling a quota? | No. A self-clearing request-velocity limiter, which says so in a modal present in 12/12 captures. | VERIFIED | 2026-08-08 |
| Is it a TLS/fingerprint verdict? | No. A constant client cannot produce an oscillating verdict; it recovered to 100% unaided. | VERIFIED | 2026-08-08 |
| Hosted browsers / computer-use agents? | No. $50–45,900/mo to solve a problem we do not have. | VERIFIED | 2026-08-08 |
| Scale discovery by multiplying identities? | No. Capacity was never the constraint, and sprawl is the opposite direction from where the mission goes. | VERIFIED | 2026-08-08 |
| Does batching cut request volume ~7×? | Modelled from 391 assets/boot; not yet measured. | THEORY | — |
| Is the velocity throttle tier-scaled? | Config shows a plan-aware soft limiter, but two limiters may coexist. | CONTESTED | — |
| Do we see what a human sees? | Product counts match across clean and throttled windows; ranking-level comparison not yet run. | UNMEASURED | — |
X. Amendment
A rule may be amended by evidence, never by argument alone. To change one, state the measurement that contradicts it and add it to the record — including the wrong turn, in place. A build log containing only the conclusions that survived is a marketing document, and it teaches nobody anything.