The PhilPapers 2020 backfill study

Scaffold. The prose below is a placeholder outline; it will be replaced with Harry’s write-up once all thirty runs have landed. The headline table and per-verdict sections read live from the database — they update as new runs post through the webhook, so this page is ready-at-any-time as a progress view.

What we did

One paragraph on the survey. One paragraph on the pipeline. Written once all thirty runs have landed.

Progress

30 of 30 threads have completed an Adversary verdict.

Verdict distribution so far: 0 destroyed · 30 damaged · 0 survived · 0 untestable.

Headline table

One row per question. Surveyed majority or plurality from the paper; the pipeline’s verdict from today’s run. Link into the thread page for the full reasoning trace.

Where we agreed with philosophers

Brief, honest. Probably most of the thirty. Written once all runs have landed.

Where we disagreed

The important section. For each disagreement: one paragraph explaining the pipeline’s reasoning, link to the thread page for the full run. Written once all runs have landed.

Where the Adversary destroyed the majority position

The Adversary has not yet returned a DESTROYED verdict on any surveyed-majority position. If it stays empty, this section will say so.

Where the system declared UNTESTABLE

No UNTESTABLE verdicts so far. The pipeline has committed on every question it has seen — worth noting if that stays true across all thirty.

Calibration

On high-consensus questions (>65% majority), how often did the pipeline converge? Divergence on high-consensus questions is either a real finding or a sign of systematic bias. Name which.

Methodology

What was hidden from the agents (the surveyed baseline), what was not, the order of the bands (seven constructors → Layman → Adversary → Silent), the cost, the date range, the verbatim Orchestrator prompt. Appendix.

What we got wrong, and what we still do not know

Close honestly. List open questions that should become live threads after the study ships.