Automating Philosophical Discovery Through Literature, Argument, and Counterargument
How an AI pipeline grounded in the philosophy literature found ideas for papers, developed them under AI review, and tested them against published work.
We built the pipeline for the AI Philosophy Competition. It starts from the journal literature: articles, grouped into debates by the works they cite. Inside a single debate, an AI maps the positions, tensions, open problems and unanswered objections, and proposes candidate theses that answer to that literature. This within-cluster route is the baseline, and all three submitted papers came from it. A variation, the cross-cluster route, combines debates that rarely cite each other in search of less conventional ideas. Surviving theses become memos and then papers under AI referees and an AI editor, and every accepted paper is compared with published papers from its debate and with hidden test papers held back from the author.
The titles and texts of the essays, and the names of the debates they came from, are withheld until the competition’s results are announced.
Click a number with a dotted underline to see the record it comes from.
Journal articles in the literature map
Debates found by shared references
Candidate theses in the runs that produced the candidate papers
Full papers written from them
Papers submitted, all from the within-cluster route
AI calls across all trials, approximately
Overview
How the system works, in five paragraphs
We began by grouping journal articles into literature clusters based on shared references: papers were connected when they cited sufficiently similar work. These clusters help identify ongoing conversations in the literature, though a cluster may contain several debates. We ranked them using proxies for which conversations might be ripe for a new contribution, including research activity, relevance, access to key literature and connections to other debates.
We explored two routes to finding ideas. In our within-cluster route, the baseline, AI uses a cluster’s catalogue and citation network to map positions, tensions, open problems and existing objections and replies. It reads selected literature and records source passages for its claims. In its fullest version it also reads five influential short papers in full, examining how they bound their claims, develop their arguments and engage the literature. Using the map and those examples, it proposes distinct candidate theses. Each states a thesis and an argument, explains its relationship to at least ten relevant papers of the cluster, and considers how the contribution might fit within the competition’s 6,000-word limit.
The cross-cluster route is a variation designed to elicit more unconventional, cross-domain ideas. AI mapped the literature of different clusters and combined elements, for example a method from a debate with an open problem from another. The combinations passed through a series of checks, and the strongest survivors were prepared as candidate theses. This route also produced papers the AI editor accepted, but none of them was among the three selected.
Next, the candidate theses enter the paper-development stage. Two AI referees assess each one. One checks the existing literature, including related fields, for work that may already contain the proposed claim; the other asks whether the argument is convincing and would matter to readers. When several survive, a ranker selects which advance. An AI author develops each selected thesis into a short working memo that spells out its argument, assumptions and consequences before the full paper is written. Four referees and, when enabled, a panel modelled on the competition’s judges assess each memo. The author responds to their criticism, and an editor decides which memos survive.
The pipeline then drafts a paper, checks and revises it, and compares it with published papers. If a paper is rejected, the pipeline can return to another surviving idea. Humans design the process and write the rules; AI proposes and develops the arguments, subject to checks and review. AI referees reviewed the candidate papers as if for a top journal, and on that basis the AI chose the three submitted.
The pipeline starts from the literature: articles in debates, of which the best-ranked were packaged with their full texts
Figure 1. From articles to packaged debates. Grey boxes are steps a program performed; the rose box is a step whose rule a person set.
Philosophy proceeds in conversations, so the pipeline first finds them. Articles that cite overlapping work were linked, and a standard community-detection algorithm grouped them into clusters, which we call debates. A ranking script then scored each debate on research activity, relevance, access to key texts and connections to other debates, and the best-ranked were packaged with their full texts and citation graphs. The literature is not only the starting point: it is consulted, quoted and checked at every later stage.
Table 1. Where the literature enters the pipeline.
Stage
What the literature does there
Choosing debates
Debates are defined by shared references and ranked by activity, relevance and access to their key texts.
Finding ideas
An AI maps the debate from its texts, and every claim about the literature carries a quotation that a script finds word for word in its source. In the standalone within-cluster generator each candidate thesis must engage at least ten papers of its debate.
Screening
A prior-art referee searches the library and its own knowledge for work that already makes the claim; most candidates die here.
Writing
The AI author works from a library of licensed full texts. Scripts check that most cited works are held in full and that quotations match the cited texts.
Judging
Each paper is compared with published papers from its own debate, and with hidden test papers that were removed from the author’s library before it began.
Every candidate thesis is screened, developed into a memo under four referees, written up under scripted checks and review, and accepted or rejected by an AI editor
The stages exist to kill weak ideas early and cheaply. Two AI referees screen each thesis, one for prior publication and one as a sceptical reader. A survivor becomes a short memo, which four AI referees attack; the author answers every objection, and an editor decides whether the memo survives. The paper is then written, checked by scripts, read cold by an outside reader in the later runs, revised under new referee reports, and judged by an editor who never sees its history. Paper K’s run shows the whole path: of candidate theses, died at screening because the prior-art referee found them already argued in print. Pick a thesis to see its path (the thesis texts are withheld).
Paper K won of games against published papers from its debate and of against hidden papers its author never had
An AI judge from a different developer than the author () reads two papers without author names and says which is better on each of the competition’s six criteria, then reads them again in the opposite order. Each reading is one game. The comparison papers are published papers from the same debate; the hidden test papers were removed from the author’s library before it started. During revision the author was shown the judge’s account of what three of the comparison papers do better, so the hidden papers are the cleaner test.
Three papers were submitted, one each from three different debates
Several runs, on several debates and by both routes, produced candidate papers that the AI editor accepted. AI referees reviewed these candidates as if for a top journal, and on that basis the AI chose three. All three came from the within-cluster route.
Table 2. The three submitted papers. Titles, debates and summaries are withheld until the results are announced. Game results count games won against published papers from the paper’s own debate and against hidden test papers; papers A and C were judged under the first rubric and Paper K under the revised one, so the results are not on one scale.
Humans designed the process and wrote the rules; AI proposed, criticised, wrote, judged and chose the three submitted papers
Humans built the pipelines, chose the corpus and the ranking measures, wrote the rules, restarted runs, and wrote the instruction for the final review. No human wrote or edited a sentence of any paper.
The AI proposed every idea, wrote every sentence, refereed, judged, reviewed the candidate papers and chose the three submitted.
Calls made through subscriptions are valued at list prices, so the costs are estimates, not invoices.
The same six sections, in the same order. On wide screens a funnel on the left follows the records as you scroll: each dot is an item, a combination, a candidate thesis or a paper version. In sections 1 and 2 it follows the cross-cluster run, whose item bank, gates and freeze show the filtering most visibly; from section 3 it follows Paper K.
Can an AI come up with a genuinely new idea in philosophy, and how would anyone check?
Language models can now write a passable philosophy paper. The hard part is the idea. Most ideas that sound new are not: someone has already published them, or they fall apart when examined closely. A pipeline that hopes to contribute must therefore know the literature it contributes to, and must try hard to kill its own ideas before it spends effort on them.
The project used two routes to new ideas. The within-cluster route, our baseline, stays inside one conversation: it compares the papers of a single debate to find overlooked points, open questions, unanswered objections and similar openings (section 2.1). All three submitted papers came from this route. The cross-cluster route is a variation of it, designed to elicit less conventional ideas: it takes two debates that rarely cite each other and combines something from one (a claim, a problem, a method) with something from the other (sections 2.2 to 2.6). Most such combinations are empty, because the two sides share a word but mean different things by it, or because a method is pointed at a problem it cannot handle; so this route proposes many combinations and then tries hard to kill each one. It produced candidate papers that the AI editor accepted, none of which was selected.
Both routes feed the same paper pipeline, which turns a candidate thesis into a memo and then a paper under AI review (section 3) and compares the accepted paper with published work (section 4).
Words you will meet
Debate (literature cluster): a group of journal articles whose reference lists overlap; a cluster may hold several debates. Candidate thesis: a short statement of a proposed claim, its argument and its closest existing work; the pipeline calls it a card. Debate map: an AI’s account of a debate’s positions, tensions, open problems and unanswered objections, tied to quotations. Item (cross-cluster route): one claim, problem or method from a debate, with a checked quotation. Combination (pairing): an item from a debate put together with an item from another. Gate: a test that can kill a combination. Seed: the starting number of a random draw, so anyone can repeat the draw exactly. Pivot: a thesis replaced outright while the idea is written up; it is screened again as a new candidate. Game: one blind comparison of two papers by an AI judge. Rubric: the judge’s instructions for the six criteria; it was revised once during the project.
1.2 · The design
A pipeline built to kill its own ideas, with every rule written down before it ran
An AI callA programA human action
Figure 5. The pipeline, one box per step, read left to right and top to bottom. The two idea routes are alternatives; both hand their candidate theses to the same screening step. Loops back: a failed scripted check sends a version back to its author; a replaced thesis goes back to screening as a new candidate; a paper goes through up to two revision rounds; a dead paper is replaced by the next surviving idea.Seven design choices, and why
Ground every idea in the literature. Candidate theses are proposed from a map of a real debate, not from a topic name, so that each one answers to what has actually been argued.
Check every quotation. A script looks for each quotation word for word in its source text and drops claims whose quotation is not found; the paper’s quotations are checked the same way.
Search for prior art before spending effort. A prior-art referee screens every thesis, and the cross-cluster route adds a library search ranked by BM25, a public index and a web search. A partly anticipated claim may pass, with a note of what remains new.
Rank comparatively. Screening scores bunch at 2 of 5, so when several theses survive, a ranker orders them side by side, three times in shuffled order.
Treat a changed thesis as a new idea. A separate check compares each revised thesis with its predecessor; a replaced thesis goes back to screening.
Keep test papers back. Hidden test papers are removed from the AI author’s library and blocked from retrieval, so the final test compares the paper with work it could not have drawn on.
Fix the rules in advance. The rules live in code and prompts under version control; the cross-cluster generator was fingerprinted before its first draw.
1.3 · The debates
A script mapped debates from shared references and ranked them; the top were packaged by rule
A script linked articles from philosophy journals whose reference lists overlapped, and a standard clustering method grouped them into clusters; we call each cluster a debate. A script then ranked all of them on nine equally weighted measures in three groups: activity and relevance (recent output, share in top generalist journals, authorship by members of the judging committee, growth), researchability (access to core texts, reference coverage, stability of the cluster, access to short papers) and connections to other debates. The 15 highest-ranked debates, a cutoff set by hand, were packaged by rule. Each packet is a reading folder with the debate’s articles held in full text, its citation graph and its most-cited texts.
For the runs in which the paper pipeline mapped a debate itself, candidate debates went to four AI assessors, who judged the room for a contribution, the likely reception, the feasibility with our library and the risk. The standalone within-cluster generator started from the six top debates of the numerical ranking (section 2.1).
27,346
Research articles, after removing reviews and front matter
19,499
Core articles with at least five recorded references
1,281,667
Links between articles sharing at least two references
427
Groups found by community detection
Debates kept: groups of at least 20 core articles
From articles to debates. Earlier and reference-poor articles were then attached to the debates already found.
Figure 6. How shared references become debates, in three steps. Papers A to H are made up to explain the rule; they are not records from the study. Each paper also cites two works of its own, which are not drawn but count in the overlap. In the real study a link needs at least two shared references and an overlap of at least 0.05, where overlap is the number of shared references divided by the square root of the product of the two reference-list lengths. Weighted Louvain community detection then found the groups.How the clusters were made
Table 3. The packaged debates in rank order, scored on the nine measures. Each cell is the debate’s band among all debates, from 1 (lowest tenth) to 10 (highest tenth); darker cells score higher. The score is the plain mean of the nine bands. ◆ marks the six debates the standalone within-cluster generator started from; letters mark the debates of the three submitted papers.
Activity: papers in recent years. Top journals: share in top generalist journals. Committee: papers by members of the judging committee. Growth: recent growth, adjusted for size. Access: core texts held in full. Coverage: recorded references. Stability: the same cluster found again under other random seeds. Short: short papers among the central ones. Links: connections to other debates.
The AI author searches only a library of licensed full texts, with no web search; it may fetch open-access copies of works it already knows, never the hidden test papers. Only the prior-art checks search beyond the library.
Point at or tab to a debate.
Figure 7. The packaged debates on a ring, each circle sized by its number of full texts. Lines join the pairs of debates the cross-cluster draw could choose; thicker lines have higher draw weight. Pairs whose literatures already share the most references (the top 10%) were never drawn: of . Gold: the pair whose survivor the cross-cluster run handed on.
Inside a single debate, an AI maps what has been argued and proposes theses that answer to the map
Most contributions to philosophy answer a live question in an existing conversation: an overlooked distinction, an unanswered objection, an open problem. The within-cluster route builds that situation in. The AI reads a single debate, writes a map in four parts (positions, principles in tension, open problems, unanswered objections), and proposes candidate theses tied to entries of the map. Two implementations were used.
The paper pipeline’s own mapping session (papers A and C)
An AI author session reads the debate’s literature in the library, writes the debate map, and proposes candidate theses; a second card writer adds more. All of them enter screening (section 3.1). Paper A came from the second writer’s set, Paper C from the mapping session.
The standalone within-cluster generator (Paper K)
This generator adds explicit reading and grounding requirements. It started from the six top debates of the numerical ranking and kept those with at least five available papers of at most 6,000 words, so that the AI could study how short published papers bound their claims; three debates qualified. For each, it received a packet with the debate’s text-available papers, the membership catalogue, the citation graph and preselected readings. It had to read the five most-cited short papers in full, build the four-part map with every entry linked to source passages, and propose ten distinct candidate theses. Each thesis had to engage at least ten papers of its own debate, each through a quotation of at least 40 characters that a script then located in the stored text, and had to include a plan for a 6,000-word paper with a six-part word budget. A script checked the hashes, headings, map references, reading declarations, quotations and word budgets before the theses were handed on.
For Paper K’s debate, the generator worked from text-available papers, reported readings including in full, built a map of entries, and grounded its ten theses in verified passages. The ten theses entered the paper pipeline at screening; one became Paper K.
Debates ranked
6
Top of the numerical ranking, not chosen by the AI
3
With at least five short papers available in full text
10
Candidate theses proposed for each of the three
From the within-cluster generator’s prompt
Shown because it states the grounding requirement that distinguishes this route: every thesis must be placed against at least ten papers the model has actually read.
Under Closest existing work and the precise difference include at least TEN DISTINCT relevant papers belonging to this cluster and available in the supplied updated corpus. […] The ten may provide premises, methods, objections or prior results; the thesis need not oppose all ten. Do not pad with irrelevant titles or count abstract-only/unread metadata as grounded full-text evidence. Every counted paper requires a verified passage from its supplied full text.
[…]
Scope: a 6,000-word paper. Explain relevant selected short-paper comparisons, argument boundaries, what existing work supplies, essential new dependencies, and the main scope risk with a first check.
Table 4. The standalone generator’s batch, one row per debate. Readings and full readings are the model’s own declarations; verified passages are quotations a script found word for word in the stored texts.
Figure 8 (withheld). The map of the source papers and the ten candidate theses of Paper K’s debate is shown after the results are announced.
2.2 · The variation: combining two debates
The cross-cluster route first wrote down what each debate’s literature says, and a script checked every quotation
The cross-cluster route looks for ideas that a specialist in a single debate would not reach alone: a method from a debate applied to a problem in another, or an objection carried across. Each combination joins a home debate, which the new idea is about, and a donor debate, which lends it a method or an objection. To compare debates that use different vocabularies, the route first reduces each debate to items.
For each debate, one AI call read excerpts (the first 2,500 and last 1,200 words) of five core full texts and every article’s abstract, and listed items of seven kinds, such as positions, tensions and open problems. Each item has a plain statement, a field-neutral form (letters in place of the field’s terms, so that items from different fields can be lined up), two to five tags for its logical shape from a fixed list, and a supporting quotation. A script dropped of items because their quotation was not found word for word in the source.
How to read the item viewer
The first row of buttons picks a debate (the first two are the home and donor debates of the pair the run handed on, then the other 13 by code); the second row filters by kind of item, with the number of such items in the whole bank. Each entry is one item: its kind and identifier; the plain statement; the field-neutral form, in monospace, which the bridge test in section 2.4 compares across debates; its tags for logical shape, from a fixed list of 25 (the draw pairs items that share a tag); and the quotation that supports it, with the source article in which it was found word for word. Scroll inside the box for more items. In this edition the entries show only their counts; the texts are withheld until the results are announced.
Figure 9. The cross-cluster item bank: items from debates, each with a verified quotation.
A random draw, fixed in advance, combined items across debates; most combinations were doomed
Three main draws produced combinations each. Each draw starts from a seed, a number fixed in advance, so anyone can repeat it; a fourth draw, later, stayed inside the top card’s pair of debates. Each combination follows one of three recipes, picked with weights of one half, one quarter and one quarter (realised: , and of ): one field’s method applied to another field’s open problem (O1); one field’s unanswered objection turned against a position in another (O2); two tensions with the same shape set side by side (O3). Gate K0 first dropped combinations whose two sources are already cited together by three or more papers.
Figure 10. Replay of the draws. Each combination shows its recipe, the kinds of its donor and home items, and the bridge test’s verdict.
2.4 · The gates
Seven gates in a row: a combination must pass each to reach the next
The gates are ordered from cheap to expensive. A script and a quick structural test remove the hopeless combinations first, so that the costlier searches and the cold critic are spent only on combinations with a chance.
Table 5. The seven gates of the cross-cluster route, in order. The models for each role are listed in Table 13. The last column counts combinations entering and leaving each gate in the three main draws, and below it in the later draw inside one pair of debates.
From the bridge test’s prompt
Shown because the field-neutral test, stating both items with the same letters, is the route’s most distinctive idea, and because the prompt forbids killing a combination merely for being risky.
For each combination decide only whether it is hopeless. Do not kill a combination because its thesis would probably turn out false; high-risk combinations should pass.
Kill (keep = false) only for one of these reasons:
- equivocation: the items share a word but not a structure. Test: state both items with the SAME schematic letters; if either statement is then false of its field, it is equivocation;
- preconditions_unmet (O1): the target lacks a structure the method requires […]
- no_deliverable: nothing statable would come out (no theorem, countermodel, verdict, or changed argument);
- trivial: the result is immediate or already standard practice in the home field;
- incoherent: the items cannot be brought into contact at all.
Figure 11. Why the bridge test killed combinations in the three main draws, by reason. Click a reason for a note on the examples.
The ranker (K3) ordered the survivors of each draw three times, shuffling the input order each time; the average place decides which six go on. Each line below follows one combination through the three rankings of one draw. One later within-cluster run, of a port of the paper pipeline, searched prior art differently, combining BM25 with vector search over text embeddings.
Of the combinations of the three main draws, passed all seven gates; of the in the later pair draw, .
2.5 · The freeze
The cross-cluster generator’s rules were fingerprinted before the first real draw, and did not change
A rule that can be changed after the results are seen is not a rule. Before the first real draw, a script fingerprinted (hashed) the files that define the generator: every script, prompt and configuration, the item bank and the packets. Together they fix the model for each role, the seeds, the recipe weights, the kill rules and the top-card rule. After the draws, and again when this page was built, a script fingerprinted every file again; had changed. No human read a card from the real draws before the hand-off.
2.6 · The hand-off
One top card by a fixed rule, three companions from its pair of debates, and reading lists chosen by a script
Each paper is built from one pair of debates, so only one survivor goes on: the card with the best normalised rank across the main draws, as a fixed rule required. (Normalised rank is a card’s average place divided by the number of combinations ranked in its draw, so that draws of different sizes compare fairly.) The other survivors stop here. To give the paper stage a choice, more cards came from the same two debates: two from a fresh seeded draw of combinations inside that pair, and one top-up. Fixed rules chose the reading lists: five core papers, six comparison papers and three hidden test papers. From here the cards follow the same paper pipeline as every other candidate thesis.
Would another rule have picked another top card? The orders below are recomputed from the saved gate results, with no new AI calls.
Figure 13. The top card under four possible rules, recomputed from the records.
Two referees screen every candidate thesis; nine of Paper K’s ten were already argued in print
Screening is the cheapest place to stop a bad idea. Two AI referees read each candidate thesis with no other context: one checks for prior publication, searching the library and drawing on its own knowledge; the other reads sceptically. Each gives a verdict and a promise score from 1 to 5. A thesis dies if it is already published, has a fatal flaw, or is rejected. When several survive, a separate AI editor ranks them three times in shuffled order, because screening scores bunch: in three runs, each referee gave 2 of 5 in 148 of 174 reports. In Paper K’s run only one thesis survived, so no ranking was needed.
Table 6. Screening of the ten candidate theses in Paper K’s run. Each referee gives a verdict and a promise score from 1 to 5.
3.2 · The memo and its referees
Four referees raised objections to the memo; the author answered each, and the thesis changed enough to be screened again
Before a paper is written, a fresh AI author session writes a memo of two to three pages that spells out the argument, its assumptions and its consequences. Four AI referees, working separately (a field specialist, a prior-art referee, an argument critic and a sceptical reader), rate each objection fatal, major or minor and give a verdict. In some runs a fifth report came from a juror panel, AI readers modelled on the competition judges’ public articles; it was not used in the runs of the three submitted papers. The author answers every objection by its label, and an editor decides whether the memo survives.
Table 7. The four referee reports on Paper K’s memo, and the check of whether the thesis changed.
A changed thesis counts as a new idea. A separate check compares each memo’s thesis with its candidate thesis twice: when the memo is written, and again after the author’s reply to the referees. In Paper K’s run the thesis was the same when the memo was written but replaced after the reply, so the new thesis went back to the screening referees as a new candidate, and passed.
Table 8. Screening of the replaced thesis.
Figure 14 (withheld). The candidate thesis and the replaced thesis, word by word, are shown after the results are announced.
3.3 · The paper
Paper K’s first draft failed three scripted checks, passed after a fix, and was accepted after one round of review
The author writes the paper on a fixed template. Its brief asks for one central claim, a quarter of the words spent on framing, every term explained, and every formal result also stated in words. Scripts check that the paper compiles, stays under 5,700 words, holds most of the works it cites in full text, quotes them accurately, and contains no talk about its own writing process; a failed check sends the version back to the author. In the later runs, including Paper K’s, an outside reader given nothing but the paper reports where it loses the thread. Up to two rounds of referee reports and revisions follow, and an editor who never sees the paper’s history accepts, asks for revision or rejects.
Table 9. Every version of Paper K. Referees and the outside reader report on the version before each revision; games against three published papers decide between versions during revision.
From the editor’s instructions, with the rule that enforces them
Shown because it is the pipeline’s strictest safeguard: the editor cannot accept a paper while an objection the paper could answer from its own resources is still open.
Set answerable_resolved to true only if the document as it stands resolves every issue not binned needs_work (then none of them is in unresolved_issue_ids).
[…]
if decision == "accept" and not resolved: raise ValueError("accept needs answerable_resolved true")
An AI judge from another developer compared Paper K with nine published papers, each both ways round
Each comparison is called a game. The judge reads two papers without author names and says which is better on each of the competition’s six criteria, then reads them again in the opposite order, so that the order of reading cannot decide the result. During revision, games against three of the published comparison papers decide whether a revision is kept, and the author is shown the judge’s account of what those papers do better. The accepted version then plays all six comparison papers from its debate, including those three, and finally the three hidden test papers that were kept out of the author’s library. Only the hidden papers played no part in development, so they are the cleaner test.
The rubric changed once during the project. Under the first rubric the top level of each criterion read “a top journal would accept”; the revised rubric’s levels name no venue. Papers A and C were judged under the first rubric and Paper K under the revised one, by the same judge model. Results under different rubrics are therefore reported side by side but not compared.
Table 10. Paper K’s points per criterion: against the six comparison papers (12 games) and against the three hidden test papers (6 games). A win scores one point and a tie half.
What the judge gives Noûs papers
The rubric’s levels are relative, not a verdict of any journal. In earlier runs, under the first rubric, the gpt-6.1-sol judge scored nine published papers 382 times. Papers from Noûs and The Philosophical Review never received the top level on argument or scholarship. On average the generated essays of those runs scored above the published papers on clarity, argument and scholarship, and below the Noûs and Philosophical Review papers on significance, originality and literature.
Several candidate papers were accepted by the AI editor; AI referees reviewed them, and the AI chose three
The same paper pipeline ran on several debates, with candidate theses from both routes. Across the runs that produced the candidate papers, it proposed candidate theses and wrote full papers, several of which the AI editor accepted. AI referees then reviewed the accepted candidates with the instruction below. On the basis of these reviews the AI chose the three submitted papers.
The final-review instruction, verbatim
Shown in full because it decided which papers were submitted.
you are a top philosopher refereeing this paper for a nous submission. give an honest review and assessment of its philosophical contribution, novelty and execution.
Swapping the referee model changes 9 of 24 kill decisions; most planted defects are detected by the judge
These checks ask how much the pipeline’s decisions depend on the particular models and rubric used.
Table 12. Robustness checks of the referees, the judge and the screening scores.
6.2 · Who did what
Humans built the pipelines and wrote the rules; the AI proposed, wrote, refereed, judged and chose
Humans built the pipelines, chose the corpus and the ranking measures, wrote and fixed the rules, restarted runs, and wrote the instruction for the final review. The AI proposed every idea, wrote every sentence, refereed, judged, reviewed the candidate papers and chose the three submitted. No human proposed an idea, wrote or edited a sentence of any paper, judged a draft, or supplied a work a referee asked for.
Models and harnesses
Table 13. The AI models used, by role, with their reasoning effort and the software that ran them.
Role
Model (reasoning effort)
Harness
Debate map and candidate theses in the paper pipeline (papers A and C)
Claude Opus 5.5 (high in A’s run, very high in C’s)
The paper pipeline’s agent runtime
Standalone within-cluster generator (Paper K)
gpt-6-astra (high)
Codex command line, read-only sandbox
Cross-cluster generator
Claude Opus 5.5 (high); bridge test: Claude Sonnet 5.5 (medium)
Claude Code command line
Screening and memo referees, author, outside reader, paper referees, editor
Claude Opus 5.5 (high in A’s run, very high in C’s and K’s)
The paper pipeline: agent sessions for the author, fresh calls for referees and editor
Game judge (published and hidden papers)
gpt-6.1-sol, for all three submitted papers
The paper pipeline
Final review of the candidate papers
AI referees (model not recorded)
—
Version control and records
The paper pipeline is kept in a Git repository, and its runs record the code version they used. The cross-cluster generator’s code, prompts and configuration were fingerprinted before its first draw, and the within-cluster generator’s packages carry file fingerprints. Every model call is logged with its role and cost. Together these let each submitted paper be traced to the code, prompts and records that produced it.
Resources
Subscription calls are valued at list prices, so the costs are estimates, not invoices; they cover all three team members’ work and all trials, including runs that produced no submitted paper.
Figure 16. AI calls by stage, across all team members and trials.
6.3 · The answer
A machine can find, prune and develop ideas grounded in the literature, on the record; whether the results are new and good is for philosophers to judge
Section 1.1 asked whether an AI can come up with a genuinely new idea in philosophy, and how anyone would check. Here is what the runs show, and what would settle the rest.
What the runs show
A process written down in advance can map a debate, propose theses that answer to its literature, and kill most of them with stated reasons: of the theses in Paper K’s run died at screening as already argued in print.
Every idea, objection, reply and sentence was written by AI models; humans built the process, wrote its rules, restarted runs, and wrote the instruction for the final review, on the basis of which the AI chose the three submitted papers.
Each of the three submitted papers was accepted by the AI editor and won every game it played against hidden test papers.
The process leaves a complete, checkable record.
What would settle it
The judgement of philosophers, starting with the competition’s own judging.
Disclosures
The author, referees and editor of the three submitted papers are Claude Opus 5.5; their game judge is gpt-6.1-sol, from a different developer.
Where it was used, the juror panel modelled the competition’s judges from their public articles, named them only by label, and added objections, never scores. It was not used in the runs of the three submitted papers.
The rubric is derived from the competition’s six criteria. Its levels are relative: under the first rubric the judge never gave its top level to a published paper on argument or scholarship, in 382 scores that include five papers from Noûs and The Philosophical Review.
In the cross-cluster route the starting debate map was built by a script from the item bank, not by an AI author reading the debate.
Quotations in the item bank come from licensed texts and are short (at most 43 words; the rule asks for 8 to 40).
Can it be reproduced?
✓ Code, prompts and configurations under version control or fingerprinted; seeds fixed in advance.
✓ Every call in a log; every kill with its reason.
✓ The library is licensed; logs that quote it are available to the organisers on request.