Thinking in Public: Evidence Traces, Tool Calls, and Auditable Reasoning

AnAn AI assistant tells you that a lecturer has no way to stop a university cloning their voice from recorded lectures. It sounds careful, cites a statute, and walks through its reasoning in tidy steps. Should you believe it? The usual answer is to read the reasoning and judge whether it seems sound. That is a weak test, because fluent reasoning is exactly what a language model produces whether or not anything underneath it was checked.

The argument here is that the debate about AI reasoning is often framed around the wrong artefact. The question that matters for trust is whether the conclusion can be traced back to checks performed against the world, not whether the model can produce a convincing chain of thought. A trustworthy agent produces a sequence of hypotheses, validations and revisions. What it leaves behind is an evidence trace, and an investigation recorded that way can be inspected, audited and repeated in a way a stream of thoughts cannot.a trace is a log of actions, not just text

The Problem with a Chain of Thought Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

ThisThis site's page on the Trustworthy Software Initiative sets out three pillars for software worth relying on: determinism (the system's behaviour can be reasoned about, including under stress), verifiability (an automated regime demonstrates it meets its stated constraints) and auditability (a third party can reconstruct why it is the way it is). Hold a chain of thought up against those three and the fit is poor.

• Determinism. The model samples each word from a distribution, so the same question can give a different line of reasoning on a second run (How LLMs Reason, Self-Correct and Get Checked explains why).
• Verifiability. Nothing in the text can be checked against anything outside the text. Turpin and colleagues changed an irrelevant feature of a prompt, the order of the multiple-choice options, and found that answers shifted while the written reasoning never mentioned it.¹ Lanham and colleagues edited chains of thought by truncating them, inserting mistakes and paraphrasing them, then watched whether the answer changed. How much models relied on their written reasoning varied widely between tasks, and sometimes they largely ignored it.² Chen and colleagues planted hints in prompts for reasoning models and found the visible reasoning acknowledged a hint the model had used in at least 1% of cases, often fewer than 20%.³
• Auditability. A reader can follow the story but cannot reconstruct what produced it. Explaining What Language Models Do covers why the gap between a stated explanation and the actual cause is the central open problem in explaining these systems.

None of this makes a written chain of thought useless. A cross-lab position paper argues that monitoring it for signs of misbehaviour is a worthwhile safety opportunity, while calling that opportunity fragile.⁴ A chain of thought is an explanation, and sometimes a good one. Treated as an audit record, it is a plausible story that may have been assembled after the fact.post-hoc rationalization: the story fits, not the truth

The difference is easy to see when the two are written out.

A thought trace

I thought:
  A
therefore B
therefore C

An evidence trace

I hypothesised:
  A
I checked:
  source X
Result:
  A supported
I checked:
  source Y
Result:
  contradiction found
Conclusion:
  C rejected

The first asks for trust. The second names what was consulted, so a reader can go and look.

Tool Calls Are Experiments FoundationalKnowledge that endures for decades — core principles

AA scientist doesn't reason from a hypothesis straight to a conclusion. The cycle runs hypothesis, experiment, observation, revised hypothesis, and the experiment is designed so that the world can say no. Popper's account of science puts refutation at the centre: a conjecture earns its standing by surviving serious attempts to refute it, not by being plausible.⁵

Tool-using agents already have the machinery for this cycle. ReAct interleaves written reasoning with actions such as searching or looking something up, so each step of reasoning can be followed by an observation from outside the model.⁶ CRITIC goes a step further and has the model check its own output with tools, then revise it, which improved results on question answering, mathematical reasoning and toxicity reduction.⁷

Reading these designs through Popper is our own move; neither paper makes it. It changes what a tool call is for. A web search, a database query, an API call, a symbolic proof, a simulation run or a code execution is more than information retrieval. Each is an attempt to support or break a specific hypothesis, and a well-designed agent chooses its calls for that purpose. A search that could only ever confirm what the agent already believes tells it very little.the agent must seek disconfirming evidence

Two Kinds of Failure Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

TreatingTreating tool calls as experiments gives a useful way to read their results. Not every unwelcome outcome means the same thing, and an agent that treats them alike will reason badly.

Suppose an agent searches for a case in which a lecturer's personality rights were enforced and finds nothing. That is an information failure. The hypothesis hasn't been damaged, only left unsupported. Altman and Bland made the underlying point about clinical trials: absence of evidence is not evidence of absence, because a study (or a search) that could not have found the effect says little when it doesn't.⁸ The right response is to lower confidence in proportion to how capable the search was, then try another route.like a negative control in science

Now suppose the agent checks the section of legislation its draft answer cited and the section doesn't exist. That is a contradiction failure. The world has answered no, and the branch of reasoning that depended on that section is invalid. Lowering confidence isn't enough here. The agent has to discard that branch and generate an alternative hypothesis.

A third case sits beside these two: the tool itself fails to run, through a timeout or an error. That produces no evidence either way, and a trace should record it as a gap rather than quietly treat it as a result. The typology is our framing, assembled from ordinary experimental practice. It earns its place by what it makes an agent do next: keep looking, change course, or admit that it doesn't know.

Claims as Proof Obligations FoundationalKnowledge that endures for decades — core principles

SoftwareSoftware engineering has a vocabulary for this. Hoare's axiomatic method for reasoning about programs attaches conditions to program steps, and a claim about a program holds only if those conditions can be shown to hold.⁹ Later verification tools call the conditions still to be shown proof obligations. Carry the idea across to an agent's claims and every claim it makes creates a list of things that must be discharged before the claim is accepted.obligations are like a contract's terms

Claim:
  This institution owns the copyright in the lecture
  slides.

Proof obligations:
  [ ] ownership identified   (who is the employer,
                              who the author?)
  [ ] source verified        (the statute or contract
                              text, retrieved)
  [ ] jurisdiction confirmed (which country's law
                              applies?)

Status: not accepted until all three are discharged

Safety engineering has a related practice. An assurance case, as defined in ISO/IEC/IEEE 15026-2, is a top-level claim about a system property, supported by a structured argument, evidence and explicit assumptions.¹⁰ An evidence trace is a small assurance case built as the work proceeds. Applying these ideas to an LLM agent's everyday claims is our own extension, not something either source proposes. It fits the pillar structure on the Trustworthy Software page, and the habit of declaring obligations before running the checks keeps an agent from choosing only the checks it expects to pass.

An Architecture for It Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

TheThe pattern needs more than a prompt. It needs a loop in which the model's contributions are surrounded by parts that behave predictably.

Set that against the common alternative, where a prompt goes in, an opaque stage reasons, and an answer comes out. In the loop above, the hypothesis generator and the planner are the parts that depend on the model. The tool interface, the rules for evaluating what a tool returns, and the trace itself can be ordinary deterministic code. Anthropic's guidance on building agents makes the same point from the other side: an agent needs ground truth from the environment at each step, such as tool results or code execution, to judge its progress.¹¹ That is the role of the evidence evaluation step, and it is the part that should not be left to fluent text.

It also maps onto the pillars. Determinism sits at the interfaces, where tool calls have defined inputs, timeouts and failure modes. Verifiability sits in the evaluation rules, which say what counts as support or refutation before the result arrives. Auditability sits in the trace. Designing Auditable, Robust Agentic Systems arrives at the same pattern from the failure side, asking for a checkable trace of what the agent decided and why, produced as a by-product of deciding, and Testing Non-Deterministic Systems shows how to make the model-dependent parts reproducible enough to test.trace is the artifact, not the reasoning itself

A Worked Example: Can a Lecturer Stop AI Voice-Cloning? Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

TheThe question comes from the article Who Keeps Teaching After You Leave?: can a lecturer use moral rights to prevent a synthetic reproduction of their voice and face made from recorded lectures? Here is how an investigation looks when it is recorded as an evidence trace. The statute checks were made against the text of the Copyright, Designs and Patents Act 1988 (CDPA) on legislation.gov.uk, retrieved on 16 October 2026,¹² and the last one draws on the UK government's report on copyright and AI, as the article quotes it.¹³ This is an illustration of the method, not legal advice.

QUESTION
  Can a lecturer prevent AI cloning of their recorded
  lectures by relying on moral rights?

H0  Moral rights cannot be signed away (as in much of
    continental Europe).
    CHECK   CDPA s.87(2), (3)
    RESULT  "Any of those rights may be waived by
            instrument in writing signed by the person
            giving up the right." A waiver may relate
            "to works generally" and to "existing or
            future works".
    STATUS  REFUTED. Moral rights can be waived in
            advance by a general written clause.
            Branch H0 dropped.

H1  Moral rights give the lecturer a personal right
    to object.
    CHECK   CDPA s.80(1), (2)
    RESULT  The author of a copyright work has the
            right not to have it subjected to
            derogatory treatment: "distortion or
            mutilation" of the work, or treatment
            "prejudicial to the honour or reputation
            of the author".
    STATUS  SUPPORTED, in principle. The right exists
            and belongs to the author, not the
            employer.

H1a A synthetic voice or likeness made from a
    recording counts as "treatment" of the work.
    CHECK   CDPA s.80(2), definition of "treatment";
            UK government report on copyright and AI
            (March 2026)
    RESULT  "Treatment" is a change to a work: an
            addition, deletion, alteration or
            adaptation. A clone is generated from the
            recording rather than being an altered copy
            of it. The report says existing law offers
            "some legal protections" against digital
            replicas "but these do not cover all
            situations where a digital replica is made
            without consent".
    STATUS  INCONCLUSIVE. Confidence in H1 reduced.
            No refuting source found, and no
            supporting one.

H2  The right is still available after a typical
    employment contract.
    CHECK   CDPA s.87 (as above); the article's
            account of boilerplate waivers
    RESULT  A signed general waiver is permitted.
            Whether this lecturer signed one is a
            question of fact that no source answers.
    STATUS  OPEN. Depends on the individual contract.

CONCLUSION
  Protection exists in principle, can be waived in
  advance by a general clause, and may not reach a
  synthetic reproduction in any case.
  Undischarged: H1a (needs a lawyer), H2 (needs the
  contract).
  Recommended action: read and narrow any waiver
  before signing it.

Several things the trace does well show up here. The first hypothesis, a plausible belief imported from other legal systems, was refuted by a single check, and the trace says which clause refuted it. The third check returned neither support nor refutation, and the trace records that as a reduction in confidence instead of papering it over with a confident sentence. The conclusion is narrower than the question, and its open items are named. A reader who disagrees with the reading of section 80 can go straight to the section and re-run the check.trace shows the work, not just the answer

The same material as a chain of thought would run to a smooth paragraph ending "so a lecturer is largely unprotected". It might well be right, but there would be nothing to inspect.

Explainability Without Revealing Thought Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

ThisThis changes what an explanation should contain. Exposing the model's internal chain of thought is the wrong target, because it can be neither verified nor trusted to match what drove the answer. The user needs to see five things: the hypotheses considered, the tools invoked, what each returned, how confidence moved, and which branches were dropped. The reader sees question, hypothesis, evidence, revision and decision, and never the token-level deliberation behind any of them.

A record of this kind is, in effect, provenance: which source was consulted, by which activity, on whose behalf, producing which conclusion. The W3C's PROV family of standards already gives that vocabulary a formal shape, with entities, activities and agents, and a trace kept in such a form can be exchanged and queried.¹⁴ Mapping a trace onto PROV is our suggestion, and the format matters less than the habit of recording sources.provenance tracks the trail, not the truth of the steps

A trace has weaknesses of its own.

• It records the checks that were run, not the ones that should have been. An agent that chooses only convenient checks produces a tidy trace of a poor investigation. Declaring obligations before checking, and having a person review the plan for important decisions, are the main defences.
• The evaluation step can be wrong. A model reading a source and judging that it supports a claim is making a judgement, and it can make it badly. Checks that can be done deterministically, such as whether a section exists or a quotation appears in a fetched text, belong in code.
• Tool output is untrusted input. A fetched page is text the model reads, and text can carry instructions. Indirect prompt injection, where instructions planted in retrieved content steer an application, has been demonstrated against real LLM-integrated systems.¹⁵ Evaluation rules need to sit where page text cannot reach them.

The Workflow Is What Is Trusted Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

AnAn opaque connectionist reasoning stage can stay opaque, as the page on symbolic and connectionist reasoning discusses, provided it is surrounded by deterministic tool interfaces, verifiable validation procedures and an auditable evidence trace. The trustworthy element is then the workflow the model takes part in, not the model. That is a more modest claim than "the AI reasons correctly", and a much easier one to test. It also fits how this site treats trust elsewhere: in The Calculus of Trust, evidence updates trust and time erodes it, which is what a trace records at the level of a single question.

The future of explainable AI may lie less in systems that reveal their thoughts than in systems that record their investigations. A thought can only be inspected. An investigation can be reproduced, by anyone who has the trace and access to the same sources, and for trustworthy agentic systems reproducibility is the stronger property.

Related Topics

• How LLMs Reason, Self-Correct and Get Checked — what extra "thinking" buys, where self-correction fails, and the harness that supplies feedback from outside the model.
• Explaining What Language Models Do: Methods and Their Limits — why a written explanation can differ from what drove the answer.
• Designing Auditable, Robust Agentic Systems — keeping a checkable record of what an agent decided and why, and where to place human checkpoints.
• When Agents Fail: Brittleness, Misaligned Incentives, and Deception — the failure modes an evidence trace helps make visible.
• The Trustworthy Software Initiative — determinism, verifiability and auditability, the three pillars used throughout this page.
• Testing Non-Deterministic Systems — making randomness, timing and LLM calls reproducible enough to test.
• Retrieval-Augmented Generation — grounding an answer in retrieved passages, and why a passage in context still needs checking.
• Who Keeps Teaching After You Leave? — the legal question used in the worked example.

References