Evidence & methodology

The paper trail.

What the experiment demonstrates, how it was measured, and where its claims stop. The recorded experiment results come from the same saved reports as the walkthrough; the diagrams are labelled as drawings. Current project evidence is dated and linked separately below.

01 · The comparison

Two ways to delete a document.

GhostCache runs the same deletion two ways: once naively, as a deliberately broken control, and once by following provenance. The source document is emptied either way. The difference is everything that was derived from it.

Broken controlSource-only deletion

Source-only deletion. The source d-vault is withdrawn. Its chunk, three summaries and saved answer stay active and stale, still holding the marker. A reworded question retrieves them and the answer reveals the marker. d-vaultWITHDRAWN c-vault s-vault s-mixed s-handoff saved answer d-garden d-archive REWORDED QUESTION “Which access phrase opens the vault?” ◆ MARKER REVEALED 1 2 3
  1. The source is emptied and its title replaced. Only the document changes.
  2. Its chunk, three summaries and the saved answer stay active: stale, and still holding the marker.
  3. A reworded question retrieves those copies, and the model repeats the marker.

CleanupProvenance-aware cleanup

Provenance-aware cleanup. Cleanup walks every recorded descendant of d-vault and turns each into a content-free tombstone. One summary is rebuilt from the surviving documents d-garden, d-maintenance and d-shuttle. The independent d-archive keeps the reading-room fact. d-vaultWITHDRAWN c-vaultPURGED s-vaultPURGED s-mixedPURGED s-handoffPURGED saved answer ↻ s-rebuilt d-garden d-maintenance d-shuttle d-archive ✓ LANTERN HALL ANSWERS 1 2 3 4
  1. Cleanup walks every recorded descendant of the source, in one transaction.
  2. Bodies are emptied and titles replaced; IDs, links and roots stay as inspectable tombstones.
  3. One summary is rebuilt from the surviving original documents, never from the old mixed prose.
  4. An independent source keeps a shared fact answerable.
Fig. 1 Diagram drawn from GhostCache's seed design (the same in every run), simplified: the other documents' chunks and some links are omitted. Not recorded data. Diagram

02 · The controlled corpus

Five fictional documents, one fresh secret.

Every run seeds the same five fictional documents about an observatory. Only the passphrase changes: each reset generates a new MOTH- marker, so a pass can never rely on an old one.

  • d-vaultVault access note

    The source that is withdrawn. Holds the fictional passphrase and names the reading room.

  • d-gardenRoof garden timetable

    An unrelated opening time, 07:40, that must stay answerable.

  • d-archiveIndependent archive directory

    Names the reading room, Lantern Hall, independently of the vault note, so that fact must survive.

  • d-maintenanceTelescope maintenance plan

    An unrelated schedule. Its chunk feeds the handoff summary.

  • d-shuttleVisitor shuttle guide

    An unrelated departure that retrieval pulls in beside the vault question.

From them, seeding derives five search chunks (one per document), three model-written summaries chained s-vault → s-mixed → s-handoff, and one saved answer: fourteen artifacts. Every generated artifact records every artifact it was given as a parent. The summary prompt asks the model to keep the passphrase on purpose, which builds a known, traceable propagation path; GhostCache does not measure how often secrets leak naturally.

03 · Recorded results

Three live runs, read straight from their reports.

Each recorded run is a full ten-check evaluation of local Gemma (gemma4:e2b-it-qat through Ollama) on a fresh in-memory database. The counts below are the reports' own audits at each stage.

Loading the recorded results…

Three runs on one laptop over two days are not a success rate. A deterministic fixture report (string rules, no model call) also passed all ten checks; it stays in the repository and is never shown here as AI output.

04 · The balance

Remove the secret. Keep the facts.

A cleanup that destroys everything would pass a "secret is gone" test trivially. So every evaluation asks six recorded questions: two must reveal the marker (proving the failure was real), two must not, and two must still answer facts that should survive.

Loading the recorded answers…

05 · Timing evidence

What the clock recorded.

Each report records the duration of every model call. The October 28 runs came one after the other: the first right after Ollama was started (cold), the second straight afterwards (warm). The October 27 run was a separate session.

Loading the recorded timings…

The pass/fail result hides this: all three runs passed every check. The slow first call is consistent with Ollama loading the model on its first request after starting, but the reports don't record Ollama's load time, so the cause is inferred, not measured.

06 · Scope and limits

Where the claims stop.

  • Application-managed memory onlyCleanup covers GhostCache's SQLite titles and bodies, its recorded provenance and its retrieval. It is not model-weight unlearning or secure disk erasure.
  • The evaluator keeps the markerThe expected marker stays in evaluator state, saved reports, screenshots and this site, on purpose: they are the evidence.
  • Bounded, exact-match checksOne literal marker across four fixed questions. That is not proof that every paraphrase or inference is gone.
  • Lexical retrievalTerm overlap, not embeddings; the reworded question still shares words with its target.
  • A deliberate propagation pathThe summary prompt asks the model to keep the passphrase, so the copies are expected, not discovered.
  • Three runs are not a rateOne model, one laptop, two days. Failures from future runs belong in the record too.
  • Provenance has to be capturedA copy made outside the store is outside the graph, and no traversal can find it.
  • Fictional data onlyNo uploads, no real documents, no connectors to other RAG systems, and no claim about any other product.

07 · Current project evidence · October 30, 2026

The work behind the recorded runs.

This is the project's development record, separate from the October 27–28 model results above. Updating the app does not change those saved answers, timings or checks.

Local regression suite126 tests passed
The latest local check, on October 30, passed all 126 tests with Node and the optional Sentry SDK installed. Without the SDK, six telemetry tests skip. This tests storage, provenance, model-error handling, HTTP boundaries and UI behaviour; it is separate from each recorded run's ten evaluation checks. Test record ↗
Recorded Gemma runsThree live evaluations
The October 27 run and October 28 cold and warm runs each passed 10/10 checks using gemma4:e2b-it-qat locally through Ollama. The walkthrough reads those same three reports. Saved reports ↗
GitHub ActionsArchived CI evidence: run #17 on 00cfb10
The workflow runs the offline regression tests and a fresh fixture evaluation, then uploads that evaluation's report. The report downloaded from hosted run #3 ↗ passed 10/10 checks with the earlier 62-test suite. Hosted run #16 ↗ on commit f66977d and run #17 ↗ on commit 00cfb10 reported success for every step. Both uploaded reports are archived, each matching GitHub's artifact digest: each is a deterministic fixture evaluation, with no model call, that passed all ten checks. Commit 00cfb10 contains the October 29 content refresh and the failed-mode-switch fix, so run #17 tested them; it did not test later commits, including this October 30 update. Neither run's log was read, so the number of tests run on GitHub is unverified. Run #16 evidence ↗ · Run #17 evidence ↗ · Workflow ↗ · Run history ↗
Sentry Agent TracingReal traces, metadata only
One fixture and two live evaluations sent traces on October 28. The first summarizer model-call (chat) span took 40.45 s cold and 2.28 s warm; loading is an inferred cause, not a measured load time. Another span showed the source-only leak through synthetic context IDs and a marker-present boolean, without sending the marker, document text or answers. Two findings and screenshots ↗
Render and .techPublic static walkthrough
ghostcache.bipulchamoli.tech points to the same Render static site as its alternate address. The saved checks show a verified hostname, an issued certificate and successful HTTPS responses. The owner's Hacktoberfest promo confirmation is supported by claimed-promo and registration screenshots; they do not show which code was applied. Domain and promo evidence ↗
Local lab recoveryCurrent controller state
An unresolved summary-rebuild warning survives a reload and later ledger events. Repeating cleanup does not retry a failed rebuild; reset starts a fresh experiment. If a cleanup request fails after the purge commits, the lab reads the server's state once, keeps the original error and does not repeat the deletion. If a mode switch fails, the run-mode menu returns to the experiment's confirmed mode. Unrestorable storage has a separate reset-to-recover state. Recovery guidance

Five intended categories

  • Best Use of GemmaThe local open-weight model generates summaries and answers.
  • Best Use of GitHub CopilotThrough GitHub Actions automation; no claim that Copilot wrote the project.
  • Best Use of Sentry Agent TracingReal metadata-only traces, screenshots and explained findings.
  • Best Use of RenderHosting these static pages of recorded local evaluations.
  • Best Use of a .tech DomainThe public walkthrough uses the owner's Hacktoberfest-promo domain.

These categories are intended, not entered: the DEV submission has not been published. Render's recorded-front-end interpretation and the .tech category's use of a subdomain are not organizer-confirmed eligibility. The promo-to-registration link rests on the owner's confirmation. Category evidence and limits ↗

Real-phone, second-browser and accessibility checks remain open. The public demo video (4:55) is available; the DEV submission remains to be created. Remaining work ↗

08 · Reports and code

Check the paper trail yourself.

Prior work

Source-aware invalidation is established practice; these DEV articles describe it, and GhostCache makes the failure and the fix visible in one small experiment.