A model-launch chart can cite a familiar benchmark while leaving the exact dataset version and evaluation procedure unclear. Perplexity can help locate the original materials, but a dataset provenance list should connect each reported result to its paper, repository, split, and scoring method. This workflow investigates evidence identity rather than treating benchmark names as interchangeable.
Primary product documentation checked on 8 October 2026. The worked examples are fictional editorial examples, not claims of hands-on testing inside Gemini or Perplexity.
Extract the launch evaluation claims
Record the model identifier, benchmark label, reported metric, evaluation date, and any footnotes in the launch material. Keep separate entries for different splits or settings, even when they share a benchmark family name. A result using a special subset or multiple attempts is not automatically comparable with a single-attempt result on the full dataset.
In a fictional launch review, one chart says Benchmark Q while a technical appendix says Benchmark Q hard subset. The provenance list should preserve both labels and flag the relationship for checking. Do not normalize them into the same row merely to simplify the table.
Trace each benchmark to its original materials
Ask Perplexity to locate the benchmark’s original paper and official author repository, then open those sources. Follow the launch document’s own references when available. Record which source defines the dataset and which source describes the vendor’s evaluation implementation. They may be different projects with different release histories.
OpenAI’s simple-evals repository illustrates how an evaluation collection can reference original benchmark papers and datasets. Use those references to trace provenance, then inspect the original materials relevant to the particular result. The presence of a benchmark in a repository does not prove a launch evaluation used that repository’s current code unchanged.
Identify version, split, and preprocessing
Capture dataset version or commit, split name, sample count where verified, inclusion rules, transformations, and any filtering described by the evaluator. If the exact version is absent, mark it unknown. Do not substitute the latest public release for an unspecified historical evaluation without explaining that assumption.
Check whether prompts were reformatted, answer options changed, or examples removed. These details can affect comparability even when the benchmark name remains familiar. Ask Perplexity to help find the relevant methodological passages, but verify them against the paper and code documentation. Keep inferred relationships explicitly labeled as inference.
Inspect scoring and reproducibility limits
Record the scoring method, number of attempts, tool access, judge model or rubric when documented, and aggregation rule. Distinguish dataset provenance from evaluation reproducibility. You may identify the original dataset while still lacking the prompts or configuration needed to reproduce the reported score.
For the fictional chart, a result based on several generated answers and an aggregation rule should not be presented as an ordinary single-answer accuracy. The provenance list should state the actual documented metric and explain the missing details. Avoid declaring a result invalid solely because complete reproduction is unavailable; instead describe the specific evidential limitation.
Publish a clear provenance list
Use one row per evaluation claim with launch reference, original dataset source, version or unknown state, split, metric, implementation source, and remaining questions. Include checked dates and working links. Preserve author and license information where it affects permitted reuse. Do not assume a publicly accessible dataset has unrestricted usage rights.
Before handover, choose two rows and follow their links from the launch claim to the original definition. If the route breaks or lands on an unrelated dataset, correct it. The finished list should help readers judge whether results are comparable and what they would need for a reproduction attempt, without exaggerating what the research established.
Validate the evidence and hand over the result
Run the review in two passes. First, check the source records without reading the generated conclusions. Select a small but representative sample: a clear record, a contradictory record, an incomplete record, and a record that changed after the review began. Write down the expected treatment of each. This gives you a reference for judging the output instead of accepting a plausible narrative because it looks organized.
Then inspect the generated result against those expectations. Open every source supporting a consequential conclusion. Check the surrounding paragraph, the relevant table heading, and any qualifying note. A link to the correct document is only the beginning of verification. The document may support a narrower claim, describe a different version, or use a different unit. Record a correction beside the affected item rather than quietly changing its meaning.
Count the inputs and outputs where a one-to-one relationship is expected. Explain every omitted record and every merged record. When several source items become one conclusion, retain the individual references so the reviewer can reverse that grouping. When one source item produces several actions, preserve the connection so a later source correction can update all affected actions. Never interpret an empty output as evidence that there are no problems until you have checked source coverage.
Keep proposed wording separate from approved wording. An AI system can suggest how to express a finding, but the source owner still needs to confirm the underlying facts when records disagree. Use explicit states such as supported, disputed, missing evidence, and awaiting owner confirmation. Avoid a single confidence score that disguises those different reasons for uncertainty. A disputed statement with two strong sources needs a different next step from a statement with no accessible source.
Before handover, ask a colleague to use the result without your explanation. Can they locate the original evidence, identify unresolved items, and understand what changed since the last version? Fix the places where they need to guess. Save the source manifest, the prompt, the reviewed result, and the correction history together. A useful deliverable should remain understandable after the conversation that produced it has ended, and it should make the next review cheaper because the evidence trail is still intact.
Frequently asked questions
Does a benchmark name uniquely identify a dataset?
No. Versions, splits, subsets, and transformations may differ.
Can I use the latest repository version for an old result?
Only as an explicitly labeled assumption when the historical version is unknown.
Are provenance and reproducibility the same?
No. Dataset identity can be known while evaluation configuration remains incomplete.
Should multiple-attempt scores be compared with single-attempt scores?
Preserve the evaluation settings and explain the difference before comparing them.
Does public access permit unrestricted reuse?
No. Check the dataset’s license and applicable conditions separately.
