Responsible Research

How to Detect Fake AI Citations: A Researcher’s Verification Checklist

Speaker

Joanne Donoghue (Director, Clinical Research, NYIT)

Joanne Donoghue (Director, Clinical Research, NYIT)36 min

Quick answer

When SciSpace or another AI system suggests a reference, confirm that the source exists and supports the claim before using it. Resolve the title, authors, venue, year, pages, and DOI in trusted records; open the source; check the relevant passage; and remove or correct any citation that is fabricated, mismatched, or irrelevant.

About this session

Joanne Donoghue explains how to identify AI-generated citation hallucinations by manually verifying that referenced papers exist and accurately support claims [L5-L33]. She emphasizes that researchers must audit reference lists from faculty and students to counter AI's tendency to predict, rather than think, when generating look-alike citations [L14-L33].

AI-generated references can look plausible while combining a real author, an incorrect title, a nonexistent journal article, or a DOI that belongs to something else. This expert playbook shows how to verify both the identity of a source and whether the source actually supports the claim attached to it.

What this tutorial covers

Recognize common citation-hallucination patterns before they enter a manuscript or review.
Validate bibliographic metadata across the DOI, publisher page, indexing record, and full text.
Check claim-to-source alignment rather than stopping when a paper merely exists.
Create a repeatable reference audit for AI-assisted literature reviews and writing.

Key concepts explained

Existence check: Verify that the cited work is real and retrievable.
Identity check: Title, authors, venue, year, volume, pages, and DOI must refer to the same work.
Entailment check: The paper must support the specific claim, population, direction, and magnitude stated.
Provenance: Record where each reference came from and retain a link or source document.

How to try it in SciSpace

1
Export or collect every AI-suggested reference in a single audit table.
Collect every AI-suggested reference in one audit table with the claim it is meant to support. Give each item a status field and do not cite it while the status is unverified.
2
Resolve the DOI or search the exact title and compare all metadata fields.
Resolve the DOI or search the exact title, then compare authors, title, venue, year, volume, pages, and URL. A near match is not the same source.
3
Open the original source and locate the passage, table, or result supporting the claim.
Open the original work and locate the passage, table, or result that supports the claim. Record the evidence location and note whether the source is primary or secondary.
4
Flag references that are missing, mismatched, secondary when primary evidence is needed, or irrelevant.
Flag references that are missing, mismatched, irrelevant, retracted, or secondary when primary evidence is required. Also flag a real source that does not support the stated claim.
5
Correct the claim or citation and retain a verification status for each reference.
Correct or remove the claim and citation, then update the audit status. Keep the table with the manuscript so future edits do not reintroduce the unverified reference.

Worked example

An AI-generated draft contains twelve references. The researcher searches each title and DOI in trusted databases and discovers that three references do not exist, two have incorrect author details, and one real article does not support the claim attached to it. The invalid references are removed, the metadata are corrected, and the affected sentence is rewritten using verifiable evidence. The final reference list is generated only after this source audit.

Practice prompt

Audit this reference list. For each item, verify title, authors, venue, year, DOI, and retrievability; then check whether the cited source supports the attached claim. Return Verified, Metadata Mismatch, Unsupported Claim, Secondary Source, or Not Found, with an explanation.

Key takeaway

A real-looking citation is not enough: verify the source identity and the exact evidentiary relationship between the paper and the claim.