Judging a source

Lesson 2 36 minFinding & judging the literature

In this lesson

By the end, you’ll be able to

  • Tell primary research from reviews and non-research sources, and know when each belongs
  • Judge a source by peer-review status, venue, and authority
  • Recognize the warning signs of a predatory journal or a retracted paper

Sign in to save your progress

You can keep reading without an account, but completed lessons won’t be saved.

Sign in

Not every source is the same kind of thing, and a review that cites the wrong kind quietly loses its footing.

A primary source reports original research: the authors collected or analyzed data and are telling you what they found. This is the evidence your review is actually built on.

A secondary source digests other people's primary work: a literature review, a systematic review, a meta-analysis, a textbook. These are invaluable for orienting yourself and for finding primary sources (chase their reference lists), but citing a review for a specific finding, when you could cite the study that found it, puts a layer of someone else's interpretation between you and the evidence.

Then there are sources that are not research at all: editorials, opinion pieces, news articles, press releases. They can point you toward research, but they are not evidence. The move is simple: build on primary sources, use secondary sources to navigate, and keep opinion out of the evidence base.

Summaryshow you understood the studyMajor concernswhat threatens the conclusionsMinor pointsfixable detailsconstructive tone throughoutcritique the work, not the author
A review article is a secondary source: it sits on top of many primary studies, synthesizing them. Use a review to find and orient, then go down to the primary studies for the evidence itself.

Once you know what kind of source you have, you have to decide whether to trust it. Three questions carry most of the weight.

Was it peer reviewed? Peer review is imperfect, but it means at least some independent experts checked the work before publication. A preprint or a blog post has skipped that step.

Where was it published? Legitimate journals have editorial boards, transparent processes, and a track record. Predatory journals mimic the look of real ones but will publish almost anything for a fee, with little or no real review. Their tells: aggressive email solicitations, promises of review in days, fake or padded editorial boards, and names that imitate established journals.

Has it held up? Papers are sometimes retracted for error or misconduct. A retracted paper can keep collecting citations for years from people who never checked, so before you lean on a key source, make sure it has not been withdrawn.

Categorize

Sort each source into primary research (original data), a secondary synthesis (digests others' work), or not a research source at all.

Primary = the authors generated or analyzed the data. Secondary = they digested other studies. Non-research = opinion or journalism.

A randomized trial reporting new patient outcomes
A meta-analysis pooling results from 30 trials
A narrative review summarizing a field
A newspaper article describing a new study
A journal paper analyzing the authors' own dataset
An editorial arguing for a policy position
How sure?

Exercise

A key source you want to cite appeared in a journal that emailed you unsolicited, promised peer review within 72 hours for a publication fee, and has an editorial board you cannot verify. What is the right read?

Which signals (speed, fees, solicitation, unverifiable board) line up with predatory publishing, and what does that imply you must do?

Answer options
How sure?

Signature interactive

Citation Verifier

Three claims, each with a citation attached. You have already done the checking; the result is shown. Deliver the verdict for each: does the source support the claim, is it real but off-point, or is it not a real source at all?

Claim. Spacing study sessions over time improves long-term retention compared with massing them.

Cepeda NJ, Pashler H, Vul E, Wixted JT, Rohrer D. Distributed practice in verbal recall tasks. Psychol Bull. 2006;132(3):354-380.

When checked: The paper is found, and its abstract reviews exactly this: distributed practice reliably beats massed practice for retention.

Claim. Students who used our tutoring app scored higher because the app improved their motivation.

Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation. Am Psychol. 2000;55(1):68-78.

When checked: The paper is real and widely cited, but it is a theory of motivation in general. It never studied this app, and says nothing about it raising test scores.

Claim. A 2021 meta-analysis found that gamified quizzes raise retention by 30% in secondary students.

Nguyen TP, Alvarez M. Gamification and retention: a meta-analysis. Journal of Learning Sciences. 2021;29(4):511-538.

When checked: No database returns this article. The DOI does not resolve, and that journal has no volume 29, issue 4 matching it. The AI draft that produced it likely invented it.

Constructed example. The point is the reasoning, not these specific references.

Select all that apply

Which of these are warning signs that a journal may be predatory? Select all that apply.

Three of these describe pressure, speed, or imitation. Two describe transparency and vetting. Only the first three are red flags.

How sure?

Checkpoint · item 1 of 4

Which of these is a primary source?

Which one reports data the authors themselves produced?

How sure?

Reflection