systematic review and meta-analysis، Users’ Guides to the Medical Literature

●  Evidence synthesis, explained 

JAMAevidence for Systematic Reviews and Meta-Analysis

Learn how the Users’ Guides, the calculators and the worksheets in JAMAevidence guide you through every stage of evidence synthesis, from the first question to the final recommendation.

The review pipeline

Seven stages, and where JAMAevidence helps at each one.
  1. Frame the question
  2. Register a protocol and search
  3. Screen and select studies
  4. Appraise and extract
  5. Synthesize the results
  6. Grade the certainty of evidence
  7. Report and apply
Why this matters

A systematic review is only as trustworthy as the person reading it.

A systematic review uses a predefined, transparent method to find, select, appraise and combine the studies that answer a focused question. Some reviews include a meta-analysis, a statistical pooling of compatible results that yields a single best estimate.
Reviews sit at the top of the evidence hierarchy, but a review can be done well or badly. Readers need to judge the question, the search, the appraisal and the pooling before trusting the conclusion.
JAMAevidence, built by McGraw Hill with the JAMA Network, teaches those judgments. Its Users’ Guides give a whole part of the book to summarizing evidence, with worksheets and calculators to practice on.

Two ideas that are easy to blur

The two are linked, but they are not the same thing.

Evidence-based medicine

A practice. Ask a clinical question, acquire the evidence, appraise it, apply it with the patient, and assess the result. Its unit of work is the clinical decision.

Systematic review

A research method. A structured study of existing studies, designed to answer one question reproducibly. Its unit of work is the research question.

A systematic review is one of the best sources an evidence-based practitioner can use. JAMAevidence teaches both sides: how to do and read a review, and how to use it for the patient in front of you.

The pipeline

From question to recommendation in seven stages

Each stage explains what happens and why, the standards and tools involved, a common pitfall, the matching chapters of Users’ Guides to the Medical Literature (3rd edition) and the sources behind it. Chapter numbers follow the published 3rd edition table of contents, and some chapters are available only to JAMAevidence subscribers.
01

Frame the question

A review is only as good as the question it asks. Write the question first, in a form that can be searched, screened and answered.
Most reviews of interventions frame the question with PICO: the Population (who), the Intervention or exposure (what is being tested), the Comparator (what it is compared with) and the Outcomes (what is measured). The extension PICOTS adds Timing (how long participants are followed) and Setting (where care is delivered). The framework is a drafting tool: each element becomes an eligibility criterion, a block of search terms and a column in the data-extraction form.
Other question types use adapted structures. A review of diagnostic accuracy states the target condition, the index test and the reference standard. A prognosis review states the population, the prognostic factor or model, and the outcome over time. Whatever the type, the question is settled before searching and recorded in the protocol.
Key points
Standards and tools
In JAMAevidence. The Foundations, Chapter 4: What Is the Question? covers how to frame the clinical question. Chapter 2 (What Is Evidence-Based Medicine?) and Chapter 3 (Evidence-Based Medicine and the Theory of Knowledge) give the framework behind it. See the 3rd edition table of contents.
02

Register a protocol and search

Decide the rules before you see the results, then search widely enough that the review does not depend on a single database.
A protocol records the question, the eligibility criteria, the search strategy, how studies will be selected and appraised, and how results will be synthesized. Writing it before screening begins means later decisions cannot be steered by what the studies show. PRISMA-P provides a 17-item checklist for protocols. Registering the protocol publicly, for example in PROSPERO (the international register of systematic reviews with a health-related outcome), makes the plan visible, lets readers detect undeclared changes and helps prevent duplicate reviews.
No single database indexes the whole literature. Reviews of interventions usually search MEDLINE (through PubMed or Ovid), Embase and the Cochrane Central Register of Controlled Trials, then add multidisciplinary sources such as Scopus and Web of Science, trial registries such as ClinicalTrials.gov, and the reference lists of included studies. Each strategy combines controlled vocabulary (MeSH in MEDLINE, Emtree in Embase) with free-text synonyms. It is reported in full, with the database, platform, date and exact search string, so another team could repeat it. PRISMA-S defines 16 items for reporting searches.
Key points
Standards and tools
In JAMAevidence. The Foundations, Chapter 5: Finding Current Best Evidence covers how to search the literature and use preappraised resources, an emphasis of the 3rd edition. The information cycle worksheets in JAMAevidence help structure the path from question to answer.
03

Screen and select studies

Apply the eligibility criteria the same way to every record, and show how many studies survived each step.
Records from all databases are merged and duplicates removed. Screening then happens in two passes: titles and abstracts first, erring on the side of inclusion, then the full text of every remaining report against the eligibility criteria. At least two reviewers screen independently and settle disagreements by discussion or with a third reviewer; independent screening reduces both errors and personal bias. A pilot round on a small sample of records tests whether the criteria are clear enough to apply consistently.
The unit of analysis is the study, not the report. Several publications can describe one study, and they must be linked so the study is counted once. For each full-text report that is excluded, the main reason is recorded. The counts at each step (records identified, screened, assessed for eligibility, included) are shown in the PRISMA 2020 flow diagram, so readers can see how the final set of studies was reached.
Key points
Standards and tools
In JAMAevidence. Summarizing the Evidence, Chapter 22: The Process of a Systematic Review and Meta-analysis walks through how a review is conducted from question to synthesis. Its online version includes a discussion of the process by M. Hassan Murad (chapter page, subscribers).
04

Appraise and extract

Judge how far each study can be trusted before combining anything, and extract the data so another person could check them.
Data are extracted into a structured form that was tested on a few studies first. It captures the design, setting, participants, interventions, outcome definitions and time points, numerical results and funding source. Two people extract independently, or one extracts and a second checks, because extraction errors can change a pooled result. When a study does not report what the review needs, its authors are contacted.
Risk of bias asks whether flaws in design, conduct or reporting could have pushed a result away from the truth. The tool depends on the design. RoB 2 is used for randomized trials and examines five domains: the randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome and selection of the reported result. Each result is judged low risk, some concerns or high risk. ROBINS-I is used for non-randomized studies of interventions; it compares each study with an ideal target trial across seven domains, starting with confounding, and judgments range from low to critical risk. QUADAS-2 is used for studies of diagnostic accuracy. These judgments are used later to explore why results differ and to rate the certainty of the evidence.
Key points
Standards and tools
In JAMAevidence. Separate parts teach appraisal for each design: Therapy (Chapter 7, Therapy (Randomized Trials), and Chapter 11, Advanced Topics in the Risk of Bias of Therapy Trials), Harm (Chapter 14, Harm (Observational Studies)), Diagnosis (Chapter 18, Diagnostic Tests) and Prognosis (Chapter 20). Chapter 6, Why Study Results Mislead: Bias and Random Error, explains the principles. The critical appraisal worksheets guide the judgment of validity, importance and applicability.
05

Synthesize the results

Combine only what can sensibly be combined, and show how uncertain the combined result is.
Studies are first grouped by comparable populations, interventions and outcomes. When they are similar enough, a meta-analysis pools their results: each study’s effect (for example a risk ratio, odds ratio, hazard ratio or mean difference) is weighted, usually by its precision, and the weighted average is reported with a confidence interval. A forest plot shows every study’s estimate and the pooled result on one graph. Larger studies receive more weight, so one big trial can dominate the pooled estimate.
Studies rarely agree exactly. Heterogeneity is variation in results beyond chance. It is judged by inspecting the forest plot, with Cochran’s Q and with I², which estimates the proportion of variability due to heterogeneity rather than chance. A fixed-effect model assumes all studies estimate one common true effect. A random-effects model assumes the true effect differs between studies and estimates its average; a prediction interval shows the range in which the effect in a new setting may fall. Reasons for heterogeneity are explored with prespecified subgroup analyses, which must be interpreted cautiously, and sensitivity analyses test whether conclusions change when, for example, high-risk-of-bias studies are removed.
Network meta-analysis compares three or more interventions in one model by combining direct comparisons with indirect ones through shared comparators. It relies on transitivity (studies are similar enough in factors that modify the effect) and consistency (direct and indirect evidence agree). When studies cannot be pooled, results are synthesized without meta-analysis using structured, transparent methods (SWiM). With ten or more studies, funnel plots can be used to look for small-study effects such as publication bias.
Key points
Standards and tools
In JAMAevidence. Summarizing the Evidence: Chapter 23 (Understanding and Applying the Results of a Systematic Review and Meta-analysis), Chapter 24 (Network Meta-analysis) and Chapter 25 (Advanced Topics in Systematic Reviews), including 25.1 Fixed-Effects and Random-Effects Models and 25.2 How to Use a Subgroup Analysis. Chapter 10 (Confidence Intervals: Was the Single Study or Meta-analysis Large Enough?) and Section 13.2 (Numbers Needed to Treat) help you judge precision and translate relative effects into absolute ones. The matching JAMA article is Murad et al., 2014.
06

Grade the certainty of the evidence

A pooled number is not yet a conclusion. Say how much confidence it deserves.
GRADE (Grading of Recommendations Assessment, Development and Evaluation) rates the certainty of evidence for each outcome as high, moderate, low or very low. In the standard approach, randomized trials start at high certainty and observational studies at low, and the rating is then lowered for concerns or, for observational studies, raised.
Five reasons lower certainty: risk of bias in the studies, inconsistency of results, indirectness (the evidence does not match the question’s population, intervention, comparator or outcome), imprecision (wide confidence intervals) and publication bias. Three reasons can raise certainty in observational studies: a large effect, a dose-response gradient, and plausible residual confounding that would reduce the observed effect. Each judgment is explained. Results are shown in a Summary of findings table that lists, for each important outcome, the absolute and relative effects, the number of participants and studies, and the certainty rating.
Certainty of evidence is separate from the strength of a recommendation. A recommendation also weighs the balance of benefits and harms, patients’ values and preferences, and the use of resources, which is why GRADE distinguishes strong from conditional recommendations.
Key points
Standards and tools
In JAMAevidence. Moving From Evidence to Action, Section 28.1: Assessing the Strength of Recommendations, The GRADE Approach. Chapter 26 (How to Use a Patient Management Recommendation: Clinical Practice Guidelines and Decision Analyses) shows how guidelines use these ratings. The Murad et al., 2014 article lists the same factors that raise or lower confidence.
07

Report and apply

Report the review so others can check it, then ask whether it applies to the patient in front of you.
PRISMA 2020 is the reporting standard for systematic reviews. It has a 27-item checklist covering the title, abstract, rationale, eligibility criteria, information sources, search strategy, selection, data collection, risk of bias, synthesis methods, certainty of evidence, results, discussion, registration and protocol, support, and availability of data. It also provides a 12-item abstract checklist and a flow diagram. The PRISMA website lists 17 official extensions for particular review types and purposes, such as PRISMA-NMA for network meta-analyses, PRISMA-DTA for diagnostic accuracy reviews, PRISMA-ScR for scoping reviews, PRISMA-IPD for individual participant data and PRISMA-S for searches.
A well-reported review still has to be applied with judgment. The questions are: do the studied patients resemble mine; were the outcomes ones that matter to patients; how large are the benefits and harms in absolute terms; how certain is the evidence; and what do this patient’s values and the available resources suggest? Absolute risks and the number needed to treat usually communicate better than relative measures. A review reports an average effect, and an individual patient may differ, so applying evidence remains a shared decision.
Key points
Standards and tools
In JAMAevidence. Moving From Evidence to Action: Chapter 26 (How to Use a Patient Management Recommendation: Clinical Practice Guidelines and Decision Analyses) and Chapter 27 (Decision Making and the Patient). Therapy sections 13.1 (Applying Results to Individual Patients) and 13.2 (Numbers Needed to Treat) help translate results to one patient. The JAMAevidence calculators support the same step.

Inside the book

Four chapters on systematic reviews

The Summarizing the Evidence part of Users’ Guides to the Medical Literature (3rd edition) is where review methods are taught. Here is what each chapter helps you do.

Reading a published review

Six questions to ask of any systematic review

These questions follow the validity, results and applicability logic used throughout the Users’ Guides.

A clear question names the population, intervention or exposure, comparator and outcomes. If you cannot state the question in one sentence, the review may be too broad to interpret.

Look for several databases, a described search strategy, explicit eligibility criteria and two independent reviewers. A narrow search can miss studies and bias the result.

The review should say how it judged bias in each study and use that judgment when interpreting the pooled result. Pooling flawed studies produces a precise but misleading answer.

Look at heterogeneity. If studies disagree widely, ask why before accepting one pooled number, and check whether the review explored the reasons.

Read the confidence interval, not only the point estimate. A wide interval means the true effect could be much smaller or larger than the headline number.

Compare the studied populations, interventions and outcomes with your situation, and weigh benefits against harms and costs. Certainty ratings such as GRADE help you decide how firmly to act.

The platform

One platform, four books and a set of tools

Systematic review skills are one thread. The same platform covers the clinical side of evidence-based practice.

Users’ Guides to the Medical Literature

A Manual for Evidence-Based Clinical Practice, 3rd edition (2015), by Guyatt, Rennie, Meade and Cook. Seven parts: The Foundations, Therapy, Harm, Diagnosis, Prognosis, Summarizing the Evidence, and Moving From Evidence to Action.

The Rational Clinical Examination

Evidence-Based Clinical Diagnosis. Summarizes the evidence on how well signs, symptoms and tests change the likelihood of a diagnosis, including diagnostic accuracy.

JAMA Guide to Statistics and Methods

Short explanations of the statistical and methodological concepts used in published research.

Care at the Close of Life

Evidence and Experience. A resource on caring for patients at the end of life.

Learning tools

Who it is for

Four readers, four ways to use it

Review authors and researchers

Plan your methods, practice appraisal and understand the statistics before you pool.

Residents and students

Prepare journal clubs and assignments, and learn to judge a review rather than quote its abstract.

Clinicians

Decide how far to trust the reviews and guidelines that shape your practice.

Faculty and educators

Teach evidence synthesis with ready-made slides and durable links to chapters.

Access

Study the whole pipeline with your own JAMAevidence account

AliMed offers McGraw Hill Medical Access resources through flexible subscriptions. JAMAevidence is one of the individual platforms you can choose, and an ALL in ONE package is also available.

How it works

  1. Choose JAMAevidence on the McGraw Hill Medical Access page, or pick the ALL in ONE package.
  2. Read the product details before you order, then complete your purchase.
  3. Receive a dedicated account with direct access to the platform. [Add: activation time]

Questions and answers

Frequently asked questions

JAMAevidence is an online resource for teaching, learning and applying evidence-based medicine, built by McGraw Hill with the JAMA Network. It combines four textbooks with calculators, worksheets, education guides and a glossary.

Yes. The Summarizing the Evidence part of the Users’ Guides (3rd edition) has chapters on the process of a systematic review, understanding and applying results, network meta-analysis, and advanced topics such as fixed-effects and random-effects models.

It is a learning and decision-support resource. It teaches the methods and judgments behind a review. You would still use databases, screening and extraction workflows, and reporting standards such as PRISMA 2020 to carry one out.

Evidence-based medicine is a practice of making clinical decisions with the best evidence. A systematic review is a research method that summarizes studies on one question. Reviews are one of the best sources EBM practitioners use.

Users’ Guides to the Medical Literature (3rd edition), The Rational Clinical Examination, the JAMA Guide to Statistics and Methods and Care at the Close of Life.

Education Guides, calculators and nomograms, critical appraisal and information cycle worksheets, a glossary of more than 900 terms and self-assessment quizzes.

No. JAMA Network publishes medical journals. JAMAevidence is a separate educational platform on evidence-based medicine.

A free MyAccess profile lets you save worksheet notes, take quizzes and return to saved content. Through AliMed you also receive a dedicated account for access.

Yes. JAMAevidence is available as an individual option on the AliMed McGraw Hill Medical Access page, and an ALL in ONE package is also offered.

Leave a Reply