How Biomedical Researchers Configure PICO Criteria in Elicit for Semi-Automated Title and Abstract Screening

Biomedical researchers conducting systematic reviews configure PICO criteria in Elicit by defining clinical parameters within Elicit's dedicated Systematic Review workflow, translating Population, Intervention, Comparator, and Outcome elements into structured, rule-based screening criteria, and deploying dual-reviewer or AI-assisted blind screening. This workflow replaces purely manual dual screening of thousands of citations with a semi-automated, PRISMA 2020-compliant process that provides sentence-level justification quotes for every inclusion and exclusion decision.

Conducting a rigorous systematic review often requires screening 3,000 to 10,000 titles and abstracts. Under standard Cochrane and PRISMA methodologies, this stage demands two independent reviewers working in blind conditions to prevent selection bias, consuming hundreds of research hours before full texts are even retrieved. Elicit automates the initial appraisal while keeping human researchers in control of threshold arbitration and methodological compliance.

1. Prerequisites and Workflow Selection in Elicit

Elicit organizes research tasks across distinct modes, including general literature search, automated research reports, and a specialized Systematic Review workflow (Review mode). Access to end-to-end systematic review screening, custom screening columns, and dual-review management requires an Elicit subscription (Pro, Scale, or Enterprise tiers).

Before configuring screening rules, research teams must prepare their systematic review protocol:

• Protocol Registration: Register the review protocol on PROSPERO or Open Science Framework (OSF) with predefined eligibility criteria.
• Operational PICO Decomposition: Break down the clinical research question into discrete, testable boundaries rather than open-ended descriptive sentences.
• Search Strategy Execution: Finalize whether the citation corpus will be retrieved natively through Elicit or imported from external biomedical databases.

2. Step-by-Step Configuration: From PICO to Machine-Actionable Screening

Step 1: Initialize the Systematic Review and Provide Protocol Context

1. Navigate to the Elicit workspace and select Systematic Review from the workflow selector.
2. Enter the primary review question in the main prompt field (for example: What is the clinical efficacy and safety of GLP-1 receptor agonists compared to SGLT-2 inhibitors in adults with type 2 diabetes and chronic kidney disease?).
3. Open the Additional Context configuration box on the Setup page. This free-text parameter directly shapes Elicit's criterion generation. Enter your operational protocol details explicitly structured under PICO headings:
   • Population: Target patient demographic, clinical stage, comorbidities, age boundaries, and explicit exclusions (such as pediatric cohorts or end-stage renal disease on dialysis).
   • Intervention: Qualifying active agents, drug classes, dosages, and administration routes.
   • Comparator: Eligible active controls, standard of care, or placebo regimens.
   • Outcomes: Primary and secondary clinical endpoints (such as progression of eGFR decline, major adverse cardiovascular events, all-cause mortality, and specific adverse drug reactions).
   • Study Design: Permitted methodological types (such as randomized controlled trials with a minimum follow-up duration of 12 weeks; excluding animal models, in vitro studies, editorials, and narrative reviews).

Step 2: Assemble the Citation Corpus (The Dual-Track Search Strategy)

Elicit allows researchers to gather literature either through its native semantic index (indexing over 126 million papers powered by Semantic Scholar) or by importing external citation packages.

For biomedical reviews adhering to PRISMA 2020 standards, teams should not rely exclusively on AI semantic searching. Empirical evaluations demonstrate that while Elicit's native search delivers high precision (averaging ~41.8%), its search sensitivity averages ~39.5%, compared to over 94% in conventional multi-database searches across PubMed, Embase, and Cochrane CENTRAL.

The methodologically robust approach is:

1. Conduct comprehensive Boolean searches in PubMed/MEDLINE, Embase, CINAHL, and CENTRAL through an information specialist.
2. Deduplicate records using reference management tools (such as EndNote or Covidence).
3. Export the final deduplicated citation library in standard RIS or BibTeX format.
4. In Elicit's Gather phase, upload the RIS file to populate the screening queue with the complete, PRISMA-compliant citation corpus.

Step 3: Configure and Calibrate PICO Screening Criteria

Once citations are loaded, Elicit analyzes your protocol context and proposes an initial set of screening criteria. Researchers must review and refine these criteria into discrete, verifiable parameters:

To optimize classification accuracy:

• Keep Criteria Atomic: Avoid combining multiple conditions into one rule (for example, separate "Adult population" from "CKD Stage 3-4" into two individual criteria if mixed cohorts require subset analysis).
• Define Explicit Exclusions: Add dedicated exclusion checks (e.g., "Exclude if study is a conference abstract without peer-reviewed full text", "Exclude if animal or pre-clinical model").
• Set Criterion Importance: Assign weighting flags in Elicit's setup to differentiate absolute eligibility gates (such as study design and human subjects) from secondary criteria that may require full-text inspection to confirm.

Step 4: Execute Title and Abstract Screening and Audit Evidence Justifications

When screening is initiated, Elicit processes titles and abstracts across the citation table:

1. Rule Evaluation: For each citation, Elicit evaluates every configured PICO column independently, returning a status of Met, Not Met, or Uncertain.
2. Quote Grounding: Clicking into any cell reveals the exact sentence extracted from the abstract that led to the determination. If a paper is marked "Excluded on Intervention," Elicit highlights the excerpt showing an unapproved combination therapy or non-qualifying agent.
3. Composite Scoring and Recommendation: Elicit calculates an overall relevance score based on weighted criterion satisfaction and displays an automated recommendation: Include, Exclude, or Review.
4. Interactive Spreadsheet Review: The interface organizes records into a sortable table. Reviewers can filter papers by individual criteria (e.g., viewing all papers where "Comparator" was marked "Uncertain") to accelerate targeted checks.

Step 5: Implement Dual-Review Blind Screening and Conflict Arbitration

PRISMA 2020 guidelines require minimizing screening errors through independent assessments. Elicit accommodates two screening operational models:

• Model A: Human Reviewer + Elicit as Second Reviewer: One human reviewer screens the citations while Elicit acts as the independent second screener. The platform flags discordant decisions (for instance, where the human included a paper but Elicit recommended exclusion, or vice versa) for human re-adjudication.
• Model B: Dual Human Blind Review with AI Prioritization: Two human reviewers screen independently within the team workspace without viewing each other's decisions. Elicit's relevance score is used to rank-order the queue, presenting high-probability inclusions first. Once both reviewers complete their screening, the workspace reveals inter-rater agreement statistics and consensus discrepancies.

In both models, researchers retain full manual override capability. Overriding a decision allows the reviewer to document a specific PRISMA exclusion category (such as Wrong Population, Wrong Outcome, or Ineligible Publication Type).

Step 6: Export PRISMA 2020 Data and Transition to Full-Text Screening

Following title and abstract resolution, export the complete screening dataset:

• Export Formats: Download the decision table as a CSV or Excel file containing bibliographic metadata, criterion-by-criterion evaluations, verbatim quote justifications, screener overrides, and final eligibility determinations.
• PRISMA Flow Diagram Documentation: Use the aggregated export numbers to populate the PRISMA 2020 Flow Diagram:
  • Total records imported from databases and registries.
  • Duplicates removed.
  • Total records screened at title/abstract stage.
  • Records excluded with itemized categorical reasons.
  • Candidate records advanced to full-text retrieval.
• Full-Text Transition: Eligible records move to Elicit's secondary full-text screening and structured data extraction phase. Reviewers can install the Elicit Browser Extension to streamline institutional paywall retrieval.

3. Methodological Performance, Trade-Offs, and Limitations

While Elicit accelerates title and abstract appraisal, biomedical researchers must understand its performance profile to prevent methodological pitfalls.

High Precision vs. Variable Sensitivity

Empirical research evaluating Elicit against Cochrane-standard systematic reviews shows marked distinctions between screening precision and search sensitivity:

• High Precision (~42%): Elicit excels at filtering out irrelevant studies based on clearly defined PICO rules, yielding far higher precision than standard broad search returns (which frequently display precision below 8%-10%).
• Sensitivity Variations by Clinical Complexity: In reviews with binary pharmacological interventions (e.g., single-agent trials with clear RCT designs), Elicit demonstrates robust sensitivity (approaching 70%-94%). However, in complex public health interventions with heterogeneous populations or multi-component interventions, sensitivity can drop significantly if abstract texts omit explicit PICO terminology.

Managing Borderline Scoring Artifacts

Elicit relies on composite scoring thresholds to generate automated Include/Exclude recommendations. Reviewers must observe caution around boundary thresholds:

• Score Boundary Auditing: Independent evaluations have noted instances where studies with identical borderline scores (e.g., 4.8 on a 5-point scale) were bifurcated into inclusion and exclusion due to secondary weighting nuances.
• Mandatory Discrepancy Audits: Biomedical teams should establish a protocol rule that all citations scored within +/- 10% of the inclusion threshold undergo mandatory human inspection rather than automated bulk exclusion.

Compliance with the RAISE Framework

Under the Responsible AI in Evidence Synthesis (RAISE) framework, AI systems must not operate as unverified arbiters in high-stakes healthcare research. Elicit satisfies the requirement for traceability because every judgment is tethered to verbatim abstract text. However, researchers remain accountable for final selection integrity. All systematic reviews utilizing Elicit must disclose the tool's use, model versions, configured PICO prompts, and human verification procedures in their published methodology.

4. Best-Practice Checklist for Systematic Review Teams

1. Separate Search from Screening: Run your gold-standard comprehensive search across PubMed, Embase, and specialty databases; import the deduplicated corpus into Elicit rather than relying on single-source AI discovery.
2. Pilot Test Criteria on 50 Citations: Before running automated screening across the full dataset, calibrate your PICO rules on a 50-paper pilot sample to confirm that rules accurately mirror human expert appraisal.
3. Audit All Exclusions: Because the clinical risk of a false exclusion (missing an eligible trial) outweighs the labor cost of reviewing false inclusions, inspect all AI-excluded abstracts or audit a 20% random sample plus all borderline cases.
4. Preserve Raw Decision Exports: Maintain version-controlled exports of Elicit's screening tables to provide an auditable paper trail for peer reviewers and journal editors.

References

• Blog | Elicit: AI for research & decision-making
• Comparison of Elicit AI and Traditional Literature Searching in Evidence Syntheses Using Four Case Studies - PMC
• Using artificial intelligence for systematic review: the example of elicit - PMC
• The PRISMA 2020 statement: an updated guideline for reporting systematic reviews - PMC
• Development and evaluation of prompts for a large language model to screen titles and abstracts in a living systematic review - PMC