3 Tools

3.1 Entrez Programming Utilities

The Entrez Programming Utilities (E-utilities) are a set of nine programs which provide an API to the Entrez database system of the NCBI. Comprehensive information about the E-utilifies is available in the official E-utilities Help.

3.1.1 E-utilities via URL

These programs are server-based and can be deployed using different URLs. These URLs to query Entrez are composed of the base URL https://eutils.ncbi.nlm.nih.gov/entrez/eutils/ to which a string with the desired utility and the query is attached, for instance esearch.fcgi?db=pubmed&term=BMJ[journal]+AND+hernia+AND+2010[pdat]

The results for these queries are usually returned as XML-structured data.

3.1.2 Entrez Direct

The E-utilities are also available directly on a Unix shell under Linux or macOS by using the package Entrez Direct (EDirect).

One of the main advantages of EDirect are the commands that are available on the shell, for example grep, sort, uniq or wc, which allow the results to be processed directly. Also shell scripts can be used to automate the processing.

Example: Retrieve similar articles

The command

elink -db pubmed -id 25554246,29463298 -related -cmd neighbor \
| xtract -pattern LinkSetDb -element Id

will return all PMIDs of publications which PubMed identifies as Similar Articles for the PMIDs 25554246 and 29463298.

Figure 3.1: Similar Articles via Entrez Direct

On PubMed the Similar Articles search can be carried out for only one PMID at a time, whereas using the E-utilities it is possible to search with many PMIDs in a single query.

Unix Shell on Windows

There is a quick and easy way to get access to Linux and its shell even when one is running a (modern) Windows PC: The Windows Subsystem for Linux (WSL) provides an environment to install Linux from within Windows (10 or later).

For more on this, see

EDirect-related resources

3.2 FINER

FINER is an acronym for the five criteria feasible, interesting, novel, ethical and relevant which can be used as guidance when formulating a good clinical research question. See Table 3.1. (Cummings, Browner, and Hulley (2013))

Table 3.1: The FINER criteria

feasible	The research question should not exceed the available resources, i.e. participants, expertise, time and money).
interesting	The research question should in itself be interesting, not only to the researcher, but also to a broader public.
novel	The gain of knowledge and new information should be at the heart of the question. Research usually does not only reiterate already established data unless it is designed as a confirmatory study.
ethical	Proper research has to be ethical and must not bear unacceptable risks for its participants.
relevant	The expected impact of the research on scientific knowledge and clinical processes can be a good measurement parameter for the relevance.

3.3 PICO

PICO is a mnemonic for the concepts Population, Intervention, Comparison and Outcome. Concepts like these are used for defining research questions as well as eligibility criteria for the studies relevant for answering the question.

There are other concepts and mnemonics for different kinds of research questions, such as SPIDER, ECLIPSE, PIRD or PICOS. (Munn et al. (2018), Methley et al. (2014), Eriksen and Frandsen (2018))

3.4 PRESS

Peer Review of Electronic Search Strategies (PRESS) by McGowan et al. (2016) is an evidence-based guideline for the peer review of search strategies. It is mainly focused on systematic review projects, health technology assesments (HTAs) and other kinds of reviews. A main tool for PRESS is a checklist which guides the reader through the process of peer review.

3.5 PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) by Page, McKenzie, et al. (2021) is a set of minimum requirements for the reporting and documentation of systematic reviews. The goal of PRISMA is to set standards for reporting and in doing so increase the overall quality of systematic reviews. A detailed explanatory paper was published by Page, Moher, et al. (2021).

PRISMA also provides tools such as a checklist and a flowchart, which can be used for creating the recommended tables and figures for publication. (M. Rethlefsen and Page (2021))

There are additional extensions of PRISMA for various purposes, such as PRISMA-S for the documentation of systematic literature searches (M. L. Rethlefsen et al. (2021)) or PRISMA-ScR for scoping reviews (Tricco et al. (2018)).

3.6 PubReMiner

The PubMed PubReMiner is an online tool that enables the user to extract information from the results of a PubMed query.

Upon entering search terms (in valid PubMed syntax), PubReMiner summarizes the results of the query by creating tables with the following information:

year of publication
abbreviated journal name
author
free text terms
MeSH terms
substances
publication type
country

The tool can be used to extract data from a set of seed papers simply by entering their PMIDs as a search query.

The PubReMiner is maintained by Jan Koster from the Amsterdam University Medical Centers (Amsterdam UMC).

3.7 Reference Management software

Bibliographic records and similar sets of data such as clinical study metadata can be stored and managed using reference management programs (also called citation management software).

These programs are usually able to perform tasks with the records, such as

import and export of various formats (e.g. RIS, NBIB, BIB, TXT, CSV, XML)
creation, editing and updating
deduplication
retrieval of full-texts
output of citations in various styles

Examples of reference managers

For comparisons of reference management software see also https://mediatum.ub.tum.de/1320978 and https://en.wikipedia.org/wiki/Comparison_of_reference_management_software.

3.8 searchbuildR

searchbuildR is a tool written in R. Its purpose is to support the development of search strategies by identifing search terms from a set of seed papers.(Kapp et al. (2024))

searchbuildR takes a set of references (e.g. as a RIS file) as input and identifies terms, which are overrepresented within this set compared with a large corpus of PubMed references.

Installing and Running searchbuildR

searchbuildR is an R package, which can be installed from GitHub and run locally as a Shiny App in R:

# install and load the "remotes" package which is required to install remotely from github
install.packages("remotes")
library(remotes)

# install and load the searchbuildR package
remotes::install_github("https://github.com/IQWiG/searchbuildR")
library(searchbuildR)

# run the shiny app
run_app()

searchbuildR is developed and maintained by IQWiG.

3.9 TARCiS

The TARCiS Statement by Hirt et al. (2024) provides guidance on Terminology, Applications, and the Reporting of Citation Searching in the context of systematic searching. It offers recommendations as well as a checklist for the most imporant items required to report in a review. Please refer to the section on citation searching for more information about the techniques.

3.10 Yale MeSH Analyzer

The Yale MeSH Analyzer is an online tool for analyzing the bibliographic information of up to 20 PubMed records at once. It can be used to identify relevant search terms from the a set of seed papers.

In principle, upon entering PMIDs into the tool, it creates a summary table containing MeSH terms and author keywords for each publication (see Table 3.2).

Table 3.2: Example of a table created by the Yale MeSH Analyzer

PMID	38644158	30916023	38093118
Title	Ventilatory efficiency as a prognostic factor for postoperative…	Ventilatory inefficiency adversely affects outcomes and longer-…	A retrospective analysis of the association of effort-independe…
Author (Year)	Vetsch T (2024)	Wilson RJT (2019)	Franssen RFW (2023)
MeSH Headings		Aged Aged, 80 and over	Aged
		Carbon Dioxide / me Colorectal Neoplasms / pp Colorectal Neoplasms / su*	Colorectal Surgery*
	Elective Surgical Procedures* / ae Exercise Test / mt	Exercise Test / mt* Exercise Tolerance	Exercise Test
		Female	Female
	Humans	Humans	Heart Failure* Humans
		Lung / pp*
		Male	Male
		Oxygen Consumption / ph	Oxygen Consumption
	Postoperative Complications* / ep Prognosis	Postoperative Complications / di Postoperative Complications / ep* Postoperative Complications / pp Pulmonary Ventilation / ph*	Postoperative Complications / ep Prognosis
		Retrospective Studies Risk Factors	Retrospective Studies
		Survival Analysis
		United Kingdom / ep
Author Keywords	VE/VCO(2) exercise testing major surgery postoperative complications postoperative mortality prediction models preoperative assessment ventilatory efficiency	cardiopulmonary exercise testing colorectal surgery mortality perioperative risk factors pre-operative evaluation ventilatory inefficiency	Abdominal surgery Anaerobic threshold Cardiopulmonary exercise testing Oxygen uptake efficiency slope Peak oxygen uptake Preoperative care Preoperative risk assessment

The Yale MeSH Analyzer is maintained by the Cushing/Whitney Medical Library at Yale University.

Cummings, Steven R., Warren S. Browner, and Stephen B. Hulley. 2013. “Conceiving the Research Question and Developing the Study Plan.” In Designing Clinical Research, edited by Stephen B. Hulley, Steven R. Cummings, Warren S. Browner, Deborah G. Grady, and Thomas B. Newman, 4. ed., 14–22. Philadelphia: Lippincott Williams & Wilkins. https://swisscovery.slsp.ch/permalink/41SLSP_UBE/99pfpl/alma99117040391205511.

Eriksen, Mette Brandt, and Tove Faber Frandsen. 2018. “The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review.” J Med Libr Assoc 106 (4): 420–31. https://doi.org/10.5195/jmla.2018.345.

Hirt, Julian, Thomas Nordhausen, Thomas Fuerst, Hannah Ewald, and Christian Appenzeller-Herzog. 2024. “Guidance on terminology, application, and reporting of citation searching: the TARCiS statement.” BMJ 385 (e078384). https://doi.org/10.1136/bmj-2023-078384.

Kans, Jonathan. 2013. “Entrez Direct: E-utilities on the Unix Command Line.” In Entrez Programming Utilities Help [Internet]. Bethesda (MD): National Center for Biotechnology Information (US).

Kapp, Claudia, Naomi Fujita-Rohwerder, Jona Lilienthal, Wiebke Sieben, Siw Waffenschmidt, and Elke Hausner. 2024. “The searchbuildR shiny app: A new implementation of the objective approach for search strategy development in systematic reviews.” Cochrane Evidence Synthesis and Methods 2 (6): e12078. https://doi.org/10.1002/cesm.12078.

McGowan, Jessie, Margaret Sampson, Douglas M. Salzwedel, Elise Cogo, Vicki Foerster, and Carol Lefebvre. 2016. “PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement.” J Clin Epidemiol 75: 40–46. https://doi.org/10.1016/j.jclinepi.2016.01.021.

Methley, Abigail M., Stephen Campbell, Carolyn Chew-Graham, Rosalind McNally, and Sudeh Cheraghi-Sohi. 2014. “PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews.” BMC Health Serv Res 14 (1): 579. https://doi.org/10.1186/s12913-014-0579-0.

Munn, Z., C. Stern, E. Aromataris, C. Lockwood, and Z. Jordan. 2018. “What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences.” Journal Article. BMC Med Res Methodol 18 (1): 5. https://doi.org/10.1186/s12874-017-0468-4.

Page, Matthew J., Joanne E. McKenzie, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, Larissa Shamseer, et al. 2021. “The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.” BMJ 372. https://doi.org/10.1136/bmj.n71.

Page, Matthew J., David Moher, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, Larissa Shamseer, et al. 2021. “PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews.” BMJ 372. https://doi.org/10.1136/bmj.n160.

Rethlefsen, Melissa L., Shona Kirtley, Siw Waffenschmidt, Ana Patricia Ayala, David Moher, Matthew J. Page, Jonathan B. Koffel, et al. 2021. “PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews.” Syst Rev 10 (1): 39. https://doi.org/10.1186/s13643-020-01542-z.

Rethlefsen, Melissa, and Matthew J. Page. 2021. “PRISMA 2020 and PRISMA-S: Common Questions on Tracking Records and the Flow Diagram.” MetaArXiv. https://doi.org/10.31222/osf.io/439ju.

Tricco, A. C., E. Lillie, W. Zarin, K. K. O’Brien, H. Colquhoun, D. Levac, D. Moher, et al. 2018. “PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation.” Journal Article. Ann Intern Med 169 (7): 467–73. https://doi.org/10.7326/M18-0850.