Systematic Reviews, Scoping Reviews & Other Evidence Synthesis Projects

Before you even start the first step of a systematic review project, you need to ask yourself three questions:

1. Do I have time? A systematic review can take an average of 18 months of preparation and work. These reviews are not quick projects.
2. Do I have a team? Systematic reviews cannot be completed alone! In general, your team should consist of subject experts; reviewers who will screen abstracts and read full text; a project leader to coordinate and write the final report; and perhaps other members with specific expertise, such as a librarian or a statistician
3. Have I created a protocol? Once you have figured out your research question and defined your inclusion and exclusion criteria, you need to create and ideally register a protocol, which is a plan for your review. Once finalized, this plan should not be changed. Your protocol should be finalized by the time you are ready to start data extraction. Check out the Protocol tab on this guide to find out more about protocols.

JBI outlines the 8 steps that it takes to complete a systematic review:

1. Formulating a review question
2. Defining inclusion and exclusion criteria
3. Locating studies through searching
4. Selecting studies for inclusion
5. Assessing the quality of studies
6. Extracting data
7. Analyzing and synthesizing the relevant studies
8. Presenting and interpreting the results, potentially including a process to establish certainty in the body of evidence (through systems such as GRADE)

You need a clear research question in order to do a systematic review. The question will dictate how you construct your search strategy.

Use the PICO (Population, Intervention, Comparison, Outcome) framework to identify key concepts and clarify the question your systematic review is asking, as well as your rational for your questions.

You may have more than four concepts for your research and that is okay as long as they are appropriate for your research question.

Your inclusion and exclusion criteria define the boundaries of your systematic review. When you are in the review stage of your review, you will use the inclusion and exclusion criteria to determine whether or not a study should be included in your final analysis. Some elements that are often covered in inclusion and exclusion criteria are:

• Study types (RCTs, clinical trial studies, case studies, observational studies, etc.)
• Population (e.g. human studies vs. animal studies)
• Concepts in your research 
• Specific interventions/treatments/comparisons

Your research question will dictate your search terms. The following are some searching best practices:

• You should have multiple search terms per concept. Most of the time there are multiple ways to refer to the same topic or concept and you should attempt to exhaust all possible ways to refer to your term.
• Use field codes. 
  • Field codes tell databases where to search for your term. Common field codes for systematic searching are text word (will search most places in a record) and controlled vocabulary/subject headings (e.g. Mesh Terms)
    • Check out our FAQ on why you should use both controlled vocabulary AND text word searching for each concept.
• Test your search terms
  • Take some time to test the individual terms you want to search to see if they are bringing back the kind of results you are intending for them to bring back.
• Keep track of your searching
  • Create a document where you can list your searches and what works and doesn't work. The systematic review process is a long one and you will need your searching documentation to keep track of your terms and how your results change over time. 
• Ask a librarian for help! Librarians are expert searchers. They can help you develop your search. Click on the icon in this box to schedule a virtual appointment for help.

Once you have your search results from multiple sources, you can start going through them to determine which will be included in your review. Generally this is done in two steps:

1. A title/abstract review to quickly determine whether or not a source has any relevance to your topic
2. A full text screening for inclusion or exclusion based on the criteria you created in your protocol.

Best practice suggests that TWO reviewers blindly screen the studies and a third reviewer be available as a blinded tie-breaker.

There is both free and payment-based software that can help you with this process. Below are some popular options.

Each study that is included in your review should be assessed for quality using critical appraisal guides or quality checklists. There are many critical appraisal guides out there and you can choose the one that is appropriate for your review. You can find general checklists, as well as guides for assessing the quality of certain types of studies.

The JBI Manual has critical appraisal guides for systematic reviews on various types of evidence. 

You can extract the relevant data from your studies using systematic review software or something like a spreadsheet. Some common data elements that are extracted are:

• Title
• Author
• Year
• Journal
• Research question 
• Methodology
• Conceptual framework
• Conclusions

The next step is to analyze the data that you extracted from all of your studies. This step is where you will be able to come to conclusions regarding your research question. Ultimately, you will need to combine the data into a coherent narrative that will help the reader understand the body of evidence that you analyzed. 

The Boxer Library and JBI recommend following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) to report your systematic review.