Report mixed studies reviews

This page presents a proposal for a template to report mixed studies reviews (adapted from the PRISMA statement).

With regard to meta-narrative and realist syntheses, researchers can use the RAMESES templates (Wong et al., 2013a; Wong et al., 2013b).

 

TITLE

 

• Propose a short take-home title. The title should explicitly state that the review included different type of evidence. You can use different terms such as "mixed studies review, “review of qualitative and quantitative evidence”, "mixed methods review", etc. 

 

ABSTRACT

 

• Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.

 

INTRODUCTION AND OBJECTIVES

 

• Describe the rationale for the review (e.g., a health problem) in the context of what is already known (e.g., an existing literature review paper or a reference book chapter).
• Formulate questions and/or objectives (qualitative (QUAL) or quantitative (QUAN) or both) being addressed by your review (see 'formulate a review question' for more details)

 

For example, you can include the key elements below (when applicable):

• Context: Setting, population or participants
• Process: Program, intervention, exposure 
• Outcomes: Process-related outcomes, final outcomes
• Comparison [when enough studies]
• Process duration or length of follow-up
• Other elements when needed, e.g., supporting a qualitative question

 

METHODS

 

Protocol and registration

• Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number. 

 

Justification

• Justify the use of a review of qualitative and quantitative evidence. 

 

Eligibility criteria

• Specify the inclusion and exclusion criteria and the rationale for supporting these criteria. 

 

Information sources

• Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched. 

 

Search 

• Present full electronic search strategy for at least one database (e.g., in an appendix), including any limits used, such that it could be reproduce. 
• Describe the process for removing duplicates.
• Specify the involvement of a librarian, if applicable.

 

Study selection  

• Describe the process for selecting studies (e.g., screening based on titles and abstracts, and eligibility based on full-text, number of reviewers, software used). 

 

Data collection process

• Describe the method of data extraction from included studies (e.g., number of reviewers involved, piloted forms, etc.).
• List the data extracted.
• If applicable, state any processes for obtaining and confirming data from investigators of included studies (e.g., initial email to the first author and reminder email).

 

Appraisal

• Describe (steps and key elements of) the process for appraising included studies (e.g., tool(s) used, number of reviewers involved), and specifically for assessing the methodological quality of included qualitative, quantitative, and mixed methods studies. 
• Specify how results of this appraisal are used in the synthesis. For example, include all studies with description of their methodological quality or contrast synthesis of ‘lower quality’ studies vs. ‘higher quality’ studies using sensitivity analysis.

 

Synthesis  

• Describe the synthesis design used. 
• Describe and justify the synthesis method(s) used (e.g., quantitative content analysis, meta-analysis, thematic synthesis, etc.).

 

Additional analysis

• Describe methods of additional analyses (e.g., sensitivity or subgroup analyses), if done. 

 

RESULTS or PRELIMINARY RESULTS (e.g. preliminary results based on the synthesis of a small sample of included studies)

 

Study selection

• Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage.
• Summarize this in a flow diagram (see Illustration below).
• Give numbers of quantitative, qualitative and mixed methods studies included.

 

Study characteristics

• For each study, present characteristics for which data were extracted (e.g., tables of characteristics of included studies – see Appendices) and provide the citations.
• Specify common information across all included studies.  
• Describe the studies including their heterogeneity (variability associated with differences between studies).  

  

Result of appraisal

• Present data on the methodological quality or risk of bias of included studies based on the appraisal done.

 

Results of synthesis

Present results of synthesis. 

If QUAL synthesis

• In the text, briefly summarize the main themes or categories and refer to the appendix. 
• Appendix (table, figure, or matrix): For each study, present the themes or categories identified.

 

If QUAN synthesis

• In the text, briefly summarize the data and refer to the appendix. 
• Appendix (table, figure, or matrix): For all key variables, present, for each study: (a) simple summary data for each intervention group and (b) effect estimates and confidence intervals, ideally with a forest plot.

 

If QUAL and QUAN syntheses

• Present both
• If applicable, present the results of the integration of both syntheses.

 

Additional analysis

• Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses).

 

DISCUSSION

 

Summary of evidence 

• Provide an overall summary of results (take-home messages) from the qualitative and/or quantitative synthesis.
• State the main results for each main theme or category, and/or key process/outcome variable.
• Consider their relevance and importance for knowledge users (e.g., health care providers, managers, and decision/policy makers).
• Take into account the methodological quality across studies (when applicable).
• Describe insight gained from the integration of qualitative and quantitative evidence.

 

Contribution

• Describe the contribution of the present review (compared to what is already known) with respect to:
• Review methods,
• Scientific knowledge,
• Practice, program planning and evaluation, policy making, or else.

Limitations 

• Specify any element that may affect the cumulative evidence.
• Discuss limitations at the study and process/outcome levels, e.g., lack of rich data for QUAL synthesis, and risk of bias for QUAN synthesis, and their potential consequences on the results
• Discuss limitations at the review level, e.g., incomplete retrieval of relevant studies (selective publication of reports regarding studies with positive results), and limited reporting (selective reporting of information about included studies), and their potential consequences on the results.

 

Conclusions

• Provide a general interpretation of the results in the context of other evidence, including implications for knowledge users, e.g., a major recommendation
• State implications for future research

 

ACKNOWLEDGEMENTS

 

• Describe sources of funding and other support (e.g., supply of data) and the role of funders in the review (methods and/or interpretation of results) when applicable
• Acknowledge any information about potential conflict of interest

 

REFERENCES

 

• List all the references cited in the text.

 

APPENCICES (tables, matrices, figures, boxes)

 

In addition to tables presenting synthesis results and other relevant information, two illustrations are required: a flow diagram, and a 'Characteristics of the included studies' table.

 

FLOW DIAGRAM

 

Suggested flow diagram for mixed studies reviews. The proposed diagram below was adapted for mixed studies reviews from Moher et al. (2009). 

 

 

 

DESCRIPTION OF INCLUDED STUDIES

 

Summarize key characteristics of the included studies in a table, called ‘Characteristics of included studies’ table. For each study, this may be developed using a descriptive paragraph. While the table must be attached to any publication, the textual description may be provided as an appendix (online supplementary document). ‘Characteristics of included studies’ tables and descriptive paragraphs must contain information on the characteristics for all included studies such as: 

• Sources: First author, Year, and ID number of the study, e.g., from the reference manager software.
• Study design
• Number of participants and participants (age, sex, other)
• Setting (e.g., types and number of organizations)
• Intervention/exposure (if applicable), or relevant considerations for non-intervention studies
• Outcomes (if applicable), or relevant considerations for non-intervention studies (study results should be presented in a separate table if space allows)
• Quality rating (if applicable)
• Further comments on key characteristics of the study (according to the review question)

 

Different tables may be needed: For example, tables for quantitative studies, for qualitative studies, and for mixed methods studies.

 

Templates for a ‘Characteristics of included studies’ table 

First author Journal Year ID #	Study design Setting Participants Data collection Data analysis	Intervention or Program Comparison intervention (if applicable) Or relevant information (e.g., exposure, condition)	Outcomes (if applicable)  Or relevant information	Critical appraisal

 

 

TABLES AND FIGURES ON RESULTS OF THE SYNTHESIS 

• Provide illustrations for results of the synthesis such as:
• Quantitative synthesis: statistical summary, descriptive table, forest plots, etc.
• Qualitative synthesis: list of categories, list of themes and subthemes, concept maps, framework, etc.

 

Click here to download a PDF version of this guidance. 

 

 

REFERENCES

• Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews andMeta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097
• Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. 2013. RAMESES publication standards: meta-narrative reviews. BMC Medicine. 11:20
• Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. 2013. RAMESES publication standards: realist syntheses. BMC Medicine. 11:21