There are many types of systematic review. What they all have in common is the use of transparent and reproducible methods that are defined before the search begins. There is no ‘best’ way to synthesise systematic review evidence, and the most suitable approach will depend on factors such as the nature of the review question, the type of intervention and the outcomes of interest.
Systematic review is a specific methodology that locates existing studies, selects and evaluates contributions, analyses and synthesizes data, and reports the evidence in such a way that allows reasonably clear conclusions to be reached about what is and is not known. A systematic review should not be regarded as a literature review in the traditional sense, but as a self-contained research project in itself that explores a clearly specified question, usually derived from a policy or practice problem (Denyer & Tranfield, 2009).
Refer to Review Methodologies:
Producing a systematic review. Buchanan & Bryman (Eds.), The Sage handbook of organizational research methods (pp. 671–689). Sage Publications Ltd.
Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family : exploring review types and associated information retrieval requirements. https://doi.org/10.1111/hir.12276
Meta-analysis Framework
PRISMA Framework- PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.
This Shinyapps tool allows you to produce a flow diagram for your own review that conforms to the PRISMA2020 Statement. You can provide the numbers in the data entry section of the 'Create flow diagram' tab.
PRISMA Meta-Analysis - Checklist of Items to Include When Reporting A Systematic Review Involving a Network Meta-analysis.
The three main databases:
Grey literature is information such as reports, conference papers, theses (see below) and other documents which have not been published commercially. You might become aware of relevant documents as you conduct your searches, or your supervisor or another researcher might suggest documents to you. You can also conduct searches specifically aimed at finding grey literature using tools such as:
SSRN - Social Science Research Network, is a repository for preprints devoted to the rapid dissemination of scholarly research in the social sciences, humanities, life sciences, health sciences, and more.
OAISter - A search tool for open access materials, harvested worldwide from the repositories of universities, governmental bodies, other research institutions etc. Google OAIster to find it.
Proquest Dissertations & Theses Global - Also considered a form of grey literature, it is worth seeing if any theses relevant to your topic have already been produced.
Web searches - Many such documents produced recently can be found on the web in pdf format. Using Google, you can add filetype:pdf to the end of your search to find only pdf files.
In some fields books will be a more important source of information than in other fields. Good sources to search for books include Bond Library, WorldCat, Trove and Google Books.
The bibliography of any relevant paper you find is a potential source of other relevant papers. Go through these bibliographies systematically. Tools like Scopus and the Web of Science can be useful for this.
Another source of relevant papers is to follow the citation trail forward in time, ie. to identify any papers which have cited the papers you know are relevant. Tools which allow you to do this include Google Scholar, Scopus, Web of Science, and the Bond Library website.
Compiled from the article: The next generation discovery citation indexes — a review of the landscape in 2020 (I) by Aaron Tay, Academic librarian Singapore Management University.
scite Search is a Brooklyn-based startup that helps researchers better discover and evaluate scientific articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contradicting evidence.
Semantic Scholar - AI-Powered Research Tool - provides free, AI-driven search and discovery tools, and open resources for the global research community. We index over 200 million academic papers sourced from publisher partnerships, data providers, and web crawls.
Scilit - Scientific Literature - Rankings and comparisons available between journals and publishers. Developed and maintained by the open access publisher MDPI.Scilit is a comprehensive, free database for scientists using a new method to collate data and indexing scientific material. Our crawlers extract the latest data from CrossRef and PubMed on a daily basis.
Scinapse - Academic search engine - Scinapse's algorithm offers the best search results for academic papers based on the published date, citations, publisher, and many more
The Lens serves global patent and scholarly knowledge as a public resource to make science- and technology-enabled problem solving more effective, efficient and inclusive.
A common procedure within the area of business management and accounting follows a systematic literature review approach together with a bibliometric analysis in order to identify the relevant papers and the most important themes in the field. (see, Bartolacci et al., 2020; Caputo et al., 2018; Kumar et al., 2019; Pizzi et al., 2020; Xu et al., 2018).
The systematic review follows a rigorous, replicable method that minimises bias (Tranfield et al., 2003). This example adopts a systematic literature review based on the three steps outlined by Tranfield et al. (2003):
⁃Planning the review: establishing the research question and developing a review protocol
⁃Conducting a review: searching and selecting relevant papers using the inclusion and exclusion criteria
⁃Reporting and dissemination: data extraction and analysis
This approach is found in other systematic reviews (see, Boiral et al., 2018a, Boiral et al., 2018b; Delbufalo, 2012; Klewitz and Hansen, 2014; Silva et al., 2019). The aim of the systematic literature review is to detect the main studies of the field of knowledge and identify the possible research gaps (Tranfield et al., 2003).
Refer to Denyer, D., & Tranfield, D. (2009). Producing a systematic review. In D. A. Buchanan & A. Bryman (Eds.), The Sage handbook of organizational research methods (pp. 671–689). Sage Publications Ltd.
Key research questions defined:
RQ1: Which are the main peer-reviewed publications within the current literature in this field?
RQ2: Who are the most influential authors and journals in this field?
RQ3: What is the intellectual structure of research in this field?
RQ4: Which are the main research themes in this field?
RQ5: How can the research move forward in this field?
Databases
List the databases appropriate to search for your research topic below.
Search terms
The first phase of the systematic review process is creating the most effective search string of terms. Read other SLR articles on how authors have crafted their most effective search keywords. Here is an example from Grover, I., O’Reilly-Wapstra, J., Suitor, S., & Hatton MacDonald, D. (2023). Not seeing the accounts for the forest: A systematic literature review of ecosystem accounting for forest resource management purposes. Ecological Economics, 212. https://doi.org/10.1016/j.ecolecon.2023.107922
Search string of terms: (“System of Environmental-Economic Accounting” OR “SEEA”) AND (“ecosystem accounts”) AND (“forest ecosystem” OR “forest estate” OR “forestry”) AND (“timber” OR “harvest”). Search string terms included both (“System of Environmental-Economic Accounting” and “SEEA”) to capture publications that may use full name or only the acronym in isolation. To focus on SEEA EA consistent applications in forest ecosystem contexts, the search had to return with one of the following: (“forest ecosystem” OR “forest estate” OR “forestry”) as “forest” is too broad a search term. The terms (“timber” OR “harvest”) were included to narrow the result field to publications that evaluated forest ecosystems where timber processing was occurring.
Summarise your research topic in one or two sentences:
|
Example: |
What are the main search terms you can identify in your research topic?:
|
Example: |
What are synonyms, alternative spellings for those terms?
|
Example: health behavior nutrition diet |
You can now start searching the databases you've listed using the search terms you've come up with. At this point you are just exploring the literature. When you find relevant articles look for any good search terms you might have missed, and add them to your list above. Then try adding those new terms to your searches. Keep going until you've identified all the relevant search terms, i.e. you're not coming across any new terms you haven't already identified.
If you're not sure about using the databases, or don't seem to be finding the results you were expecting, get in touch with your Faculty Librarian.
When you're finished your scoping searches you’ll have a complete list of terms to search on:
means-end theory, laddering, means end chain approach, value based marketing, health behaviour, health behavior, nutrition, weight loss, diet, food, exercise, physical activity. |
Database searches
Search strategy
You can now make a few simple modifications to your list of search terms to help improve the results you'll get from database searches.
Truncation
Allows you to use a symbol (such as an asterisk) so your search will pick up alternative word endings.
For example, searching on diet* will also find results which use the words diets, dietary, dietetics, and dietician.
Phrase searching
If you only want results which mention a particular phrase (i.e. the terms must appear together and in the right order) put double quotes around the phrase, e.g. "value based marketing".
Group the synonyms together
You'll notice that a lot of the search terms you've identified are alternative ways of saying the same thing, i.e. they're synonyms for the same concept. Group these using OR, and brackets:
("means end theory" OR laddering OR "means end chain" OR "value based marketing") (health behav* OR nutrition OR "weight loss" OR diet* OR food OR exercise OR "physical activity") |
Join it all together
Put AND between your concepts to join them together.
("means end theory" OR laddering OR "means end chain" OR "value based marketing") AND (health behav* OR nutrition OR "weight loss" OR diet* OR food OR exercise OR "physical activity") |
This is a good representation of your ideal search, that most databases will understand. This is called a ‘search strategy”.
Subject headings
Many databases have a standardised system of subject headings. Each paper that is about a particular concept will be tagged with the same subject heading, regardless of the terminology the author has used. Examples of databases which have a thesaurus feature are EBSCO Business Source Complete and Proquest. Searching the Business Source Complete thesaurus for 'value based marketing' reveals that the closest subject heading is 'value-based management'. If the database you're searching has a thesaurus, search it to find which of your search terms has a standardised subject heading, and incorporate these in your search.
Bibliometrix package in R is an open-source tool for quantitative research in scientometrics and bibliometric method analysis. Bibliometrix package provides various routines for importing bibliographic data from SCOPUS, Clarivate Analytics' Web of Science, PubMed, Digital Science Dimensions and Cochrane databases, performing bibliometric analysis and building data matrices for co-citation, coupling, scientific collaboration analysis and co-word analysis. Developed by Massimo Aria and Corrado Cuccurullo.
Covidence - is a web-based software platform that streamlines the production of systematic reviews and other research reviews that require screening citations and full text, assessing risk of bias, or extracting study characteristics and outcomes.(via Bond Library - use Bond email for creating an account). Read: Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey
VOSviewer - is a bibliometric software tool for constructing and visualizing networks. These networks may for instance include journals, researchers, or individual publications, and they can be constructed based on citation, bibliographic coupling, co-citation, or co-authorship relations. VOSviewer also offers text mining functionality that can be used to construct and visualize co-occurrence networks of important terms extracted from a body of scientific literature. (Open Access)
As stated in Gutiérrez-Nieto et al., 2022, bibliometric studies perform statistical analyses of scientific publications (Pritchard, 1969) to obtain objective, impartial information on a specific field of research (Zupic and Čater, 2015). Moral-Muñoz et al. (2020) analyze different software tools for conducting bibliometric analysis: Bibexcel, Biblioshiny, BiblioMaps, CiteSpace, CitNetExplorer, SciMAT, Sci2 Tool and VOSviewer. The choice of the VOSviewer software was motivated by the quality and the visualization of the final rendering and by the variety of the supported format for the input and the output of data. VOSviewer is a tool for creating maps based on bibliographic databases and for visualizing and exploring these maps. It has been developed in Java programming language. It can be freely downloaded from www.vosviewer.com (Van Eck and Waltman, 2010).
Publish or Perish is a desktop application that analyzes academic citations using data from Google Scholar. It can also use a variety of data sources to obtain the raw citations, then analyzes these and presents a range of citation metrics, including the number of papers, total citations and the h-index. Developed by Anne-Wil Harzing.
Jamovi - is a free and open-source graphical user interface for the R software that targets beginners looking to point-and-click their way through analyses. It is available for Windows, Mac, Linux, and even ChromeOS.
The following provide a list of review processes and the selected databases to be used for the literature searches. An idea is to look at recent SLR's as a guide to using "search string of terms" - what databases were used - preferred reporting items for SRL & methodologies ie Meta-Analyses (PRISMA).
The first example uses the databases Scopus, Business Source Complete, and Web of Science.
Wenker, Kilian. (2022). A systematic literature review on persuasive technology at the workplace. Patterns. 8(3), 1-9. https://doi.org/10.1016/j.patter.2022.100545
Carrera-Rivera, A., Ochoa-Agurto, W., Larrinaga, F., & Lasa, G. (2022). How-to conduct a systematic literature review: A quick guide for computer science research. MethodsX, , 101895. https://10.1016/j.mex.2022.101895
Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285-296. https://10.1016/j.jbusres.2021.04.0706x
Nikki Cornwell, Christopher Bilson, Adrian Gepp, Steven Stern & Bruce J Vanstone (2022) The role of data analytics within operational risk management: A systematic review from the financial services and energy sectors, Journal of the Operational Research Society, DOI: 10.1080/01605682.2022.2041373
Maitland, Hills, L.& Rhind, D.(2015).Organisational culture in sport – A systematic review. Sport Management Review, 18(4), 501–516. https://doi.org/10.1016/j.smr.2014.11.004
Adams, R., Jeanrenaud, S., Bessant, J., Denyer, D. and Overy, P. (2016), Sustainability-oriented Innovation: A Systematic Review. International Journal of Management Reviews, 18: 180-205. https://doi-org.ezproxy.bond.edu.au/10.1111/ijmr.12068
Tod, E., Shipton, D., McCartney, G., Sarica, S., Scobie, G., Parkinson, J., Bagnall, A., Manley, J., Cumbers, A., Deas, S., & de le Vingne, J. (2022). What is the potential for plural ownership to support a more inclusive economy? A systematic review protocol. Systematic Reviews, 11, 1-9. https://doi.org/10.1186/s13643-022-01955-y
Toorajipour, R., Sohrabpour, V., Nazarpour, A., Oghazi, P., & Fischl, M. (2021). Artificial intelligence in supply chain management: A systematic literature review. Journal of Business Research, 122, 502–517. https://doi.org/10.1016/j.jbusres.2020.09.009
Iranmanesh, M., Ghobakhloo, M., Nilashi, M., Tseng, M.-L., Yadegaridehkordi, E., & Leung, N. (2022). Applications of disruptive digital technologies in hotel industry: A systematic review. International Journal of Hospitality Management, 107. https://doi.org/10.1016/j.ijhm.2022.103304
Mapping impact investing: A bibliometric analysis - We select the sample using a three-step process. In the first step, we identify our sample of papers, based on searches on the Elsevier Scopus database. Due to more extensive coverage over peer-reviewed articles from 1970, the Scopus database is widely used in bibliometric studies (Ball and Tunger, 2006; Fahimnia et al., 2015; Feng et al., 2017, Mishra et al., 2018). Moreover, Scopus is particularly comprehensive, since it covers several publishers and fields of study (Vieira and Gomes, 2009).
In the spirit of reconciliation, Bond University acknowledges the Kombumerri people, the traditional Owners and Custodians of the land on which the university now stands. We pay respect to Elders past, present and emerging. Read more
CRICOS Provider Code 00017B | TEQSA Provider ID PRV12072