How to Evaluate and Select Literature
This Resource Page will help you:
- Understand the purpose of having inclusion and exclusion criteria for evaluating literature
- Learn the main steps to form inclusion and exclusion criteria for source evaluation
- Develop your own inclusion and exclusion criteria for selecting literature
Introduction
As you begin searching for sources for your assignment, you might notice that there are many available sources – sometimes too many. After doing an initial search, it is important to establish evaluation criteria to select only relevant and high-quality sources.
This page will provide you with practical strategies to develop inclusion and exclusion criteria and help you make informed decisions when selecting the sources for your assignments.
If you haven't yet started looking for sources or your search hasn't been satisfactory, please refer to our resource page on How to Find Sources for Assignments.
What are Inclusion and Exclusion Criteria?
Inclusion and exclusion criteria are parameters used to determine which sources are relevant for inclusion in a literature review or research project.
Inclusion criteria specify the characteristics that must be present in the studies for them to be considered, such as type of participants, interventions, outcomes, or study designs.
Exclusion criteria are the factors that disqualify studies from being included, like date of publication, language, or lack of peer review.
Why Do You Need Inclusion and Exclusion Criteria?
It is very important to have clear inclusion and exclusion criteria for your literature because they:
- Ensure the quality of your selected literature
- Reduce the number of sources you need to read during your source searching phase
- Ensure that the literature is relevant to your research focus and appropriate in scope
- Increase consistency in your search methodology
Using the RADAR Model and Other Criteria to Evaluate Literature
The RADAR model (Mandalios, 2013) is a framework used for evaluating sources. It provides a structured approach to help you assess important characteristics of a source, such as relevance to your research and accuracy of information. It's designed to help you make informed decisions on which sources to include or exclude. Below is what each component of the RADAR model stands for:
R: Relevance
A: Authority
D: Date
A: Accuracy
R: Rationale
In the following sub-sections, we will guide you to develop inclusion and exclusion criteria using the RADAR model and other elements to help you address these two crucial questions for source selection: 1) Can I trust this source? And 2) Is the source content relevant to my topic and scope?
Step 1: Can I Trust this Source?
The first step is to evaluate the reliability of your searched source. For this purpose, you can use the “Authority,” “Accuracy” and “Rationale” criteria from the RADAR model.
- Assess the qualifications and expertise of the author or organization responsible for the source.
- Evaluate the publishers to ensure the credibility of the source.
Guiding questions:
- What type of work is this? (e.g., scholarly article, policy document, blog, news, etc.)
- Where is this work published? (e.g., type of publication, credible publisher)
- Is this work published in a peer-reviewed journal?
- Who are the authors?
- How is the author's expertise specifically relevant to the research topic? (Note: In certain fields or theoretical approaches, lived experience matters more than formal qualifications, and can be seen as a form of expertise.)
- What other works does the publisher present?
Focus on the reliability and truthfulness of the information in the source.
Guiding questions:
- Are the authors’ claims and arguments sufficiently supported by reliable evidence?
- Is there any unverified and biased information included in the source?
- How was the information gathered and presented?
Determine the authors’ purposes and objectives.
Guiding questions:
- What is the purpose of this publication? Why was it published?
- To inform, teach, sell, entertain, or persuade?
Step 2: Is the Source Content Relevant to my Topic and Scope?
Once you have established that your source is reliable, you can determine the relevance of the source to your research topic and scope. For this purpose, you can use the “Relevance” and “Date” criteria from the RADAR model, along with other criteria listed below, to form your inclusion and exclusion criteria.
Determine how well the source addresses your specific research needs.
Guiding questions:
- Does the source have information relevant to my research topic?
- To what extent does the source fit into the scope of my topic?
Determine how current the source is for your topic.
Pay attention to the year when the source was published and set a timeframe (e.g., the most recent 10-20 years). IMPORTANT: Depending on your research topic, you might need to set different timeframes. For example:
- If your topic is on a particularly current or recent issue (e.g., the use of Artificial Intelligence in the classroom; post-Covid issues in education), you may want to include more recent publications.
- If your assignment is about a historical perspective on a topic (e.g., how the idea of critical pedagogy has evolved over time), you may want to include older works.
Guiding questions:
- When was the source published?
- In what way is this relevant for my topic?
- Is the source up-to-date, or does it include outdated information?
Tip: Do not discard a source only because it’s from a few decades ago and regardless of your research topic. You may want to keep older sources if they have been very influential in the field. Many influential works in education and psychology aren’t recent, such as the works of renowned scholars Paulo Freire on critical pedagogy (1960s) or those of Jean Piaget on developmental psychology (1920s-1970s).
Check what language the authors used for their work. For example, you may want to exclude sources written in languages for which it is difficult to arrange a reliable translation
In some cases, you may want to include studies that employ a specific methodology (e.g., qualitative or quantitative studies only) or research designs (e.g., longitudinal studies).
You may want to include only studies conducted in certain settings. This can refer to
- geographical locations (e.g., North America)
- specific contexts (e.g., universities).
Tip: When you find too many sources on your topic, you may need to narrow the scope of your topic (e.g., from schools in Canada to schools in Ontario). In the opposite case, you may need to expand the scope of your topic (e.g., from schools in Ontario to Canada).
You may set criteria for participant demographics depending on the scope of your topic, such as
- Education level (e.g., only participants from post-secondary institutions)
- Gender
- Age group
- Cultural / geographical background
- Other specific criteria (profession, family status, financial status, etc.)
You may want to include works adopting a specific theoretical or conceptual framework. For instance, you may only include studies framed by sociocultural theories, a focus on gender, Indigenous perspectives, etc.
Next Step
For the next step after source evaluation, please refer to our resource page on How to Organize Literature.