28 Evaluating for Relevancy

Relevant sources are those that pertain to your research question. You’ll be able to identify them fairly quickly by reading or skimming particular parts of sources and maybe jotting down little tables that help you keep track. We’ll show you how below, including where to look in specific kinds of sources and what questions to ask yourself as you do.

One thing to consider early on as you make inferences about relevancy is the effect that timeliness– called a source’s currency–should have on deciding whether a source is relevant. Sometimes timeliness has a lot to do with relevancy; sometimes it is less important. Your research question and your discipline will determine that.

For instance, if your research question is about the life sciences, you probably should consider only the most recent sources relevant for citing because the life sciences are changing so quickly. There is a good chance that anything but the most recent sources may be out of date. So it’s a good idea to aim for life sciences sources no more than 5 years old. (An example of a discipline that calls for even newer sources is computer security.)

Sometimes emergencies change the schedule of what is recent enough. For instance, when the Covid-19 pandemic started, it was incredibility important for scientists to share their research information as quickly as possible. At that time, scientific information about Covid-19 could become outdated in weeks or months–before the peer review process was barely started.

Lives were at stake and for that reason, scientists started publishing their new research results on Covid-19 as preprints—publications of results that had not yet been peer-reviewed–in an attempt to have them be useful faster. Nonetheless, after preprint publication, the peer review process continued for much of that research.

But pre-prints didn’t start with the Covid pandemic. Around for more than 30 years and now at Cornell University, arXiv is a free distribution service and an open-access archive for more than two million scholarly articles first published as preprints in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics. Materials on the site are not peer-reviewed by arXiv itself. (arXiv is pronounced archive.)

Before using preprints as sources, talk with your professor about whether she or he recommends their use in your situation.

Many sciences other than life sciences primarily use newer content under 10 years old. But not always. That’s because the history department is not alone in valuing older content. For instance, mathematics is a discipline that makes heavy use of older content. So how important the currency of your sources is will depend on your research question and your discipline. Your professor can guide you about your own situation.

In most cases, it’s best not to use a hard and fast rule about how current your sources have to be. Instead, consider your discipline and research question and do some critical thinking. For example, suppose your research question is about the Edo Period in Japan (1603-1868) or about Robert Falcon Scott, who explored the Antarctic from 1901-1913. In these cases, an item from 1918 might be just as useful as an item from 2018 (although new information may have been found in the 100-year gap). But something from 1899 about Antarctica or from 1597 about Japan would not be current enough for these research questions.

These examples also give you two more clues about how to treat the timeliness or currency of sources as you consider relevance:

• Because of how long ago they lived or occurred, it would be unusual for many sources on Robert Scott or the Edo Period to have been published very recently. So, unlike sources for the life sciences, whether a source is very recent should probably not determine its relevancy to research questions about Scott or the Edo Period.
• Primary sources might be considered especially relevant to many humanities and other non-science research questions. For disciplines in the humanities, the phrase primary sources refers to sources created at the same time as something under study—in this case, things such as Scott’s diaries and expedition photographs, as well as paintings, literature, clothing, and household items from the Edo Period. They go a long way to explain faraway people and times. On the other hand, when science disciplines use the phrase primary source, they usually mean where they primarily find the information they consider valid—in research journals.

Time-Saving Tips

Instead of thinking you have to read all of every source in order to figure out whether it’s relevant, read or skim only parts of each source. If you’re looking at the right parts, that should give you enough information to make an educated guess about relevancy and currency.

But what should you be looking for as you do that reading and skimming? One way to figure that out is to first parse your research question so that you can figure out its main concepts. (This is like identifying main concepts in your research question in order to search precisely, as we advise in Chapter 4.)

For instance, suppose your research question is: How does having diverse members in a group increase the critical thinking of the group?

What are this question’s main concepts? Our answer is: group diversity and critical thinking.

So when trying to judge which sources are relevant to these main concepts, you would assess whether each source you’ve found pertains to at least one of these main concepts. We recommend you jot down a little table like the one in the example below to keep track of which sources address each main concept.

To be considered relevant to your research question, a source wouldn’t necessarily have to cover all of your main concepts. But finding sources that do is ideal. Otherwise, you just have to make do with what you’ve got. Don’t forget that each source would have to pass the currency test, too, if the currency is important to your research question. So it saves time to record your decisions about the sources’ currency on your tables, too.

If you do make little tables for relevance, it’s probably a good idea to hang on to them. You might find them helpful later in your research process.

Where to Look in Websites, Articles, and Books

The information below tells where to look and what questions to ask yourself to assess the relevancy of articles, books, and websites. The name of a source seldom tells you enough about its relevance, so whatever you do, don’t stop evaluating after looking only at a website’s name or the title of another source.

Save time by looking in particular places in sources for information that will help you figure out whether the source is relevant to your research project. Much of our advice below comes from “Speedy Reading” in The Craft of Research, second edition, by Wayne Booth, Gregory Colomb, and Joseph Williams, University of Chicago Press, 2003, pp. 108-109.

On a website, check the name of the website and its articles for clues that they contain material relevant to your research question. Consider whether time should have an impact on what information can be considered relevant to your research question. If so:

• Skim any dates, datelines, What’s New pages, and press releases to see whether any website content works with the time considerations you need.
• Check for page creation or revision dates that you find. What you’ve already learned from other sources can also help. For instance, you may know that the information covered by a particular website, which seems relevant, is no longer considered the latest thinking. In that case, you could mark it irrelevant on your little table.
• Skim any site map and index on the website for key words related to your research question.
• Try the key words of your research question in the search box. Do you see enough content about your keywords to make you think parts of the website could be helpful?

For a research journal article, magazine article, or newspaper article, think about the title. Does it have something to do with your research question? Consider whether time should have an impact on what sources can be considered relevant. If so:

• Is the publication date of any of these three kinds of articles within your parameters?
• Skim the glossary of a journal article to see whether the article works with the time considerations you need. For instance, if there is a time period in your research question, does the article address the same time period or was it created during that time period?
• Look at the abstract and section headings in a journal article or the early parts of a newspaper or magazine article to locate the problem or question that the article addresses, its solution, and the outline of the article’s argument for its main claim. Can those help answer your research question? Do they make it seem as if the article will give you information about what others have written about your research question? Do they offer a description of the situation surrounding your research question?
• Do the journal article’s introduction and conclusion sections help you answer your research question and/or offer a description of the situation surrounding your question so you can explain in your final product why the question is important?
• Check whether the journal article’s bibliography contains keywords related to your research question. Do the sources cited by the bibliography pertain to your research question? (Bibliographies are especially good places to look for sources.)
• If you decide the newspaper or magazine article is relevant, look at sources quoted or otherwise identified within it. Those may be additional sources for you.

For a book (perhaps in its library catalog listing), check whether the title and/or subtitle indicates the book could be about your research question. You can find a lot of such information about the book from its listing in a library catalog. Consider whether time should have an impact on what sources can be considered relevant.

• Is the publication date or copyright date (usually listed in the library catalog or on the back of the book’s title page) too early or late for any time constraints in your research question? Maybe it’s just right.
• Skim some of the preface and introduction to see whether the book works with the time considerations you need.
• Check the bibliography to see whether the sources cited are about your research question.
• Skim the book’s table of contents and any summary chapters to locate the problem or question that the book addresses, its solution, and the broad outline of the book’s argument for its main claim. Will any of that be helpful in answering your research question?
• Skim some of the preface and introduction to see whether the book works with the time considerations you need.
• Do those sections give you information about what others have written about your research question?
• Do they offer a description of the situation surrounding your research question?
• Look for your key words in the bibliography. Do the sources cited pertain to your research question?
• Skim the index for topics with the most page references. Do the topics with the most page references pertain to your research question?