I've just watched the eight episodes of a Netflix documentary Lenox Hill, filmed in 2018 at a Manhattan hospital. It follows four specialists in neurosurgery, emergency medicine, and obstetrics/gynecology as they meet with patients and families, perform surgeries, and discuss cases with their peers. I came to admire their unvarying commitment to patients' best outcomes and empathetic responses to those faced with hard choices. I've gained a better understanding of how doctors weigh information from questions and observations; how they factor in ongoing scientific research and clinical trials, as well as discussions with other medical personnel; and how they consider individual circumstances and personal support networks. Knowledge in science develops differently than in history or literature. Observing the process doctors use to evaluate information can reshape your teaching of source evaluation in science.

A Hypothesis, not a Thesis

Scientists develop knowledge through an iterative process. In one sequence, a surgeon describes what he sees on an x-ray, discusses possible treatments with his team, and explains his best judgment about options to a patient and family. He continually adjusts his decisions right through the post-operative treatment. The scientific method builds upon logic, observations, and accumulated evidence, repeatedly proving, advancing, and even discarding what is known as new information is acquired.
Like a doctor, students can learn to consider their judgments as conditional. In evaluating credibility and authority, the task is not to land on an immutable decision but rather to critically synthesize observations, evidence, scientific facts, and contextual relevance into a conditional hypothesis. While logically arrived at, trusted information builds evidence for a hypothesis, not a thesis.

Building Students' Skills

Students are at a disadvantage in evaluating scientific claims. Poor data-literacy skills and inaccurate or uncritical reporting contribute to confusion, misinformation, and a growing mistrust of health research throughout society. Making sense of science research is daunting, as an epidemiologist with a background in medical statistics asserts:

"…people are confronted with an overwhelming amount of scientific evidence, and while most of that evidence will be based on robust, well-designed research, some of it will arise from ill-designed, poorly implemented, inappropriately analyzed or selectively reported studies...and, when two pieces of evidence contradict each other, it can be hard for people to know which to act on" (Alba).

However, health and medical data are often the basis for government regulation, public policy, and personal health decisions (Goldstein), so we are obliged to help students learn to evaluate this information. Three questions can draw students' attention to the disciplinary thinking used by medical researchers, reporters, and writers to assess credibility.

Questions of Trust

1. Does the science author evaluate a range of outcomes?

For instance, credible epidemiologists use current data to create multiple models to predict the spread and severity of COVID-19 ("A Compendium"). Since data can support a range of interpretations, readers should be given an explanation of why projections differ and which outcomes are more likely than others (Best and Boyce).

• Ask students to identify whether their source discusses multiple scenarios or promotes a single conclusion.

2. Is the science researcher transparent about funding source(s)?

It has been shown that medically related industry sponsorship can influence research agendas (Fabbri et al.) and experimental results can be contaminated (Corb). Consciously or unconsciously, a researcher may design studies or interpret evidence to support a funder's interests.

• Ask students if the author is transparent about funding sources for research results published in a journal article, a press release, or even on social media (Gill).

3. Does the science journalist contextualize the reporting?

To report accurately, a journalist needs to develop background knowledge in order to critically analyze science claims and evidence (McBride).

• Ask students to tally how many experts the journalist has interviewed, then look for evidence that what has been learned is used to elaborate on or contextualize the reporting.

Adding Friction

Teaching students to evaluate health information has real-world relevance and is also a good fit for hybrid learning during the pandemic. Early into her school's transition to the Cloud, Tasha Bergson-Michelson developed a two-day lesson with a 7th grade biology teacher. The assignment asked students to draw upon their knowledge of health and body systems to evaluate home remedies for coronavirus promoted on social media.

As anxiety about the coronavirus escalated, misinformation thrived. Tasha wanted her students to feel more confident in evaluating claims ("Can You Spot") but was reluctant to send 7th graders into the "wilds" of social media. Instead, she directed students toward a few credible sources and curated a set of claims to evaluate on a slide deck.

Tailoring Instruction to Goals

Tasha has a repertoire of evaluation strategies she wants students to learn to apply. As she works with middle- and high-school classes, through practice that is spaced in time and alternated by type ("Spaced and Interleaved"), she has been systematically building her students' skills at identifying cues that signal source type, point of view, and discipline. Likewise, the biology teacher looks for opportunities for students to recognize that classroom science is relevant to their lives. The biology teacher's immediate goal was that students apply what they've been learning about the human body to the evidence and reasoning used in social media claims about coronavirus remedies.

Since this assignment would be rolled out over two consecutive days, Tasha designed a tightly focused activity with minimal homework supported by synchronous Zoom work periods with access to teachers for questions.

Modeling an Argument

Since the assignment focused on coronavirus remedies that are being endorsed in social media posts, the class practiced in the virtual classroom on the first day to evaluate one post.

They identified the claim…

• "…a few sips of water every 15 mins at least" would prevent the coronavirus from entering your windpipe and lungs."

discussed the evidence…

• "Even if the virus gets into your mouth…drinking water or other liquids will wash them down through your esophagus and into the stomach. Once there in tummy…your stomach acid will kill all the virus."

and evaluated the reasoning that connected the claim to the evidence:

• "If you don't drink enough water regularly, the virus can enter your windpipe and into the LUNGS. That's very dangerous."

Highly structured activities often require considerable planning. Tasha selected a series of supposed remedies that included all three elements of an argument—a claim, evidence, and reasoning. Each student was assigned a single slide which contained a screenshot of the social media post. Tasha showed students how to search the CDC and WHO, the sources she required them to use. However, she recognized many posts included words like estimate, percentage, correlate, or cause, terms that would require data literacy skills. So she also curated a list of fact-checking sites that were likely to discuss the data, listed here in order of their usefulness.

• Coronavirus Alliance Database (https://www.poynter.org/ifcn-covid-19-misinformation) was launched in January 2020 by Poynter to fight the "infodemic." It is a database of pandemic-specific fact-checking information, with entries written by a global consortium of over one hundred reporters from seventy countries.
• FactCheck (https://www.factcheck.org/), supported by the Annenberg Public Policy Center, identifies "statements based on facts" made by politicians of both parties and rebuts misleading or inaccurate claims. SciCheck focuses on science-based claims made by public figures to influence public policy.
• PolitiFact (https://www.politifact.com/), owned by the Poynter Institute, a nonprofit school for journalists, evaluates political or newsworthy facts. They outline the process they use to fact-check and correct statements with a preference for verifying information using primary sources such as "government reports, academic studies, and other data." Claims are rated on a Truth-O-Meter scale from "True" to "Pants on Fire."
• AP Fact Check (https://apnews.com/APFactCheck) is part of the Associated Press, a global news organization. The fact-checking articles are in timeline order on a separate page of their website but are actually integral to their newswire articles and conform to their reporting standards.
• Snopes (https://www.snopes.com/fact-check/), supported by ads, initially debunked urban legends and hoaxes but has expanded to include trending search topics and social media posts. Since the entries are based on the public's interest, subjects are somewhat hit-or-miss for educational purposes, but they do have a small coronavirus collection. Snopes is transparent about how they evaluate and correct information, which is labeled with defined terms like "scam," "outdated," "true," "mostly true," "false," "unproven," etc. The site also includes AP stories, clearly labeled, which helps them respond to more questions.

Students' searches began as individual on-screen work, with the teachers available for help, and were completed by the end of the first afternoon. For the few students who weren't able to locate information in the curated sources, Tasha showed how to mine language within the claim to construct a synonymic search to use on the open web:

[fact check coronavirus OR covid-19 OR 2019-nCoV drinking water]

Flexible Hybrid Assignment and Assessment

The homework asked students to complete the analysis of their assigned remedy and

1. Post a headline that summarized the fact-checking article
2. Label the social media claim as True, False, or Misleading (partially true)
3. Add the headline as a comment to their assigned slide in the deck
4. Comment on at least two other student posts.

Most students were engaged and completed the project, which provided ample documentation of students' understanding. Hybrid learning is most successful with up-front preparation, explicit goals, sequential instructions, and both teachers coaching at the point of need. If students had returned to campus during this assignment, the teachers could have shifted from peer-to-peer commenting online to short presentations and discussion in class.

For the foreseeable future, the news will continue to be dominated by health and medical reporting. During this pandemic, there are obvious long-term benefits for teaching students to assess the credibility of health information, given its relevance to personal concerns, public policies and global issues. Hybrid lesson design, effectively planned and executed, is an ideal structure for teaching students to evaluate health information.

Works Cited (MLA Style)

Alba, Sandra. "Time to Talk about Trust." Significance, vol. 17, no. 2, Apr. 2020, pp. 12-13, doi:10.1111/1740-9713.01371.

Best, Ryan, and Jay Boyce. "Where the Latest COVID-19 Models Think We're Headed—And Why They Disagree." FiveThirtyEight, ABC News Internet Ventures, 9 June 2020, projects.fivethirtyeight.com/covid-forecasts/.

"Can You Spot COVID-19 Misinformation?" Thomson Reuters, May 2020, www.thomsonreuters.com/content/dam/ewp-m/documents/thomsonreuters/en/pdf/social-impact/tr-infographic-misinformation.pdf. Infographic.

"A Compendium of Models That Predict the Spread of COVID-19." American Hospital Association, Apr. 2020, www.aha.org/guidesreports/2020-04-09-compendium-models-predict-spread-covid-19.

Corb, Benjamin. "This Is What Happens When Coronavirus Research Funding Gets Political." CNN, Cable News Network, 30 Apr. 2020, www.cnn.com/2020/04/30/opinions/nih-daszak-coronavirus-funding-cut-opinion-corb/index.html.

Fabbri, Alice, et al. "The Influence of Industry Sponsorship on the Research Agenda: A Scoping Review." American Journal of Public Health, vol. 108, no. 11, Nov. 2018, pp. e9-e16, doi:10.2105/AJPH.2018.304677.

Gill, Ranpal. "Funding Acknowledgements: Requirements and Boilerplate Language." Xerox DocuShare, Xerox, 16 Apr. 2020, docushare.lsst.org/docushare/dsweb/Get/Document-3607/.

Goldstein, Harvey. "The Role of Evidence in Public Policy Making." Harvey Goldstein, 15 Dec. 2019, harveygoldstein.co.uk/2019/12/15/the-role-of-evidencein-public-policy-making/.

McBride, Rebekah. "The Stories behind the Data." Significance, vol. 17, no. 2, Apr. 2020, pp. 10-11. Royal Statistical Society, doi:10.1111/1740-9713.01370.

"Spaced and Interleaved Practice." MIT Open Learning, 2020, openlearning.mit.edu/mit-faculty/research-based-learning-findings/spaced-and-interleaved

You can download and modify a NoodleTools annotated bibliography of sources which could be used as a template for a fact-checking assignment at https://my.noodletools.com/public/200624201902371659641130.