Skip to main content
 

point of view

How I Inadvertently Misled Some Students

Journal of College Science Teaching—July/August 2023 (Volume 52, Issue 6)

By Mark Milanick

Clear key results and simple figures have clear advantages when teaching. However, these same properties can mislead some students. I provide examples from my presentation on the body mass index, Centers for Disease Control and Prevention maps of the prevalence of obesity, changes in blood flow to organs when someone moves from rest to exercise, and nutritional values of food. These examples show how simple figures and an emphasis on only major points actually hinder some students’ ability to appreciate other dimensions of an issue as they examine data.

 

I value speakers who clearly identify key points; I try to do the same when I teach. However, when I provide the key points, I inadvertently mislead some students. In this article, I provide a few examples of how I have inadvertently misled students.

When discussing sensory physiology, I stress ways in which our senses can mislead us. My favorite example is that a person’s two hands sense water in the same beaker as different in temperature if one hand has been in hot water and the other one in cold water for at least a minute just before touching the beaker. This is why physiologists like their instruments! However, I now realize that some students think this preference for objective measurements implies that physical measurements and instruments always result in objective categories. I realized this idea was misleading when students indicated that they felt a diagnosis of attention deficit hyperactivity disorder (ADHD) was much less objective than a diagnosis of a person being overweight based on body mass index (BMI).

Other researchers have discussed the many limitations of BMI (Centers for Disease Control and Prevention, n.d.); my consideration in this article concerns how categories are determined. I certainly agree that one’s weight and height are objective measurements made by using a physical instrument. By contrast, a clinical or educational evaluation is not as easily measured by a physical instrument. I point out to students that it is misleading to suggest that the BMI cutoffs are objective because there are no clear break points in the actual data of hazard ratio versus BMI. As an analogy, when one is stepping up a set of stairs, it is clear which step a foot is on. However, if one is walking up a ramp, the number of steps depends on the person’s stride, so there is some subjective choice about where we draw the line between Step 2 and Step 3. If we had used the units of ounces and inches instead of meters and kilograms, we might have picked cutoffs of 200,000; 300,000; 350,000; and 400,000 ounces/per inches squared (corresponding to metric BMI of 18, 27, 31, and 36). BMI categories can be misinterpreted by students who think these are discrete categories, even though BMI is a continuous parameter in most populations. The plot of health hazard versus BMI is more like a ramp than stairs, but students often misinterpret the use of categories as implying that the data look like stairs instead of the ramp.

I have shown students the Centers for Disease Control and Prevention (CDC) maps of the prevalence of obesity by state to illustrate that the prevalence has increased in nearly every state over the years (Centers for Disease Control and Prevention, 2020). However, I recently realized that the abrupt color changes in the maps were misinterpreted by some students as reflecting discrete differences in prevalence, even though the data show that there are no discrete breaks, which is similar to the issue of having separate BMI categories.

Eventually, I realized that a major problem with the maps was the choice of what to display—the prevalence of obesity. I initially thought that targeting states with the highest prevalence was a natural approach. However, I now recognize that targeting the states with the highest total number of people who are obese might make more public policy sense, depending on the economics of distribution of a particular intervention. Not surprisingly, the most populous states (i.e., California, Texas, Florida, New York) have the greatest numbers of obese people, but these are not the states with the highest prevalence (which are Mississippi, West Virginia, Alabama, and Louisiana; Centers for Disease Control and Prevention, 2020). I point out to students that although prevalence is plotted, perhaps the total number of people who are obese would be a more relevant parameter to consider when making public policy.

I have told students that the fraction of cardiac output to skeletal muscles greatly increases during exercise. I now realize that considering only the data as the fraction of cardiac output to a particular organ might cause the students to miss the fact that total blood flow to the brain does not actually change during exercise. Even if I present the data as the total amount of blood flow to each organ, I might mislead the students into missing the fact that the heart and kidneys actually have more blood flow than muscle does per weight, even during exercise.

In both the obesity prevalence case and the blood flow case, there is an additional complication: I am treating a state and an organ system as one unit. The variation of prevalence by county, within many states, is actually greater than the variation of prevalence by state. Even during exercise, there is substantial variability in blood flow between different muscle groups.

I compared food nutritional value for students by using the traditional unit of serving size. When I compared foods for iron, I realized that this could be misleading. Consider a hamburger, lettuce, and a Snickers Marathon Double Chocolate Nut Bar (hereafter referred to as “Snickers”).

If we compare iron per serving size for each of these foods, Snickers has the most iron per serving (calculated using My Food Data, n.d.): Snickers > hamburger > lettuce. If we compare the iron content per 200 calories, we get an expected order: hamburger > lettuce > Snickers. Interestingly, even if we compare iron per 100 grams of each food, Snickers is “better” than lettuce: hamburger > Snickers > lettuce. Focusing on just one nutrient, though, is misleading. One needs to consider all of the nutrients in a particular food. A major problem with Snickers is that it also contains essentially all of the recommended daily allotment of added sugar (U.S. Department of Agriculture & U.S. Department of Health and Human Services, 2020).

It is difficult to always strike the “best” balance between having a few clear points that the whole class can grasp versus trying to avoid inadvertently misleading some of the students by reminding them of alternative choices for how to present data. In cases where categories are color coded but the underlying variable is continuous, for example, I now remind the students that other choices are possible for where to separate the categories. In the case of disease, I point out that whether one shows the prevalence or the total number of people represents a choice, and remembering the other options might be important. When we discuss nutrition, I now try to suggest a more holistic approach.

How often should we point out the other choices or their implications? I believe we should do so about as often as we stress that correlation does not imply causation. As a reminder, there are four general ways that the correlation of X with Y can occur without X actually causing Y (Lee, 2021):

  1. If Z causes both X and Y, then X and Y correlate, but X is not causing Y. For example, hot weather causes people to wear shorts and hot weather causes people to eat ice cream, but wearing shorts does not cause eating ice cream, even though the number of people wearing shorts correlates with the number of people eating ice cream.
  2. If Y causes X, then X and Y correlate, but X is not causing Y. Falling rain correlates with open umbrellas, but open umbrellas did not cause the rain.
  3. If the sample is not random, then a correlation may appear, whereas there is no correlation with a random sample, in which case X and Y might correlate because the sample is not random. For example, when you are comparing male and female study participants in a survey, if your sampling approach only samples nonpregnant female participants, one might observe a correlation without causation.
  4. Particularly when sampling people, you might see a correlation because people who do not feel comfortable answering truthfully will not do so. The cause of the correlation is thus not that X causes Y but that Y causes people not to answer about X truthfully. One might observe a correlation between those who do lots of background research before betting on a sports outcome. Those who do a lot of research might be uncomfortable admitting that they usually lost the bet, so successful betting would correlate with doing lots of research, but there is not actually a cause and effect present.

In addition, beware that when students hear the saying “correlation does not equal causation,” they may infer (incorrectly) that if there is no correlation between A and B, then A does not cause B (Penn, 2018). For example, there are many situations in which the response to a nutrient or a drug saturates at high doses. If you only sample doses of A above the saturation point, there will be no correlation with the response B because the response is maxed out. One might find no correlation between iron intake and iron status if everyone in the population is taking more than enough iron. On the flip side, there may be a minimal dose of A to detect response B, and if all of the sample doses of A are below the minimal dose, there will be no correlation. One might find no correlation between amount of study time and exam scores if the amount of study time only varied between participants from 0 to 5 minutes but at least an hour of study would have been needed to do well on the exam.

To sum up, there is the trade-off between making a clear, single point with a data presentation or visual plot and having a student be aware that the plot or major point might obscure other possibilities.


Mark Milanick (milanickm@umsystem.edu) is a professor emeritus in the Department of Medical Pharmacology and Physiology at the University of Missouri in Columbia, Missouri.

References

Centers for Disease Control and Prevention. (n.d.). Body mass index: Considerations for practitioners. https://www.cdc.gov/obesity/downloads/bmiforpactitioners.pdf

Centers for Disease Control and Prevention. (2020). Prevalence of self-reported obesity among U.S. adults by race/ethnicity, state and territory, BRFSS. https://www.cdc.gov/obesity/data/maps/2020/obesity-prevalence-map-by-race-ethnicity-2011-2019-508.pdf

Lee, I. (2021, April 18). 4 reasons why correlation does NOT imply causation. Towards Data Science. https://towardsdatascience.com/4-reasons-why-correlation-does-not-imply-causation-f202f69fe979

My Food Data. (n.d.). Nutrition facts search tool. https://tools.myfooddata.com/nutrition-facts

Penn, C. S. (2018, August 10). Can causation exist without correlation? Yes! https://www.christopherspenn.com/2018/08/can-causation-exist-without-correlation/#:~:text=Causation%20can%20occur%20without%20correlation,most%20often%20with%20insufficient%20samples

U.S. Department of Agriculture & U.S. Department of Health and Human Services. (2020). Dietary guidelines for Americans, 2020–2025 (9th ed.). United States Department of Agriculture. https://www.dietaryguidelines.gov/sites/default/files/2020-12/Dietary_Guidelines_for_Americans_2020-2025.pdf#page=31

Pedagogy Teacher Preparation Teaching Strategies Postsecondary

Asset 2