9.4.2A Evaluating Data Reports
Evaluate reports based on data published in the media by identifying the source of the data, the design of the study, and the way the data are analyzed and displayed. Show how graphs and data can be distorted to support different points of view. Know how to use spreadsheet tables and graphs or graphing technology to recognize and analyze distortions in data displays.
For example: Shifting data on the vertical axis can make relative changes appear deceptively large.
Identify and explain misleading uses of data; recognize when arguments based on data confuse correlation and causation.
Overview
Standard 9.4.2 Essential Understandings
Students have designed simple experiments and collected data. They have chosen appropriate data displays. Through the essential ideas in this standard students "learn more-sophisticated ways to collect and analyze data and draw conclusions from data in order to answer questions or make informed decisions in workplace and everyday situations" (NCTM, 2000). Students will evaluate published reports in the media by learning to ask questions about the source of the study, the design of the study, possible biases, and the way the data was analyzed and displayed. Students will use technology to recognize and analyze distortions in data displays. "Every high-school graduate should be able to use sound statistical reasoning to intelligently cope with the requirements of citizenship, employment, and family and to be prepared for a healthy, happy, and productive life." (ASA, 2007, p. 1)
All Standard Benchmarks
9.4.2.1 Evaluate reports based on data published in the media by identifying the source of the data, the design of the study, and the way the data are analyzed and displayed. Show how graphs and data can be distorted to support different points of view. Know how to use spreadsheet tables and graphs or graphing technology to recognize and analyze distortions in data displays. For example: Displaying only part of a vertical axis can make differences in data appear deceptively large.
9.4.2.2 Identify and explain misleading uses of data; recognize when arguments based on data confuse correlation and causation.
9.4.2.3 Design simple experiments and explain the impact of sampling methods, bias and the phrasing of questions asked during data collection.
Benchmark Group A
Benchmark 9.4.2.1 Evaluate reports based on data published in the media by identifying the source of the data, the design of the study, and the way the data are analyzed and displayed. Show how graphs and data can be distorted to support different points of view. Know how to use spreadsheet tables and graphs or graphing technology to recognize and analyze distortions in data displays.
Benchmark 9.4.2.2 Identify and explain misleading uses of data; recognize when arguments based on data confuse correlation and causation.
Design simple experiments and explain the impact of sampling methods, bias and the phrasing of questions asked during data collection.
What students should know and be able to do [at a mastery level] related to these benchmarks:
- Students will recognize the importance of understanding data to be able to answer a variety of questions.
- Students will develop the ability to ask good questions, consider the implications of sampling methods, and identify potential biases for a study design. "The conditions under which data are collected are important in drawing conclusions from the data; in critically reviewing uses of statistics in public media and other reports, it is important to consider the study design, how the data were gathered, and the analyses employed as well as the data summaries and the conclusions drawn." (Common Core Standards, 2010, p.80)
- Students will know through the experience of designing a simple experiment that a well-conceived design for collecting data is a major component to a solid investigation.
- Students will be able to use technology flexibly to create and change graphs based on real world data.
- Students will understand that graphs should facilitate data analysis and communication. "A graph is not complete until students write a summary or interpretation of the information displayed. The question, method of data collection, and conclusions reached by an individual or class should be clear to a visitor observing a data display posted in the classroom or hallway." (Minnesota Mathematics Frameworks, 1997, p. 4).
- Students will understand the use and effect of random sampling and random assignment.
- Students will understand that a statistically significant outcome is one that is unlikely to be due to chance alone, and this can be evaluated only under the condition of randomness.
Work from previous grades that supports this new learning includes:
- Students are familiar with different graphical representations of data including histograms, bar graphs, circle graphs, and scatterplots.
- Students have worked with spreadsheets and graphing calculators to have experience with scaling graphs and can recognize distortions or bias in graphs.
- Students can draw on their prior experience of reading and interpreting graphs from statistical reports from different media outlets.
- Students have had experience with calculating and understanding measures of center and spread.
- Students are used to providing justification and reasoning behind their ideas.
- Students have conducted simple experiments and collected data.
- Students have calculated and interpreted mean, median, and range and demonstrate an intuitive sense for spread and deviation.
- Students are careful to select and construct representations most appropriate for the data to avoid misleading representations.
- Students have worked with linear functions.
NCTM Standards
Data Analysis and Probability Standards
formulate questions that can be addressed with data and collect, organize and display relevant data to answer them
- understand the differences among various kinds of studies and which types of inferences can legitimately be drawn from each
- know the characteristics of well- designed studies, including the role of randomization in surveys and experiments
develop and evaluate inferences and predictions that are based on data
- evaluate published reports that are based on data by examining the design of the study, the appropriateness of the data analysis, and the validity of conclusions
- understand how basic statistical techniques are used to monitor process characteristics in the workplace.
Common Core State Standards (CCSS)
Interpreting Categorical and Quantitative data S-ID
Interpret linear models
9. Distinguish between correlation and causation.
Making Inferences and Justifying Conclusions S-IC
Understand and evaluate random processes underlying statistical experiments
1. Understand statistics as a process for making inferences about population parameters based on a random sample from that population.
Make inferences and justify conclusions from sample surveys, experiments, and observational studies
3. Recognize the purposes of and differences among sample surveys, experiments, and observational studies; explain how randomization relates to each.
6. Evaluate reports based on data.
Misconceptions
Student Misconceptions and Common Errors
- Confusing correlation and causation.
- Students have trouble with properly scaling axes of graphs.
- Students may have trouble in writing good data collection questions and tend to write leading questions or asking two questions at once.
- Confusing random sampling and random assignment.
- Confusing the statistical meaning of words with their everyday usage for example the words normal, significant, sample, and random. "Rather studying statistics is akin to studying foreign language, for students need lots of practice to become comfortable using these terms correctly." (Rossman, Chance, & Medina, 2006, p. 10).
Vignette
In the Classroom
In this lesson students will get to work with data for different countries that involve life expectancy and the number of people per television. This lesson focuses on students understanding correlation and causation, and that correlation does not imply causation. Previously, students would have had experiences with verbally and graphically exploring data and working with lines of best fit. The data in this example is older but still can provide for rich discussion and investigation.
Teacher: We are going to explore the relationship between life expectancy and people per television for various countries. Life Expectancy is the average life expectancy in years for a country. People Per TV is the ratio of people to TVs that are owned in the country. A country with a high People Per TV ratio indicates that very few people own a TV. A country with a low People Per TV ratio indicates that many individual people own a TV.
To start individually, make estimates for 2011 and for 1993 for the life expectancy and people per television for the countries that are listed. Then discuss in your group of three.
|
|
Life Expectancy |
Life Expectancy |
People per Television |
People per Television |
|
Country |
1993 |
2011 |
1993 |
2011 |
|
U.S.A. |
|
|
|
|
|
France |
|
|
|
|
|
Haiti |
|
|
|
|
|
Russia |
|
|
|
|
|
South Africa |
|
|
|
|
|
Mexico |
|
|
|
|
(Teacher has some students share their estimates and their discussion)
Teacher: Now I want you to talk in your group whether you think there is a relationship between life expectancy and people per television. If so, would the association be positive or negative?
(Students discuss)
Student: I think that there would be a relationship because the smaller number of people per television the richer the country. If people have more money they might be able to take better care of themselves.
Teacher: Would it be a positive or negative relationship?
Student: It would be negative because as life expectancy goes up, people per tv would go down.
Teacher: We will look at the data. We do not have the people per tv for 2011 but the other data is there. In this activity we will focus on the 1993 data. Go ahead and create a scatterplot of the data with life expectancy and people per television. Describe the data. What type of relationship seems to exist?
|
Country |
Life expectancy (1993) |
Life Expectancy (2011) |
People per TV |
|
Angola |
44 |
38.2 |
200 |
|
Australia |
77 |
81.6 |
2 |
|
Cambodia |
50 |
59.7 |
177 |
|
Canada |
77 |
80.7 |
2 |
|
China |
70 |
73 |
8 |
|
Egypt |
61 |
71.3 |
15 |
|
France |
78 |
80.7 |
3 |
|
Haiti |
54 |
60.9 |
234 |
|
Iraq |
67 |
59.5 |
18 |
|
Japan |
79 |
82.6 |
2 |
|
Madagascar |
53 |
59.4 |
92 |
|
Mexico |
72 |
76 |
7 |
|
Morocco |
65 |
71.2 |
21 |
|
Pakistan |
57 |
65.5 |
73 |
|
Russia |
69 |
65.5 |
3 |
|
South Africa |
64 |
49.3 |
11 |
|
Sri Lanka |
72 |
75.14 |
28 |
|
Uganda |
51 |
51.5 |
191 |
|
United Kingdom |
76 |
79 |
3 |
|
United States |
76 |
78.3 |
1 |
|
Vietnam |
65 |
74.2 |
29 |
|
Yemen |
50 |
62.7 |
38 |
Scatter plot
Student: The data is somewhat linear.
Teacher: Let's look at the linear model and calculate the correlation coefficient.
Student: It is -0.801.
Teacher: What does the correlation coefficient mean in this situation? Would increasing the number of televisions per people work to increase a country's life expectancy?
Student: That does not seem to make sense. Watching a lot of television does not make you live longer.
Teacher: Since shipping televisions to other countries will not increase their life expectancy, what might be another explanation for this relationship?
Student: Countries where more people have televisions probably have more money and are able to afford better food and health care. People per television might have a relationship for other reasons.
Teacher: This is an important point. Even though there is a relationship between two variables it is always important to think through what is happening and use your reasoning skills.
(Teachers can show the following Youtube video at this time and discuss how average life expectancy has changed throughout history: Sojo Hans Roslings 200 countries, 200 years, 4 minutes- The Joy of Stats).
Exit Questions
- Does correlation imply causation? Explain.
- How would you respond to a media report that said the number of people per television in a country was related to life expectancy?
Rossman, A. (1994). Televisions, physicians, and life expectancy. Journal of Statistics Education, 2(2).
http://en.wikipedia.org/wiki/List_of_countries_by_life_expectancy
Resources
Teacher Notes
- Students may need support in vocabulary. Teachers should model effective use of vocabulary terms and give students opportunities to discuss and represent vocabulary concepts in a variety of ways. This can be a teaching opportunity to show students terms in statistics carry specific meanings and thus correct this misuse of terms.
- Students may need support in further development of questioning how data was collected. Teachers should model the type of questions that students should ask when reading a media report that references statistics.
-
https://www.keypress.com/x2814.xml Collections of Data Sets
While Fathom Dynamic Data comes equipped with more than 300 diverse data sets, the software is also compatible with a wide range of independent data. The links at this website have a variety of collections of data-ranging from lunar eclipses to a baseball player's batting average-that can be easily integrated into the Fathom software for further exploration of real-world statistics. (Note: Fathom software is not free).
https://illuminations.nctm.org/Search.aspx?view=search&st=d (Web Links for Data Analysis and Probability).
Chance Database. L. Snell, The Chance Project, Mathematics Department, Dartmouth College, Hanover, NH 03755. Web site: https://chance.dartmouth.edu/
- Chance News. L. Snell, The Chance Project, Mathematics Department, Dartmouth College, Hanover, NH 03755. Web site: https://chance.dartmouth.edu/chance_news/news.html
This is a website for a quantitative literacy course based on case studies, with the aim of making students more informed and critical readers of current news that uses probability and statistics as reported in daily newspapers.
- Journal of Statistics Education. https://amstat.tandfonline.com/toc/ujse21/current An international journal on the teaching and learning of statistics
- Activities from the Texas Instruments Classroom Activities Exchange can be used to supplement lessons on the concepts in Gr. 9-12 Data Analysis and Probability. (https://www.education.ti.com)
Websites for real world data
Statistical literacy is "understanding the basic language of statistics (e.g., knowing what statistical terms and symbols mean and being able to read statistical graphs) and fundamental ideas of statistics" (ASA, 2007, p.14). Statisticians are practical problem-solvers. Often clients present a problem (Is there a treatment effect present?). "Different statisticians may come up with somewhat different analyses of a given set of data, but will usually agree on the main conclusions and only worry about minor points if those points matter to the client" (ASA, 2007, p. 15). Suggestions for teachers include modeling statistical thinking for students, use technology, have students practice statistical thinking, plenty of practice with choosing appropriate questions and techniques, and give feedback on statistical thinking. There are many types of real data including archival, classroom-generated and simulated data. "Few data sets interest all students, so one should use data from a variety of contexts" (ASA, 2007, p. 16).
The Machine Learning Repository (https://archive.ics.uci.edu/) includes data sets both large and small, and those listed under CS/Engineering cover some topics on themes raised in this paper, such as fraud detection in hacker attacks on computers
- Theinfo.org
http://www.gapminder.org/data/
- Data and story library
- Australian data and story library
http://www.statsci.org/data/index.html
- Bureau of Justice Statistics:
https://bjs.ojp.usdoj.gov/index.cfm?ty=daa
City Data Site:
- https://www.census.gov/
- https://www.city-data.com/
- State and county statistics sites
- State and national Dept's of Education
- County tax assessment records
- Applebee's:https://www.applebees.com/downloads/nutritional_info.html
- Arby's: https://www.arbys.com/nutrition/Arbys_Nutrition_Website.pdf
- NFL Historical Stats: https://www.nfl.com/history
- Cost/Prices; e.g., Kelley Blue Book:https://www.kbb.com/
- Consumer Report ratings: https://www.consumerreports.org/cro/index.htm
causation: performing certain actions causes certain effects--understanding that there is a relationship between cause and effect
correlation: a measure of the relationship between two data sets (positive, negative or no correlation)
display distortion: displaying data on a graph so that it distorts the data in a way that makes it misleading; for example, showing only a portion of the vertical axis can make small changes seem significant.
sample: a subset of a population
Reflection - Critical Questions regarding the teaching and learning of these benchmarks
- In what ways did students critically analyze statistics in media?
- Do students have a better understanding of how statistical thinking can be used to draw inferences, make predictions, and justify conclusions?
- Was the statistical data relevant to the lives of students?
- What difficulties did students have in designing simple experiments? How can I facilitate their learning better in the future to provide scaffolding for these difficulties?
Materials
American Statistical Association. (2007). Guidelines for assessment and instruction in statistics education. Retrieved December 14th, 2010 from http://www.amstat.org/education/gaise/index.cfm
This is the K-12 portion of the American Statistical Association (ASA) website. They have workshops for teachers, online resources for teachers, useful websites, student competitions, and a list of publications in statistics education. http://www.amstat.org/education/publications.cfm
Ben-Zvi, D. (2009). Toward understanding the role of technological tools in statistical learning. Mathematical Thinking and Learning, 2(1), 127-155.
Haberman, M. (1991). The pedagogy of poverty versus good teaching. Phi Delta Kapan, (December). 291-294.
Peck, R., Starnes, D., Kranendank, & H., Morita, J. (2009). Making sense of statistical studies: Teacher's module. Alexandria, VA: American Statistical Association.
This book consists of 15 hands-on investigations that provide students with valuable experience in designing and analyzing statistical studies. It is written for an upper middle-school or high-school audience. Each investigation includes a descriptive overview, prior knowledge that students need, learning objectives, teaching tips, references, possible extensions, and suggested answers.
One of the goals of the American Statistical Association is to improve statistics education at all levels. Through the Statistics Education Web (STEW), the ASA plans to reach out to K-12 mathematics and science teachers who teach statistics concepts in their classrooms. STEW will be an online resource for peer-reviewed lesson plans and resources for K-12 teachers. The web site will be maintained by the ASA and accessible to K-12 teachers throughout the world.
American Statistical Association (2007). Guidelines for Assessment and Instruction in Statistics Education. Retrieved December 14th, 2010 from http://www.amstat.org/education/gaise/index.cfm
Ben-Zvi, D. (2009). Toward understanding the role of technological tools in statistical learning. Mathematical Thinking and Learning, 2(1), 127-155.
Metz, M. (2010). Using GAISE and NCTM standards as frameworks for teaching probability and statistics to pre-service elementary and middle school mathematics teachers. Journal of Statistics Education, 18(3), 1-27.
National Council of Teachers of Mathematics (2000). Principles and Standards for School Mathematics. Reston, VA: NCTM.
Rossman, A., Chance, B., & Medina, E. (2006). Important comparisons between statistics and mathematics, and why teachers should care. In Thinking and reasoning with data and chance. Reston, VA: National Council of Teachers of Mathematics.
Assessment
DOK level 3: Strategic Thinking
1. The bar graph below shows the average yearly temperatures in degrees Celsius for England from the year 1275 to the year 1975 in increments of 50 years.
FCAT 2006 Mathematics released test, Florida Department of Education, http://fcat.fldoe.org/pdf/releasepdf/06/FL06_Rel_G10M_TB_Cwf001.pdf p. 41
Correct Answer: I
DOK level 2: Basic Reasoning
2. The population for the years 1996 to 2005 of a small town is shown on the graph below.
MD Department of Education, 2009, Algebra/data analysis public release, p.14, http://mdk12.org/assessments/high_school/look_like/2009/algebra/hsaAlgebra.pdf
http://mdk12.org/assessments/high_school/look_like/algebra/intro.html
Correct Answer: J
3. Javier wants to collect data about the number of hours students in his school exercise each week. Based on the principles of simple random sampling, which of these is the best method for Javier to collect his data?
A. Randomly select 20 students from both the girls' and boys' soccer teams.
B. Randomly select 10 freshmen, 10 sophomores, 10 juniors, and 10 seniors.
C. Choose any 3 buses, and randomly select a total of 40 students from those buses.
D. Number all students in the school, and randomly select the numbers of 40 students.
MD Department of Education, 2009, Algebra/data analysis public release, p.28, http://mdk12.org/assessments/high_school/look_like/2009/algebra/hsaAlgebra.pdf
http://mdk12.org/assessments/high_school/look_like/algebra/intro.html
Correct Answer: D
DOK level 2: Basic Reasoning
4. Minh used the graph below to show the average cost of a school lunch is increasing at a rapid rate.
MD Department of Education, 2009, Algebra/data analysis public release, p.33,
http://mdk12.org/assessments/high_school/look_like/algebra/intro.html
Correct answer: J
DOK level 2: Basic Reasoning
5. A school wants to conduct a survey of its 1200 students to decide on a school song. Based on the principles of simple random sampling, which of these is the best method for the school to use to conduct its survey?
A Assign a different number to each student who has been at the school for at least one year and select 30 students using a random number generator.
B Assign a different number to each student whose age is 16 years or older, and select 30 students using a random number generator.
C Assign a different number to each student, and select 15 males and 15 females using a random number generator.
D Assign a different number to each student, and select 30 students using a random number generator. (Maryland State Department of Education, 2009, Algebra/data analysis public release, p.35,
http://mdk12.org/assessments/high_school/look_like/2009/algebra/hsaAlgebra.pdf
http://mdk12.org/assessments/high_school/look_like/algebra/intro.html
Correct Answer: D
DOK level 2: Basic Reasoning
6. A television station will conduct a survey to determine the most popular television show in its viewing area. Which of these methods provides the most representative sample?
A Survey 50 randomly selected residents in its viewing area.
B Survey 50 randomly selected people who leave a video rental store.
C Survey 50 randomly selected people who enter the television station.
D Survey 50 randomly selected residents from the local telephone book.
MD Department of Education, 2009, Algebra/data analysis public release, p.38, http://mdk12.org/assessments/high_school/look_like/2009/algebra/hsaAlgebra.pdf
http://mdk12.org/assessments/high_school/look_like/algebra/intro.html
Correct answer A
DOK level 2: Basic Reasoning
7. The graph below shows the number of new houses built in a town from 1970 to 2000.
MD Department of Education, 2008, Algebra/data analysis public release, p.13, http://mdk12.org/assessments/high_school/look_like/2008/algebra/hsaAlgebra.pdf
http://mdk12.org/assessments/high_school/look_like/algebra/intro.html
Correct answer: J
DOK level 3: Strategic thinking
8. The president of the student government wants to survey the students in the school about their satisfaction with the 36 after-school activities. There are 1,000 students in the school-200 freshmen, 200 sophomores, 300 juniors, and 300 seniors. The president suggested three different sampling methods.
- Method A: Randomly choose three students from each of the 36 after-school activities for the survey.
- Method B: Randomly select 100 students from the honor roll list to survey.
- Method C:Randomly select 20 freshmen, 20 sophomores, 30 juniors, and 30 seniors for the survey
- Which method provides the most representative sample of the student population? Use mathematics to justify your answer.
- Use mathematics to justify why each of the other two methods does not provide a representative sample.
MD State Department of Education, 2008, Algebra/data analysis public release, p.23 http://mdk12.org/assessments/high_school/look_like/2008/algebra/hsaAlgebra.pdf
http://mdk12.org/assessments/high_school/look_like/algebra/intro.html
Minnesota Comprehensive Assessment (MCA) III Test Specifications
Standard 9.4.2: Explain the uses of data and statistical thinking to draw inferences, make predictions and justify conclusions.
Benchmarks
9.4.2.1 Evaluate reports based on data published in the media by identifying the source of the data, the design of the study, and the way the data are analyzed and displayed. Show how graphs and data can be distorted to support different points of view. Know how to use spreadsheet tables and graphs or graphing technology to recognize and analyze distortions in data displays.
Item Specifications: Not assessed on the MCA-III
9.4.2.2 Identify and explain misleading uses of data; recognize when arguments based on data confuse correlation and causation.
Item Specifications: Vocabulary allowed in items: causation and vocabulary given at previous grades
9.4.2.3 Design simple experiments and explain the impact of sampling methods, bias and the phrasing of questions asked during data collection.
Item Specifications: Items do not require students to design experiments. Vocabulary allowed in items: vocabulary given at previous grades
(Note: These Test Specifications are in DRAFT form as of May 4, 2011 (updated Test Specifications will be available: http://education.state.mn.us/MDE/Accountability_Programs/Assessment_and_Testing/Assessments/MCA/TestSpecs/index.html)
Differentiation
- Strategies: Real world problem solving, multiple entry points, vary teaching methods, group work, teach problem solving strategies.
- Challenges: motivation, slower processing, reading and writing ability, organization, and behavior issues and coping strategies.
Teachers must be explicit in how they talk of vocabulary terms and use vocabulary in context. Teachers should use vocabulary terms often so that students will become familiar hearing them in context. Students should also be allowed to practice the use of vocabulary in small groups.
- Strategies: Model vocabulary, manipulatives, speak slowly, visuals, variety of assessments, group work, verbalize reasoning, understanding context or concept, making personal dictionaries.
- Challenges: Vocabulary and Reading ability, standardized tests, how to approach problem solving, cultural differences
http://www.mcgt.net/resources.html (Minnesota Council for the Gifted and Talented)
- Strategies: Tiered objectives, open-ended problem solving, grouping (heterogeneous and homogeneous), curriculum compacting, and independent investigations.
Students can be given more challenging work by using activities developed by the University of Minnesota for an introductory statistics course. All of the activities, lesson plans, and data are available at http://www.tc.umn.edu/~aims/index.htm
- Challenges: Motivation, acceleration and attitude associated with this for students, maturity, isolation and social issues, and not wanting to be moved outside of age group.
Parents/Admin
Administrative/Peer Classroom Observation
|
Students are: (descriptive list) |
Teachers are: (descriptive list) |
|
posing questions. |
allowing students to share ideas and pose questions. |
|
hypothesizing about possible outcomes. |
asking students questions about their ideas. |
|
choosing tools and methods of inquiry. |
selecting quality and rich data examples for students to investigate and critique. |
|
designing or changing representations. |
structuring class activities for discussion, justification, and exploration. |
|
interpreting results. |
having students refine or revise their thinking or work. |
|
drawing conclusions and justifying them. |
providing or allowing students to find relevant data for students to investigate. |
|
investigating real world problems which allow for many interactions and collaboration in the classroom. |
helping students see major concepts, big ideas and general principles and not merely having students engage in isolated facts. |
|
technology is being used to contribute to making sense of data and constructing meanings of basic statistical concepts as well as to facilitate the use of multiple data representations. (ben-Zvi, 2009). |
helping students connect new information to prior knowledge |
|
determining an audience for their results. |
letting students build on each other's ideas. |
|
using technology to go beyond computations and procedures to more statistical reasoning based on the ability to interpret, evaluate, and test conjectures. |
|
Parent Resources
- How to Lie With Statistics by Darryl Huff
Parents can discuss statistical graphs or information presented in the newspaper or other media with their children. Parents can ask their children what the graphs or analysis means, what information is not included that might be helpful, what conclusions are supported by the data, and what are the limitations of the statistics presented.
- This is an article about the growing variety of jobs in statistics that are available due to the advancements in technology. Lohr, Steve. (2009). For Today's Graduate, Just one word: Statistics. New York Times.
This website has a variety of applets and activities for students to explore patterns and investigate probability.
This website has summary information of other websites that can be helpful for further information, practice, and exploration for students.
- Search You Tube or Video Google for instructional videos on probability and statistics.