EDDiE: Electronic Document Disagreement Evaluation

What is EDDiE

EDDiE (Electronic Documents Disagreements Evaluation) is a computer-supported collaborative learning (CSCL) system that allows several learners to read, discuss, and analyze a set of documents, even if the learners are far apart.

EDDiE helps learners weigh and integrate information from multiple documents on the Internet and beyond when they are engaged in inquiry to find out answers to their questions about issues. In other words, when learners want to do research on any issue on the Internet, EDDiE can support them in developing and using effective ways of evaluating, weighing, organizing, and integrating information from multiple, often conflicting sources of information. The goal is to help learners develop better ways to reason about multiple documents.

The application enables learners to create a visual representation on a tableau to compare and contrast information from multiple conflicting documents. The tableau incorporates scaffolds that support learners as they identify and evaluate sources, the views of experts, and evidence that bears on an issue. Additional scaffolds help students identify and resolve disagreements among the documents to reach sound conclusions.

EDDiE is designed to help students learn to reason well about conflicting viewpoints and information, which are pervasive in the so-called “post-truth” digital world.

How to Learn using EDDiE

Theoretical Grounding

Conflicting viewpoints and information are pervasive in digital information in the post-truth world. In order to successfully deal with such conflicts, it is important to evaluate the sources of information, but also scrutinize their content, carefully analyze why the disagreements exist, and work out how to resolve the disagreements. These processes all involve appraising the evidence that supports various claims, along with examining the views of experts who have studied the issue.

EDDiE’s design is grounded in the Grasp of Evidence framework (Duncan et al., 2018). This framework is useful for examining the use of evidence by laypeople (evidentially practices) as well as by experts. The Grasp of Evidence Framework identifies five dimensions of reasoning about evidence, as shown in the table below. EDDiE is designed to promote better reasoning practices along each of these five dimensions.

Epistemic scaffolds based on Grasp of Evidence Framework and students’ reasoning practices.

GoE Dimensions	Definition	Interface component and students’ reasoning practices
1. Evidence Analysis	Breaking down the evidence into its component parts and comprehend how the parts fit together.	Students collaboratively extract, analyze and record important elements of evidence (e.g., sample size, critical comparisons, results) in the analysis tableau by dragging and dropping sentences in each document.
2. Evidence Evaluation	Scrutinizing the methodological qualityof the studies that produce evidence.	Students evaluate the quality of evidence via the color of evidence circles. High-quality evidence is colored dense green; poor evidence is colored yellow.
3. Evidence Interpretation	Examining how strongly evidence supports or contradicts different explanations.	Arrow thickness between evidence and claims reflects evidence strength. Thick arrows mean strong evidence for the claim. Thin arrows mean weak evidence. Dotted arrows mark disagreements. Colored dotted arrows mean counter-evidence to the same-colored claims.
4. Evidence Integration	Considering how large, diverse, and often conflicting bodies of evidence relate to competing explanations	The size and shape of evidence circles mark evidence quantity. The color of arrows denotes which claim the evidence supports. Students easily recognize consistencies or inconsistencies of evidence across the documents and each evidence quality.
5. Lay use of Evidence	Examining evidence in light of laypeople’s bounded knowledge and limited expertise; appraising source trustworthiness, expert consensus, etc	The knowledgeable supporters circles and arrows (size and color of circles, color and width of arrows) denote lay evaluation of quality and consensus of experts that support the document’s claims.

Research and Publications

What have we found in the research program?

Our research over the past two years has employed iterative cycles of design-based research to improve the EDDiE design. Our findings overall have demonstrated EDDiE’s strong potential in supporting students as they reason about evidence, sources, and expert consensus. Changes to the interface and to the surrounding curriculum have been made in response to our iterative findings. For example, data from the first implementation revealed that students often analyzed and evaluated evidence and identified disagreements without actually using the products of their reasoning to resolve the disagreements. Subsequent implementations have sought to provide better scaffolds to support the actual resolution of the disagreements. Early findings have also suggested the need to develop tasks that highlight one dimension of a grasp of evidence at a time as students build their understanding of how to think about evidence dimension by dimension.

Some of our findings have been reported in publications listed below.

Conference Presentations:

Mochizuki, T., Chinn, C.A., Zimmerman, R.M., Myat Min Swe, Htay Min Khaung, & Sekine S., (2021). Grasping Evidence with EDDiE: A CSCL Tool to Support Collaborative Reasoning about Disagreements in Multiple Documents. In C. Hmelo-Silver, B. de Wever, & J. Oshima (Eds.), 14th International Conference on Computer-Supported Collaborative Learning – CSCL 2021 (pp. 271–272). Bochem, Germany: International Society of the Learning Sciences.

Mochizuki, T., Chinn, C. A., Zimmerman, R. M., Myat Min Swe, Htay Min Khaung, & Sekine, S. (2022). Promoting Collaborative Productive Epistemic Discourse for Disagreement Resolution among Multiple Documents: How Epistemic Scaffolds and Epistemic Scripts in CSCL Worked. In Weinberger, A., Chen, W., Hernandez-Leo, D., & Chen, B. (Eds.) 15th International Conference on Computer-Supported Collaborative Learning – CSCL 2022 (pp. 340–343). Hiroshima, Japan: International Society of the Learning Sciences.

Mochizuki, T., Chinn, C. A., O’Dwyer, E.P., Zimmerman, R.M., Myat Min Swe, Htay Min Khaung, Sekine, S. (2022). Collaborative Inquiry with EDDiE: A CSCL system for resolving disagreements among multiple documents. ICT Demonstration presented at the the Joint 2022 conference of the EARLI Special Interest Groups 20 (Inquiry Learning) and 26 (Argumentation, Dialogue and Reasoning).

Paper:

Mochizuki, T., Chinn, C. A., Yamaguchi, E., & Oura, H. (2022). Information Literacy Based on Epistemic Cognition and Its Learning Environments Design in Post-Truth Era. Transactions of Japanese Society for Information Systems in Education, 39(1), 17-34. (in Japanese)

Workshops for the public:

Chinn, C. A. (2020). Improving students’ thinking in a post-truth world: Growth of epistemic cognition. JSET (Japan Society for Educational Technology) SIG-06 (Collaborative Learning and the Learning Sciences) & 08 (Medial Literacy) Joint workshop.

Clark A. Chinn introduced epistemic cognition and his AIR framework to improve students’ thinking in the post-truth era, where information of various quality, including fake news, is all over the place. The participants engaged in some analytic activities to understand how AIR (Aims and values, Ideals, and Reliable processes) is crucial to understanding the information, and discussed how we should enhance epistemic cognition in the digital era.

Mochizuki, T., Chinn, C. A., Yamaguchi, E., & Ohishi, T. (2022). Designing learning activities to develop skills to read and understand conflicting information: Toward the development of information literacy in the post-truth era. New Education Expo 2022@Tokyo. (in Japanese)

This was the first workshop to introduce EDDiE to the public. We introduced how we think evidence-based comprehension is important as scientific information literacy, how the GoE framework works importantly for understanding the information, and how we taught reasoning about conflicting information in a high school classroom as a prototype before designing EDDiE. Teachers and government officials who are interested in information literacy in the post-truth era participated in trying it out to experience how students collaboratively worked with the multiple conflicting documents with EDDiE.

(A summary in Japanese produced by Kawai-juku is here)