Given the increasing presence of multimedia in K-12 schools, higher education, and professional training settings, it is appropriate to study the impact of multimedia on learning. This edited book represents works in multimedia that include both theoretical and empirical research. Its purpose is to explicate the phenomenon related to multimedia learning and provide a better understanding of how multimedia effects learning affectively and cognitively.
One of the heavily studied topics with respect to multimedia learning is human information processing capacity. Over the last twenty years much of the research in multimedia learning has been informed by the work of memory research, particularly the research on working memory by Baddeley and his colleagues (1986; 1999; Baddeley & Hitch, 1974). For example, Cognitive Load Theory (Sweller & Chandler, 1991, 1994; Sweller, van Merrienboer, & Pass, 1998) and Cognitive Theory of Multimedia Learning (Mayer, 2001) consider the impact of limited capacity of working memory in learning.
Recently, research has focused on human cognitive architecture and multiple representations, including manipulation, in multimedia learning environments (Reed, 2006). This volume includes a collection of works that focuses on multimedia learning and working memory, cognitive architecture and instructional design, and cognitive demands of multimedia learning in a Web-based environment. Readers will find an array of interesting topics ranging from theoretical framework for designing manipulative multimedia to cognitive functionality of multimedia through the lens of semiotics. Other topics explore cognitive mechanisms involving visual and non-visual information representations as well as the complex relationship among learners’ prior knowledge, motivation, context, and task in a simulation learning environment.
Although much is known about the cognitive aspects of multimedia, little research has been done to understand the affective aspects in multimedia learning (Astleitner & Wiesner, 2004; Keller, 1987). This volume contains two chapters on motivation with the first chapter focusing on the motivational theories and models of multimedia learning, followed by a chapter on case study that investigates students’ engagement with multimedia enhanced problem-based learning. Both chapters identify the intrinsic and extrinsic motivational factors related to multimedia learning.
There has been a tremendous amount of interest among researchers and educators who see multimedia as a viable tool for teaching and learning. Researchers are interested in understanding how multimedia contributes to deeper learning and knowledge transfer, when and what external representations should be used to promote such deeper learning, how multimedia should be designed to facilitate information distribution among the external representations in order to better assist learners’ cognitive learning, etc. Readers will find in this volume chapters that focus on the practical aspects of multimedia learning. The topics covered include knowledge transfer in multimedia learning consisting of both vertical and horizontal knowledge transfers, use of multimedia to facilitate students’ research through a goal-tracing methodology, development of scalable, cognitively-grounded multimedia learning tools customized for individual students, integration of multimedia to support linguistically challenged students, and use of multimedia to facilitate analogical learning in a high school physics course.
THE CONTRIBUTION OF THIS BOOK
This book addresses pressing needs in multimedia learning and cognition by (a) identifying the role and function of multimedia in learning, (b) bringing together research in multimedia study with a focus on cognitive functionality, and (c) bridging the theories with practices in multimedia by focusing on the effective use of multimedia in teaching and learning. The book targets educators globally with an emphasis on diverse aspects of multimedia learning and cognition. A major contribution of this book is to bring together multimedia learning theories and practices with an emphasis on cognitive effects in multimedia learning. Thus, the book is significant both theoretically and practically. At the theoretical level, it contributes to the knowledge base in multimedia research. It enhances our understanding of the cognitive functionality related to multimedia learning. At the practical level, the book provides an array of instructional strategies in multimedia learning ranging from knowledge transfer, goal tracing, to scalable, multimedia learning tools which, among others, readers will find beneficial in the design and development of multimedia learning.
The book also reflects the collective effort of multimedia learning theorists and practitioners who challenge the traditional theoretical boundaries pertaining to multimedia and cognition, identify parameters critical to multimedia design, and propose theoretical frameworks that would lead to new research and practice in multimedia learning. Fortunately, we are able to bring together a group of excellent authors who represent various perspectives in multimedia learning and cognition from a broad range of academic institutions and research organizations – from private to public comprehensive and from state and national to international. This book thus appeals to readers from the United States to the international educational community. Researchers and educators will find this book a useful companion as they discover helpful information representing new perspectives on cognitive effects in multimedia learning.
THE ORGANIZATION OF THIS BOOK
The three sections of this book are organized to maximize the value for the readers as they move from the theoretical to the practical and from a focus on cognitive and affective to specific issues of teaching and learning with multimedia.
Section I presents a theoretical perspective on multimedia learning and cognition. It contains eight chapters that cover a wide range of topics on multimedia and cognition that include cognitive architecture, working memory, manipulation, visualization, specialty, and deep learning in multimedia.
In Chapter I, Renee Low, Putai Jin and John Sweller, University of New South Wales, Australia, analyze human cognitive architecture within an evolutionary framework. Using the framework as a basis for understanding cognitive load involved in learning the authors argue that human cognition can be characterized as a natural information processing system which operates on five principles described as a) Information store principle, b) Borrowing principle and reorganizing principle, c) Randomness as genesis principle, d) Narrow limits of change principle, and e) Environment organizing and linking principle. The authors propose that instructional design need to consider these five principles in terms of their effects on cognitive load in learning.
In Chapter II, the concept of control attention in multimedia learning is discussed by Peter Doolittle, Krista Terry, and Gina Mariano of Virginia Tech University, who addresses the role of working memory capacity (WMC) that plays in multimedia environments. Specifically, the authors examine the relationship between individual differences, working memory and attentional control in learning. In their findings the authors discover that students with high WMC perform better than those with low WMC in terms of attentional control measured by recall and transfer tests. The authors thus suggest that individual differences in working memory capacity should be taken into consideration when creating and implementing multimedia instructional environments.
Chapter III raises an important issue related to cognitive load research: the measurement of cognitive load. Anne Cook, Robert Zheng, and Jacquelyn Blaz, University of Utah, review the existing approaches to cognitive load measurement by (a) defining multimedia learning and describing its effects on cognitive load; (b) describing theoretical definitions of cognitive load; and (c) mapping definitions of cognitive load onto commonly used measurement techniques. The authors propose using convergent measures to measure cognitive load which would allow researchers to map different constructs of cognitive load onto their respective behavioral, affective, and physiological components. This would also allow researchers to understand the associations and dissociations among different aspects of cognitive load.
Chapter IV presents a relevant topic in multimedia research, that is, manipulating multimedia materials, which has been, for some reason, understudied. Stephen Reed, San Diego State University, discusses a theoretical framework by Engelkemp (1998) who sees manipulation or haptic learning as a different encoding system that requires a different approach in terms of defining research parameters. Reed points out that although the assumptions of Engelkamp’s model should be helpful for instructional design, they are not complete enough to include the additional demands of multimedia learning. He thus explores the instructional use of manipulative multimedia where the ability to integrate schematic knowledge is highlighted, compared to the ability to recall from action phrases based on which Engelkamp’s theory is formed.
In Chapter V, Katherina Scheiter of University of Tuebingen, Germany, Eric Wiebe, North Carolina State University, and Jana Holsanova, Lund University, Sweden, use a semiotics approach to provide a definition of visualizations as a specific form of external representation, and discuss the differences between verbal and visual representations in how they represent information and how meaning is achieved when learning with them. The authors then discuss basic perceptual and cognitive processes relevant to learning with visualizations which is further used to specify the instructional functions that visualizations have either as self-contained instructional messages or as text adjuncts. Moreover, the role of individual differences in processing visualizations is highlighted.
Chapter VI continues the discussion of visualization but with a different focus: the management of visual split attention in multimedia learning. Florian Schmidt-Weigand, University of Kassel, Germany, explores the viewing behavior in learning from dynamic vs. static visualizations as well as issues related to the pacing of visual presentation. The author concludes that the negative effect of visual split attention can be reduced by implementing a user interaction that allows the learner to adapt the material to perceptual and individual characteristics.
In Chapter VII, Tad Brunye of US Army Natick Soldier Research and Tufts University, Tali Ditman, Tufts University and Massachusetts General Hospital, Jason Augustyn, US Army Natick Soldier Research, and Caroline Mahoney, US Army Natick Soldier Research and Tufts University, offer a discussion the cognitive mechanisms that accounts for the advantages associated with multimedia learning. The authors are specially interested in knowing (a) what effects, if any, do format and modality manipulations have on eventuating memory form and function, (b) which working memory mechanisms are involved in the processing, manipulation and integration of multi-format and multi-modality information, (c) how does the effectiveness of manipulations vary as a function of learning material types, and (d) what, if any, individual differences predict the success of various media combinations? The authors conclude that multimedia is advantageous to learning because well-designed multimedia aligns with the structure and capacity of human working memory. They claim that complementing images when well-designed, reduce cognitive loads and allow resources to be devoted to higher-level integration which facilitates mental model development. Finally, multimedia is more engaging and more likely to accommodate different cognitive and learning styles.
The section concludes with Chapter VIII on the application of multimedia in Web learning. Mike DeSchryver and Rand Spiro, Michigan State University, explore the cognitive load considerations associated with several constructs of deep and extended learning on the Web. They also examine how adjunct online tools and the role of learner motivation may help ameliorate cognitive load concerns when immersed in Web environments. The authors propose a need for a re-conceptualization of Cognitive Load Theory for more ill-structured conceptual arenas. The re-conceptualization emphasizes support for the development of flexible knowledge assembly skills through processes of organic, reciprocal, and deep Web learning.
Section II deals with issues of affective aspects in multimedia learning. The section contains two chapters that introduce motivational theories, explore motivational factors associated with multimedia learning, and suggest ways to incorporate motivational features into multimedia learning resources to optimize learners’ experience.
In Chapter IX, Renee Low and Putai Jin of University of New South Wales, Australia, focus on four major motivation theories (expectancy-value theory, self-efficacy, goal-setting and task motivation, and self-determination theory) and two motivation models (the ARCS model and the integrated model of cognitive-motivational processes). The authors analyze and discuss motivational determinants in effective multimedia learning from social-cognitive perspectives. Important aspects covered pertaining to motivation in multimedia learning including (a) theoretical development, (b) motivational features and design of multimedia instruction, (c) learner characteristics, (d) self-regulated learning strategies and motivational training, and (e) evaluating quality of motivational features in multimedia learning resources.
Chapter X introduces a case study conducted by Min Liu, Paul Toprac, and Timothy Yuen of the University of Texas. Liu et al. offers a unique perspective on looking at learners’ motivational determinants by situating them in a problem-based multimedia learning environment. Their findings reveal eleven key elements that help evoke learners’ motivation: authenticity, challenge, cognitive engagement, competence, choice, fantasy, identity, interactivity, novelty, sensory engagement, and social relations.
Section III presents research studies and conceptual papers on teaching and learning with multimedia. It entails chapters that focus on new perspectives pertaining to the use of multimedia in various educational settings.
The section opens with a theoretical investigation by Michael J. Hannafin, Richard E. West, and Graig E. Shepherd of University of Georgia, in Chapter XI, on the cognitive demands in a user-centered, Web-based multimedia environment. The chapter starts with a discussion on perception, encoding, and retrieving processes related to multimedia learning, followed by an examination on the cognitive styles and strategies as well as self-regulation involved in such learning. Finally, the authors elaborate on the cognitive demands in a student-centered multimedia learning by putting in perspective the issues discussed above. Other important issues being mentioned include metacognition, locus of knowledge, beliefs and conceptual change, prior experience, system knowledge and so forth.
Chapter XII presents a review of research on the use of interactive simulations for learning. Lloyd P. Rieber, University of Georgia, considers the relationship between presentation and interaction. The issues of mental model, constructivist learning, universal design related to simulation learning are discussed. The author supports his discussion with three empirical studies demonstrating that learning should be based on experience, such as that derived from interacting with a simulation, and supported with explanation.
In Chapter XIII, Gina Mariano, Peter Doolittle, and David Hicks, Virginia Tech University, consider the role of transfer in multimedia instructional environments. The authors reviewed the functionality of transfer and its impact on deep learning. Based on a review of 22 studies, the authors concluded that while transfer is often found in the multimedia learning research, it has yet to become a variable of interest. The authors suggest future research should focus on a coherent agenda that explores the use of multimedia to proactively foster transfer as well as the development of metacognition within a multimedia learning environment.
In Chapter XIV, Kirsten Butcher of University of Pittsburgh, Sebastian de la Chica, Faisal Ahmad, Qianyi Gu, Tamara Summer, and James Martin, University of Colorado, discuss an emerging theme in supporting effective multimedia learning: developing scalable, cognitively-grounded tools that customize learning interactions for individual students. They describe the development of a customized tool for science learning, called CLICK, that uses automatic techniques to create knowledge models that can be fed into cognitively-informed pedagogical tools.
In Chapter XV, Mingming Zhou and Philip H. Winne of Simon Fraser University, BC Canada, introduce a goal-tracing methodology to identify students’ goal patterns together with their study tactic patterns. The goal-tracing method provides a bridge to join students’ perceptions about goals with traces that reveal how they seek goals. Using a multimedia learning tool called gStudy, the authors are able to gain insights into students’ implicit goals, their motivation and metacognition in learning.
In Chapter XVI, Alan Koenig and Robert K. Atkinson, Arizona State University, explore how narrative can be used as a cognitive aid in educational video games. It discusses how narrative is currently used in games, and how that modality of presentation, when combined with instruction, is complimentary to the way humans comprehend, store, and retrieve information. The authors offer suggestions for how to instill a functional game-schema in the minds of novice players so that they can be productive in the game environment. They also propose a model for a holistic approach to games research in which a game’s cognitive prerequisites are explicitly studied alongside the more traditional pedagogical measures.
In Chapter XVII, Marian J.A.J. Verhallen and Adriana.G.Bus, Leiden University, The Netherlands, offers a new perspective on how to use multimedia to enhance language comprehension and language acquisition skills for second language learners. The authors support their discussion with empirical data from three experiments. They conclude that multimedia such as animated digital storybook promotes young linguistically disadvantaged to become more engaged in and therefore investigate more mental effort in learning.
Finally, Wolff-Michael Roth, University of Victoria, BC Canada, describes a naturalistic case study designed to investigate knowing and learning in a real classroom setting where students use computer simulation tool to learn about Newtonian motion. The study purports to show the mediatory role of the instructor and teaching strategies in digital simulation learning. It reveals how teacher assistance may help students in bridging from the multimedia context to the real world, and how teaching strategies may allow relevant structures to become salient in students' perception which allow them to generate analogies and thereby learn.