Back to my Welcome page! 

Presentation Formats in Concept Formation:
An Experimental Study
[1]

Patric Dahlqvist

Department of Computer and System Sciences
Stockholm University
July 1997

"Thinking’s a dizzy business" Dashiell Hammet.


Abstract:

This thesis presents some basic research about cognitive science as a whole and learning in particular. A study was conducted in which presentational formats and their implications on learning were studied within the domain of physics. The part of physics that was studied was the principle of equivalence. The results from this study indicate that animation does not facilitate learning in this domain, but an explanational format that is closer to real life than what is traditionally used within physics can be beneficial.


Table of Contents

1 Introduction

2 Cognitive Science

3 Learning

4 Instruction

5 Presentational Formats

6 Problems Focused

7 Method

8 Results

9 Discussion and Future Research

10 References


1. Introduction

When new media, particularly computers, become cheaper and more readily at hand there will be dramatic effects on how teaching and learning within and outside traditional school settings take place, or should take place. The opportunities to use and the pitfalls that will have to be overcome are many. We now have to study and learn how these new media can support learning. We must look into different situations of learning: distance learning, learning in a group or by yourself, etc.

The activities that a computer affords are triggers for a system-wide change in the learning environment [Salomon, 1992]. That is, it may not be just in computer-based instruction that these changes occur but rather in all of the environments where people learn and teach.

In such a context there are several interrelated aspects that have to be considered: What can we learn from existing learning theories, and do these theories need adjustment to fit into the new media? Do we need new approaches to instruction or can we adopt the existing ones? Can we learn anything from the science of cognition? If so, from a pragmatic point of view, which approach is most fruitful? The traditional approach regarding representations as something exclusively inner or mental with an emphasis on abstract symbol manipulation disregarding emotional and motivational aspects, or the new approach referred to as theories of situated cognition where situated and social aspects of cognition are emphasised? That is, on the one hand cognition as something in our heads, and on the other, cognition as something "out there".

Naturally, basic assumptions regarding human cognition have implications both for learning theories and instruction.

Learning is a central field in Cognitive Science. How do people acquire knowledge? This is interesting in several ways. If we know how people acquire knowledge then we may be able to program computers to do that too. And we may also be able to construct better instructional material; books, software, video courses etc.

1.1. Chapter Overview

In this chapter there is a general introduction, an overview of the rest of the thesis and the thanks to the people that helped me perform this work.

In chapter 2 cognitive science is described; the scientific disciplines involved are described and the consequences of such a view on knowledge in this context are discussed.

Chapter 3 describes different views on learning and the phenomenon of transfer of knowledge is explained.

Instruction is the topic of chapter 4 and the role of the computer in a context of instruction is discussed.

How to present information in instructional material is discussed in chapter 5. Three types of presentations are described: natural language, illustration, and animation.

In chapter 6 the problem of this thesis is focused and some ideas of interplay between different kinds of presentational formats are discussed.

Chapter 7 describes the study reported on in this thesis and how it was conducted.

Chapter 8 reports on the results from the study described in chapter 7.

A discussion about the results from the study and needed future research is conducted in chapter 9.

1.2. Acknowledgements

Without the support and help from several people I wouldn’t have been able to complete this thesis.

First of all I would like to thank my adviser Robban for his everlasting enthusiasm and support in times of need. Catch a big one.

I also would like to thank Ywonne Waern for the insights, the ideas, and the stimulating discussions.

The person with the great knowledge in Physics and Physics teaching is Gunnar Edvinsson. Thank you for the support, the explanations, and for putting up with a confused computer student.

A special thank to Mattias for the prooof riding.

There are several others I would like to thank all at once:
Cia for kicking my butt when I needed it.
Torleif and Anna for lodging and mind sharpening.
Majk for the statistics.
Klas for valuable insights and comments on earlier versions of this thesis.
Fredrik for reading and commenting during the work with this thesis.
Christer Nilsson and Enikö Lukas for their help with the lab arrangement.
Lars Franzén for the computer support.
Kia for helping me finding an article.
Pia for answering the phone.
And of course my Mother and Father without whom I hadn’t been at all.
Thank you all very much. It wouldn’t have worked without you.

The work has partially been financed by The Swedish board for research in the Humanities and Social Sciences.

2. Cognitive Science

In this chapter cognition and cognitive science will be described, and some of the disputes about basic assumptions regarding human cognition are also ventilated.

2.2. What Is It About?

In the literature one can find several, in many respects, diverse definitions of cognition and cognitive science (the science about cognition).

Cognition is according to [Websters, 1997]:
"the act or process of knowing including both awareness and judgement; also: a product of this act", it comes from "Latin cognition-, cognitio, from cognoscere to become acquainted with". Thus cognition is the "things" we humans do to perceive and understand the world [2] around us.

Another definition, provided by one of the more influential researchers in the area, Howard Gardner [Gardner, 1987], describes Cognitive science as "a contemporary, empirically based effort to answer long-standing epistemological questions –particularly those concerned with the nature of knowledge, its components, its sources, its development, and its deployment".

In short, Cognitive Science is the science of the mind. It is a mixture of different scientific fields in search of how the human mind works. The field of Cognitive science grew out of several different scientific disciplines with its own particular area, or subarea, of interest.

2.2. Disciplines

The human mind and its functions are central to several scientific disciplines. The disciplines most often referred to as belonging to cognitive science are: philosophy, artificial intelligence, anthropology, psychology, neuroscience, and linguistics.

The concept of knowledge and what that is, has been debated in Philosophy ever since the ancient Greeks and even before that. Descartes (though neither ancient nor Greek) formulated the modern epistemological basic question: When can knowledge not be doubted? His method was to doubt everything he saw and sensed. But eventually he came down to the thought "I think, therefore I am.", and that, he claims, can not be doubted. Many philosophers has conquered his thesis since then, but the question is still there.

In Artificial Intelligence (AI) the scientists are trying to mimic the human mind or at least to learn from it, because it is the only intelligent [3] system that we know of. For example, a great deal of work has been put into the area of problem solving [Russel & Norvig, 1995], which also is a central issue in cognitive psychology [Andersson, 1995]. Another area where much effort has been made is perception. David Marr made some great advances during his all too short research life. He changed the view on perception completely.

Anthropologists are trying to find out how the mind is affecting, and is affected by, the culture and the social setting it is in. An example is a study conducted by Marshal Segall, Donald Campbell and Melville Herskovits in the mid sixties [Segall, Campbell, & Herskovits, 1966]. In the study they found differences in visual perception between people from different cultures.

In Psychology the mind is obviously a central issue. In a field called Cognitive Therapy one uses the knowledge of how a healthy mind works to "reprogram" people with grave misconceptions of the world, e.g. phobics. In cognitive psychology human cognition is studied and this involves areas such as problem solving (mentioned above), perception, attention, learning, and memory for instance. This area has had a great influence on AI.

Neuroscience studies the nervous system whose biggest part is the brain. Most people think that the mind is in the brain, but that is not a distinct truth. The debate about the dualism of man (are body and soul two different entities?) has been around for a long time.

A Philosophical branch, reductionism, claims that when the neuroscientists are done with their quest there is nothing more to say about learning. That is to say that the secret about learning lies wholly in how the neurones work and co-operate [Gardner, 1987]. Gardner takes another point of view. He chooses to paraphrase Wittgenstein when he claims that "one can know every brain connection involved in concept formation, but that won’t help one bit in understanding what a concept is".

Linguists study Language, and there is a belief that our thinking is affected by what language we "think" in. Edward Sapir stated that a person’s processes of thinking are structured by the particular properties of the language spoken by that person [Gardner, 1987]. This is a controversial line of thought and there have been many adversaries. Noam Chomsky, for example, has put forth a theory where he claims that there is a universal "deep structure" in all languages [Gardner, 1987].

The different disciplines contribute to Cognitive Science with different views and knowledge. The interdisciplinary exchange of knowledge and ideas has been fruitful for all the parts but in quite different ways. For example, AI has developed different techniques using neural networks, directly inspired by Neuroscience knowledge of how the human brain works.

2.3. Epistemology

Within cognitive science important philosophical questions are raised. One of these questions concerns knowledge.

The nature of knowledge is essential in Cognitive Science. How we, as humans, acquire, store and use knowledge is very much a function of what knowledge looks like. Throughout history the nature of knowledge has been a constant topic in the Philosophical debate, from Aristotle to Wittgenstein and up till today. And there is still no answer to the question: What is knowledge?

Another important issue is the status of knowledge. How can we know if knowledge is objectively correct or not? Gettier [Harman, 1973] gave an example: Imagine standing in a dark room and in front of you, you see a lit candle on a table. But, the fact is that a "devious" person has been in the room and put a screen in the way of the candle and then arranged some mirrors so that the candle appears to be in the place where it actually is. Is it possible to say that you actually know where the candle is located? Are you entitled to believe that the candle is located where you see it? Some philosophers say yes, others say no. Some say that your belief is justified even though the belief is not objectively correct.

2.4. A Traditional View of Cognition

In recent years there has been a debate in the cognitive community about how to look at cognition (e.g. Cognitive Science, Vol. 17, Number 1, 1993). There are the "traditionalists" (McCloskey, Anderson etc.) who claim that cognition is something individually inner or mental. On the other side there is the situated cognition camp (Suchman, Lave & Wenger etc.) which focuses on the social and cultural aspects of human cognition.

The traditional approach to human cognition is based on some basic concepts. The concept of intention, for instance, states that for a person to perform a certain action, that person will have to have an intention to perform that action. And in that the person strives towards a goal. To be able to have such intentions the mind has to be able to represent this goal in one way or another. It is also assumed that humans need to act rationally to fulfil these goals. This view is based on the belief that human cognition is something that goes on inside the heads of humans; that it is something individual. This is a kind of mechanistic approach which claims that human cognition is basically symbol manipulation.

If one accepts this claim it raises a lot of philosophical and/or religious questions. If intelligence basically is symbol manipulation then it is theoretically possible to build intelligence, and that is a very hard thing for a lot of people to accept.

To look upon cognition as basically symbol manipulation is the traditional view and most of the work in Cognitive Science has been done in this paradigm. Therefore most of the theories and models are based on this basic assumption.

However, this view on cognition has received some criticism during the last years, regarding the assumptions that are made concerning human cognition. In the next section some of this criticism will be presented.

2.5. A Situated View of Cognition

The situated view on human cognition is not a unified theory but has been put together because of some common criticism against the "traditionalists". The criticism focuses on the lack of social aspects and the individualistic approach in the traditional theories [Ramberg & Karlgren, 1997].

It seems obvious to state that human minds develop in social situations. But cognitive theories have not, historically, taken these factors into account [Pea & Brown, 1991].

Though recently a community of researchers has raised these issues and tried to incorporate them in their research. For example, Holm and Karlgren [Holm & Karlgren, 1996] state that "Human actions are always situated in (a) a physical context, and (b) a social and cultural context." That is, you have to consider both the physical as well as the social environment when reasoning about human cognition.

The symbol manipulation assumption, of the traditional view, is challenged from several points of view. One is the sheer amount of information processed in the human mind. It seems that for every new level of information unveiled there is another to discover [Norman, 1993]. This amount of information is just too big to be handled via symbol manipulation. The neurones have to be much faster than they are for the math to add up.

Another criticism against the symbol manipulation assumption comes from Ramberg and Karlgren. They reason about our ability to follow rules and how to interpret those rules when confronted with novel situations. There has to be social agreement about how to interpret a rule otherwise the rule would no longer be a rule [Ramberg & Karlgren, 1997].

There are some good criticism on both sides of the discussion (traditionalist vs. situaters) e.g. diSessa pointed out that "individuals do – sometimes for extended periods of time – think by themselves" (cited in [Karlgren & Ramberg, 1996]), but on the other side, the situated side, the claim is that language is essentially social and much of our understanding is based on language.

3. Learning

In this chapter I try to describe the nature of learning and which kind of learning is discussed in this thesis.

One may reason about learning at different levels. One way is to see evolution within species as learning. This is probably not what we mean in our everyday language when talking about learning. Another kind of learning is tactile learning, e.g. learning to walk, that is knowledge of how to move and use the body. In this thesis, however, the focus is on the learning of cognitive skills and abilities.

In the days of the early behaviourists, learning was thought upon as relatively automatic formation of bonds between stimuli and response. Some of the early adversaries were the Gestalt psychologists, who argued that learning involves the perception of relations among events. In a famous series of experiments Tolman and his associates demonstrated that learning can occur even if there is no reinforcement. They showed that rats learned to find their way in a maze quicker if they had been in the maze before, but without the reinforcement of food in the end of the maze [Smith, 1993].

Ohlsson [Ohlsson, 1996] makes a distinction between skill acquisition and higher-order learning. In skill acquisition the outcome is competence. The outcome of higher-order learning, on the other hand, is understanding. Skill acquisition has throughout the short history of cognitive science been quite successful. There exist models like ACT-R and SOAR which are both theoretically powerful and empirically founded [Reimann & Spada, 1996]. Research about higher-order learning, on the other hand, is still in its infancy [Ohlsson, 1996].

It is quite possible to succeed in a task without understanding how and why, e.g. it is possible to throw a javelin without knowing anything about kinetic or static energy. The converse is also true. Failure does not imply lack of understanding, e.g. you can know everything there is to know about kinetic and static energy and still not be able to throw the javelin [Ohlsson, 1996].

The research about higher-order learning must turn away from goal-directed behaviour. This is because of the lack of observable indicators of understanding. Ohlsson states that "The root of the problem is that understanding is a state of mind, not a process." [Ohlsson, 1996].

3.1. Transfer of Knowledge

One interesting aspect of learning is the ability to transfer knowledge from one domain to another, e.g. when taught the law of gravity you may apply it both to planets in space and to an apple falling from a tree.

A perhaps even more interesting phenomenon is the inability to transfer certain knowledge. Carraher, Carraher and Schliemann performed a study on Brazilian school children who also worked as street vendors. They found that the children where able to calculate the right prize of e.g. five lemons at 35 cruzeiros a piece. But when asked to perform the calculation "5*35=?" the percentage of right answers was significantly lower [Andersson, 1995].

In the traditional view of cognition it is assumed that there is a definite bound on how far knowledge will transfer, and that becoming an expert in one domain will have little effect when learning another area. But we are always confronted with novel situations even within our area of expertise. That is, the boundary on how far knowledge is transferable is quite fluent[Ramberg & Karlgren, 1997].

To learn how to facilitate transfer is a major challenge to the learning research community.

4. Instruction

It is convenient to part the research community of instruction in two major schools: constructivism and instructivism.

Constructivism focuses on the learner creating knowledge of his or her own by exploring the environment, and tries to facilitate these explorations with a rich set of cognitive tools[Wasson, 1996]. This school also claims that knowledge can not be defined objectively.

Instructivism, on the other hand, lets the content of the learning material be selected, sequenced, structured and presented on behalf of the learner[Wasson, 1996].

How you pick your view on learning theory and instruction is very much based on which view on cognition you adhere to. For instance, a situated or social view of cognition holds that the traditional view is misleading or even wrong. A situated/social view emphasises social and cultural aspects of learning, cognitive apprenticeship for instance. Instruction then becomes taking part of activities in a certain community of practice.

A common view on knowledge in the area of Physics is that novices possess a theory of roughly the same quality as a scientific theory. This view, let’s call it "theory theory", calls for instruction that provokes a theory change by presenting evidence and arguments that the currently held theory is wrong[Karlgren & Ramberg, 1996]. McCloskey is probably the most visible of the theory theorists [diSessa, 1988].

Andrea diSessa [diSessa, 1988] holds a different view. He claims that "intuitive physics is a fragmented collection of ideas, loosely connected and reinforcing, having none of the commitment or systematicity that one attributes to theories.". He claims that a much broader attack has to be made. It doesn’t suffice to confront the intuitive theory with evidence of its wrongs, but rather an attack has to be made on all the pieces of knowledge.

When creating instructional material you have to be aware of what your goal is. Understanding, remembering, or applying are three different outcomes. A strategy that facilitates learning in one of these outcome classes need not facilitate learning in the others [Levin, 1989].

4.1. Computer Based Instruction (CBI)

In the view of the above stated: What is it that computers have to offer to the learning environment?

Computers support both audio and visualisation, which you can also get out of a TV. But a computer has something more to offer: it can be more interactive.

What then is interactivity? You kind of interact with the TV when writing a letter to the producer complaining about the content of a program. One definition of interactivity is provided by Steuer [Steuer, 1992] who describes it as a function of three parameters: speed, range, and mapping. From this definition you can not describe something as interactive or not, but rather as more or less interactive compared to something else. Then e.g. interacting with a computer is more interactive than interacting with a TV set.

With this ability to be more interactive than other media it is possible to build learning systems that allow exploration and in some cases elaboration. The ability to elaborate may in some cases be very fruitful, especially when there are practical obstacles which hinders the student to do the ‘real thing’ e.g. cardiac arrest (example from [Woolf & Hall, 1995]).

The interactivity of a computer makes it "easy" to construct environments that invite to exploration and lets the learner take control of the pace and sequencing of the material.

One great option is the possibility to simulate and visualise abstract processes, e.g. within the area of physics you can do simulations with gravity or watch tiny particles collide. This ability is probably a good way to enhance understanding of these quite complex and abstract concepts.

Stella Vosniadou claims that computer-based learning environments have a distinct advantage over traditional methods when providing instruction about counter-intuitive phenomena, e.g. the earth is spinning but we don’t feel this movement [Vosniadou, 1992].

So called microworlds invite the student to free exploration of a restricted conceptual domain [Kommers, 1991]. This is a popular approach when constructing Computer-based learning environments.

Cognitive apprenticeship [Lave & Wenger, 1991] is an approach to learning which introduces the learner to a culture by authentic activities and social interaction. This approach is taken in TPLC [Ramberg & Karlgren, 1997], a system developed to teach sales personnel at Pharmacia Biotech protein purification. In this system the learner is confronted with a sales situation and is to find out what kind of problem the made up character has and to solve this problem (that is to sell a machine). The learner is to be (it is not yet implemented) accompanied by a "role model" and this mentor should be able to take over the interaction with the buyer. The mentor is not to analyse the faults in the learner’s behaviour or to give explanations, but the learner is to observe how the mentor handles a situation and solves problems.

5. Presentational Formats

When constructing instructional material, whether it concerns books, videocourses, or educational software, there are many different ways to present the learning material. The choice of media puts different restrictions on which presentational formats are possible. For example, it is not possible to use sound in an ordinary book. But there are still many choices to be made: text or picture, diagram or images, etc.

5.1. Natural Language

One choice to make is between natural or a more formalised language. For example, in Physics a phenomenon can often be described in a narrative way or through mathematics.

Karlgren and Ramberg [Karlgren & Ramberg, 1996] used the concept of language games to study how people change their conceptual ideas about, in this case, protein purification and Newtonian physics. In their view, a scientifically correct Newtonian description of an idealised situation is a specialised form of speaking. This may be counter-intuitive viewed from a common sense perspective of the world that has entirely different goals. Learning a scientific language is not viewed as replacing an old and "naive" knowledge structure (representation), as much as learning new activities and a new specialised language. In learning, people have to learn how to use a specific language by taking part in the activities in which the language is used, since the meaning of the new concepts are rooted in these activities. Understanding the concepts is thus coupled with participation in the new activities. Thus, this view on learning, as being the capability to use the language in the domain, emphasises language use and taking part in activities.

5.2. Illustrations

The use of pictures in prose has been thoroughly studied by many research programs (for a longer summary see [Willows & Houghton, 1987] or [Mandl & Levin, 1989]) and it is quite clear that, used with some common sense, pictures facilitate learning from text. For example, Levin [Levin, 1989] states that "In cases where text-embedded illustrations are relevant to (i.e., largely overlapping or redundant with) the to-be-remembered content, moderate to substantial prose-learning gains can be expected.".

Brooks [Brooks, 1967] found that if people were given a complex message that needed to be visualised, recall of the message was enhanced if it was presented solely in auditory mode rather than in a combination of auditory and visual modes in which people simultaneously had to listen and read the message. This suggests that the visualisation process interferes with reading. It also shows that it is not just a matter of adding more pictures to your learning material, but that it is a rather complex issue on how to combine different modalities in different contexts.

Levin [Levin, 1989] has classified pictures in prose learning in five different functions:

    1. Decoration, where the pictures are not related to the text and may be assumed simply to "decorate" it (this includes making a text more attractive to capture the readers attention).
    2. Representation, where the pictures basically overlap with the text, thereby making the text more concrete.
    3. Organisation, where the pictures add structure to an otherwise poorly or weakly organised text passage.
    4. Interpretation, where the pictures help to make a difficult-to-process text more comprehensible.
    5. Transformation, where the pictures are designed explicitly to enhance the memorability of a text by transforming it into a more mnemonically powerful form.

The choice of illustration must be made with caution. The illustration must contain the right amount of information. Designers have frequently found that realistic pictures carry too much information for effective instruction [Winn, 1987].

There is no uniform theory or even design principle that states how to use pictures in all learning situations. It is very much dependent on the task. The saying that "a picture is worth more than a thousand words" is not always true [Winn, 1987]. Levin [Levin, 1989] joins the cautious choir and states: "Two things that we have learned from research on pictures in text are that pictures are not uniformly effective in all prose-learning situations, and that not all types of pictures are equally effective."

Mayer and Sims [Mayer & Sims, 1994] also found that there are differences between people with different cognitive abilities, e.g. people with high spatial ability seem to a greater extent to be more able to enhance learning with the help of visualisations.

5.3. Animations

In many computer-based instructional products animations have become popular. Unfortunately, the animations are often used to impress rather than to teach [Rieber, 1990a].

There is a lack of theoretical foundation for the use of animations in computer-based instruction. Animated graphics represent a subset of instructional graphics but to which extent animations depart from and coincide with static visuals is questioned [Rieber, 1990a].

There has not been very much research done on how, if at all, animations can facilitate learning. However there are some studies [ChanLin & Chan, 1996; Mayer & Anderson, 1992; Mayton, 1991; Poohkay & Szabo, 1995; Rieber, 1990b; Rieber, Boyce, & Assah, 1989]. The problem is that the results are inconsistent.

Rieber [Rieber, 1990b] shows that animations facilitate learning for children (under certain conditions) but not for adults [Rieber, Boyce, & Assah, 1989]. On the other hand there is for example the study by Mayton [Mayton, 1991] which suggests that the use of animations in computer-based tutorials can be beneficial for adults.

Even though the results from the research on animations in instructional material are mixed to some degree, there still appears to be significant potential for the use of animations in computer-based instruction [Milheim, 1993].

Palmiter and Elkerton found in a study [Palmiter & Elkerton, 1993] that in a condition of text only, users spent less time learning a different, but similar, task than did the users furnished with animations. This suggests that the ability to transfer knowledge is suppressed by the animation.

In contrast to static graphics, animated graphics can show information about two important visual attributes: motion and trajectory. Animations can provide information about an object’s motion, if it is moving, if the motion is changing, and how it is moving (path, patterns, etc.). They can also show information about which way the object is moving [Rieber, 1996].

Milheim has put together a set of guidelines [Milheim, 1993] on how to design and use animations in instructional material. Some of these guidelines are:

As can be seen these guidelines are quite common-sense though most of them founded on studies. This illustrates one of the problems with research about animations, it seems rather complicated to reach beyond what is self evident.

6. Problems Focused

It is fairly well established that visual instruction aids are very powerful means for enhancing learning [Wærn, 1995].

However, since human attention and perception is limited, different presentational formats can interfere with each other [Andersson, 1995].

6.1. Interplay Between Presentational Formats

Paivio has put forth a theory about the processing of information in the mind, the Dual-Coding theory. In this theory he claims that the human mind has two distinct (but interconnected) systems, one to process language and another for the rest of the information [Paivio, 1986].

To support this theory Farah [Farah, 1989] argues that there exist patients that understand pictures but not text (language) and vice versa, and that this is a "proof" of the mind having two subsystems. But she also gives some evidence that there is a common system deep down. It is clear that processing is not totally separate because we can describe a picture and draw a picture from a description.

These theoretical arguments promote the use of multiple presentational formats in instructional material. This claim is also founded on some empirical findings.

Mousavi, Low, and Sweller [Mousavi, Low, & Sweller, 1995] for instance, claim to have increased learning of geometry problems by mixing auditory and visual presentation modes.

As mentioned earlier, there is a shortage of studies on the effect of animations in instructional material. Mayer and his associates have conducted some studies on how to combine pictures, animation and text to increase learning [Mayer & Anderson, 1991; Mayer & Anderson, 1992; Mayer & Gallini, 1990], they all look at problems which are quite practical, e.g. a bicycle pump.

Therefore, it would be interesting to study the use of animation on a more abstract phenomenon, e.g. the principle of equivalence (a principle within Newtonian mechanics).

In line with this, it would also be interesting to compare the results with a study performed by Ramberg [Ramberg, 1996a]. In that study he examined the role of explanations and presentational formats for learning the principle of equivalence. He found little or no difference in learning between the different presentational formats, but the focus of the study was on the role of explanations rather than the presentational formats. In the concluding remarks he suggests further investigation on the role of presentational formats.

It would also be interesting to investigate if there are any pros and cons in conducting a study of this kind on paper, as performed in the study by Ramberg, as compared to on a computer, as reported on in this thesis. If the negative aspects of conducting the tests on a computer are small there should be significant effeciency gains, when there is an automated way of handling the data collected. In the future the results of these studies are to be applied on CBI, and it is preferable to have the test environment resemble the actual applicational environment as much as possible.

7. Method

A study was conducted at the department of physics at Stockholm University. It tested if there where differences in learning the principle of equivalence when provided with different kinds of illustrations.

7.1. Subjects

35 first year students counted as subjects. The principle of equivalence was part of the curriculum for the course they attended during the period of the study. The study was conducted before the part, in the course agenda, containing the principle of equivalence was reached. All the subjects participated in the study voluntarily and no compensation was offered.

7.2. Design

To perform the study a small CBI-program was constructed. This program tries to teach the students one small subfield of Newtonian mechanics: the principle of equivalence. A test was also administered in the program in order to test if the subjects had learned something from the instruction phase. The study was conducted in groups of 3 to 10 subjects at a time. A research leader was present in the room during the study. All the answers from the subjects were saved in a text file and collected when the session was over.

7.3. Material

7.3.1. Illustrations

The illustrations were of three different kinds: abstract, analogue and animated analogue. Abstract here means a classical illustration with arrows representing vectors which represent forces (gravitation and inertia) (see Figure 1). This illustration was constructed by Robert Ramberg for another study [Ramberg, 1996b] relating to the same area in physics.

Figure 1:The abstract Illustration.

The analogue illustration is a more reality based type of illustration which the student can relate to, in this case a railroad cart seen from the inside, with a helium balloon attached to the floor and a steel ball hanging from the ceiling (see Figure 2).

Figure 2: The analogue illustration.

 

The animated analogue illustration consists of the same setting as the analogue with the exception that movement is added [4]. In the animation, the sequence proceeds from:

The sequence is then repeated.

To clearly illustrate what happens, the scenario in the railroad cart has been idealised. The movement of the railroad cart is illustrated by moving the telephone poles in the background.

The animation was created using Autodeskâ Animator Studioä running under Microsoftâ WindowsNTä 3.51 on a Personal Computer (PC). The animation was saved as a Video for Windowsä (.AVI) file using Microsoftâ RLEä encoding.

7.3.2. The Program

The program [5] consists of four phases: an introduction, a learning phase, a test phase, and a debriefing phase.

The introduction consists of a text that explains some practical details about the program. The introduction also contained three background questions regarding the subjects gender, age, and previous knowledge of the principle of equivalence.

The learning phase consisted of a text and one of the illustrations described above. The text was divided in four parts containing increasingly more detailed explanations of the principle of equivalence. When the subjects were finished (with the learning phase, before the test phase) they were asked to estimate (on a scale ranging from 1 - 10) how much they felt they had understood and how much they felt they had learned.

The test phase consisted of three different types of problems:

    1. In one problem the subject’s task was to predict what would happen to grass growing on a record player (the old-fashioned kind with a turntable). This problem was varied so that in one problem the scenario consisted in a rotating turntable whereas in another, the turntable was not rotating.
    2. Another problem consisted in the subjects filling in the missing parameters in one of the abstract illustrations stipulated that a particle is accelerating to the left or to the right (two questions).
    3. In the third problem the subjects were to predict what would happen to the balloon in the railroad cart stipulated that the railroad cart is accelerating, keeping an even pace, or applying its breaks (three questions).

All the questions were multiple choice questions. After having answered the questions the subjects were asked to motivate their choice, i.e. give an explanation in plain Swedish (or English). After each question the subjects were asked to estimate (on a scale from 1 - 10) how difficult they experienced the problem to be, and also, how confident they were that they had answered it correctly.

In the debriefing phase the subjects were asked about how much they felt they had understood and learned. There were also some questions regarding the use of the program itself.

There were three different versions of the program corresponding to the three different types of illustration. Within the three different versions there were two different orderings of the test questions (called 1 & 2), summing up to six different versions. The variation in presentation order was performed to eliminate any effects of priming (that is, solving one problem having an effect on solving the next problem presented). In both versions the two questions about the grass on the record player (see above) came first. In the version called 1 the two questions about the missing parameters followed, and after that the three questions about the balloon in the railroad cart. In version 2 it was the other way around.

The program was constructed using Borlandâ Delphiä 1 on the above stated PC.

7.4.Procedure

On entering the room where the study took place the subjects randomly picked a computer with one version of the CBI-program already started. There were 10 computers in the room, and each session therefore consisted of a maximum of 10 participants. During the time the subjects ran the CBI-program the supervisor was always present in the room. The supervisor answered questions regarding the use of the program, when there were misconceptions due to language difficulties (some of the subjects did not have Swedish as their native language), or other non physical science related questions.

When seated, the subjects followed the on screen instructions. First they answered a few background questions, they proceeded to the learning material, followed by 7 multiple choice questions, and finally there were some questions regarding the subjects own opinions on the material and the program design. After the subjects were finished they just left the room to go on with their lives, and the supervisor collected the data which had been saved in a text file.

8. Results

The participants’ answers were grouped according to which kind of illustration they received during the learning phase. These groups are hereafter called: abstract, analogue, and animated. There were 35 participants divided on 11 in the abstract group and 12 in both of the analogue and animated group.

8.1. Quantitative Results

Due to the small amount of subjects (N=35) some of the planned statistical analyses could not be performed. This concerns multivariate analysis. The small N also affects the results of the analyses that have been made.

A test of significance regarding the differences between the groups estimations were calculated with an ANOVA test using p=0.05 [6].

A test of significance regarding the differences between the amount of right answers, within and between groups, was, due to the small number of participants, not calculated. It was estimated with visual inspection.

The answers were counted as correct when totally correct. In questions 3 [7] and 4 (hereafter called the abstract questions) there had to be four right statements to count as a right answer (the correct picture and three correct parameters).

On question 1 and 2 (hereafter called the grass questions) there were almost identical results in all the groups. There was almost the same number of correct answers and the mean values of the estimations, on how certain they were of the answer and how hard the question was experienced to be, were very close to each other.

On the first of the abstract questions (3), and to some extent even the second (4), the analogue group performed better than the other two groups. The analogue group had 5 right answers, the abstract group 2 and the animated 1. (On question 4 the results were 3, 2 and 1.) This, taken together with the fact that the analogue group had significantly lower estimations (p<0.05) on how certain they were of their answers as compared to what the abstract group had, made for an interesting result. [8]

On question 5, 6, and 7 (hereafter called the balloon questions) the analogue and animated groups where slightly better than the abstract group. See Table 1.

Table 1: The number of right answers for the balloon questions (5, 6 and 7).

The estimations regarding how hard the program was to use were low, the mean value was 1.8. [9] The understanding regarding what to do and how to do it was high, the mean values were 7.3 and 8.2 respectively. The participants also thought that the illustrations were quite clear, the mean estimation was 7.2. On the question regarding how much the subjects experienced the illustrations had supported their learning, the analogue group had a slightly higher estimation than the animated group which were a little bit higher than the abstract group. These differences were calculated with an ANOVA test (p<0.05) but not found to be significantly different. The mean values were: for the analogue group 8.0, for the animated group 7.1, and for the abstract group 6.4 The standard deviations were about 2.5 for all the groups.

8.2. Qualitative Results

When comparing the answers and the motivations it seems that most of the subjects that have answered correctly also have a good understanding of the principle. There were a few obvious "guessers" but they often seem to have guessed wrong.

In the first grass question (1) some of the subjects seem to have the notion of acceleration and centripetal force mixed up. It also seems that some of the subjects tend to "forget" gravitation, when reasoning about the second grass question (2) they say that "there are no forces affecting the grass". Failing to recognise that grass is striving against gravitation seems to be the major reason for not being able to get the first grass question (1) right.

It is quite clear that most of the subjects do not use the Principle of equivalence when reasoning about the second grass question (2), they just "know" that grass grows straight up. The motivations are rather ad hoc, mentioning that this is how grass grows and that is that.

On the abstract questions (3 & 4) the motivations show that many of the subjects have the notions of acceleration and inertia confused. But almost all the subjects reason about different forces and vectors and their directions.

In the balloon questions (5, 6 & 7) most of the subjects are back to more "personal" explanations like; "when I am in a train ..." and alike contrary to the abstract explanations of the abstract questions (3 & 4).

15 of the subjects chose to leave a comment on the last "page" of the program. These comments ranged from specific opinions on how the layout was designed to very general remarks on the use of CBI.

Some of the comments concerned the fact that the subjects did not receive any feedback concerning the correctness of their answers. In an instructional material with tests this is probably a good idea, but in the test phase of this study it would probably have interfered with the objective to measure the outcome of the learning phase.

There were comments on the fact that some of the subjects experienced problems reading from the computer screen as compared to from paper. This is an opinion that has to be taken into account when designing CBI-material. In this study however there is no testing on how this effects learning. For a deeper view on this issue see [Bailey, 1989].

Due to insufficient testing there appeared an error in the text files containing the answers for the first 12 subjects. This error consisted of a loss of some of the motivations and comments the subjects gave as an explanation to their answers and in the end of the program. The loss appeared when the comments were longer than 255 characters long (including spaces), and if the subjects motivations were longer the 256th character and forward were lost. After this error was detected the program was corrected and the rest of the subjects’ motivations were saved completely. My estimation, from viewing the data and being in the room when the subjects took the test, is that motivations from 4 subjects were cut short. That is, not all of the 12 subjects, that did the test before the fault was corrected, gave motivations and/or comments that long.

9. Discussion and Future Research

One question that needs to be raised is whether it is possible at all to figure out how people learn. This is connected to if it is, at all, possible to have a science about the mind. Is it possible to observe the mind?

In this thesis I neither solve this problem nor even try to do it. But this discussion is important to have in mind when conducting research about learning. Otherwise it is easy to draw hasted conclusions.

Another note that has to be made is that this study had too few subjects to draw any concrete conclusions. However, it made for a number of interesting results that are to be looked into in future studies.

One interesting result reported on in the previous chapter is that the analogue group seemed to understand the abstract problem better than the other two groups. This taken together with their low estimations on how certain they were of their answers, compared to the other groups, is rather fascinating.

One hypotheses is that the analogue group has a deeper understanding of the principle but when presented with a presentation of the problem they have never seen before they feel uncertain. This however does not explain why the animated group did not have the same low estimation of certainty.

It is reasonable to assume that the analogue and the animated groups should outperform the abstract group on the balloon questions. This because they are more familiar to the presentation of the problem. But following this line of reasoning the abstract group should outperform the analogue and animated groups on the abstract questions. This did not happen. This could be because the abstract question was too hard (small amount of correct answers), but this does not explain why the analogue group outperformed both the other groups on those questions.

The reason to have the grass questions was that it would be a different presentation format for all the groups. If any group would have had a better score on that first question I believe it would have been a reason to believe that its presentation format made for a better and deeper understanding of the principle of equivalence. This did not happen, but this does not say that there are no differences between the presentation formats. More research is needed to sort these questions out.

The use of a computer, as contrary to paper, in this kind of study shows promise to be efficient. In this study there were the above mentioned problem with comments cut short, but apart from that the approach seems to work well. The program used in this study was rather crude and could be improved in many ways. For example, the answers could be loaded directly into a database and some of the correcting could be done automatically.

There are plans to go on with these issues regarding presentational formats in future studies. These studies are planned to be performed both at the department of physics at Stockholm University (where this study was conducted) and in an elementary school. This to see if there are any differences in learning from different presentational formats in different age groups.

Co-operation with teachers at Bredbyskolan has been introduced and the plans are to perform a study in the fall of 1997.

10. References

Andersson, J. R. (1995). Cognitive Psychology and its Implications. (4th ed.): W.H.Freeman and Company.

Bailey, R. W. (1989). Human performance enginering: Using human factors/ergonomics to achieve computer system usability.: AT&T Bell Laboratories.

Brooks, L. (1967). The suppression of visualization by reading. Quarterley Journal of Experimental Psychology, 19, 289-299.

ChanLin, L.-J., & Chan, K.-C. (1996). Computer graphics and metaphorical elaboration for learning science concepts. Paper presented at the Annual convention of the Association for Educational Communication and Technology, Indianapolis.

diSessa, A. A. (1988). Knowledge in pieces. In G. Forman & P. B. Pufall (Eds.), Constructivism in the computer age. : Lawrence Erlbaum Associates, Publishers.

Farah, M. J. (1989). Knowledge from text and pictures: A neurophysiological perspective. , in [Mandl & Levin, 1989] .

Gardner, H. (1987). The mind's new science: A history of the cognitive revolution.: Basic Books.

Harman, G. (1973). Thought: Princeton University Press.

Holm, P., & Karlgren, K. (1996). Cognitive science on trial. Paper presented at the IRIS 19.

Karlgren, K., & Ramberg, R. (1996, 30 Sept.-2 Oct). Language Use & Conceptual Change in Learning. Paper presented at the EuroAIED, Lisbon, Portugal.

Kommers, P. A. M. (1991). Cognitive tools: Prospects for explanatory learning environments. In P. A. M. Kommers, D. H. Jonassen, & T. J. Mayes (Eds.), Cognitive tools for learning. : Springer-Verlag.

Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate peripheral participation.: Cambridge University Press.

Levin, J. R. (1989). A transfer-appropriate-processing perspective of pictures in prose. In H. Mandl & J. R. Levin (Eds.), Knowledge acquisition from text and pictures (Vol. 58, ): North Holland.

Mandl, H., & Levin, J. R. (Eds.). (1989). Knowledge Acquisition from Text and Pictures : Elsevier Science Publishers B.V.

Mayer, R. E., & Anderson, R. B. (1991). Animations Need Narrations: An Experimental Test of a Dual-Coding Hypothesis. Journal of Educational Psychology, 83(4), 484-490.

Mayer, R. E., & Anderson, R. B. (1992). The Instructive Animation: Helping Students Build Connections Between Words and Pictures in Multimedia Learning. Journal of Educational Psychology, 84(4), 444-452.

Mayer, R. E., & Gallini, J. K. (1990). When Is an Illustration Worth Ten Thousand Words? Journal of Educational Psychology, 82(4), 715-726.

Mayer, R. E., & Sims, V. K. (1994). For Whom Is a Picture Worth a Thousand Words? Journal of Educational Psychology, 86(3), 389-401.

Mayton, G. B. (1991). Learning dynamic processes from animated visuals in microcomputer-based instruction. Paper presented at the Annual convention of the Association for Educational Communication and Technology.

Milheim, W. D. (1993). How to use animation in computer assisted learning. British Journal of Educational Technology, 24(3), 171-178.

Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing Cognitive Load by Mixing Auditory and Visual Presentation Modes. Journal of Educational Psychology, 87(2), 319-334.

Norman, D. A. (1993). Cognition in the head and in the world: An introduction to the special issue on situated action. Cognitive Science, 17, 1-6.

Ohlsson, S. (1996). Learning to do and learning to understand: A lesson and a challenge for cognitive modeling. In P. Reiman & H. Spada (Eds.), Learning in humans and machines: Towards an interdiciplinary learning science. : Pergamon.

Paivio, A. (1986). Mental representations, A Dual Coding Approach. New York: Oxford University Press Inc.

Palmiter, S., & Elkerton, J. (1993). Animated Demonstrations for Learning Procedural Computer-Based Tasks. . Human Computer Interaction, 8, 193-216.

Pea, R. D., & Brown, J. S. (1991). Foreword to [Lave & Wenger, 1991]. .

Poohkay, B., & Szabo, M. (1995). Effects of animations & visuals on learning high school mathematics. Paper presented at the Annual convention of the Association for Educational Communication and Technology, Anaheim, California.

Ramberg, R. (1996a). Explanations for understanding and learning complex problem solving. The Journal of the European Association for Research on Learning and Instruction.

Ramberg, R. (1996b). Psychological aspects of explanations as support for problem solving and learning., Stockholm University, Stockholm.

Ramberg, R., & Karlgren, K. (1997). Fostering superficial learning. Paper presented at the PEG-97, Sozopol, Bulgaria.

Reimann, P., & Spada, H. (1996). Towards an interdiciplinary learning science. In P. Reimann & H. Spada (Eds.), Learning in humans and machines: Towards an interdiciplinary learning science. : Pergamon.

Rieber, L. P. (1990a). Animation in Computer-Based Instruction. Educational Technology Research & Development, 38(1), 77-86.

Rieber, L. P. (1990b). Effects of animated visuals on incidental learning and motivation. Paper presented at the Annual convention of the Association for Educational Communication and Technology.

Rieber, L. P. (1996). Animation as Feedback in a Computer-Based Simulation: Representation Matters. Educational Technology Research & Development, 44(1), 5-22.

Rieber, L. P., Boyce, M. J., & Assah, C. (1989). The effects of computer animated lesson presentations and cognitive practice on adult learning in physical science. Paper presented at the Annual convention of the Association for Educational Communication and Technology, Dallas, Texas.

Russel, S. J., & Norvig, P. (1995). Artificial intelligence: A modern approach. New Jersey: Prentice Hall.

Salomon, G. (1992). Effects with and of computers and the study of computer-based learning environments. In E. De Corte, M. C. Linn, H. Mandl, & L. Verschaffel (Eds.), Computer-Based Learning Environments and Problem Solving. Berlin Heidelberg: Springer-Verlag.

Segall, M., Campbell, D., & Herskovits, M. (1966). The influence of culture on visual perception. Indianapolis: Bobbs-Merill.

Smith, R. E. (1993). Psychology: West Publishing Company.

Steuer, J. (1992). Defining Virtual Reality: Dimensions Determining Telepresence. Journal of Communication (Autumn).

Vosniadou, S. (1992). Fostering Conceptual Change: The Role of Computer-Based Environments. In E. De Corte, M. C. Linn, H. Mandl, & L. Verschaffel (Eds.), Computer-Based Learning Environments and Problem Solving. . Berlin Heidelberg: Springer-Verlag.

Wærn, Y. (1995). Hypermedia learning issues (Hyperlearn). (Project description).

Wasson, B. (1996, 30 Sept.-2 Oct). Instructional Planning and Contemporary Theories of Learning: Is this a Self-Contradiction? Paper presented at the EuroAIED, Lisbon, Portugal.

Websters (1997). URL:http://www.m-w.com/netdict.htm.

Willows, D. M., & Houghton, H. A. (Eds.). (1987). The psychology of illustration (Vol. 1) : Springer-Verlag.

Winn, B. (1987). Charts, graphs, and diagrams in educational material. , in [Willows & Houghton, 1987].

Woolf, B. P., & Hall, W. (1995). Multimedia pedagogues: Interactive systems for teaching and learning. Computer (May), 74-80.

Footnotes

  1. This thesis corresponds to twenty full time working weeks.
  2. World is here used in the widest sense of the word, including imaginary and/or made up worlds.
  3. Intelligent, as in human intelligence. I here use the word without a more precise definition. The discussion about what is intelligent and what is not is left as an assignment for the reader.
  4. In fact it is the other way around. The analogue illustration is a snapshot from the animation, with the two arrows depicting acceleration (A) and velocity (v) added.
  5. For a detailed description see the screen shots with included translations in Appendix A.
  6. ANOVA is a standard statistical function in behavioural science. "p" stands for probability.
  7. To view the actual questions refer to Appendix A.
  8. For a discussion on this result see the next chapter.
  9. For scale and meaning of end values see Appendix A.