Academic Matters

Nora/ Norhanita

Division of Information Studies
School of Communication and Information
Nanyang Technological University, Singapore

During the past years, technological advances in the computers as well as in computer vision research had facilitated the study of Augmented Reality (AR) prototype systems for education and learning. Augmented reality is the ability to overlay computer graphics onto the real world. In other words, AR is a set of technologies that operate together with the objective of merging virtual information with the real environment. It is possible for augmentation to take on various forms. The main objective is to create a system in which the user cannot differentiate between the real world and the virtual augmentation of it. In an AR interface, the user sees the world through a handheld device or head mounted display (HMD) that could be see-through or overlays graphics on video of the surrounding environment. AR interfaces provide enhancement of the real world experience that is unlike other computer interfaces that draw the users away from the real world and onto the screen.

Research in conceptual education in immersive virtual environments is a considerably new field but growing very fast. Virtual Reality (VR) offers to raise interest and motivation in students with a high potential to enhance their learning experience. However, the practical potential of VR is still being studied and understanding the application of VR technology to enhance learning activities presents a great challenge for the designers and evaluators of this learning technology. This paper will focus on the use of collaborative augmented reality for educational purposes. AR allows the user to see the real world, with virtual objects superimposed upon with the real world. Thus, AR supplements reality, rather than totally replacing it. The scenario would be ideal if it appears to the user that the virtual and real objects coexist in the same space.

The purpose of augmented reality is to combine multiple technologies together into a single system. The advances in augmented reality systems are mainly due to the main fields of computer vision, graphics and user interfaces.

The user in a standard virtual reality system is supposed to be completely engulfed in a computer-generated environment. The system has a reference structure registered with the computer graphic system that generates rendering of the virtual world. The environment is therefore sustained by this system. For the user immersion to be successful, the main frame of reference usually controlled by the user’s body and brain need to be registered with the virtual world. Any changes done by the user will lead to the changes in the perceived virtual world. In this case, a connection must be created between the two reference frames. However, there is no natural relationship between them because the user is looking at a virtual world. Thus, an augmented reality system may be considered the utmost immersive system.

In an AR system, an imaging device such as a video camera is used to view the scene. The camera projects the 3D world onto a 2D image plane. The projection on the image plane is dependant on the focal length, lens distortion, position and pose of the camera. The virtual image is done by a standard computer graphics system while the virtual items are represented in an object reference frame. For it to correctly render these objects, the graphics system requires details about the imaging of the real scene. The camera used to produce the image of the virtual objects is controlled by this data. The AR image is the result of merging the image with the image of the real scene.

The video imaging and graphic rendering mentioned above is somewhat straightforward. The research studies in AR focus on ways to register the two different sets of images and ensure they are registered in real time and the display technology for merging the two images.

2 Augmented Reality Interaction

Rekimoto and Nagao (1995) introduced Augmented Interaction, a style of human-computer interaction which reduces computer manipulations. This style enables the user to interact with the real world and having the computer automatically recognizing the user’s situation. The main role of the computer is thus to assist and enhance interaction between humans and the real world. This concept can be applied with methods such as time, location and object recognition and placing some marks (barcodes for examples) on the real environment.

Inkpen (1997) noted that students working together in groups around a single computer performed better than working alone. Students tend to spontaneously cluster in pairs and trios around a single computer when they are assigned to individual work (Watson 91, Strommen 93). The students’ attention is often focused on the desktop screen when working with a computer and thus impeding the communication cues available in group work context. In this instance, AR can be used to solve the lack of communication space, seen in the classroom encounter of using computer desktop for learning. Working on screen separated the interpersonal communication space and students are focusing only on the screen and completing the task. AR enables students to communicate among one another and learning using computing technologies at the same time. Kiyokawa (2002) hypothesized that AR enabled students to attend lectures and watch virtual objects floating in their midst thus resulting in conversational behavior similar to face-to-face communication.

3 Augmented Reality Applications In Education And Learning

3.1 MagicBook

The MagicBook, developed at the HITLab , is the first of its kind to replace books as static information. MagicBook presents the normal book as the main interface and users are able to read the book without AR technologies. Users are presented with the option to see three-dimensional animated figures acting out the story on the real page with a handheld AR display by moving themselves or the book. The use of Augmented Reality on traditionally static printed page thus becomes a means to enable student to understand the lectures without getting bored and at their own pace.

3.2 Astonomy Education

Shelton and Hedley (2002), presented at the First IEEE International Augmented Reality Toolkit Workshop in Germany, a research about using AR to teach students astronomy. In the project, the orientation of the virtual sun and earth are controlled on a platform coordinated to the students’ view and position. The students received a significant understanding in the AR exercise and they concluded that by letting the students to manipulate the 3D objects, there is little misunderstanding in grasping the subject. The usage of visual and sensory information creates a powerful learning experience for the students. Shelton (2002) hypothesize that 2D diagrams shown for learning and education creates a cognitive filter and is researching further to understand the learning content based on AR interaction using the direct cognitive path.

3.3 Sciences

Collaboration between the HITLab and the Scripps Research Institute involved the traditional physical models used to teach chemistry and molecular biology were replaced by AR graphics superimposed on fabricated physical models to show the complex increase of the molecular structures. The models are manipulated using voice commands and haptic feedback response.  The multi-modal interaction provides students and teachers an intuitive tactile learning interface. The students involved in this project attested that they experienced a compelling sense of realism of the 3D graphics which promotes better interaction and greater exchanges of ideas.

Fjeld and Voegtli (2002) presented the report on their Augmented Chemistry (AC), a Tangible User Interface application.  Students are able to construct atoms by using their hands through drag-and-drop method. Using aural application, signals are sent to the users to indicate the activity level of the molecule and the permissible step of creating a new structure. The name identifying the structure appears when students have created a completed molecule.  

3.4 Arts

The researchers at the National University of Singapore (2002) developed a system to capture live human model with cameras and transmit the images using the AR interface at another location in real time. The unidirectional system allowed different angles of the live models to be captured and viewed multiple times. Besides using the systems as 3D video conferencing systems, students can critique or analysis literature plays, martial arts training instructions, dance instructions and live educational experiments.

3.5 Paleontology

The study of past geological climate and extinct animals has been made possible by a group of researchers from Germany and the US. The AR content for the system named Virtual Showcase, present the fossilized skull and 3D graphics information of dinosaur to the users. AR enables the missing parts, such as tissues, muscles and skin, of the dinosaurs’ fossils to be reconstructed. Viewers are required to wear special glasses which track the position of the head and allowing the display to be seen by different people from several angles simultaneously.  This system can also be use on life sciences education.

An AR system must run in real time for a user’s freedom of movement within the scene and view a well-rendered augmented image. There are two main concerns on the system. Developers are working on the update rate for generating the augmenting image and the accuracy of the registration of the two types of images.

When it comes to visual aspect, the user is affected when he views an augmented image where the virtual parts are rendered with no apparent jumps. The 10 times per second rendering of the virtual setting is a standard guide for the image to appear perfectly. For a more realistic scene, a photo-realistic graphics rendering system is required. Unfortunately, complicated scenes that has shaded or ray-traced images are not fully supported by the current graphic technology. Luckily, there are many applications for augmented reality whereby the virtual aspect is neither very complex nor require a high level of photo-realism.

Noise in the system could affect the registration of the real and virtual scene. Essentially, the real scene must detect the position and pose of the camera. Errors in the registration of the virtual and real image will be reflected when there is any noise occurrence in the system.

Jittering in the image occurs as a result of fluctuating values when the system is busy. Our sense of sight is extremely sensitive to visual errors. In this aspect, the user would think that the virtual object is not stationary in the actual scene or improperly positioned. The user can even detect misregistration of a pixel under the proper conditions. Time lag is another problem with an AR system. As mentioned earlier, a minimum cycle time of 0.1 seconds is required for satisfactory real-time performance. Delays in detecting the position and alignment of the graphics camera will cause the augmented objects to lag behind movements in the real scene. The system design has to minimize the time lags to maintain overall system delay within the specifications for real time performance.

Developers of AR systems are faced with the new technical challenge of merging real and virtual images into a single image. This is why other display technologies are needed to create the sense of realism in the scene created by the display.

The AR developers are currently working on two types of HMDs namely video see-through and optical see-through. The user would want to see the real world view appear instantly before him while wearing the HMD. This is the see-through concept. The user becomes isolated from the surrounding environment when he uses the standard HMD in a virtual reality system. This requires the system to use video cameras that are aligned with the display to achieve the view of the real world.

The optical see through HMD removes the video channel that is viewing the real scene. The merging of real and virtual world is done optically in the presence of the user. Military airplane cockpits use this technology that is also known as heads up display (HUD). The two images are optically merged on the HUD display.

There are performance issues related to the above displays. To view the real world, both displays need a video camera. The result is a forced delay of up to one frame time to perform the video merging. The user will experience potentially a 33.33 millisecond delay in his view. This could be resolved by correctly timing the other paths in the AR system. However, the real scene could be delayed if the other paths are slower than the video itself. An optical see-through display the view of the real world immediately, so it is not possible to compensate for system delays in other areas. Another way is to use a monitor and video see-through displays to view the real scene. An advantage is that tracking information is available when there is graphic image generated by the camera in the system.

4.1 Collaboration

One of the most crucial objectives of an educational environment is to encourage social interaction among users located in the same physical area. In collaborative AR, multiple users can access a shared area populated by virtual objects, while staying grounded in the real world. This method is especially powerful for educational purposes when users are assembled and can use natural ways of communication for example, speech and gestures, but can also be combined successfully with immersive virtual reality or remote collaboration. Another crucial psychological factor is that some users feel unsafe if their vision is “locked” in an immersive virtual world whereas AR allow them to “be in control”, to see the real world around them. Thus, safety issues are important in collaborative mobile systems that are for direct use in classrooms where AR is apparently used to provide mobile users the freedom of sight required to move about. Another factor to consider is the relationship between emotions and learning, but how feelings such as insecurity and emotions generally influence learning is a subject of ongoing research. However, AR developers have to take into account the above-mentioned issues when constructing their ideal learning environment. Augmented reality cannot possibly be the perfect solution for all educational application needs but it is a choice to consider. The technology implemented always has to rely on the academic goals and needs of the educational application and the target audience.

It is essential to take into account the technological challenges that exist when introducing AR for educational and learning purposes. A sound educational or learning AR system should have the following features:

a. It must be easy to use and dynamic as a tool.

b. It has to provide the user with clear and detailed information.

c. It should allow the educator to put up information in a simple and efficient way.

d. It must enable learners to interact with one another easily.

e. It has to make complicated procedures transparent to the learner.

f. It has to be cost effective and easy to install and maintain.

Individuals have different learning styles and various ways of communication but AR technology facilitates in promoting education and learning in many different methods.

Since there are advances in the development of educational concepts, applications and technology, and a concurrent decline in hardware costs, the use of small scale or mobile immersive virtual or augmented reality systems could become possible for educational institutions within this decade. This is assuming that the ongoing development is at the same rate. However, the potential of each AR aspect requires thorough study so that it can actually be converted into educational efficacy. The point is not about whether or not AR is useful to enhance learning. It is the understanding how to effectively exploit the potential of AR.

Augmented Reality brings a new dimension to education and learning models with enhancing a user’s understanding and learning environment. Inclusion of all senses in learning invokes an individual awareness on the subject and AR can help individuals who are lacking in a particular sensory organ to understand better. Individuals who are not bestowed with the gift of sight can make use of aural augmentation interface as a tool for effective learning.

With evolving technologies, computing power is currently able to be in use with smaller systems and thus enhance mobility in learning. AR system can be developed exhaustively to include education in areas such as sports and linguistics. Although AR technology is not recent, it’s potential in education is just about to be explored. Unlike other computing technologies, AR interfaces provide seamless interaction between the real and virtual worlds, a tangible interface metaphor and a method for transitioning between real and virtual worlds. Educators should work closely with researchers in the field to explore further how these characteristics can best be applied in a school environment.