Augmented reality

In 2013 I wrote the concluding chapter for Experiential Learning in Virtual Worlds (edited by me and Greg Withnail). I predicted what would happen in the development of virtual worlds over the following five years. I made six different predictions. The best I did was I got one of them half-right. The rest were almost entirely wrong.

This year, I’m developing a course on Educational Futures in which I’m looking at what makes an effective, or a poor, prediction. Rather than make someone else look like an idiot, I’m looking at the predictions I made. The idea is for students to look at the text and work out how I got it so badly wrong in most of the cases.

The following is not entirely the text from the book, but I’ve only tweaked it so it will work on its own rather than as part of a concluding chapter. I’ve also added a prescience factor at the end, to sum up how well I did.

Augmented reality. One function of many mobile devices is that they can combine the camera images with an overlay of additional information. In the same way that a global position and orientation can be used to calculate the position of stars as seen from a particular viewpoint, these can also be used to determine at which geographical location the tablet is being pointed. These data can then be combined with a database of information to create an overlay of text to explain, for example, the historical background of a building, or the direction and distance of the nearest Underground station or Irish pub. Locations can be digitally tagged, either with additional information (such as in a learning exercise with students adding their own content to locations), artwork, or even graffiti[i]. As with the astronomy apps described above, this provides learning in situ, and provides a kinaesthetic element to the activity.

The potential of combining geotagged images onto the physical world is indicated by augmented reality games such as Paranormal Activity: Sanctuary^[ii]. In this, images of ghosts are located at particular physical world co-ordinates, which can be seen with a dedicated iphone app that overlays these images onto a camera image. Players can create sanctuaries, or cast spells, at locations which then influence the experience of other players. The game therefore becomes a massive multiplayer roleplay game played in a blending of the physical and a virtual world.

Greater precision than that enabled by global positioning can be provided through Radio Frequency Identification (RFID) tags, the technology for recognising which will soon be available on mobile technology[iii]. By placing an RFID tag in clothing, or furniture, or on a person, information about that object or person (i.e. metadata) are then always available, whenever a device is pointed at them. For example, products could be linked directly to their user manual; simply hold your tablet PC over your oven and pop-up boxes appear over the knobs decoding the icons, or attend a conference and each person there could have information linked to them, such as name, institution and research interests, which is revealed by holding up your phone and tapping their image on the screen. Several museums and exhibitions already have augmented reality exhibits; when a room is looked at through an AR viewer, the physical objects in the room are overlain with animations or animated characters, bringing the static displays to life[iv]. A further enhancement of augmented reality is achieved by enabling the animated characters to address the attendee directly, with their gaze following the attendee around the room, as they are tracked through the use of an RFID bracelet[v]. The characters can address many attendees simultaneously since, from the perspective of each, the character is looking at them individually, a transformed social interaction known as non-zero sum mutual gaze[vi]. These interactions can be made more seamless by plans to create AR projections within glasses[vii]. Rather than clicking on a screen, input can be through the detection of hand movements[viii] or, for the mobility-impaired, deliberate blinking[ix].

If this is possible with pre-recorded characters, then it is only a short leap to enabling this to take place with avatars or bots in realtime, by layering the virtual world image onto the physical as it is created. This activity resembles the mixed reality performances created by Joff Chafer and Ian Upton; originally these performances used images from a virtual world projected onto a gauze, so that they could share the stage with physical world actors, and more recently Chafer and Upton have used 3D imaging to bring the virtual world images out from the screen and into a physical space[x]. Capturing the images of avatars in the virtual world, and geotagging them, would enable people with the appropriate AR viewer to see avatars moving and communicating all around them. As the sophistication of bots develop, then the use of them as companion agents, guiding learners through virtual learning scenarios, could be brought into the physical world as guides and mentors seen only by the learner through their AR viewer. With ways of imaging the avatars through something as immersive as AR glasses, physical world participants and avatars could interact on an equal footing.

For learning and teaching, the advantages of blending the functionality and flexibility of the virtual and the real are enormous. For the learners who see virtual learning as inauthentic, relating the virtual world learning directly to the physical may overcome many of their objections. The integration of an object and its metadata as well as data providing context for that object (called paradata) is easily done in a virtual world; AR in combination with RFID tagging enables this feature to be deployed in the physical world too, since information, ideas and artefacts can be intrinsically and easily linked. User generated content, which again is simply created and shared in the virtual, can also be introduced to the physical. Participation at a distance, on an equivalent footing with participation face-to-face, could be achieved by the appearance of avatars in the physical environment and RFID tagging the physically-present participants and objects.

[i] New Scientist, Augmented reality offers a new layer of intrigue, 25^th May, 2012. http://www.newscientist.com/article/mg21428652.600-augmented-reality-offers-a-new-layer-of-intrigue.html,

[ii] ‘Ogmento Reality Reinvented, Paranormal Activity: Sanctuary’, 22^nd May 2012. http://www.ogmento.com/games/paranormal-activity-sanctuary

[iii] Marketing Vox, ‘Married to RFID, What Can AR Do for Marketers?’, 4^th March, 2010. http://www.marketingvox.com/married-to-rfid-what-can-ar-do-for-marketers-046365/

[iv] Canterbury Museum, ‘Augmented reality technology brings artefacts to life’, 28^th September 2009. http://www.canterburymuseum.com/news/13/augmented-reality-technology-brings-artefacts-to-life,

[v] A. Smith, ‘In South Korea, Kinect and RFID power an augmented reality theme park’,  Springwise,  20^th February, 2012. http://www.springwise.com/entertainment/south-korea-kinect-rfid-power-augmented-reality-theme-park/

[vi] J. Bailenson, A. Beall and M. Turk, ‘Transformed Social Interaction, p. 432

[vii] S. Reardon, ‘Google hints at new AR glasses in video’, New Scientist, 4^th April, 2012. http://www.newscientist.com/blogs/onepercent/2012/04/google-hints-at-new-ar-glasses.html

[viii]C. de Lange, ‘What life in augmented reality could look like’,  New Scientist, 24^th May, 2012. http://www.newscientist.com/blogs/nstv/2012/05/what-life-in-augmented-reality-will-be-like.html

[ix] Eduardo Iáñez^, , Andrés Úbeda, José Azorín, Carlos Pérez, Assistive robot application based on a RFID control architecture and a wireless EOG interface Science Direct, Available Online 21^st May, 2012. http://www.sciencedirect.com/science/article/pii/S0921889012000620

[x] Joff Chafer and Ian Upton, Insert / Extract: Mixed Reality Research Workshop, November 2011. http://vimeo.com/32502129

Prescience Factor: 0/10. Despite AR apps becoming more popular since 2013, AR is still not really a thing in that it’s not an embedded part of what we do. Linking AR and virtual worlds in the way I’ve described here isn’t any further along (as far as normal practice) than it was when I wrote the above.

Gone to mobiles every one

In 2013 I wrote the concluding chapter for Experiential Learning in Virtual Worlds (edited by me and Greg Withnail). I predicted what would happen in the development of virtual worlds over the following five years. I made six different predictions. The best I did was I got one of them half-right. The rest were almost entirely wrong.

This year, I’m developing a course on Educational Futures in which I’m looking at what makes an effective, or a poor, prediction. Rather than make someone else look like an idiot, I’m looking at the predictions I made. The idea is for students to look at the text and work out how I got it so badly wrong in most of the cases.

The following is not entirely the text from the book, but I’ve only tweaked it so it will work on its own rather than as part of a concluding chapter. I’ve also added a prescience factor at the end, to sum up how well I did.

Gone to mobiles every one. As noted above, the rate of take-up of virtual worlds anticipated by Gartner in 2007 has not been realised. Some predictions also state that the rate of development of the high end graphics technology required for virtual worlds will be slowed by the adoption of mobile technology. Essid[i] notes that the tablet PCs owned by students cannot run the viewers required for Second Life, and these are now the predominant technology with which students access online learning. In addition, many apps provide innovative and offline education, such as the use of Google Sky, Zenith or Sky Safari for learning astronomy. In these apps, the learner holds up their tablet PC and through global positioning and inbuilt sensors that detect orientation, the tablet displays the position of stars, planets and Messier objects as they appear in the sky in the direction in which the tablet is pointed. This provides learning that is interactive, kinaesthetic, and in situ. Essid’s prediction is that the predominant use of mobile technology as the new wave of learning will stall the uptake of virtual worlds. As Essid states in his blog post on the subject:

One does not wish to be on the wrong side of history, and I think SL evangelists are clearly on the wrong side, unless they are early in their careers and have a Plan B for research and teaching.

[i] J. Essid, ‘Mobile: Shiny? Yes. Hyped? Yes. Fad? No’, 3rd May, 2010, http://iggyo.blogspot.co.uk/2012/05/mobile-shiny-yes-hyped-yes-fad-no.html

Prescience factor: 8/10. To be fair, not my prediction really, but Joe Essid’s. The increasing usage of mobile devices has meant that learning can take place anywhere, but it has caused the development of some technologies to slow down because as a platform they are more limited,  in terms of the processing power when compared to PCs, but also in due to the speed of input (two thumbs are never as fast as 10 fingers) and the readability of the screen. It’s not 10 out of 10, because I think both Joe and I underestimated the capacity and functionality that smartphones would attain by 2018. Moore’s Law is severely difficult to anticipate because it’s a geometrical increase. This example shows why it’s impossible to get your head around geometrical increases.

Moves to games consoles

In 2013 I wrote the concluding chapter for Experiential Learning in Virtual Worlds (edited by me and Greg Withnail). I predicted what would happen in the development of virtual worlds over the following five years. I made six different predictions. The best I did was I got one of them half-right. The rest were almost entirely wrong.

This year, I’m developing a course on Educational Futures in which I’m looking at what makes an effective, or a poor, prediction. Rather than make someone else look like an idiot, I’m looking at the predictions I made. The idea is for students to look at the text and work out how I got it so badly wrong in most of the cases.

The following is not entirely the text from the book, but I’ve only tweaked it so it will work on its own rather than as part of a concluding chapter. I’ve also added a prescience factor at the end, to sum up how well I did.

Move to games consoles. A move in the other direction, to more sophisticated technologies, is the repositioning of virtual worlds to run on games consoles such as the Playstation 3, or the Xbox 360. Games consoles have very sophisticated graphics processors, and the quality of the rendering of games is much higher than is available using most PCs. Many Massive Multiplayer Online Games are already available on games consoles, and shared virtual worlds such as Minecraft, previously running on PCs have made the transition to this technology. In the Minecraft case this has proved immensely popular[i]. The advantages of running virtual worlds on games consoles is due not to just the more sophisticated graphics available, but also the control devices. Many people find games controllers a more intuitive mechanism to control the movement of an avatar than keys on a keyboard. However, text chat and a drag and drop functionality are less well integrated.

The next generation of games controllers offer even more interactivity as they allow detect physical interaction by the users, through the use of cameras and motion detectors. Devices such as the Xbox 360 Kinect controller have already been used to animate avatars. There are two ways in which this can be done, either avatars can be animated inworld through physical actions triggering pre-set animations, (for example, the act of raising your hand triggers a hand-raising animation) or, as in the work of Fumi Iseki and a team at Tokyo University[ii], the animations are used to animate avatars in realtime, but in a local viewer only. Because avatars are animated inworld using preloaded animation files, there is no way with current technology to map motion capture to inworld movements of avatars in realtime.

This opens up the potential to a new, closer relationship between user and avatar. As Jelena Guga notes[iii], this will be the next step change in the developing degrees of immersion that have been enabled by the changes in technology. Although the sense of immersion may be increased, requiring the user to be physically active may also, simultaneously, make the user more aware of their physical body while interacting inworld, so their sense of embodiment may actually be reduced. The individual experience of virtual worlds varies enormously, and a likely discovery will be that whether physical operation of an avatar increases or reduces the sense of engagement inworld will be different depending on the person. Another consideration is that a one-to-one correspondence between physical action and resulting motion of the avatar is, as Stelarc points out,[iv] possibly the least interesting way in which to use motion recognition to animate avatars. In his performances, Stelarc uses his body to create inworld performances, but his gestures cause his avatar to fly, float, operate cyborg attachments and so on.

From a learning point of view, a move to games consoles could have advantages and disadvantages. The move would overcome some of the objections to virtual worlds with regard to the low resolution graphics, and technical issues such as slow rendering times and lag, however, they could marginalise activity even further, since few computer suites in universities have games consoles, and it cannot be guaranteed that all users will have access to them. Developing motion controlled interfaces would address some of the issues that some users find; that operating within a virtual world is too sedentary an experience. Offering the opportunity to operate avatars through physical motion may appeal to these users, though indications are that these users actually find the virtual nature of these experiences intrinsically problematic, equating the virtual with inauthentic. However, the use of a motion recognition system will have interesting opportunities for performance.

[i] M. Hawkins, ‘‘Minecraft‘ on Xbox Live a smash success, MSNBC, May 12th, 2012http://www.ingame.msnbc.msn.com/technology/ingame/minecraft–xbox–live–smash–success-766955,

[ii] Second Lie, ‘Kinect Hack Brings Real Time Animation To Second Life’, November 2011http://second-lie.blogspot.co.uk/2011/11/kinect-hack-brings-real-time-animation.html,

[iii] J. Guga, ‘Redefining Embodiment through Hyperterminality’, Virtual Futures 2.0, University of Warwick, 18^th – 19^th June, 2011.

[iv] Stelarc, Keynote, From Black Box to Second Life: Theatre and Performance in Virtual Worlds, University of Hull, Scarborough, May 20^th, 2011

Prescience Factor 4/10. The only thing I nailed here was that consoles would become more of a platform for interacting in a social world way. Lots of RPGs now allow users to build spaces in a shared virtual environment, and not necessarily in service of the game directly, but just to settle a permanent online 3D space. The flexibility of the spaces and avatar interactions in games like, for example, Conan Exiles or Fortnite Creative is more limited than a full social virtual world, but you could potentially create a home and then invite someone round for a chat.

A virtual world in your browser

In 2013 I wrote the concluding chapter for Experiential Learning in Virtual Worlds (edited by me and Greg Withnail). I predicted what would happen in the development of virtual worlds over the following five years. I made six different predictions. The best I did was I got one of them half-right. The rest were almost entirely wrong.

This year, I’m developing a course on Educational Futures in which I’m looking at what makes an effective, or a poor, prediction. Rather than make someone else look like an idiot, I’m looking at the predictions I made. The idea is for students to look at the text and work out how I got it so badly wrong in most of the cases.

The following is not entirely the text from the book, but I’ve only tweaked it so it will work on its own rather than as part of a concluding chapter. I’ve also added a prescience factor at the end, to sum up how well I did.

A virtual world in your browser.  There are numerous legitimate reasons for using standard web browsers for access to virtual worlds. The first of these is that the processing power, particularly of a graphics card, required to run a virtual world viewer is beyond the capacity of the technology available to many people, and particularly of institutions. Secondly, the bureaucratic hurdles many practitioners face when requiring additional software to be downloaded and installed preclude the use of virtual worlds in many institutions, suffering as they do from the obstructive policies of their IT departments. Finally, enabling virtual worlds to be viewable from within a web browser means that the practice of accessing virtual worlds can be easily integrated into the majority of people’s normal internet usage, and so potentially widen the demographic of users. The initial effort required to begin using them in an educational situation would consequently be reduced.

It would be reasonable to anticipate that these factors would lead to the usage of virtual worlds becoming much more widespread. Making virtual worlds viewable through the web should have been very successful, in effect though, Lively only lasted for the second half of 2008. Newer virtual worlds, such as Kitely, although trying to widen the demographic of potential users by offering other platforms such as Facebook and Twitter for access, have returned to the use of the viewer-based technology rather than be browser-based.

The reasons for the failure of Lively are still being discussed. The direct experience of those contributing to this chapter, however, is that reducing the functionality of the virtual world in order to enable it to work within a browser removed the elements that made a virtual world worth pursuing. The sense of immersion was reduced, the opportunities to create and interact with virtual artefacts within the world were lessened, and consequently the rapid adoption by the marketplace, needed for the survival of any social medium, did not materialise. Lively disappeared before many people realised it had been launched, and new web-based viewers have not emerged to take its place.

Prescience Factor: 0/10. A total overestimation of the versatility and processing power of browsers now.

The Metaverse Lives

In 2013 I wrote the concluding chapter for Experiential Learning in Virtual Worlds (edited by me and Greg Withnail). I predicted what would happen in the development of virtual worlds over the following five years. I made six different predictions. The best I did was I got one of them half-right. The rest were almost entirely wrong.

This year, I’m developing a course on Educational Futures in which I’m looking at what makes an effective, or a poor, prediction. Rather than make someone else look like an idiot, I’m looking at the predictions I made. The idea is for students to look at the text and work out how I got it so badly wrong in most of the cases.

The following is not entirely the text from the book, but I’ve only tweaked it so it will work on its own rather than as part of a concluding chapter. I’ve also added a prescience factor at the end, to sum up how well I did.

The metaverse lives. Of the chapters in the book, four chapters use Second Life , one uses OpenSim, one World of Warcraft, one uses a 2D multimedia website and one began with Second Life and then, due to the price increases imposed by Linden Lab, moved to OpenSim. From this (admittedly small) sample, it appears that Second Life is still the strongest contender for a platform to host virtual world activity, but that educators are more becoming more likely to consider alternative, though similar, platforms, with OpenSim leading the way.

Educators’ dissatisfaction with, and the expense of, Second Life is beginning to cause fragmentation of the virtual world community. Whereas before it was almost guaranteed that educators would share a single grid, increasingly they are becoming spread across a range of different platforms. One saving grace of this diaspora is that many of the most popular of these virtual worlds use the same viewer. Whether one uses the Second Life viewer, Imprudence, Phoenix or Firestorm or any of a number of others, once a user has learned to interact with the world using that particular interface, then it is of little difficulty to switch to another one. This is particularly important with virtual world as a technology (moreso than, for example, with a word-processing package, or an online forum); since what is required for an effective learning opportunity is immediacy of experience rather than hypermediacy; any changes in the interface are extremely disruptive, since this makes the technology more visible and reduces the transparent nature of the interaction.

However, although they are operated in the same manner, the grids remain separate. The step that will reintegrate this fragmented community, and enable educators to once again easily share and visit their educational resources will be the successful employment of hypergridding. Hypergridding is the connecting of these separate virtual worlds to create a collection of linked worlds, an example of Stephenson’s metaverse. Once it becomes possible to move not only avatars, but also their inventories, from world to world, then these separate grids will perform as a single platform; so, for example, objects purchased from within Second Life (which has a thriving creators’ market) could be employed within OpenSim (which gives institutions greater control over privacy and ownership of the space). This would greatly expand the choices, and the flexibility of using virtual worlds for educators, and to a large extent enable far more effective collaboration. Simple and effective hypergridding is close to deployment, but, as of writing in 2012, has not been realised.

Prescience factor 0/10. Hypergridding is not a thing.