Invited talk at Espacios Educativos Especulares (Educational Reflected Spaces) EEE Anual Meeting
27-28 May 2013 - Universidad Carlos III de Madrid, Spain
1 of 34
More Related Content
Towards blended reality on collaborative laboratory activities using smart objects
1. Towards blended reality on
collaborative laboratory activities
using smart objects
School of Computer Science & Electronic Engineering
University of Essex, UK
Anasol Pe単 a-R鱈os
acpena@essex.ac.uk
1
2. Introduction
Blended learning landscapes
Scenario: Collaborative laboratory activities
Interreality environment (Conceptual model &
implementation)
Blended reality distributed system
Collaboration projects
Summary
Overview of the presentation
2
5. Use of technology in academic settings
(Some challenges)
Promote adoption of new technologiesadoption of new technologies
by academic faculty members and students
Adapt technologies, pedagogicalAdapt technologies, pedagogical
methods and evaluationsmethods and evaluations enabling this
integrated solutions for people in
geographically dispersedgeographically dispersed locations
Change the way to experienceexperience
education from one-dimensionaleducation from one-dimensional
(physical) to multi-dimensionalmulti-dimensional (physical
and virtual) education in an integrated way
BlendedBlended
LearningLearning
LandscapesLandscapes
7
6. Blended learning landscapes
Virtual
Learning
Environmen
t
Blended
Learning
Environment
Real Learning
Environment
An infrastructure
that works as a link
between this
environments able to
reflect information
(data, changes, etc)
on real time
Learning activities that
complements virtual
and real world making
possible for students
and tutors to move
between these two in a
seamless way
8
7. Blended
Reality
This might be limited by the capacity of users presence and engagement to a single reality
at a time (Lifton's "Vacancy problem"1) a consequence of the users real immersion.
9
User's interaction in real-
time with two different
environments (real/local
and virtual/distant) extend
them to work as if they
were one, by blending
traces of one into the other
unconsciously (often
seemingly as simultaneously).
1. J. Lifton and J. Paradiso, Dual Reality: Merging the Real and Virtual, Lecture Notes of the Institute for Computer Sciences,
Social Informatics and Telecommunications Engineering, vol. 33, no. 1, pp. 12-28, 2010
9. Laboratory activities for distance learners
Which are the options for remote laboratory activities?
Simulations
Virtual laboratories
Remote laboratories
To test conceptual knowledge
To work collaboratively
To interact with equipment
To performing analysis on
experimental data
Laboratory
activities
11
10. Examples
University of Bristol - LabSkills
University of Leeds - Virtual Labs
Challenges for current technology:
Limited interaction with real equipment
Use of idealised datasets
Restricted collaborative interaction
Freie Universitat Berlin
Universit Technology Enhanced Textbook
12
11. Interreality Environment
1 - J. van Kokswijk, Hum@n: Telecoms and Internet as Interface to Interreality : a Search for Adaptive Technology and
Defining Users, Bergboek, 2003
A collection of interrelated devices in the real world, a virtual
environment and software agents that allow users to complete activities
at any point of Milgrams Virtuality Continuum to achieve user
perception of integration between the physical and the virtual world.
13
Interreality.- a hybrid total experience between reality
and virtuality1
12. Interreality Portal (Definition)
In the educational context can be used:
As a learning environment
For geographically dispersed students
Using mixed reality laboratory activities
14
Human-computer interface (HCI) which captures data
obtained in real-time, and processes this data so
it can be mirrored in the virtual world linking
both worlds.
14. A "real" immersive
environment
Gives to the learners the
sensation of "being there"
A 3D virtual environment
3D GUI made on Unity3D, a
cross-platform game engine
used to create interactive 3D
content.
xReality objects
1- Immersive Display Group (www.immersivedisplay.co.uk)
2 - V. Callaghan, 束Tales From a Pod損 on Creative Science Workshop,
Nothingham, 2010.
1, 2
ImmersaStation
Interreality Portal (implementation)
16
15. 17
Smart objects
coupled to their virtual
representation (dual
reality state)
updated and maintained
in real time
1.- Kortuem, F. Kawsar, D. Fitton and V. Sundramoorthy, Smart objects as building blocks for the Internet of
things, IEEE Internet Computing, vol. 14, no. 1, pp. 44-51, 2010
Autonomous physical/digital objects
augmented with sensing, processing, and
network capabilities which can
interpret their local situation and
status, and can communicate with other
smart objects and interact with human
users 1
+Xreality
objects
=Smart
objects
16. Interaction between physical and virtual
elements within an environment
Each object, (real and virtual), is
complete by itself and can exist
without the other.
Dual reality state enriches
both objects by the bi-directional
process that can reflect, influence
and merge real-time information.
MIT Media Laboratory Dual reality
lab 2008
18
CrossReality (xReality)
17. xReality objects & virtual objects
Real world objects that have
a virtual representation
Dual reality state
xReality objects Virtual objects
Only exist inside a virtual environment
Rules and behaviours but these depend
on the existence of the virtual world.
19
Anasol Pe単 a-R鱈os, V. Callaghan, M. Gardner & M. J. Alhaddad, 束Remote mixed reality collaborative laboratory activities: Learning
activities within the InterReality Portal損on Web Intelligence and Intelligent Agent Technology (WI-IAT), 2012 IEEE/WIC/ACM
International Conferences - The Intelligent Campus International Symposium (IC12), vol.3, no., pp.362,366, 4-7, Macau, China, 2012.
18. xReality objects (Implementation)
* Fortito (www.fortito.mx)
20
**Raspberry Pi Foundation (www.raspberrypi.org)
Main module Components
Fortitos* Buzz-Board Educational Toolkit
30 pluggable boards that can
interconnect and communicate between
each others via I2C protocol
Comprises different sensors and
actuators to allow the creation of diverse
physical mashups
Raspberry Pi **
Detects other components and
works as a hub to connect
them to the interreality system
20. Implementation (architecture) (single dual
reality)
22
Anasol Pe単 a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, 束Developing xReality objects for mixed-
reality environments損on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.
21. Implementation (single dual reality)
23
Anasol Pe単 a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, 束Developing xReality objects for mixed-
reality environments損on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.
22. Multiple dual realities
24
Management of multiple dual reality states
Addition of a collaborative layer to allow two or more learners in
different geographical locations participate on the laboratory activity
+Anasol Pe単 a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, 束End-user programming & deconstrutionalism for collaborative mixed
reality laboratory co-creative activities損on 2nd European Immersive Education Summit (EiED12), Paris, France, 2012.
23. Blended reality distributed system
(Conceptual model)
25
Anasol Pe単 a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, 束Developing xReality objects for mixed-
reality environments損on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.
24. End-user programming
The interaction between users within collaborative mixed
reality learning activities can be analysed from two
different angles:
From the learners view
From the instructors view.
The learning environment should allow users to create
and execute learning activities regardless their expertise
on computers.
26
25. Homogeneity & structure to
the creation of mixed reality
learning activities
Adding collaborative
learning to the use of
xReality objects
Learner
s view
Instructor
s view
3D
Learning
environmen
t
End-user programming
27
26. Learning Activities
Learning activities within the InterReality Portal are
structured as a sequence of activities based on the IMS
Learning Design specification (Units of Learning-UoL).
CollaborativeCollaborative
interactioninteraction
betweenbetween
learnerslearners
EmphasiseEmphasise
ComputingComputing
fundamentalsfundamentals
Problem-Problem-
basedbased
learninglearning
VirtualVirtual
objectsobjects
(SW)(SW)
Internet-of-Internet-of-
ThingsThings
applicationsapplications
xRealityxReality
objectsobjects
(HW)(HW)
ComputerComputer
sciencescience
projectsprojects
28
27. Deconstructed model
Based on the disaggregation of physical/logical
devices and services.
Propose the creation/identification of a number of
elementary services (atomic functions) which can
be combined in various ways to create complex
functions (nuclear functions).
29
28. Deconstructionism
Role Atomic function Nuclear function
Learner Objects available in the
environment, actions
available (programming
statements).
An Internet-of-Things
project
Instructor Resources available in
the environment,
activities available
(sequence of activities).
A Unit of Learning
(UoL)
Technical Infrastructure Processes, threads,
processors or FPGA
xReality toolkit and
system
30
Anasol Pe単 a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, 束End-user programming & deconstrutionalism for collaborative
mixed reality laboratory co-creative activities損on 2nd European Immersive Education Summit (EiED12), Paris, France, 2012.
29. Implementation phases
31
Anasol Pe単 a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, 束Towards the Next Generation of Learning Environments: An
InterReality Learning Portal and Model損on 8th International Conference on Intelligent Environments 2012 (IE'12), Guanajuato,
Mexico, 2012.
30. Scale Up project
A collaboration between King Abdulaziz University, KSA and
the University of Essex,UK.
32
31. Summary
Core elements of our mixed reality activities
Combine them using end-user programming
to create an educational mixed reality object
Immersive mixed reality learning environment
Grounded on collaborative, constructionalist,
problem-based learning theories
Offers the possibility of creating collaborative
laboratory activities for geographically
dispersed students
InterReality
Portal
xReality &
Virtual objects
34
32. Challenges
Pedagogical challenge: a constructionist student-centred
method to create laboratory activities for distance
learners
A representation of
a deconstructed
worldTechnical
challenges
Learners
Instructors
A set of components that
can be shared and
combined
Distribution of objects
between different immersive
environments
Blended
reality
distributed
system
35
33. Challenges (2)
To solve this we have proposed the use of
a deconstructionist architecture.
Physical model of
distributed xReality
objects in an
immersive learning
environment
Pedagogical
model of
constructionist
laboratory
activities
A solution forA solution for
distributeddistributed
mixed realitymixed reality
laboratorieslaboratories
36
Me PhD Candidate and Information Officer at the Immersive Education initiative European chapter. My research focused on Intelligent Learning Environments, mixed reality and immersive education. This research is part of a collaborative project between Essex and King Abdulaziz Uni to scale-up existing technology in intelligent environments, from smaller facilities such as the University of Essex iSpace to a complete campus with multi-story large footprint buildings.
Survey conducted by The Economist magazine in 2008 with 289 executives responded the survey, 189 participants came from higher education and 100 came from corporate settings
A virtual learning environment (web/ 3D world) A physical learning environment (real world)
ejemplo Blended Reality: use of videoconferences (Skype) ejemplo Vacancy problem: everyday activities as run
But what happen with laboratory activities?
University of California Davis School of Medicine - Virtual Hallucinations Project: http://www.ucdmc.ucdavis.edu/welcome/features/20070404_virtual_psych/index.html Video (a bit disturbing): http://www.youtube.com/watch?v=s33Y5nI5Wbc Sakshat Amrita Virtual Lab (Amrita University) Project: http://amrita.vlab.co.in/ Video: http://www.youtube.com/watch?v=ViqHtlZSOjM Ohio University College of Osteopathic Medicine - Virtual Haptic Back Project for palpatory training http://www.ohio.edu/people/williar4/html/VHB/VHB.html University of Bristol - LabSkills http://www.bristol.ac.uk/news/2009/6154.html Software was developed by Bristol Uni and currently it is used in Durham Uni and Notthingham Uni (each unigave a talk about the use of this software in their courses) http://www.labskills.co.uk/ University of Leeds - Virtual Labs http://www.virtual-labs.leeds.ac.uk/pres/index.php Freie Universitat Berlin - UniversitTechnology Enhanced Textbook (TET) http://didaktik.physik.fu-berlin.de/projekte/tet/ http://didaktik.physik.fu-berlin.de/IMPAL/show/demo.php Durham University - Interactive screen experiments - physics http://level1.physics.dur.ac.uk/general/index.php
The interaction between the elements can be defined as Cross-Reality (xReality) or dual reality
The implementation of the InterReality Portal is based on three major components: A real environment A 3d virtual environment xReality objects and Virtual objects
we combine 1) a semi-spherical sectioned screen that includes a desk allowing the user to be sit in a natural position to perform learning activities, with a free-range of head movement without the need of any intrusive body instrumentation, based on a specification from the sci-fi story Tales from a Pod 2) a camera that allows students to interact between them and the environment, 3) a network of sensors and actuators to obtain real-time information for object identification and replication for dual reality states.
Following the ideas of smart objects on the Internet-of-Things (IoT), an xReality object has a unique ID, a list of available services (e.g. to get data or receive data) and in some cases rules (e.g. a certain object cannot work without fulfilling some preconditions). In a similar way each virtual object has a unique ID, one or more behaviours attached (e.g. the virtual object must behave as a solid object according to physical variables, such as weight, gravity, etc.) and rules similarly to the xReality object
formed by a group of interchangeable pluggable components, to create a mashup with a main component which identifies and integrates the others
As long as the session continues, changes in any of the objects will be managed by the Context-Awareness agent and the Mixed Reality agent considering the following scenarios: A change in any Virtual object of a given InterReality Portal results in identical changes to all subscribing InterReality portals. A change in an xReality object of a given InterReality Portal results in changes in the representation of the real device on all subscribing InterReality portals.
Learners view: During the learning session when they interact with the environment and between them, they are programming a series of actions to be executed by the objects. Instructors view: While they are creating a UoL, they are establishing a sequence of activities to be performed by the learners during the educational session.
is to built computer science projects combining hardware (xReality objects) and software modules (virtual objects) to create Internet-of-Things applications emphasising computing fundamentals grounded on co-creative and collaborative interaction between learners using problem-based learning (PBL). A diversity of resources may be available to support learning however this process cannot be complete without IMS = Instructional Management Systems (IMS) Global Learning Consortium BENEFITS portability and reusability Problem-based learning is a constructionist method that allow students to construct their own knowledge by the correlation between concepts and proposed solutions to real world problems performed in realistic settings
From the instructors perspective the creation of UoL is based on activities (atomic functions) which can be combined to create nuclear functions (a complete UoL). From the learners perspective xReality and virtual objects (atomic functions) can be considered as part of a deconstructed set of components that students can reconstruct in any combination to generate their own unique xReality project or to reconstruct a xReality project that is prescribed by the instructor (nuclear functions). From a technical (system architecture viewpoint) the deconstructed elements become sets of autonomous networked resources, that may be inter-connected to form different combinations (constructed) forming a variety of student projects or UoL as required by the teacher or student.
our research is moving from phase 1 to phase 2, integrating the InterReality Portal implementation with the end-user programming concepts discussed within this paper. The possibility of combining virtual and xReality objects with deconstructionism and the use of end-user programming to create mixed reality learning activities that can be constructed and shared by teams of geographically dispersed students is our final goal.
We presented
We presented a series of challenges for constructing this model