Page 167 - Kaleidoscope Academic Conference Proceedings 2021

Connecting physical and virtual worlds

Microsoft Mixed-Reality Toolkit is used to provide client 1. Aroaro is a multi-user environment, in which tools for
services. Data sets, currently .csv and .json formats, are collaboration have been incorporated. Collaboration can
supported allowing connection to cloud and local databases happen either synchronously or asynchronously. A team of
and API services. As Figure 1 shows, Aroaro links to several decision makers can meet in Aroaro from geographically
cloud services, which mainly provide access to data. Data is distant places. If the session happens with all members being
reached directly on the machine Aroaro runs on, i.e., a PC or simultaneously in the virtual environment, then interaction
HMD for the VR application, or HoloLens for the AR will occur in a synchronous manner. Decision makers can
application. also rely on individual immersive analytics and leave their
annotations on virtual or real objects of interest so that other
A cloud relay, Photon, enables multiple users to work on the team members can continue their tasks later.
same tasks and interact with objects in the same virtual space
at the same time. Users from distant, different places can be 2. Aroaro’s virtual reality facility provides a user with total
present in the same collaboration space viewing the same spatial immersion and in doing so it is hoped users gain
visualizations from different viewpoints and communicating better understanding of the data. User interaction with
synchronously with speech or asynchronously with shared network data happens as the user “flies into the network”,
notes. while being able to explore the object with a sense of
unconstrained space. The environment is a full 3D
Microsoft Azure Cognitive Services are used for speech to visualization of Aroaro’s interpretation of abstract data
text, enabling users to avoid awkward typing on a virtual contained in a network data set.
keyboard. Messages sent to the server are returned to the user
with an interpretation of the text. Complementing Azure, 3. Multi-user experience requires a means of communication
Aroaro uses PlayFab as the back-end tool to provide user within the virtual space offered by Aroaro. Similar to
management. It is also used to store scene objects. presence under a collaborative environment, the ability to
hear other decision makers’ voices can correspond to either
Aroaro currently runs on HoloLens 1, HoloLens 2, Windows synchronous or asynchronous modes. In Aroaro, multi-
Mixed Reality and Android with iOS and Oculus Quest soon sensory capability is embodied into the audio tool, which
to be released. An alpha version of Aroaro for Android is contemplates the ability for users to leave recordings on
available at https://aroaro.auckland.ac.nz/aroaro-on-android/. designated objects in the virtual space for their collaborators
HoloLens 2 is a particularly welcome enhancement to to listen to some time later.
HoloLens 1, which can track the movement of hands and
eyes allowing the user to virtually move the objects by using 4. The combination and utilization of features that take
the fingers. advantage of the opportunities listed above facilitate a higher
level of engagement. This is of particular importance to
Heroku is used to host the code that calculates network decision-making supported on network data visualization,
measures such as centrality and other degree-based node which is the business task of interest studied in this paper.
properties. Using a cloud-based approach with Heroku
allows Aroaro to offload the processing from the XR client. Increasingly, more business activities and processes as well
as human organizations and business structures can be
4. IMMERSIVE ANALYTICS described as networks. Increasingly too, organizations need
access to information contained in the connectivity
According to Dwyer et al. [8], Immersive Analytics (IA) is properties of a network, as recorded in a network data set. It
“the use of engaging, embodied analysis tools to support is important to realize this type of data set is inherently richer
data understanding and decision making.” Immersive and more complex than a data set where, a priori, there is no
analytics builds upon several features such as data seeming relation between the entities/subjects/names that
visualization, visual analytics, virtual reality, computer make up the data set.
graphics, human-computer interaction to support data
understanding, and either individual or collaborative Aroaro has been endowed with a facility that allows the user
decision-making. IA is a new technology that uses head- to display the connectivity information inherent to a network
mounted VR/AR or hand-held equipment to provide new data set as a graph that “floats on the virtual space”. We have
kinds of display and interaction technologies such as sensor chosen to use known measurements of node centrality, which
technologies as well as machine learning [9]. are pre-computed when the network data set is stored in
Aroaro for the first time. Such measurements constitute the
4.1 Opportunities afforded by IA building blocks of the analysis to be carried out by decision
makers able to access the network data visualizations module
We find it adequate to introduce IA to Aroaro as it offers of Aroaro.
opportunities beyond traditional analytics such as those
identified in [7]: situated analytics, embodied data 4.2 IA in the context of network data visualization
exploration, collaboration, spatial immersion, multi-sensory
presentation and engagement. Let us highlight how Aroaro We argue network information is becoming of heightened
takes on some of these opportunities. strategic importance to many business organizations.

– 105 –