08. Collaboration in Mixed Reality

Tuckman’s model of group formation:

1. Forming

• Orientating themselves around the task at hand
• Become acquainted with each other
• Testing group behaviors
• Establishing common viewpoints, values
• Establishing initial ground rules

2. Storming

• Marked by intense team conflicts
• Leadership and roles determined
• Project and tasks redefined
• Characteristics:
  • Disagreements
  • Resistance to task demands
  • Venting of disagreements
  • High level of uncertainty about the goals

3. Norming

• Team roles cleared up
• Agreement on how the team can work with each other
• Clear expectations and consensus on group behaviors and norms
• Consensus on group goals, quality standards
• Forming the basis for behavior for the remainder of the project

4. Performing

• Active work on a project
• Clearly understood roles, tasks, and well-defined norms
• Sufficient interest and energy from all team members

5. Adjourning

• Dissolution of the team: team tasks are accomplished and the team disbands
• Possible feelings of regret

Drexler’s team performance model:

1. Orientation: why I am here?
2. Trust building: who are you?
3. Goal clarification: what are we doing?
4. Commitment: how are we doing it?
5. Implementation: who does what, when, where?
6. High performance
7. Renewal

Collaboration

Definitions

• Wood and Gray, 1991: a process that occurs when a group of stakeholders engage in an interactive process using shared rules, norms and structures to act or decided on issues related to that domain
• Terveen, 1995: a process in which two or more agents work together to achieve a shared goal
• Knoll and Lukosch, 2013: an interactive process in which a group of individual group members use shared rules, norms and structures to create or share knowledge in order to perform a collaborative task

Designing collaboration

Collaboration is affected by internal and external factors:

• The group: size, proximity, experience
• Task: type and complexity
• Context: organizational culture and environment
• Process: interactive process, shared rules, norms
• Tools: technology and their limitations

Collaboration outcomes:

• Creative ideas for activities
• Shared understanding
• Commitment
• Consensus
• Sharing perspectives and visions
• More objective evaluation
• Acceptance
• Mutual learning
• Shared responsibility

e.g. using AR to help people understand impacts of climate change.

Collaboration Challenges

Piirainen et al., 2012 - group perspective:

• Shared understanding:
  • Ensure the team has a shared understanding and mental models of:
    • The problem
    • The current state of the system
    • The envisioned solution
• Satisfying quality requirements/constraints
• Balancing rigor and relevance
  • The more formal the process, the slower you go but the more you can involve and understand stakeholders
• Organizing and ensuring effective, efficient interaction between actors
• Ensuring ownership
  • Team members must pick up tasks and take ownership of them

Nunamaker et al. 1997 - process perspective:

• Free riding
  • Especially in larger groups
• Dominance
  • Both the amount of work done and of decision-making power
• Group think
• Hidden agenda
• Fixed design
  • Process limits the design space the group can explore
• Lack of expert facilitators

Haake et al., 2010 and Olson and Olson, 2000 - tool perspective:

• Google Docs, email, video conferencing, etc.
• No regular use
• Variety
• Not intuitive
• Difficult to adapt to group needs
• Collaboration awareness
  • Being aware of when other people have made changes
• Co- and spatial referencing

Collaboration Design from a Tool Perspective

Time-space matrix of Computer-Supported Cooperative Work (CSCW):

• Same place, same time (synchronous interaction): face-to-face interaction
• Same place, different time (asynchronous interaction): shared files, team rooms etc.
• Different place, same time (synchronous distributed): video calls, shared editors etc.
• Different place, different place (asynchronous distributed): email, newsgroups etc.

In AR:

• Synchronous, co-located: AR shared space
• Synchronous, remote: AR telepresence
• Asynchronous, co-located: AR annotations/browsing (in-situ)
• Asynchronous, remote: generic sharing

3C model:

• Communication: information exchange to facilitate a shared understanding
• Coordination: arranging task-oriented activities
• Collaboration: working together towards a shared goal
• Group awareness mediates relation: none of the other 3Cs are possible without it

Human-Computer-Human Interaction Design

• Software design: software interacts with other software
• Human-computer interaction design: humans interacting with computers
• Human-computer-human interaction design: several humans in front of several computing devices working together towards a shared task
  • Computers must interact with each other, and humans must interact with each other as well

Oregon Software Development Process (OSDP) (Lukosch, 2007):

• Oregon Experiment, Christopher Alexander:
  • University campus did not put down any footpaths initially, but waited to see what trails the students would make
• Patterns for computer-mediated interaction:
  • High-level patterns:
    • Focus on issues and solutions targeted at end users
    • Empower end users to shape their groupware application
  • Low-level patterns:
    • Describe issues/solutions targeted at software developers
    • Focus on system implementation and includes technical details
  • Example: remote field of vision
    • Collaborative whiteboard/canvas: need to know where team members are and where they are looking at
    • Possible solution: multi-user scrollbar (multiple narrow scrollbars)
      • Understand where their team members are both globally and relative to themselves
      • Users can see roughly how much of their screen space intersects with another user’s, and where
• Iterations follow design -> implementation -> test/usage -> planning cycle
• Conceptual iteration
  • Talking to users, understanding the problem space, creating prototypes
  • Use of patterns to discuss high-level ideas with users
  • Developers use the low-level patterns tODO
• Development iteration: TODO
  • Requirements analysis
  • Low-level patterns used to plan and design groupware
  • Functional tests
• Tailoring iteration:
  • Users have used the prototypes and have provided feedback

Workspace Awareness in Collaborative AR

Types:

• Informal awareness:
  • General sense of who is around and what they are up to
  • Not necessarily related to project work
• Social awareness:
  • Understanding of the person:
    • What they are interested in
    • Their emotional state
    • What they are paying attention to
• Group-structural awareness:
  • Knowledge about the group structure:
    • Roles/responsibilities/status
    • Positions on issues
    • Group processes
• Workspace awareness:
  • Understanding of the task space
  • Interaction of others with the space and its artifacts

Awareness categories and elements:

• Who:
  • Presence: is anyone in the workspace?
  • Identity: who is participating?
  • Authorship: who is doing that?
• What:
  • Action: what are they doing?
  • Intention: what is their goal?
    • They are doing x in order to achieve y
  • Artifact: what object are they working on
• Where:
  • Location: where are they working?
  • Gaze: where are they looking?
  • View: where can they see?
  • Reach: where can they reach?
    • Can children or short people access it?

Workspace awareness:

• Knowledge: who/what/when/when/how
  • is used to determine what to look for next
• Exploration
  • is used to gather perceptual information
• The environment
  • aids in interpreting the perceptual information
• Knowledge
• is used to help with collaboration
  • Coordination of activities
  • Anticipation of events
• which impacts the environment

Case Studies

Workspace awareness in collaborative AR:

• Remote expert can see what the player can see and give hints on how to complete the puzzle
• Expert given a gray box which represents the size of the Hololen’s display
• Expert can freeze the view:
  • View is continually changes, which makes it difficult to focus
  • Hence, they should be able to freeze the view (and annotate it), possibly in a separate window
• The remote person must be able to communicate to the local person that they have made changes or annotated something: workspace awareness
• They can, of course, talk, but the paper tried adding automatic notifications:
  • Aural: TTS when the remote user adds/selects/deletes and object, or when they freeze/unfreeze the view
  • Visual: small blinking icon
• Results:
  • Audio is much more noticeable, but also more annoying
  • Participants preferred visual notifications

A collaborative game to study presence and situational awareness in a physical and an augmented reality environment:

• Game played in two environments (physical/augmented reality)
• Investigate how a remote person can try to help a local team
• Three players had to build a Lego tower following certain constraints:
  • Each player has access to a subset of the constraints
  • Each player could only move certain-colored blocks
    • But also some blocks that everyone could move
• Two players co-located, one player remote
  • Co-located users wearing HMDs, remote user viewing laptop
• The same group also played it physically (with randomized order)
• Asked AR presence questionnaire:
  • Interaction/immersion
  • Interference/distraction
  • Audio/tactile experiment
  • Moving in environment
• Results:
  • Mental demand not significantly different
  • Physical demand in AR higher
    • Finger has to hover in midair
  • Slower in AR
  • Presence:
    • Interaction in AR much more difficult, and impacts concentration
    • Difficult for remote player to understand and foresee the other people’s actions
    • Co-located AR players reported tactile experience (even though it was completely virtual)

CSI The Hague:

• Collaboration with Hague police circa. 2009
• Special skills required to secure evidence
• Need to capture evidence early on, but collector is likely not an expert
• Expert could remotely help the on-site person
• Video see-through HMD
• Two webcams used for SLAM:
  • 3D pose estimation
  • Dense 3D map
    • Remote user could explore the space in VR
• Bare hand tracking for gesture-based interaction
• Evaluation:
  • Lack of protocol for collaboration
  • High mutual understanding
  • Picture-oriented information exchange
  • High consensus: both parties can see the same video stream
• Data integrity: how do you ensure it has not been modified
• Responsibility: if the crime scene gets messed up, who is responsible - the local person or the expert?

Burkhardt et al., 2009: seven dimensions of collaboration:

• Fluidity of collaboration: verbal turns (cues?)/actions
• Sustaining mutual understanding
• Information exchange for problem-solving
• Argumentation and reaching consensus
• Task/time management
• Cooperative/collaborative spirit in the team
• Awareness of their individual tasks and contribution