ISMAR 2024 Tutorial: Participating in the Testing of VERA

Overview

The Virtual Experience Research Accelerator (VERA) is currently undergoing development, but by ISMAR 2024 we expect to have an early version of the system available for the running of real or realistic user studies, for the purpose of testing the VERA systems and procedures. On 21 OCT 2024 we will conduct an ISMAR 2024 tutorial to introduce attendees to the requirements and procedures for getting involved in such testing, what we expect in terms of tester effort, and what we expect in terms of test system functionality. We will update this website with materials relevant to the tutorial prior to the conference, so that we may use the in-person setting to focus on demos, adapting studies, and discussion.

Motivation

When carrying out traditional laboratory-based human subjects research for XR, researchers are inherently constrained to relatively slow data collection with relatively small and demographically narrow “convenience” samples of the population. With VERA, researchers will have the opportunity to push their study out to a standing participant pool and gain human subjects data from a relatively large population, with controlled demographics, in a relatively short time. VERA seeks to provide the XR researcher community with a one-of-a-kind tool that will support more sophisticated, robust, and impactful XR research, benefiting the field of XR and society as a whole.

Testing at early stages will not involve paid participants. When testing does involve paid participants, it is our intent to subsidize some of the participant costs, with priority given to those with demonstrated committed engagement in the testing process. With or without paid participants, we anticipate that the testing will be a net-positive experience for the testers: in exchange for engagement in the testing process they will get early and free or reduced-cost use of VERA for what we expect will be publishable results. They will also be helping to develop a resource that will eventually be available to all XR researchers, i.e. a chance to “do good” for the XR research community. We will publicly express our appreciation to all institutions participating in the testing.

Organizers

Ali HASKINS LISLE, Ph.D.
Director of Development and Operations, VERA
University of Central Florida
ahaskins@ucf.edu

Presenters

Gerd BRUDER, Ph.D.
Research Associate Professor
University of Central Florida

Ali HASKINS LISLE, Ph.D.
Director of Development & Operations, VERA
University of Central Florida

Greg WELCH, Ph.D.
Pegasus Professor and AdventHealth Endowed Chair in Simulation
University of Central Florida

Agenda (Tentative)

1. Introduction (10 Minutes)

• Motivation
  • Discuss the limitations of traditional lab-based studies in XR research, such as small, homogenous sample sizes and logistical challenges.
  • Explain how VERA will addresses some of these limitations by providing access to a large, diverse participant pool for remote XR experiments.
  • Highlight the potential impact of VERA on advancing XR research and fostering innovation in the field.
• Infrastructure Overview
  • Provide context on NSF Infrastructure Grant
  • Provide an overview of the VERA platform, including its architecture and key components.
  • Discuss the technology stack used to build VERA, such as the VERA App, API, frontend, Unity plugin, and database.
  • Highlight the capabilities of each component and how they contribute to the overall functionality of VERA.

2. A Tour of VERA (30 Minutes)

• Users and Community
  • Introduce the various user roles within the VERA ecosystem, including researchers, participants, and administrators.
  • Discuss the importance of community engagement and mentorship within the VERA community.
  • Highlight how users can collaborate within VERA.
• Software and Processes
  • Discuss phased MVP approach and release schedule
  • Discuss the user permissions models within VERA, including access levels for researchers (e.g., faculty and students), individuals in the VERA participant pool, and VERA administrators.
  • Provide guidance on navigating IRB approvals and institutional agreements when using VERA for research.
  • Walk through the processes involved in conducting a study using VERA, from study creation to participant recruitment and data collection.

3. Testing VERA (2.5 hours, with breaks)

• Overview of the Roles, Resources, and Responsibilities
  • Overview of researchers vs.participant workflows.
  • Overview of the specific resources available to researchers and participants.
  • Review of the potential benefits to the researchers participating in the testing.
  • Review of the expectations for the engagement and efforts of researchers participating in the testing.
• Permissions and IRB Processes
  • Provide detailed guidance on obtaining permissions for researchers, students, and lab managers to access VERA resources.
  • Discuss best practices for ensuring compliance with IRB regulations when conducting research using VERA, including data privacy and participant consent.
  • Explain the steps involved in establishing institutional agreements with VERA.
• Example Experiment Working Session
  • Work through multiple examples of studies being run through VERA
  • Expose known risks and / or unexpected workflow pathways
  • Address best practices
  • Provide examples for attendees to work through themselves. (We will attempt to assemble a volunteer group of remote participants for this purpose, or simulate the same.)
• Catered Assistance, Questions, Feedback
  • Offer personalized assistance to tutorial attendees, addressing any questions or concerns they may have about using VERA for their research projects.
  • Solicit feedback from attendees about VERA and any suggestions for improving the platform.
  • Provide resources for ongoing support and community engagement, such as user forums, documentation, and tutorials.

02 JUL 2024

This material is based upon work supported primarily by the National Science Foundation under Grant Number 2235066.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.