On January 26th, 2022, Jeremy Nelson presented the development and projects of the Extended Reality Initiative at the University of Michigan to the XR ERA community.
About Jeremy Nelson
Jeremy Nelson is the director of the Extended Reality Initiative at the University of Michigan, where he works alongside a team of developers and student fellows to create extended reality (XR) content for faculty members to include in their courses. Projects funded by the Extended Reality Initiative range from development-heavy video game-style virtual reality (VR) experiences to 360-video immersive content, augmented reality (AR), and mixed reality (MR) in healthcare. Jeremy also consults on XR-related projects across the University of Michigan.
The Extended Reality Initiative at the University of Michigan
The Extended Reality Initiative launched in the fall of 2019 with the aim to actively engage the University of Michigan community with knowledge and content through immersive technology. In “bringing XR technologies campus-wide,” told Nelson, there were three objectives:
- Integrate XR into residential curricula for all nineteen schools and colleges that are a part of the institution.
- Integrate XR into online curricula.
- Create innovative public/private partnerships.
Currently, the initiative works to fund and provide consultation for projects across all nineteen schools and colleges. The integration of XR into online curricula is done by a team of learning experience designers, media designers, and behavioral scientists at the Center of Academic Innovation. Massive open online courses have been developed for hybrid online degree programs delivered through partner platforms such as Coursera and FutureLearn. To bring XR technologies into online spaces is a particularly interesting challenge, considering the variability of access to devices, explained Nelson.
The XR Curriculum at the University of Michigan
Any student in a graduate program is eligible to enroll in a certificate program as part of the XR curriculum at the University of Michigan. Nelson gave the example of a three-course series XR For Everybody, first launched in December of 2020, providing a broad learning experience for students of various backgrounds. Revolving around knowing, doing, and shaping the future of XR, the starting point within the course is “how to get started in XR.” The course contents consist of a series of videos, assignments, and projects that the students work through at their own pace.
First Round of Projects
In the first year, the Extended Reality Initiative received a grant to expand the team. Thirty-six students from different graduate programs were hired, assisting with storyboarding, prototyping, and interaction design and receiving training in XR. The first round in 2020 had eight projects, four of which were large VR projects that involved “building everything from scratch.” One of those projects was XR Nuclear Reactor Laboratory (seen in the video below), which with the advent of COVID-19 regulations, aimed to allow student participation in experiments feasible without physical attendance. It entailed creating a digital twin of the on-campus nuclear reactor, enabling students to learn how it functions by exploring the space and running simulations.
Another of the first round of projects was Augmented Tectonics (seen in the video below), which enabled architecture students to enter VR to learn how to build with various materials. Met with design challenges, students explore how the properties of materials such as steel, glass, concrete, and wood yield different potential for what is possible to build.
Designing a Process for Project Development
Based on what had been learned in the first round, one of the aims was to design a process for project development. Nelson, coming from a background of scalability and repeatability, emphasized that the process was to be designed such that it could be repeated. Additionally, it needed to facilitate students transitioning in and out so they could be “plugged into” any part of the process and make considerable progress. The process comprises the following steps:
- Discovery (1 week)
- Flow diagram
- Schedule/timeline
- RACI Matrix
- XR Design Document
- Pre-production (2 weeks)
- Script
- Storyboard
- Prototypes
- Moodboard/design guide
- Production (1-6 months)
- Reach Minimum Viable Product (MVP)
- Post-production (1 week) [iterate steps 2 and 3]
- Final export
- Distribution (3-4 days)
- Published content
- Launch (4 days)
- Feedback capture
- Form
- Issue tracking
- Document process
- Maintenance
Second Round of Projects
The second round consisted of fourteen projects, one of which resulted in a social VR event called Virtual Diag with around two hundred attendees worldwide. The project was a collaboration between an architecture student and an art student, who recreated the center of the university campus where students would gather and socialize.
Nelson then recounted how the nursing school faculty came to him with the problem of having too many students and not enough faculty to train and teach them procedural skills. Having just begun exploring the Hololens 2, he got the idea that the nursing school faculty could utilize it to allow students to have an AR training experience. Most of the training is guided, so learning procedural skills such as a lumbar puncture or catheterization would involve practicing on mannequins with someone guiding the student through the steps. By incorporating XR technologies, that process became a self-guided training experience in AR.
Nelson ended with the prospective goals of the Extended Reality Initiative:
- Continue to fund XR Projects
- Integrate XR into more online courses
- Explore virtual production
- Expand Michigan Metaverse
- XR Summit 2022
Q&A Session
The first inquiry was about the decision-making process behind selecting projects for funding. “Can people from any domain submit project proposals, or is it dependent on costs or the people involved?” For the first round of projects, the open call was “wide open” to “just see what we get” and get an idea of “what people want to do,” explained Nelson. The second round was much more targeted because of limited funding and not having the development capacity to continue to build such intense and large game engine applications. Subsequently, a requirement was that the faculty include a commercial tool in the proposal, enabling more scalable and sustainable projects.
A participant inquired about how the process of the project rounds is managed, asking whether applicants are expected to complete their proposal within a set period. Is it within the duration of the school year, or are applicants given a longer period? Nelson responded that applicants are generally given eighteen months, but depending on the scope of the project, sometimes extensions are given. For the large VR projects, “we were willing to extend it” because, though “we wanted to try to control the scope […] we also wanted them to look really good.”
A participant who works with XR within Leiden University asked how the initiative manages and organizes all of these projects since working with such “novel technologies can be interesting to manage” and is incredibly time-consuming. Nelson responded that his role at the university is entirely dedicated to managing all of the projects, creating communities, and connecting with other institutions. Having such a nonfragmented role and student participation is “what makes it all happen.”
Lastly, Nelson was asked whether there is the “possibility, or momentum, to exchange productions between universities?” Nelson responded, “we are open to it, whether it is a project or sharing ideas, or how we did this or that?”
