As shared in a previous article in January 2021, we organized a fruitful community meetup, in a virtual reconstruction of the vanished neighbourhood Vlooienburg in Amsterdam. The meetup’s goal was to collaboratively explore the potential of remote teaching using virtual reality and 3D reconstructions. This meetup could not have happened without the existing work of 4D Research Lab and full commitment of them, SURF and TU Delft to make 4D Research Lab’s models fully experienceable, enabling visitors to wander through a life-size 3D reconstruction in VR of the Amsterdam neighbourhood Vlooienburg in the 17th century.

Tijm Lanjouw, Jitte Waagen, Paul Melis and Huu Dat Nguyen were so kind as to share some insights into the creation process, their motivations and future plans through a written interview. Enjoy their explanations and thoughts and don’t hesitate to get in touch with them or us at XR ERA to discuss ideas for collaboration. 

If you haven’t done so, we’d recommend you to first read the meetup recap. 

Interviewees

Jitte Waagen – Coordinator at the University of Amsterdam, 4D Research Lab

Tijm Lanjouw – 3D Modelling for Archaeology at the University of Amsterdam, 4D Research Lab

Paul Melis – Senior Consultant Visualization at SURF

Huu Dat Nguyen – Virtual Reality Developer at TU Delft, VR-Zone

Tijm, did you already use your 3D models in VR environments before? / Did you already think about the opportunity of using your reconstructions in VR environments before? 

“Not actual virtual reality, but we have some experience using augmented reality using smartphones and Microsoft Hololens in other projects at the Allard Pierson museum (the Amsterdam University museum with a large archaeological collection). Although the current Vlooienburg model is displayed as a touchscreen application in the Jewish Historical museum, we have always wished to make a version in virtual reality as well, so this was a great opportunity.”

Participants wandering around the model of Vlooienburg in Mozzila Hubs, guided by JItte Waagen and Tijm Lanjouw of the 4D Research Lab.

What steps were taken in transforming the model 4D Research Lab provided into the lecture environment?

Paul: “After we contacted the 4D Lab and learned about the existing 3D model of Vlooienburg we knew it could serve as a basis for the virtual tour. The model was really detailed, looked great and gave a unique historical feel. However, to use the model in the Hubs environment meant transforming it to something a bit less detailed and graphically less complex. This was needed to keep the hardware requirements for the participants manageable. At SURF we did some tests in Hubs to get a feel for the limitations on the model, which was in turn used by 4D Research Lab to provide a simplified model. 

All in all the number of iterations needed was minimal and it was actually quite surprising how smoothly we succeeded to display the model in in a Mozilla Hubs Room. Apart from the Vlooienburg model itself we needed to set up things like the starting point in the model for the tour, added a skydome showing some nice clouds and making sure participants walking around the model could navigate to where they wanted to go. The Vlooienburg scene was a separate room in Hubs, while the introductory museum setting was another room, which was smaller in scope.”     

Huu Dat, what was your role in this project?

“My role was to realize the museum setting, which was also the room where al participants started the tour. I had the freedom to start from scratch and based on some preliminary notes from the 4D Research Lab where they expressed their needs and ideas I started the creation process. The setting needed to be small and simple, so that the visitors could get the hang of the theme and get acquainted with navigation in Mozilla Hubs before they entered Vlooienburg, which was the main course.

Although no one had experience creating custom environments in Mozilla Hubs before, it was not that hard to realize the setup and make sure both rooms complemented each other. There was good communication between the teams when we were all still figuring out how to get it to work. By quickly sharing knowledge and ideas, we saved a lot of time.“

How does this project and how does this work inspire you for other projects at TU Delft?

Huu Dat: “Experimenting with virtual and playful environments is something we do a lot in the VR-Zone. Compared to many other workflows that we use to create VR environments, Mozilla Hubs is relatively accessible. It is relatively easy to setup for the developer and friendly to the end-user. Seeing how this project turned out out made me see that there are a lot of possibilities with Mozilla Hubs for other projects at TU Delft.”

What challenges / obstacles did you come across in preparing the environment and tour?

Paul: “One technical limitation that we were aware of from the start was the maximum number of participants in a Mozilla Hubs room, which is recommended not to go over 25. We actually had a few more in the XR ERA session, which probably caused some of the audio issues we experienced. Other than that I don’t think we suffered issues from the high number of participants. Interestingly, people in the virtual environment kept quite a bit more distance than they would in a physical tour, but there was lots of room in the model so this wasn’t a problem.

The audio settings for the room were a bit of a challenge during the preparation. By default audio from a participant’s avatar falls off quite quickly as you get further away. This allows people close together to talk to each other without interfering with others further away in the room. However, for the guided tour we wanted to make sure that you could always figure out where the group was by listening to where the guide’s speech was coming from. So we changed the room settings so that basically anyone can hear everyone else, regardless of how far apart they are. This worked fine for the goal we had for it, but made it impossible to “privately” talk amongst participants.

One final issue that we investigated was how the 3D models uploaded to Mozilla Hubs would be treated from a privacy and commercial standpoint, as the platform is cloud-based. But since it is developed and hosted by the non-profit Mozilla Foundation it turned out that uploaded data would not be sold or otherwise be made available without permission from the uploader, so that was great”.

Paul, how do you see the value of this type of lecturing in light of (higher) education / the type of education the SURF members are involved in and what are the main obstacles?

“Using a 3D virtual environment for lecturing will most likely add value for topics where spatial relationships and/or interaction is inherent to the subject. A 3D representation, especially when it’s interactive, can offer a much more natural presentation for such cases. For example, the way the Vlooienburg area was shown and experienced as a life-size 3D environment is hard to match, even when using (say) an animated video of the same environment. And being able to interact freely as a participant has upsides as well, in terms of being able to choose your focus, with whom you (virtually) engage during the presentation, etc. In this way it is similar to a real-life physical environment.

But there is a flipside as well, in that not all topics or content are a good fit for presentation in a 3D virtual environment. One participant remarked that the introductory presentation in the museum setting (using images on museum walls) could also have been done using a regular slide presentation in a video-conferencing application. That is a fair point, but we felt that starting in the museum room would allow the participants to get a feel for the way the Mozzilla Hubs environment worked, before we moved on to the Vlooienburg scene. Also, it can take quite a bit of effort to prepare the content and setup for a lecture in virtual reality. But one nice thing is that Mozilla Hubs rooms can be kept alive for longer periods of time, allowing people to visit them more than once and/or at their own chosen moment.

The current features in the Mozilla Hubs environment offer a low threshold for usage, even for less technically-inclined people. A URL to the Mozilla Hubs room and a web browser on a standard PC or laptop are enough for anyone to join the environment, even without needing an account, which is great. Plus it’s free and open-source.

But some usability improvements would definitely help:

• For example, teleporting in a Hubs room is currently done using unnamed icons shown hovering in the 3D environment, with no option to teleport to a named location (or the location of a specific person) in the room. This made it hard for a few of the participants to quickly get back to the group after they got lost in the neighbourhood’s model. 
• It is also not very clear which person is speaking, as that person’s avatar is only slightly scaled which can easily be missed or misunderstood. It would be better to show a clear visual indicator of which avatar is speaking. All in all, such usability issues can make a virtual session more challenging for some of the participants, especially if they are less familiar with such 3D environments or the technology involved.
• One final limitation to note is that Mozilla Hubs scenes are pretty much static, with no options to do even simple variations, especially based on interaction. For example, we initially thought about showing the Vlooienburg area at multiple points in history, switching between several 3D models using something like a button press. However, this turned out to be technically infeasible in the current platform, which really limits the type of content and experiences that can be built. Hopefully this will improve in future”.

Tijm and Jitte, how did you experience giving a lecture in a virtual environment? What was easier than you expected / what was harder than you expected? Were there any unforeseen positive aspects you discovered?

Tijm: “For me, it was much more fun than giving a traditional slide presentation. The tour-guide approach to presenting our models really works well. It felt more natural and easygoing than a traditional presentation where everybody is forced to look at one person and presentation. I practiced a couple of times the days ahead of the presentation to familiarize myself with the navigation and the virtual pointing stick/drawing tool, so I could use it without thinking too much. Above all, I found it exciting to observe other people, or their avatars, explore Vlooienburg and to see what attracts their attention. People being able to move around freely, while still being able to listen to the tour-guide is also an advantage over real city tours where you have to stay near your group to hear the guide. Of course, the downside is that there’s no way of checking whether people are really listening“.

Jitte: “I think I should indeed emphasize that we are very enthusiastic about it! As for the functionality of the virtual environment, I would certainly say that over the whole, doing a presentation in this way was much easier and more intuitive than I expected. It takes just a short time to get acquainted with the navigation after which I quite effortlessly slipped into a quite natural mode of talking to a crowd. The hubs environment really provided the possibilities to engage with the 3D reconstruction in an inspiring manner. A pitfall that we experienced with past touchscreen interfaces and AR apps is that they can be limiting in the number of people enjoying them, and thereby limit a more communicating and collaborative experience. Of course there are other solutions to this, but the hubs environment brought group interaction and a VR experience smoothly together.

A downside that I experienced is that personally, I usually get a lot of feedback through nonverbal communication, observing members in the audience, which I completely missed here. So it is difficult to gauge whether the information is actually landing, or whether one should elaborate or skip through some parts of a presentation. This is in line with what Paul mentions above: it is difficult to understand who is speaking at moments. 

As for the positive aspects on communicating the subject of our presentation, I would say that the second part, the guided tour through the neighbourhood, was really effective. Walking through the 3D reconstruction with Tijm as a guide gave a sort of embodied sense of presence and made the experience really concrete. As in many other fields, in the archaeological and historical sciences, getting your students up-and-close to the material under study is fundamental. Usually, a tour like this would take place in the presence where you’d have to imagine the past situation. Here it was the other way around, providing a quite a powerful experience and a unique perspective: “you’re now standing at what later will be the Waterlooplein square”.

Jitte, are you still going to use the setup for other sessions / purposes? Or do you already have bigger plans for more stuff like this?

“Absolutely. Especially in this lockdown period, we thought it would be an excellent plan to repeat this presentation for colleagues and students on events like online research meetings and social events. This is actually going to take place in February this year, happily making use of the still existing VR setup for Vlooienburg in hubs. We are also developing plans for other setups as well, e.g. we have drone-acquired 3D models of excavation trenches in Italy as well as a the site of Troy in Turkey, which could provide a cool environment for educating students on excavation techniques, important sites and archaeological interpretations. And we can think of so much more. We are keen to explore more systematic incorporation of hubs in education here at ACASA and 4D Research Lab courses”.

Join future meetings

Does this taste like more? Join the XR ERA community to connect with fellows and stay up to date on future content and meetups. Would you like to know more about this specific project? Get in touch with SURF (Paul Melis) or the 4D Research Lab (Jitte Waagen).

Image credits: 4D Research Lab, Mikko Kriek and Tijm Lanjouw