MAPPING THE PORT OF OSWEGO
AR Tour Guide Application
OVERVIEW
This project was part of the overall Mapping Oswego Project, an initiative to document in a modern way some of the rich and often under-recognized history of Oswego, NY. Oswego has a deep history, being home to the famous Fort Ontario, Safe Haven Holocaust Refugee Center, several stops on the underground railroad and as a massive hub for maritime trade. This project specifically focused on the history of the Port of Oswego, looking into the West Pierhead Lighthouse, the Oswego Maritime Museum and the Oswego Canal. The mission of this project is to bridge the gap between the city of Oswego and SUNY Oswego students by teaching students more about the city’s rich maritime history. In order to reach this goal, my group decided to create an Augmented Reality (AR) tour guide application that could be used to teach college students in a fun and interactive way about various landmarks associated with Oswego Maritime. We also created a corresponding website and informative flyer.
Context: Hypermedia and Multimedia Design Class Project
Timeline: March - May 2019
My Role: Augmented Reality Developer/Designer, Public Outreach
Team: 6 Members
Charles Domicolo - Historical Information
Zak Gudlin, Brandon Druschel and Omari Fabor - Web Development
Kimberly Hirsch and Daniel Hufnal - AR Development
THE PROCESS
AR DEVELOPMENT
Before beginning the actual development, my partner and I drew out some wireframes for the tour guide. We envisioned an application with a simple interface and wanted the user to be able to simply hold up their camera to a landmark associated with Oswego maritime and have a virtual tour guide pop up and tell the user historical information and interesting facts about the site.
Creating Christine in the Amazon Sumerian Development Environment
STEPPING UP TO
THE PLATE
First, my team and I determined which tool would be best for our AR development. We chose Amazon Sumerian because of its built in host functionality. One of the biggest challenges while completing this project was learning how to work within Amazon Sumerian. In fact, none of my group members, including myself had ever created an AR app prior to this experience. With this said, it was rather intimidating to step up to the plate and volunteer to be part of the AR development team, when I could have stayed within my comfort zone and taken on the web development portion of the project.
INTRODUCING CHRISTINE
I quickly learned how to work within Sumerian and generated a tour guide that I named Christine. I decided to develop Christine as a persona to make her feel more lifelike and to connect her better to the user. We decided to make her female based on studies that discuss how people receive information better when it comes from a female voice, while commands are better followed from a male voice.
BRINGING CHRISTINE TO LIFE
Before integrating in Christine's speech, I worked with a teammate, who was focusing on the historical accuracy of the project, to write a few scripts for Christine. After adding in this speech component, I then integrated gestures to make the user feel more like they were being spoken to by a real human. After Christine’s speech and gestures were complete, I loaded a static background into each scene for a remote viewing experience that would be featured on the website. My partner then created the app home screen and integrated the AR scenes into the app.
CHRISTINE IN THE KITCHEN:
INITIAL TESTING
Initial testing of Christine out of her intended environment revealed some basic issues with eye contact and grounding. This type of testing also better helped me understand how the Sumerian AR hosts behave, such as how they will naturally look for a horizontal line to be grounded on when it becomes difficult to differentiate the ground from other aspects in the environment.
FINITE STATE MACHINE DEVELOPMENT
After enabling Christine to speak and gesture, I created a finite state machine for her to preform more complex actions such as waiting for her to speak until clicked on. At this point, I also enabled Christine to make "eye contact" with the user by having her gaze follow the AR scene's camera.
Creating a basic Finite State Machine for Christine to wait to speak until being clicked on by the user
TESTING
CHRISTINE
We then gave a small group of students instructions on how to download the app and sent them out to these locations to test its performance in the field. The app was generally successful during this test, with the exception of a few issues, such as some minor speech and gesturing problems with the tour guide. We also discovered during this test that the tour guide will only face the direction that the user’s camera is faced when the user clicks on the location selection. We weren’t able to overcome this issue, but simply included a reminder on our website and instructional flyer that they must hold their camera up to whatever direction they want to view before making the selection.
FINAL DELIVERABLES
The application home screen, as shown to the right, displays four different selections, one for each of the three locations that we focused on for the application, and one general selection that introduces their tour guide, Christine, and how to use the application. The user can click on their current location and view the Christine
overlaid with their camera view. We also developed a corresponding website, which contains a remote viewing option (shown below), where Christine is overlaid with a static image, but conveys the same information.
Load screen and sample shot of the remote viewing experience with static background
APPLICATION RELEASE & RECEPTION
I created an informative flyer to give potential users, which contains both detailed information on our project in general and how to download and utilize our AR app. We then released the application to the public and handed out our fliers to various local historians, professors and museum board members. The application received widely positive responses:
“As a person who has immersed themselves in the history of this town and this region and continually shares it as a storyteller, I think the content is very well done. This is an exciting resource!”
- Jonel Langenfeld, Oswego Maritime Museum Board Member
Overall, I believe that the final application has the potential to do exactly what it was intended to, teach students about richly historical sites near their college campus that they may have been previously unaware of. The extensions of this app are limitless. It can be easily modified to work on any college campus that has the same issue that Oswego has been experiencing throughout the years. This app could also be modified to become a historical walking/driving tour of any type. All in all, this app brings history into the modern era and allows users to tour historical landmarks at any time they please.
LEARNINGS
This project taught me a great deal about how to put my own biases aside while conducting user testing. Going into user testing, I thought that the majority of the critique would come from visual components and Christine's display. However, most of the criticism involved user confusion while historical information was being delivered such as the speed of Christine's voice, not being able to pause her speech and conveying too much information in a given chunk. As one of my first experiences conducting user testing, I learned that some of the most persistent issues of a program may just be those that you never thought of before.
The most challenging undertaking that I took on while working on this project was volunteering to develop AR within an environment that I had never worked in before, Amazon Sumerian. For this undertaking, the devil was certainly in the details. Small elements made all the difference, such as adjusting the scene's cameras, Christine's eye contact, modifying each scene's finite state machine, and so much more. Working on this project definitely helped me sharpen my attention to detail and ability to pick up new methods quickly. Because of the strict due date, I also didn't have time to learn every little functionality of Amazon Sumerian, so I had to learn to quickly comb the internet for very specific tutorials.
I went into this project feeling a great deal of imposer syndrome. My group was made up of half graduate students and half undergrad, with myself being in the latter group. I was rather intimidated as is, not to mention that I was the only female in the group as well. With this said, I handled this feeling of imposer syndrome as I always do, by stepping up and proving to not just others, but myself, that I have just as much if not more to contribute as anyone else.