Artifact is an immersive visitor experience for The Carnegie Museum of Art. Visitors pick their own paths to view what is most meaningful to them, contextualize art with interactive and immersive content specific to each object, and share their own ways of experiencing art with the museum community. 

Social movements like Decolonize this Place are pushing museums to represent a more accurate picture of art history. 

The Museum of Modern Art undertook a massive rehang of its collection, disturbing the old order and hanging a brutal painting of a racial riot next to Picasso’s D’amoiselles D’Avignon.

The Denver Art Museum just opened from a four year renovation, where they placed a new prominence on on fashion, textiles, and nonwestern objects.


Growing movement for art museums to reinvent and recontextualize their collections

Bringing more and more objects to one place will become less relevant, versus how you translate the knowledge, understanding, and complexities of these objects to a wider audience.

Max Hollein, Director
Metropolitan Museum of Art

We’re hoping to spark dialogue. Some may be for it, some may be against it, but the conversation is important.

Alyssa Velazquez, Curatorial Assistant
Carnegie Museum of Art

Museums are now playing an integral role in shaping and developing identities and bringing different community groups together...

Melbourne Forum

If the white cube has never been truly ‘neutral’ – it is intentionally coercive – then it is also not, or is no longer, a space in which the infrastructure and imbalances of the artworld conveniently fade out.

Martin Herbert

How do we start addressing artworks that are displaced from their original place and context?

How do we start addressing artworks that are displaced from their original place and context?


The problem of the White Cube

Problem Space : The 21st Century Museum


An immersive multi-sensory experience for museum visitors. 


MDes/MPS IxD Studio

Carnegie Mellon University

Fall 2021 (5 Weeks)


Jonah Conlin

Matt Muenzer

Yeonjin Park


Strategy / Ideation

Exploratory Research 

Concept Development

UI Wireframe

Video Production



Adobe After Effects

Adobe Premiere Pro

Adobe Illustrator

Rhinoceros 3D

How might we enrich learning at art museums, in a way that is founded on both the diverse histories behind the art and visitors’ individual curiosities?

How might we empower curators to create these learning experiences?

Problem Statement

Ticket Desk


Purchasing Passes

Museum-goers purchase Artifact passes at the ticket desk. Staff direct visitors to the Artifact Experience desk to pick up headsets. 

Forum Gallery


Borrowing Equipment

Visitors show their ticket to borrow a Hololens 2. Staff assist with fitting and calibrating their headset. 

Once set up, visitors go through with a tutorial to learn Artifact's gestural interfaces. 


Forum Gallery


Choosing Your Path

Visitors begin by choosing from: 1. a fully guided tour from the museum's curatorial staff 2. a community tour published by other museum-goers 3. a custom tour generated based on what the visitor wants to see 4. browse the collection and Artifact content independently. 


Atrium / Hallway


Getting To The Gallery

Lightweight augmented visuals guide visitors to galleries, and the pieces on their chosen pathways. 

Gallery Entrance


Discovering Interactions

Virtual indicators adjacent to wall text signal where visitors can interact with Artifact content. 



Interacting With Art

Visitors learn about works by browsing across four categories of information: General information, Artist Background, Process and Craft, Historical Context. Within each, visitors may read up on the pieces, as well as engage and interact with guided content. 



Your Personal Guide

Visitors are guided through galleries, interacting with information and experiences specific to each piece on their tour. 

Experiencing Art



Contributing Your Expertise

When following customized tours or browsing independently, visitors can assemble and submit their own path as tours in an online community. Museum staff moderate Artifact community tours and highlight them in the experience select interface. 

Forum Gallery


Remembering Your Tour

When they wrap up, visitors return their headset to the check-in desk, and receive a memento card showing their path through the museum. A QR code links to a set of digital keepsakes showing the works seen during visitors' experiences. 


The Process

1 week


Exploratory Research

  • Defining the Problem Space

  • Exploring technology in museums

1 week


Concept Development

  • Storyboarding

  • Systems' Diagram

  • Journey Map

1 week


Defining Artifact

  • Visual language

  • AR User Interface

2 weeks


Final Production

  • Shot list

  • Video Storyboarding

  • Visual Assets

  • Filming/Editing

"Pin" feature from Mixed Reality Toolkit

After being given the prompt to improve the user experience of a museum, we wanted to understand the current challenges and future opportunities of a museum experience. We’ve conducted interviews with a curator at the Carnegie Museum of Arts as well as looking at the larger discourse happening around the changing roles of museums in our society.


Our guiding questions to designing a multi-sensory experience at the museum are (1) How might we enrich learning at art museums, in a way that is founded on both the diverse histories behind the art and visitors’ individual curiosities? and (2) How might we empower curators to create these learning experiences?

Interview with CMOA curator

Visits to CMOA galleries and exhibitions 

Our team decided to use HoloLens 2 as the main device of interaction after testing the device ourselves. Its given gestural and auditory capabilities were compatible with what we strived to achieve within our designed experience. 


In defining the types of gestures for interactions within the HoloLens 2, we looked into the contents of Microsoft’s Mixed Reality Toolkit- MRTK asset library for references and found six frequently used gestures to be relevant to our project. Unlike screen based UI, augmented reality exists in three dimensional space - dividing the types of gestures into two categories - Nearby and Far Away gestures. We identified Tap, Scroll, Zoom, Grasp, Move and Open Menu, each with two sets of movement. 


Another important feature we wanted to utilize was persistent AR content, meaning that the spatial recognition capability enables the user to “pin” the UI based on the actual position within the space. These “pinned” content exist within the virtual space even if it leaves the user’s point of view.

Bridging the Gap

Museums can only communicate so much with wall text and the other tools they have. Augmented Reality can provide additional, seamlessly connected channels for what you can learn about art. Opt-in interactivity and user choice afford museum-goers a wealth of new information without overloading them.

Nimble Curation

Augmented reality can empower museums to create new ways of experiencing their collections more nimbly, and reach out to new and changing audiences. In contrast to spending years building a physical exhibition, curators and educators might gain a tool to quickly enrich visitor experiences with a diversity of pathways that can offer all at once. 

Focus on Art

Physical presence is very important to art museums and Augmented Realities do not have to interfere with this connection. Augmented information can range from peripheral to immersive, the amount of cognitive load that the AR content demands.

Responsive Museums

Museums collect rigorous data on attendance, but outside of surveys are limited in what they can learn about visitors beyond that.Augmented Reality can provide a diversity of information about what museum-goers see and how they make their way through galleries. A museum could leverage this data to responsively improve curation of not only virtual, but also real-life content.

Testing of HoloLens 2

Testing AR Interface

From speaking with museum-goers and curators, our industry research, and our own experiences at CMOA, we saw a lot of potential in developing multi-sensory experiences through the use of Augmented Reality to empower both museums and visitors to create informational learning experiences for these following reasons:

Why Augmented Reality in Museums

Defining the Problem Space

Exploratory Research

Workshop for a full user experience scenario

Rough project flow

We shaped the flow of the AR experience through storyboarding from a user’s perspective. Storyboard exercise helped us narrow down how the AR technology can be incorporated into the current museum experience as well as adding what could potentially enhance the existing experience. These are the following factors we focused on:


  1. Meeting various needs: allowing specific learning experiences for academic visitors,  offering curated guides for a first-time visitor, creating diverse guides for repeated visitors, etc

  2. Content-driven Interactivity: giving specificity to the AR interaction based on the artwork’s context

  3. Personalized reflection: offering individuals to reflect upon their experiences through reviewing and creating their own physical and digital keepsake

  4. Sharing your experience: providing an opportunity to share one’s museum experience before departing the museum


Concept Development

Defining Artifact

Visual Language


For legibility of the text in the AR environment, we chose Helvetica Neue, a geometrical typeface.


Adapting the current grayscale colors with one accent color, Artifact’s color-palette was created.


The name “Artifact” was inspired by being able to write and interpret the word in different ways.

Artifact I AR tifact I ART-(effect)


We created logo of Artifact inspired by CMOA’s current visual identity; four angle brackets animating into a cubic symbols which intuitively presents the AR aspect of the project.

Improve the visitor experience at the Carnegie Museum of Art in Pittsburgh. Possible area includes way-finding, information browsing, experiential learning and social interaction. 

Project Objective

AR User Interface

We designed visual information with high contrast colors, clear typography, and simple layout.


Fewer, more obvious interaction targets


High contrast colors, typeface and layout


Concept Video


Forging New Paths

“Giving the audience an option to forge their own path and getting to publish the tour is great!”

Aligned with our intention, she noted that experience selection would be most useful for large scale shows, giving visitors an opportunity to narrow down the field or to pick specific narrative that they want to explore. She also thought that Artifact was effective in addressing specific interest from a visitor and tailoring an experience for different visitor types.


Solving Existing Problems

“I often wished we could use google maps to navigate within the museum.”

Identified as one of the problem areas we can tackle with the augmented reality technology, she found the way-finding feature to be very helpful since it is a constant issue that the museum is trying to deal with.


Potential issues

“People touch artworks a lot more than you’d imagine.”

She mentioned that with any technological devices in museums, cost and maintenance are issues. It requires dedicated time, space, and staff to maintain. Also, she raised a concern around the issue of people touching the artworks and how this might affect existing behaviors.


Future Use Cases

“(Artifact could be for) something that it once was or what it could be in the future.”

As a curator, she spoke about how this tool can be used for “giving you something that is missing”. For instance, she thought Artifact can be used to demonstrate the conservation efforts by comparing the before and after or showing related artworks in storage. It was great to hear her ideating on Artifact’s potential use cases from a curatorial perspective.

We were able to show Artifact to a curator at CMOA and get her feedback. 


Future Works

Rounding Out The Service

The systems designed for Artifact present many opportunities to build a complete service, particularly in how to leverage data on how visitors experience CMOA. We prototyped a video wall for the atrium showing show community tours and visualizing the most popular art in the galleries. Importantly, this screen lets museumgoers without the AR experience connect with Artifact-generated content.


Artifact is a scalable product. Although we designed our solution around specific challenges at CMOA, the product’s framework can be easily applied to revigorate other public art collections.

Building Artifact

We’d love to continue building Artifact by prototyping with AR software and developing in Unity for Hololens. In the long term, CMOA’s relationship with CMU and stature in the art world put it an excellent position to work on this and related future-looking experiences.

Systems' Diagram

Thinking about the larger system behind the technology that enables the experience was a constructive exercise. It allowed us to think about the service at different scales - amount of input and output data, privacy, as well as from the data management perspective of museum staff who will moderate the AR contents.

Journey Map

© 2022 Jamie SunJae Choi