MR Navigation System for Library

We developed an MR navigation experience using Apple Vision Pro that allows users to walk through a library, designed to work in conjunction with “Xperigrapher,” a VR experience simulation system developed by the Design Association for Augmented Experience (DAAX) under the Human Augmentation Research Division of the National Institute of Advanced Industrial Science and Technology (AIST).

This project was presented at an event held in November 2025 at Fukuroi City Library in Shizuoka Prefecture. Visitors wore Apple Vision Pro and moved through the library by following navigation cues and audio displayed in the space, encountering books selected for the Christmas season along the way.

By balancing extensibility for research purposes with the quality of an experience that a broad range of visitors could enjoy, MESON provided technical support for a demonstration experiment exploring the “future library experience” enabled by spatial computing technology.

1. Building an “in-app server” that connects with an external system

We designed and developed a proprietary config server structure, or “in-app server,” that enables the Unity application to load and generate configuration data and assets output from DAAX’s independently developed system, “Xperigrapher.” This made it possible to reflect route changes, content placement updates, and resource file replacements without rebuilding the application.

This mechanism expanded the scope of work that designers could handle, enabling closer collaboration with engineers and rapid short-term iteration. The system was implemented within MESON’s “GAUGUIN” package and is reusable as a development asset for both MESON and DAAX.

2. High-precision alignment and occlusion processing in a public space

To resolve positional drift during movement through the wide library space, we placed correction markers at key points and introduced a mechanism that adjusts spatial alignment through a natural UX that does not interrupt the experience. We also implemented occlusion processing that accounts for physical elements in the library, such as pillars and staircases, creating a highly immersive experience in which digital content appears to exist within the real space.

As part of the experience design, snow falls throughout the library, and seasonal 3D elements such as Christmas trees and Santa Claus are placed in various locations. The high-precision spatial alignment and occlusion processing support the sense that these 3D objects are naturally integrated into the physical environment.

3. Spatial UI/UX design that is easy for anyone to understand and enjoy

To ensure that even first-time MR device users could experience the content without confusion, we implemented features such as a start button positioned with gaze guidance in mind and a simple, easy-to-understand tutorial. Including operational aspects such as attendant support, we designed a navigation experience that could be enjoyed safely and intuitively in an environment where general library visitors were moving through the space.

The navigation route was designed to connect books across the library’s classification system under the theme of “Christmas.” At each bookshelf, narration based on introductions selected and written by librarians is played, allowing participants to receive a preview of the book’s contents as they walk toward it. Analysis of behavioral logs confirmed that participants moved along the designed navigation route, suggesting that the spatial UI guidance worked as intended.