Mixed Reality for Architecture, Engineering, and Construction

September 13, 2017

What is Microsoft HoloLens?

Microsoft HoloLens is a wearable, self-contained holographic computer. The device features a see-through, holographic display and advanced sensors that map the physical environment. HoloLens enables users to interact with 3D holograms blended into the real world. In this Mixed Reality environment, users can pin holograms to physical objects and interact with data using GGV (gesture, gaze and voice) commands.

What is Mixed Reality?

Mixed Reality spans the purely virtual and purely real environments. In the context of the building industry and BIM modeling, this is the phase in which digital and real content co-exist, where architectural design collides with reality, and where construction teams transform digital content into physical objects. It helps users efficiently interpret physical and digital information, and the spatial relations between them.

Reality - Virtuality Continuum, Paul Milgram (1)

Reality - Virtuality Continuum, Paul Milgram (1)


In today’s practice, the interpretation of digital content and its translation to real-world objects heavily depend on the user’s spatial understanding. This is an error-prone process and demands a highly skilled workforce. Interpretation errors are common during the design and construction stages, and often result in poor quality, cost overruns, and schedule delays.

Visualizing digital content as holograms in the context of the physical world bridges the gap between virtual and real, and eliminates inefficiencies in the current workflow. In addition, while our physical world is finite, Mixed Reality presents the opportunity for an infinite environment in which additional data such as schedule, specs, and simulation can be overlaid onto the world, creating a hyper-reality environment.

Making 3D Visualization a Reality

The last few years’ transition from 2D documents to 3D BIM modeling was a natural evolution, which improved team communication and coordination. 3D models are common today, but still, interacting with volumetric data behind a 2D screen is extremely limited. Mixed Reality and holographic technology bring the models out of the screen, allowing users to engage and interact with design data more intuitively.

Also, unleashing the 3D model democratizes the data by offering a natural way to experience and understand the design. While years of education and practice might train architects to visualize their designs in 3D, other stakeholders have a hard time deciphering them. Using Mixed Reality, stakeholders can walk around and explore the design in real 3D without the need for an expert to guide them and dictate their point of view.

Mixed Reality moves along the Reality - Virtuality Continuum, making the design-build-operate workflow far more efficient. For some decisions or specific tasks, a real “unmodeled” world is required as visual input. For other tasks, a mixed reality – a “world partially modeled” – or even a completely immersive Virtual Reality mode is ideal. With Mixed Reality the user controls the “mix” and adjusts it according to what their task requires.

Moving along the Reality - Virtuality Continuum: in this proof of concept, a Trimble SketchUp model is displayed using Microsoft HoloLens. To best explore and experience the design, stakeholders can switch between a tabletop holographic view and an immersive view mode.

Mixed Reality and holographic technology also improve remote collaboration. The ability to share 3D holograms with remote stakeholders, including participants’ position and gaze, improves communication and collaboration efficiency.

Remote collaboration: in this proof of concept, Trimble Connect data is shared with remote users. Using Microsoft HoloLens’ holographic technology, the remote teams experience the ‘spatial presence’ of the design model, share views, position, and gaze information, and communicate over real 3D holograms.

Facilitating Seamless Translation

Every construction project involves design being transformed into reality, and abstract ideas being translated into physical objects. Since every project is unique, site-specific and labor-based, intensive and accurate “translation” of the design information is required.

The current translation process is a constant struggle. It’s inherently complex and allows little room for error, since wrong interpretation of data often leads to costly mistakes, quality issues, and rework. Digital 3D models can make the data less abstract, and help clarify the design intent, but a clear boundary between design and execution still exists.

Mixed Reality technology blurs the line. With Mixed Reality, the design data itself is superimposed on the physical environment, minimizing the need for translation and reducing subjective interpretation. The user visualizes the design in context, and can better understand, interact with, and execute the required actions in real time. The immediate visual feedback makes the production control process more effective and reveals any deviation in real time.

Seamless Translation: a holographic 3D BIM model from Trimble Connect is visualized on-site using Microsoft HoloLens.

How Mixed Reality Supports Integration and Collaboration

Today’s AEC industry needs to improve communication and facilitate integration of business workflows. Serious inefficiencies are still present in the silo structure and the need to re-create data when moving from one stage to the other. Project delivery methods like IPD and technology changes like 3D BIM modeling services support this trend, improving project quality, cost and schedule.

Mixed Reality technology supports integration and collaboration along three main axes:

The Project Stage Axis: Design to Build (and Operate):

Moving along this axis, Mixed Reality brings 3D data to life and puts information in the user’s hands without the need to change or adjust the data format. A sign-off 3D BIM model can be projected in context to guide construction teams on-site, and later, by facility management teams when evaluating required changes or maintenance work.

The Digital to Physical Axis:

Holographic display is not limited to 3D models. In fact, using Microsoft HoloLens, multi-layered datasets can be overlaid as holograms on the real world. A partial list includes 2D documents, energy analysis, light simulation, acoustics, layout data, and equipment metadata. Presenting data in context like that improves communication and leverages confidence in decision-making.

Office to Field Axis: Design to Build, Build to Design

Sharing up-to-date design data with on-site teams is crucial in the dynamic construction environment. Using Mixed Reality technology, construction teams can access geo-located, context-based data, which can be projected and anchored to the physical environment around them. The ability to map the as-build physical environment supports communication from the field to the office and completes the Build to Design – Design to Build loop.

The Future of Mixed Reality in the AEC Industry

Mixed Reality will have a significant impact on the AEC industry over the next few years. The technology addresses some of the industry’s major inefficiencies during the design, construction and operation stages. Integrated with quality construction layout software and using 3D BIM models as a main data source, Mixed Reality improves communication, tightens workflow integration, and enables real time collaboration with remote teams.

While it is possible to describe the benefits of Mixed Reality in words, one must experience it firsthand to appreciate its power and realize how it can transform the AEC industry.



Milgram, Paul; H. Takemura; A. Utsumi; F. Kishino (1994). "Augmented Reality: A class of displays on the reality-virtuality continuum" (pdf).

Previous Article
The Top 5 Features of a Construction Asset Management Software Solution
The Top 5 Features of a Construction Asset Management Software Solution

Tool mismanagement can impact the efficiency and productivity of the business wasting time and money. That’...

Next Article
3D BIM Modeling - Level of Detail and Model Progression Specification
3D BIM Modeling - Level of Detail and Model Progression Specification

3D BIM modeling is becoming more common in today’s construction processes. This article dives into Level of...


Sign Up to Get Tips & Articles Sent Directly To Your Inbox!

Thank you!
Error - something went wrong!