Augmented Reality and Virtual Reality Improve Project Delivery

by Vince Naviello, Virtual Design and Construction Manager & Patrick Sullivan, Virtual Design and Construction Specialist

Before 2020, technologies designed for remote work played an important but mostly supporting role. Project teams used simple video conferencing tools to meet over long distances. Digital twin technologies were evolving, guiding teams towards a deeper understanding of their facilities. Project teams who wanted to squeeze more value from their BIM data were turning towards emerging AR and VR applications. No one doubted the momentum behind these advances, but few could have predicted what came next.

As COVID-19 swept the globe, these digital technologies went from nice-to-have to mission-critical almost overnight. That’s when AR and VR started having their moment—a moment that’s quickly become the new way of working. This article will explore what that “new way” looks like, with a particular focus on:

• How BIM data and digital twin technologies drive immersive AR and VR experiences
• How VR drives efficiencies by immersing users in a virtual environment
• How AR supports smart decision-making by integrating real and digital environments
• The future of VR and AR technologies in lean project delivery

Augmented reality, virtual reality and extended reality explained:

Both augmented reality (AR) and virtual reality (VR) typically rely on wearable technology, like smart glasses, to immerse the user in a digital environment.

They differ in terms of how they integrate that digital environment with the real world. In the case of AR, users experience a hybrid environment in which digital elements are overlaid on their physical surroundings. This experience is sometimes also referred to as mixed reality. In virtual reality, the user is plunged in a fully digital environment, to the exclusion of the real world.

Collectively, these technologies are known as extended reality, or XR.

The concepts

• Augmented reality (AR)
  • Augmented reality visualizes design intent in the real world by overlaying digital elements on the user’s environment. This gives project teams the opportunity to witness exactly how their building model integrates with the existing physical infrastructure.
• Mixed reality (MR)
  • Mixed reality refers to experiences that merge real and digital environments. As a term, it’s often used interchangeably with augmented reality.
• Virtual reality (VR)
  • Virtual reality immerses the user in a fully digital environment based on the building model. This gives project teams and other stakeholders the opportunity to explore, understand and engage with the building model before construction begins.
• Extended reality (XR)
  • Extended reality is an umbrella term that encompasses all technologies designed to combine real and digital elements.
• Building information modeling (BIM)
  • BIM is the process of authoring a digital representation of a physical environment. Modern BIM software replaced traditional paper-based design workflows, giving project teams a collaborative, multi-user digital platform for generating models and sharing real-time information. By feeding BIM data into AR and VR platforms, project teams are able to experience those models in rich and interactive ways.
• Virtual design and construction (VDC)
  • This term refers to the overall process of using shared models and technology to manage and improve the entire lifecycle of a project. VR and AR are now integral components of VDC.

The tools

• Digital twin

  • The digital twin is a dynamic clone of a real-world asset, such as a building. Advanced digital twin technology extends the functionality of the BIM model by dynamically capturing real-time design, construction and operational data through sensors, scanners, AI technologies, and other inputs.

    Digital twins are a hot topic right now, especially as the AEC industry moves towards standardization and embraces tools that promote transparency and collaboration. Using AR and VR technologies to explore the digital twin environment, project teams can generate new insights and make valuable predictions at every stage in a project’s lifecycle. This translates to less rework during project delivery and improved safety and quality once the facility is operational.

• Smart glasses
  • Smart glasses are wearable, hands-free headsets capable of moving digital elements in and out of the user’s view. This technology uses multiple sensors, advanced optics and holographic processing to integrate a digital environment within the real one. At CRB, we use Microsoft’s HoloLens.
• Remote assistance
  • This is a category of software designed to share a real-world view with offsite collaborators. At CRB, we pair Microsoft’s Remote Assist platform with AR technology to create a dynamic, two-way exchange of information that enriches and improves project outcomes without the expense or delay of travel.

The context

• Industry 4.0
  • The first industrial revolution dates back to the late eighteenth century, when the mechanized factory floor displaced traditional craft production. A fourth revolution is underway today, characterized by autonomous systems, big data and analytics, and the flow of information between physical objects (the “Internet of Things,” or IoT). Extended reality is a key component of Industry 4.0.
• Pharma 4.0
  • All industries must embrace the efficiencies of extended reality to compete on the 21st-century stage, but the highly regulated nature of the pharmaceutical industry makes these cutting-edge digital tools especially vital for pharma manufacturers. That’s what Pharma 4.0 is all about: leveraging the full potential of digitalization to establish the smart factory, where teams work more closely together, information flows more easily from one function to another, and innovation happens more quickly.
• ONEsolution™
  • This is CRB’s approach to lean and integrated project delivery, and it’s the driving force behind our rollout of next-generation collaborative tools like augmented reality and remote assistance.

From BIM to XR: how teams build interactive, immersive digital models

The digital environment that fuels AR and VR experiences is a replica of the project team’s native model, which is co-authored inside their BIM software. In order to bridge that BIM data with their extended reality platforms, project teams typically follow a cyclical four-step process:

1. Build: The engineering team uses BIM software, like AutodeskⓇ RevitⓇ, to collaboratively author the building model. This generative process often involves reality capture tools, which turn real-world data into a digital model, giving project teams an accurate canvas on which to build their design.
2. Clone: They replicate that native design and push it to the VR or AR platform. Now there are two standalone models: the one built in the BIM software, and the digital twin, available as a virtual “view only” environment.
3. Explore: In the case of a VR experience, team members use a headset and hand-held controllers to move through the virtual environment. Those using AR technology see their physical surroundings, overlaid with digital elements from the building model. By navigating and interrogating these environments, team members can uncover hidden design issues.
4. Improve: The team returns to the BIM model to address those issues. The process repeats from step one.

This process allows team members to identify challenges further upstream in the project delivery lifecycle, before construction begins, thereby reducing costly rework and improving project coordination.

The pathway to these positive outcomes is shrinking as AR and VR platforms evolve. Cloud-based digital twin technologies, which capture and visualize not only dynamic modeling data but also real-time data from the field, are helping to cut out the interim steps between the BIM model and the extended reality environment. This means that project teams will “experience” the same building model, whether they’re on desktop software or using AR and VR hardware, further enabling cross-functional communication and proactive design and construction workstreams.

Augmented reality: a closer look

Anyone who remembers the Pokémon Go phenomenon of a few years ago has some idea of the basic AR experience. Through a smart device, users are able to see animated characters “interacting” with the real world around them.

In the context of project delivery, AR is much more than entertainment. It can save lives, as in cases where project teams use smart glasses to avoid underground utility strikes or other hidden dangers. It’s also an absolutely essential tool in supporting remote collaboration and inspection, giving project owners and AEC teams the opportunity to “be there” without ever leaving the boardroom.

In the past, project teams had to go on-site to accurately assess construction progress or to undertake complex inspections. AR technologies flip that arrangement—they bring the site to the project team. That way, when relocating isn’t possible, the team can still move forward with confidence.

All it takes is one person on-site with a smart device or pair of smart glasses capable of supporting remote AR connectivity. Using a conferencing platform like Remote Assist, the on-site operator can project everything they see to a team in another location, creating a witnessing environment that supports remote inspections and guided navigation.

So far, this may sound a lot like traditional video conferencing. Where AR adds value is in its ability to overlay digital elements and pull documentation into the operator’s view. Remote viewers also see those digital elements, and are able to interact with them or add their own, as needed.

Say, for example, the on-site operator is facilitating a virtual factory acceptance test (FAT), and one of the remote engineers wants a close-up view of a particular feature of the equipment. From the screen in their remote office, that engineer can overlay an arrow on the real-world environment, pointing the on-site operator to the exact location she’d like to inspect. It saves time, improves accuracy, and builds transparency into remote auditing and testing scenarios, keeping teams moving forward even when moving around the world isn’t possible.

It’s important that we balance the need to adopt these capabilities quickly with the need for due diligence. Hardhats, for example, were not designed to accommodate bulky headset rigs. Fastening smart glasses to a hardhat could invalidate the hat’s warranty; in worst-case scenarios, it could put a worker’s safety at risk.

As this technology matures, its supporting equipment will evolve as well. Already, we’re seeing a new generation of smaller, less intrusive hardware emerge. The Trimble XR10 with HoloLens 2, designed to incorporate a hardhat, is a good example.

Augmented reality in action

In the early days of the pandemic, CRB saved this client from costly delays.

A CRB client located in the U.S. was at a critical stage in assessing and selecting equipment from a European vendor when the pandemic eliminated the option of in-person FATs.

At first, the equipment vendor tried to solve this challenge with pre-recorded video, captured by an operator holding a phone. But our client couldn’t be certain of when those recordings were made, or how accurate they were. On top of that, the quality of the video was affected by an operator who had to hold their phone in one hand and conduct tests on the factory floor with the other. It wasn’t a viable solution.

That’s when our client asked CRB for help. We shipped an order of Microsoft’s HoloLens smart glasses to the equipment vendor in Europe and walked them through the set-up. Within 90 minutes of receiving the shipment, the vendor was able to provide our client with a real-time, interactive witnessing environment. The project’s schedule was saved from disaster, and our client continues to rely on AR technology as a component of their smart factory operation.

What’s happening today in augmented reality?

Deeper, richer, more interactive relationships between digital and real features are helping project teams accelerate decision-making in the field.

New AR technologies are capable of pushing rich data into the real world, not simply as overlaid elements (like interactive menus or documents) but as integrated components that “read” the live physical environment and respond to it accurately.

A construction engineer could wear a headset to visualize exactly how future piping will route through the real environment, for example. This opens up enormous potential for improving and accelerating the installation of complex MEP systems—and that’s just one example of how advanced AR applies across all design and construction activities.

The ROI of AR

Conduct virtual FATs and other site-specific assessments: When it’s not possible or efficient to conduct services like factory acceptance tests on-site, project teams can use AR to create a witnessing environment that’s accurate, interactive, and enriched with digital features that help teams navigate and assess the remote environment.

Improve safety on the jobsite: On-site teams can use AR to identify hidden hazards, such as underground utilities, avoiding a potential strike. Meanwhile, remote teams can use it to inspect and approve new and ongoing installations, or to assess the safety implications of an on-site incident immediately after it happens.

Update stakeholders on progress: AR technologies give all project stakeholders meaningful exposure to the project early on. The owner, engineers, construction managers, and other team members can more easily see how the project’s value comes together when they’re able to witness its progress on demand, without having to be on-site.

Mentor new workers on the job, in real time: Factory managers can deploy AR as a virtual training assistant for their new workforce. The trainee’s classroom is the real-world production line, and their teacher is the AR system, which provides precise digital guidance directly in front of the trainee’s eyes. Rather than sweating out scenarios on their own, this gives trainees the advantage of knowing exactly what to do, when to do it, and how to do it, efficiently and without error.

Virtual reality: a closer look

The idea of immersing users in simulated worlds first captured the imagination of computer engineers in the 1980s and 1990s, but early consumer technology wasn’t powerful enough to push VR into the mainstream.

Fast forward to today’s digitized world, and VR technology is everywhere. Gamers, surgeons, military strategists, astronauts in training—you’ll find VR in a huge variety of industries, giving users the ability to explore, experience and examine their world in low-risk, highly immersive ways. This is a game-changer in the context of project delivery and construction.

Using the headset and hand sensors, team members are able to explore a digital twin of the model. Inside the digital twin, operators can proactively identify design clashes, solve costly issues, and clear the path for expedited construction.

The ability to experience the model before construction begins is the next best thing to looking into the future. Teams can eliminate the expensive surprises that too often hide in static drawings, which means less rework and fewer delays in downstream delivery. It also supports lean PPMOF strategies (prefabrication, preassembly, modularization and offsite fabrication) by helping project teams identify suitable elements and accurately scope the cost and production schedule of modular, offsite fabrication.

This approach to VR introduces challenges, though. Early VR simulations typically supported just one user at a time, for example, making collaboration difficult inside the virtual model. Then there’s the challenge of passing information between the BIM software, where the project team authors the model, and the VR software, where they explore a clone of the model. Updates in the BIM model are not necessarily reflected in the VR model without a manual step in between, which can stymie a team’s ability to accurately capture and resolve issues. In other words, it’s possible for the simulated environment to backslide from a digital twin to something more like a digital “second cousin.”

To solve these issues, our expert team continuously assesses advanced virtual reality tools, such as the Resolve VR app for BIM. We strategically leverage these technologies to provide our clients with solutions that push the digital twin to the cloud.

What’s happening today in virtual reality?

Rather than approaching the VR model as a view-only environment, we’re using the cloud to integrate it as a live, real-time platform.

Earlier in this article, we outlined the cyclical steps traditionally involved in replicating the native model and pushing it to the extended reality platform, then routing user feedback back to the BIM software.

Advanced VR environments, powered by a digital twin that dynamically integrates real-time data, have eliminated those “middle steps”. Whether they’re inside in the simulation or on their desktop, all users interact with the same model, at the same time. If a VR user adds a markup inside the simulation, the platform will push their “live” annotations to a shared issue tracker, and it will appear in real-time inside the BIM model.

This means that VR is shifting from an observational device to an interactive design platform—an essential part of the AEC team’s collaborative toolkit. The result is an unbroken loop of feedback and continuous improvement, which drives immense efficiencies across the design and construction lifecycle.

The ROI of VR

Train your workforce: Use VR simulations to offer your staff hands-on, multi-user training and task rehearsals in a facility that hasn’t yet been built, or to safely onboard new workers without the expense and delay of a full-system shutdown.

Deepen stakeholder engagement: Invite owners and other project stakeholders to experience their future facility in a dynamic and realistic environment.

Expedite project delivery: Accelerate design, coordination and construction activities by inviting construction experts inside the dynamic simulation long before breaking ground, giving them an opportunity to provide live feedback and prepare for future project phases.

The next leap forward for AR and VR: harmonized across all platforms

The momentum behind AR and VR technologies isn’t slowing down. As more project teams adopt these platforms to drive lean delivery, we’re going to see huge advances in how AR and VR deliver value across the whole design, construction and operations value chain. In particular, the gap between AR and VR will close, which in turn will open whole new opportunities for visualizing design intent and proactively mitigating construction risks.

Right now, AR and VR experiences rely on different hardware, and the model that users experience on one platform is not necessarily the same model they experience on the other—and both may be different from the native BIM model. Teams lose efficiency when they have to juggle these discrete modeling environments and “translate” from one to the other. In the future, this will no longer be a problem. One type of headset will provide a gateway to both types of extended reality, and all team members will interact with the same live and up-to-date digital twin, whether they’re using AR, VR, their desktop software, or another application.

This also opens the door to richer multi-user experiences. One team member might be on-site in an AR environment while a remote partner might use VR to “join in.” They’ll be able to see one another, interrogate spatial data together, and co-author design changes—all without being in the same physical room.

Are you ready to take the leap forward in project delivery?

Reach out to discuss how our team can leverage virtual reality and augmented reality to eliminate traditional bottlenecks, improve remote collaboration, and ultimately accelerate your design and construction project in ways that weren’t possible until today.