A report released by the Capgemini Research Institute in September, which surveyed more than 700 executives in the automotive, manufacturing and utilities sectors from around the world, has revealed that enterprises are using augmented reality (AR) and virtual reality (VR) technologies to enhance their business operations.
AR superimposes a digital layer on to the physical world – integrating the physical, real environment with virtual details. This is typically experienced using smartphones, tablets, head-mounted displays and ‘smart’ glasses that project a small image on to a screen mounted near the eye. VR, on the other hand, creates an interactive and completely digital environment – a 100% virtual, simulated experience. This is often experienced through the use of a head-mounted device.
The report, pictured right, and available via https://is.gd/requku, reveals that organisations are finding value in immersive technology during design and assembly, training and inspection and quality assurance activities. However, the most popular use of AR and VR implementation is in repair and maintenance. So, how does this technology aid workers in their maintenance activities?
Remote assistance
Nick Boughton, digital lead at leading systems integrator Boulting Technology, explains that immersive technology has the potential to support multi-location teams that have equipment or service experts who may not be easily accessible, to help support maintenance processes, particularly in emergency or unplanned circumstances.
AR opens up the potential for off-site staff to access the same view as the maintenance engineer on site. “Linking up workers through voice calls would further ease this process, as the experts can communicate the required actions and be able to see and guide the on-site worker in a real-time view,” he explains.
“It opens the opportunity of true on-the-job learning, as well as increasing the opportunity of skills and knowledge sharing across multiple sites, or even nations.”
American-based energy corporation Chevron is one company taking advantage of remote assistance by deploying Microsoft Dynamics 365 Remote Assist and Microsoft HoloLens (pictured, far right) within its global facilities, including oil refineries, lubricants plants, and off-shore locations (video available at https://is.gd/cinawu).
The technology is revolutionising the first line worker experience and generating a measurable impact on Chevron’s bottom line, improving performance and speed of global collaboration, according to the company. Ed Moore, senior technology strategist at Chevron, explains: “Chevron uses Dynamics 365 Remote Assist for two core scenarios. The first one being ‘remote expert’, where we put any expert anywhere in the world in under a minute. The other one being remote inspection, where we allow our inspectors to inspect the construction and the operations of our facility in real time.”
So far, Chevron has acquired more than 100 HoloLenses, with plans to buy thousands in the future. Moore continues: “We have facilities all over the globe [and] we have facilities in areas that we can’t get to all of the time. Being able to digitally transform that expert into the field within a minute is foundational and just transformative.”
As well as connecting off-site staff, Boughton believes that there could be a rise in off-site suppliers supporting maintenance engineers through remote assistance technology. “Rather than a simple phone call, the AR system will allow suppliers to show their customers exactly how to service or replace a part,” he says. “By providing them access to relevant maintenance data, the supplier can suggest the best course of action, which may include sending one of their own engineers out to assist.”
Drive and control specialist Bosch Rexroth is one company making this a reality. It has introduced ‘Hägglunds Inside Intelligence’, a connectivity suite that provides customers with instant analysis, real-time advice and condition monitoring of Hägglunds hydraulic drive systems (https://is.gd/fosege).
Included in this is ‘Hägglunds InSight Live’, a diagnostic tool that uses AR to provide Bosch Rexroth service support remotely. Through a mobile device or a pair of video goggles, central service experts literally see what the onsite team sees. In real time, they can provide instruction and visually guide the customer through diagnosis and solution.
Visualisation
Another use for immersive technology in maintenance and repair activities, according to Capgemini, is through the visualisation of specific components and functions behind obstacles. In fact, 30% of respondents who use AR and VR told Capgemini that they are using immersive technology for this very purpose.
The report gives, as an example, fieldworkers at Toms River Municipal Utilities Authority, a New Jersey, USA utility company. They are using VR and AR to see concealed utilities such as water, gas, electric, and sanitary and storm water sewer conduit.
Industrial wearable computing and AR solutions provider Ubimax, and AR company DAQRI are also exploring this (https://is.gd/izikuq). Working with German car manufacturer Daimler, in Bremen, it carried out a proof of concept to successfully demonstrate how AR applications improve the internal inspection process of oil filter systems. The solution allows the visualisation of various inspection workflows, while providing automated data logging. A built-in thermal camera also speeds up the inspection processes.
Boughton adds that by using a smart phone, a pair of smart glasses or a headset, maintenance engineers can also create a digital representation of a fault or potential fault, and the solution.
“Using highly sophisticated software, the engineer can pull up a full set of diagnostics of equipment featuring real-time data, and the action that needs to be taken,” he says. “VR builds on this even further by placing the engineer in a fully-immersive digital environment. With the headset on, the engineer can speed up time, using data to visually pinpoint exactly when the system may fail. This level of maintenance allows repairs to be predicted and planned for, ensuring the correct replacement part is ordered well in advance, minimising production downtime.”
Instructions & references
Capgemini’s report also shows that immersive technology is being utilised by 29% of respondents to superimpose step-by-step instructions on their field of vision, and by 31% to view reference videos and digital manuals.
These work by the operator aiming a tablet’s camera or headset device at the equipment that needs to worked on, explains Patrice Duboe, chief technology officer for digital engineering services at Capgemini. The system will then superimpose a simplified digital image of the equipment, which will allow the operator to identify what components require attention, along with information on their status to reduce risks.
“The system can then guide the intervention step-by-step, potentially via animations of the superimposed 3D representation of the equipment, with each step acknowledged by the operator, potentially recording parameters along the way,” says Duboe.
This also means that workers wouldn’t need to rely on potentially out-of-date instructions, as documents can be automatically updated when new versions become available, Boughton adds.
At the start of the year, Boeing revealed that it is testing AR as a possible way to give technicians real-time, hands-free, interactive 3D wiring diagrams. This allows technicians to roam, while seeing where electrical wiring goes in the aircraft fuselage (video available at https://is.gd/uyaqus).
Although not maintenance-related, DHL Supply Chain, the contract logistics specialist within Deutsche Post DHL Group, is also using a ‘vision picking’ application with smart glasses from Ubimax (https://is.gd/zabiji). The application provides graphical displays for single and multi-order picking instructions information on the item’s shelf and picking cart location, leaving the picker’s hands free of paper-based instructions.
Maintenance training
The benefits that remote assistance and visual instructions can provide go even further when they are used for maintenance training.
Last year, the author tried the Microsoft HoloLens at Microsoft’s London HQ. During the demo, the HoloLens was used as a training device to fix a ‘faulty’ electric box, whereby a professional gave step-by-step instructions on a fix via remote assistance. The HoloLens was also used in another demo to look at a holographic human body that could be pulled apart and studied. Although a medical example, the technology could be equally applied to teaching technicians about industrial machinery maintenance.
Boughton adds that budding engineers can use immersive technologies to put theory into practice without impacting the day-to-day running of the plant. VR, he adds, can also be incorporated into the training room to enable apprentices and new engineers to explore the plant environment digitally.
Duboe echoes this: “VR is used for on-the-(virtual) job training. Benefits are huge in avoiding travel costs and reducing the carbon footprint. Training can be delivered anywhere without needing dedicated physical assets, especially adding value for industrial assets such as plant, aircraft, and satellites.”
As well as training, immersive technology can also be used to get young people interested in engineering. Lambert, a provider of automation systems and components, did just this in October when it invited more than 200 primary and secondary school pupils and college and university students to showcase the life of a modern-day engineer. The ‘Future of Engineering’ event was held as part of the inaugural Leeds Manufacturing Festival and allowed visitors to experience different technologies, including immersive technology.
A bright future
Duboe says that the benefits that immersive technology can bring to maintenance activities range from increased reliability and equipment uptime to decreased maintenance costs. But, the “drawback” is that this requires the availability and maintenance of high-fidelity reference data and graphical representation of key assets.
“The graphical representation, in particular, requires a 3D design of the asset, which often does not exist for brownfield plants. For all other data, any disparity will impact user experience, and might get in the way of acceptance. But artificial intelligence will remove these concerns by providing higher performance and intelligence within these remote AR assets. Additionally, operator acceptance requires a very intuitive design of the solution.”
Boughton concludes: “While the future looks bright for the use of VR and AR in engineering, there are still challenges that lie ahead, which are mainly due to the complex nature of engineering systems and models.
“That said, both technologies are already making an enormous impact on the sector, as they allow access to situations that would otherwise be difficult to visualise or might typically be inaccessible.”