The consumer Oculus Rift VR headset is now open for pre-orders - but what serious uses might it have in aerospace? (Oculus)
Augmented reality, 3D virtual reality and more – TIM ROBINSON looks at how the rise of ‘wearable’ technology is set to transform aerospace.
The Oculus Rift virtual reality headset, launched earlier this month may be the latest must-have consumer IT gadget – but this sort of wearable technology has a longer history than you might think in aviation and aerospace. Indeed, the humble wristwatch was partly invented in the first place because pilots needed to know the time, and couldn’t take their hands off the controls to fiddle with a standard pocketwatch. Watchmaker Louis Cartier reputedly made the first pilots’ wristwatch for the pilot Alberto Santos-Dumont in 1904.
'Augmented reality' has existed for decades in aerospace - with fighter and attack helicopter pilots having symbology projected onto their helmet visors (BAE Systems)
Other ‘wearable’ technology pioneered by the aviation sector included oxygen masks, radio headphones and helmets, G-suits and electrically heated clothing – all necessities developed as aircraft flew faster and higher. An astronaut’s space suit, perhaps might be thought of as the ultimate ‘wearable’ – effectively protecting a human in their own personal mini-spaceship from the most hostile environment possible.
But today, it is the consumer IT and electronics sector which dwarfs the specialist avionics industry which is driving change and allowing possibilities through augmented reality and 3D virtual reality. Lets take a look at the most exciting areas.
Thales has launched a wearable lightweight HMD aimed at airline pilots (Thales)
‘Augmented reality’ where the outside world is overlaid with useful information is, of course, not new to aerospace. The first helmet-mounted displays evolved in the 1970s and 80s, and were used by attack helicopter and fighter pilots to provide pilots with enhanced situational awareness so they could keep their eyes outside the cockpit. These HMDs, now in wide military use today, projected useful information, such as speed, altitude and targeting data. However, while the latest helmets, as used by the F-35, allow the pilot to ‘look through’ the structure of the aircraft thanks to all-round 360deg sensors, they are still highly expensive ($400,000 each) for civil use). However, thanks to consumer technology and building on experience from military systems, some companies, such as Thales and Elbit Systems, are now pitching lower-cost versions at the commercial market – to give airline pilots increased situational awareness. These dispense with the helmet – but still would allow pilots to focus on the world outside the flight deck, while delivering essential information to them.
Could the next generation of augmented reality wearables provide low-cost situational awareness for GA pilots? (Aero Glass)
GA pilots, too, are not being left out. While Google Glass has not taken off in the same way yet as the smart phone, it did make some think about innovative ways to use these low-cost AR wearables in aviation. For example, the ‘Aero Glass’ app was designed with GA pilots in mind – taking data from tablet computers or GPS systems to put flight data and maps, in the corner of the pilots eye. Expect to see more of these as the technology develops further.
Virgin Atlantic trialed Google Glagg to enhance the passenger check-in experience (Virgin Atlantic)
It is not just in the sky either. Prior to the launch of Apple’s iWatch, a number of airlines rushed out passenger ‘apps’ so that their customers could check in, select seats and be kept-up-to date on the status of their flight. Meanwhile, in 2014, Virgin Atlantic conducted a successful trial with Google Glass – with check-in staff using the eyewear to improve customer service.
Escape into virtual reality on long-haul flights? (Qantas)
Finally, wearables may make long-haul flights more comfortable and allow passengers to seal themselves off, using virtual reality to watch films, check out their destination or play games. Qantas, for example has already issued VR headsets for their first-class passengers on selected A380 routes.
Low-cost immersive simulation
Could consumer virtual reality headsets deliver low-cost mission rehearsal, tactics training or for example, landing in confined spaces simulation?
Virtual reality, of course,comes from the video gaming industry and has been promised before – but never lived up to expectations. Today, however, consumer IT produces ever more high-resolution displays (for smart phones) smaller chips and at far lower cost. The Oculus Rift VR headset, for example, is £500 to pre-order – a fraction of the cost of previous-military virtual reality systems. The Oculus, and other consumer VR systems like the HTC Vive could open up new, lower-cost simulation and training concepts – for example in training helicopter pilots, where depth perception and cueing at low-level are important. Or it could allow airline pilots to practise unfamiliar airports in the comfort of their hotel room. With HMD for fighter pilots already featuring a version of augmented reality – the next step may be to ‘inject’ virtual enemy aircraft or friendly aircraft into their visors – allowing them to train against artificial intelligence or pilots in simulators on the ground. This live-virtual construct (LVC) training is expected to get more important as militaries focus on saving scarce money.
Design and manufacturing
Airbus has been pioneering augmented reality goggles to help install seating tracks on A330s in the factory. (Airbus)
Wearables are also invading aerospace design offices and factories. Many aerospace companies already feature some sort of 3D ‘virtual reality’ immersive tool and this, using wearables, can only be expected to increase. Low-cost VR means that suppliers around the world can explore and collaborate on the same CAD model – solving problems even though they might be thousands of miles apart. In manufacturing, too, wearables are expected to provide benefits to the workforce. Airbus, for example, foresees a ‘future factory’ where a supervisor could see 3D plans and schematics overlaid to on the production process, or workers could call up key tasks without having to drop tools. Indeed, Airbus is already pioneering this technology by using AR goggles to help workers fit seating tracks precisely on A330 during assembly.
Finally, there may also be applications in the MRO or maintenance sector. There too, wearable technology can allow greater efficiency and fewer mistakes – by connecting the worker to the Internet at all times. Honeywell, for example, has been using voice recognition software for its technicians when stripping down its turbine engines and APUs for overhaul – allowing them to enter the engine data hands-free.
Research and development
A ground-breaking UK study will use wearable 'eye-tracking' technology in imporove helicopter safety. (HeliOffshore)
New wearable technology may also have important applications in R&D/. 'Eye-tracking' glasses - which monitor what a person is actually looking at, are now light and cheap enough to be used to monitor passenger experiences and can be used to optimise cabin layouts, IFE designs, emergency exits and so on, by providing raw unfiltered data of what humans look at.
In addition, there are also applications for flight safety research using this technology. Indeed the global offshore helicopter body, HeliOffshore, is to use a eye-tracking research, along with an Airbus Helicopters H225 simulator to enhance pilot performance and safety in the cockpit. The results of this ground breaking rotar-wing study will be available in May 2016.
Thought control and beyond
"You must think ... in Russian" Will Firefox's vision of a thought-controlled HMI come true one day?
Yet this wearable tech could just be the tip of the iceberg for the human-machine interface and aerospace. One idea, for example, is to use wearable technology to monitor a pilot’s cognitive state or wakefulness – especially important for long-haul flights where pilot monitoring is a growing issue. Using sensors, a headset or perhaps the seat itself could detect if the pilot was ‘zoned’ out and alert them to new information (such as weather or traffic) that needed attention.
Even more science fiction is the thought that perhaps one day they may be direct neural interfaces to control aircraft or UAVs – allowing a pilot or operator to ‘fly’ the aircraft just by thinking about it. Already scientists and engineers have used ‘mind’ controllers, which ‘read’ electrical brainwaves stimulus and convert them to commands, to control UAVs. This may start off as a gimmick or as a consumer device to 'relax' passengers scared of flying – but could also one day have applications in allowing the disabled to experience the joy of flight. Indeed, in March 2015 it was reported that in a DARPA trial a paralysed woman had ‘flown’ an F-35 in a simulator. Honeywell, too has been trialling 'mind-control' neural sensing HMIs - both in a 737 simulator, and according to a report In Aviation International News, in a King Air in 2015. Ten test flights were conducted, with flashing lights on a control grid in the cockopit demonstrating the capability to translate pilot's thoughts in recognising the patterns into control inputs. These are obviously still only very early days - but could flying aircraft be as simple one day as thinking 'up'?
In short - wearables promise to connect us to the sky in ways we can only begin to imagine.