UPDATE: this blog post got picked up by Business Insider
A robot looks around, scans his surroundings, and detects humans. Using real-time visual reports on the humans’ vital and biographical stats, he identifies which ones are enemies and efficiently eliminates the threats.
That is a scene from the movie The Terminator – except that the Augmented Reality (AR) technology featured in it is no longer a Sci-Fi fantasy. Simple AR applications have been around for years: for example, the lines superimposed on NFL broadcasts to show the line of scrimmage and first-down markers. More recently, we’ve seen AR applications that help with real-world navigation (Yelp), natural language translation (WordLens), and online commerce (virtual fitting rooms that let shoppers try on clothes before they buy them).
In order for AR to reach its full potential, several technological advancements are required:
1. Markerless object recognition needs to mature
Most AR applications currently use markers that look like small barcodes, requiring users to either print the markers themselves and attach them to objects, or else limiting users to only interact with objects that already have markers on them. Markerless AR enables applications to recognize objects without markers and thus significantly improves user experience. Microsoft’s Kinect is the best-known and most successful commercial example of markerless AR, and startups like Organic Motion have created very impressive prototypes for creating real-time AR avatars.
2. Image Processing needs to become faster and more accurate
Currently, it is fairly easy for a computer to recognize a human face in a frontal portrait shot and distinguish it from a very different object – say a car or an apple. However, it is much harder for a computer to tell the difference between similar-looking people, especially if they aren’t facing the camera or are wearing accessories. The same difficulties apply to detecting buildings and other natural landmarks. To correctly identify objects, computers need to build a 3D model of the world to understand how objects look from every possible angle – a very complex and resources intensive operation. Some interesting projects (such as Photogrammetry research) have tried to address this problem, but none have yet worked sufficiently well in a real-world tests.
3. Mobile Hardware needs to become more powerful
Truly immersive AR requires computers to be able to replace objects with matching 3D renderings of virtual objects in real-time. Mobile hardware is not powerful enough yet for this but will be soon. For example, the iPhone 4G runs on an A4 Apple processor at approximately 1GHz, similar to the speed of common desktop computers from 7 years ago. Mobile devices are likely to reach the processing power of today’s desktops — which are sufficient for advanced AR applications — in another 2-3 years.
AR is poised to significantly change the way we interact with the world and will be a major driver of technology innovation over the next decade. The Terminator Sci-Fi fantasy is just a few difficult — but tractable — technology advancements away.