It has long been rumored that Apple's tech gurus are working on smart glasses. The rumors started after it was revealed the company had plans to add augmented reality to its uber-popular iPhone. Indeed, it was later determined that Apple engineers were building a prototype for smart glasses. The details of the company's eyewear patent application have leaked. The document provides somewhat of a sneak peek at Apple's widely anticipated augmented reality glasses.
What the Patent Tells Us About Apple's AR Smart Glasses
The language of the patent explains Apple has developed a means of representing “points of interest” within a view of a realistic environment through a mobile device. Apple's legal team describes this capability in regard to operation on a smartphone as well as a semi-translucent screen connected to a head-mounted display. Examples of the “points of interest” referenced in the patent include objects in the real-world such as skyscrapers and automobiles that are connected to on-screen content. Such content could be video, text, three-dimensional images, audio etc. The user will likely interact with the content through balloons or other indicators that identify points of interest.
Tech Hurdles of Note
The patent goes on to note the interaction with on-screen overlay images could block the operator's view due to space limitations. Rather than touching the screen, the smartphone's camera might prove capable of locating the operator's finger and using it as a substitute for touching the screen. This method will likely prove critically important to functionality on a head-mounted display in which it is impractical to physically touch the screen. The camera will capture an image of the surrounding environment and might also pinpoint the positions of the operator's finger in such images. The image positions obtained through tracking the user's finger might prove to be similar to touching points on a touchscreen.
Head Mounted Display Screen Functionality
The HMD's screen will likely display virtual objects generated by computers the operator views in the form of an overlay upon the real space he observes. In this instance, the operator's eye functions similar to a smartphone's camera. The HMD's camera is tasked with identifying the position of the operator's finger and the position of annotations and balloons.
Perfecting the User Experience Design
Those who have examined the leaked patent application have determined Apple engineers are considering the practical positioning of balloons and annotations. There are potential usability issues if the operator views a building and is required to hold his hand high to point at annotations toward the top of the screen. This would block the operator's view and also prove quite uncomfortable. The suggested remedy is positioning the balloons toward the upper portion of the display and linking them in a visual manner through a line that points to relevant annotation toward the display's bottom. The operator would interact with the points of interest by touching the annotation. This design keeps the operator's arm at a more ergonomically efficient angle. It also prevents the operator's hand and fingers from obstructing his view.