Augmented Reality

Posted on by milliesc

By Arushi Gupta, 12R

THROUGH THE LOOKING GLASS: AUGMENTED REALITY IN SNAPCHATnull

By Arushi Gupta 12R

One of the most exciting features of popular disappearing photo and video sharing service Snapchat, is its incredible ‘Snapchat Lenses’. Lenses are masks, designs and graphics that are digitally superimposed on your face creating an entertaining illusion you can share with your friends. But what is the science behind it? Without our even realising it, Snapchat has introduced us to the fascinating world of Augmented Reality.

In simple terms, Augmented Reality (AR) refers to a combination of real and virtual

(computer-generated) worlds. Given a real subject, captured on video or camera, the technology ‘augments’ (= adds to) that real-world image with extra layers of digital information.

Technology Behind AR

The technology behind AR is very advanced as it is able to deal with new information immediately and superimpose it with the physical real-world environment. To understand better how exactly AR works- there are 4 tasks carried out by the system: Scene capture, Scene Identification, Scene Processing and Visualization of the augmented scene.

Scene Capture:

This can be done by devices such as computers with webcams, mobile phones and tablets with cameras, video cameras and head mounted displays. The quality of the camera and lighting conditions can affect the quality of the scene captured. For example, while using snapchat app on your mobile phone, the phone’s camera is used to capture the image.

Scene Identification:

There are some scene identification techniques which are either Marker-based or Markerless.

Marker based- Markers could be like QR codes (Quick Response code which is a machine readable barcode) which may be placed as part of the scene. For example a QR code could be placed near a picture in a museum . Once the scene is captured which contains the QR code, the code links to a web address, and the device’s browser looks up the appropriate web page with further information, which in this case could be more details about the artist of the picture. These markers are called “added reference points” which have been placed specially as part of a scene so that an AR system can add information about the scene when it is viewed. AR systems can be designed to read all kinds of other markers (or fiducial markers) and barcodes.

Markerless- This type of Scene Identification can be of three types:

  1. Image Recognition: You could point your phone at each picture or exhibit and have some kind of pattern recognition or feature-detection system try to identify it. In a museum, this would take the form of identifying a painting if you point your phone camera at it. In Snapchat, the software recognizes the image of our face by identifying features such as our eyes,nose,mouth etc and is able to apply entertaining masks over them.
  2. Geopositioning: This technique uses GPS (Global Positioning System) of the camera to identify where the image has been captured. For eg. in Snapchat special Geo-Filters get activated depending on where the picture/video is taken by calculating the latitude and longitude of the photographer’s position. If you are taking a picture in New York for instance, with the help of the phone’s GPS the scene identification part of AR will be able to recognise and a Geo-Filter related to New York will be shown over your image.
  3. Hybrid: This technique makes use of a mixture of image recognition and geopositioning.

Scene processing and visualization:

When the scene is captured and identified, the process of mixing real scenes with virtual information begins. Basically, this process superimposes any type of digital information (usually audio and video) onto the real scene. There are 2D and 3D graphical libraries that permit mixing of real and virtual information. For example OpenGL, Java 3D and NokiaCV are popular graphical libraries used in computer application development.

At the end, the system produces the combined image of the real-time object and the augmented information obtained after processing. This Augmented Reality image is then presented to the user.

Other Applications

Besides used for entertainment as in Snapchat, the applications for AR are increasing continuously. In the medical field, AR spectacles can be used by a blind person. The AR spectacles are fixed with a camera and an earphone which gives an audio commentary to the blind person of the scene around him. The popular game “Pokemon Go” is another example of augmented reality where players have to find images of Pokemon through their phone or tablet’s camera by pointing them towards specific locations. Similarly for students, new information would be at our fingertips in a fun and interactive way.

Recent Advances

Recently, Snapchat created Snapchat Glasses which can be worn over the eyes and provides the same features of the app used on glasses instead of a smartphone. Snapchat has also recently acquired an Israeli augmented reality startup Cimagine Media, indicating that the company intends to make great leaps into the field of Augmented Reality.

AR is a constantly evolving technology that will soon be more accessible to us in the future and will become a complementary part of our lives.

Bibliography:

Leave a comment