Augmented Reality Application for Book Detection with Google Colaboratory and SnapML Lens Studio

Abstract

Artificial intelligence is one of the technological breakthroughs in science, especially in the field of computers. Object detection and recognition presents a new challenge on how to make machines detect objects automatically. To detect an object, the previous system will take the classifier of an object and evaluate it at various locations and various scales in the image. Smartphones currently play an important role in everyday life, but the existence of smartphones can be one of the causes of the user's lack of interest in reading books. With this Augmented Reality-based object detector, it is hoped that users can experiment on smartphones with the Snapchat application installed to detect objects in books that are around them and the number of objects will be counted automatically, it is also expected to attract users' desire to read books. Image recognition identification is taken through the phone's camera which is recognized as a lens. Lens AR in the Lens Studio application is supported by the SnapML machine learning algorithm, this machine learning function is to display book information and the number of books detected. This research was conducted using Cloud Computing such as Google Collaboratory which uses the python programming language, the book object dataset is selected from the COCO class, the model is exported to Open Neural Network Exchange (.ONNX) and then imported into the Lens Studio application. Publish lenses in the Snapchat app. Finally, a model test was carried out to detect objects and the number of objects in books that had been made using a Smartphone. Open the Snapchat application, scan the Snapcode generated by Lens Studio, AR lenses can be used in real time. AR Book Detection has been tested by other users on Snapchat with 18,845 Lens Views, 18,737 Lens Plays and 16 Lens Shares.