Object Identification & Interaction with a Smartphone Knock
A KAIST research team led by Professor Sung-Ju Lee at the School of Computing have developed a software technology called “knocker” that identifies an object with a simple smartphone knock.
Previous solutions for object identification are largely categorized into two approaches. The first is to rely on computer vision technologies utilizing cameras; however, this solution requires users to take pictures and does not work well in poor lighting situations. The second is to rely on extra hardware such as RFID tags attached to each object for identification; however, extra cost and labor are required for this approach.
The knocker, on the other hand, can identify objects even in dark environments only with a smartphone, without requiring any specialized hardware or using a camera. Knocker utilizes the smartphone built-in sensors such as a microphone, an accelerometer, and a gyroscope to capture a unique set of responses generated when a smartphone knocks an object. Machine learning is used to analyze these responses and classify and identify objects.
The research team evaluated its Knocker technology using 23 everyday objects such as books, laptop computers, water bottles, and bicycles. In noisy environments such as a busy café or on the roadside, it achieved 83％ identification accuracy. In a quiet indoor environment, the accuracy rose to 98％.
The team also suggested a new object interaction paradigm. For example, by knocking on an empty water bottle with a smartphone, a smartphone can automatically order new water bottles from a merchant app. When integrated with IoT devices, knocking on a bed headboard before going to sleep would turn off the lights and set an alarm. The team suggested and implemented 15 use case applications in the paper.
Professor Sung-Ju Lee said “This new technology does not require any specialized sensor or hardware. It simply uses the fusion of built-in sensors on smartphones and takes advantage of the power of machine learning. It’s a software solution that everyday smartphone users could immediately benefit from.” He continued “This technology enables users to conveniently interact with their favorite objects.” The research team also consists of Taesik Gong, a Ph.D. candidate at KAIST, Hyunsung Cho, an MS candidate at KAIST, and Professor Bowon Lee from Inha University. The team recently presented this research at the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2019).