On the Fly Deep Learning Based Video Search Engine

Abstract

Retrieval of information based on the content is a complicated process and requires a combination of different techniques to produce accurate results. The increase in computational power and the ability to process large amounts of data has enabled the use of deep learning algorithms to provide solutions to difficult problems. Nowadays deep learning algorithms are being used in content-based retrieval systems but there are currently no systems that allow real-time learning. Most deep learning based content retrieval systems are limited by their inability to adapt to new classes of data that were not originally a part of the model training process. This can be a hindrance when developing a system for retrieving specific person(s) from the data. It is not possible to train a model with a class for each individual person. This project aims to provide a solution to this problem by using a relatively new concept of deep learning known as "On the fly deep learning". In this approach the system is divided into two parts: Online processing and Offline processing. The offline processing deals with preparing the data, detecting the required person(s), extracting feature vectors, and storing the feature vectors. Similarly, the online processing involves fetching positive examples at run-time from the internet based on the user query, detecting the person(s) in the examples, extracting feature vectors, and then comparing the features with stored features using a ranking algorithm. This approach yields relatively fast and accurate results. The approach is motivated by human cognition and is similar to how humans learn to recognize people. Humans make connection between the features of a person's face and certain attributes that distinguish them from others. For example: If a face exists in the human memory and a person wants to recognize someone based on that face, they would compare the attributes of both faces and discern their similarity