Abstract:
This project is about incorporating gesture and speech based input with an IoT device
to increase ease of access to similar devices. This report elaborates on the techniques
used to recognise speech and gesture based input and then using them to control an
IoT based device.
To demonstrate this a robotic vehicle built with Raspberry Pi 3 is being used.
The device is designed to be remotely controlled through a laptop over Wi-Fi using
gesture or speech based input. For this 3 main components had to be accomplished:
Firstly, the Input Application (The one running on the laptop); that reads and
processes the gesture and speech based input before transmitting it to the device, this
section has been implemented in C#.
Secondly, the Device Application (the one running on Raspberry Pi), that
utilises the incoming commands to dictate the robotic cars movements, this part of the
code is implemented in Python.
Thirdly, the hardware; that is the robotic vehicle made using Raspberry Pi 3,
DC motors, motor drivers, chassis, wheels and other components. It also utilises a
sonar sensor for obstacle detection during forward motion.
This system is designed to provide multiple modes of interaction for a device,
ensuring the availability of the same functionality regardless of the chosen interaction
mode. Recommendations for future development and conclusions are also included in
this report.
