GesTune – A Musical Magic (the fun part is we didn’t remember to give our project a name at first)
Imagine creating a musical instrument that dances to your body’s rhythm. This project fuses technology, music, and a dash of magic that captures the artistic and graceful poetry of our movements and transforms them into melodious symphonies. The key point of this project is to involve people with special needs to weave their own harmonies through the simple art of gesture and body movement. GesTune will bring a new dimension to musical expression and create a unique and immersive music-playing experience for everyone.
As I mentioned in my previous day’s blog, our initial plan was to use the mystical power of camera vision to detect the fluid motions of our arms and based on that, transmute them into musical notes. Later we replaced camera vision with sensors. So we used a Time of Flight (TOF) sensor to measure the distance from the device. With that using the esp32 board we implemented a BLE connection with our mobile app. With implementation, we created a system that calculates the distance between our body and the sensor and sends a MIDI note through BLE to our mobile app. Using the Musical Instrument Digital Interface (MIDI) app, it can play a specific note depending on the sent MIDI note from the esp32 board.
Zarin T. 