YouTuber KOALAB looks like a mad scientist in his latest video experimenting with Mixed Reality MIDI controllers. Here he combines several different technologies across macOS and iOS. The hardware is also a mix of off-the-shelf gear like Leap Motion, with off-the-fucking-wall gear like the $400 Bridge helmet for iPhone, and even some custom hardware for monitoring hand rotation. All tolled this experiment probably cost a small fortune. So it is unfortunate that the setup suffers from noticeable latency; while flat-out ignoring some of his commands! Still this gets the discchord Stupidtacular seal of approval for sheer inventiveness.
It is a virtual MIDI console that can be used with the same sense as a hard MIDI controller.
I applied for GUGEN 2017.
MR(Mixed Reality): iPhone + Bridge SDK(https://goo.gl/BU6gHJ)
Hand tracking: Leap Motion(https://goo.gl/5k2aV9)
Recognition of hand rotation and tap: a self-made hardware
Data processing of various sensors, MIDI assignment and MIDI output: a self-made Mac application