« gattobus - The Wind Forest (Analog Physical Model CELLO) | Gear News | VJ Franz K - Stranger Beats »

MRMidi - Mixed Reality MIDI Experiments

YouTuber KOALAB looks like a mad scientist in his latest video experimenting with Mixed Reality MIDI controllers. Here he combines several different technologies across macOS and iOS. The hardware is also a mix of off-the-shelf gear like Leap Motion, with off-the-fucking-wall gear like the $400 Bridge helmet for iPhone, and even some custom hardware for monitoring hand rotation. All tolled this experiment probably cost a small fortune. So it is unfortunate that the setup suffers from noticeable latency; while flat-out ignoring some of his commands! Still this gets the discchord Stupidtacular seal of approval for sheer inventiveness.

Video Description:

It is a virtual MIDI console that can be used with the same sense as a hard MIDI controller.
I applied for GUGEN 2017.

MR(Mixed Reality): iPhone + Bridge SDK(https://goo.gl/BU6gHJ)
Hand tracking: Leap Motion(https://goo.gl/5k2aV9)
Recognition of hand rotation and tap: a self-made hardware
Data processing of various sensors, MIDI assignment and MIDI output: a self-made Mac application

Reader Comments 1

The concept is pretty neat. It does sound like one key is to map out controllers to specific gestures. Reminds me a lot of Leap Motion, which has been making a pretty big push for AR/VR.
But, as you say, this demo suffers from the “close but not quite” problem. Can be inspiring, but the current implementation doesn’t sound that useful.
Those types of project keep cropping up, so it must be “in the air”, what with the VR hype and people’s hopes shifting to AR. Does sound, yet again, that Augmented Reality will have a broader impact that Virtual Reality.

Wonder how many people are experimenting with similar things. University departments probably have too many of these projects to count.
November 14, 2017  | person_outline Alex Enkerli

  Post a New Comment

Anonymous comments are closed for old articles. If you'd like to comment please Register.


Do not use HTML in your comments. Tags: [b] Bold Text [/b] [i] Italic Text [/i]
Links will be generated if you include http:// or https:// at the beginning of a URL.