Nice write up! I love seeing novel uses for game engine tools. I lead a media group at a large museum and have mostly standardized on unity for all sorts of interactives and media experiences. Bookmarking this to checkout the specifics of the pipeline later.

This is pretty nice, I used to make an app for projection mapping which this seems to replicate. But I used syphon a lot, along with Quartz Composer for inputs, now that QC. is almost dead, I know a quite a few people have to moved to generating live visuals with other tools like Unity and Blender, I've been meaning to look into it myself. But I also got myself a job developing another app now. I wish I had the time to get my previous app up to date

I’m impressed technically and still don’t really think the effect is super convincing.

It does seem like this technique could work with some refinement. Perhaps just making the mobiles about 33% smaller would help.

The fact that multiple camera angles sync a virtual object is really impressive. Keep at it there is something valuable that’s almost captured.