[UE5] Proof of Concept – Puppeteering – VR

[UE5] Proof of Concept - Multiplayer Puppeteering - VR

This project, led by my former producer Simon, aimed to create a puppeteering proof of concept inspired by The Jim Henson’s Company style. It involved developing a multiplayer VR experience for recording using Take Recorder.
As a relatively inexperienced developer, this project was a big challenge for me.

Project's Breakdown

Proof of Concept:

Simon wanted to use Unreal Engine to streamline the animation process and achieve a visual style reminiscent of The Jim Henson’s Company productions. His approach involved using VR knuckles for character manipulation rather than a traditional control rig. The concept immediately captured my interest.

He envisioned recording multiple puppets simultaneously using the Take Recorder.
This required the development of what essentially amounted to a “multiplayer VR game” where Simon and I would control the puppets, while my colleague, Aleks, handled the recording and framing.

After a lot of searching, I found some really helpful tools like the VR Expansion Plugin and codehawk64‘s impressive VRE-DragonIK-VRIK example project.
These tools, along with their respective Discord communities, provided me with essential guidance and support.

(Me and Simon testing out the puppets in our 'virtual office')

 I set up a remote VPN using Tailscale, enabling Simon to join my gaming sessions by simply entering my IP address in the Widget.

Episode 1:

In the initial episode, we introduced Jeff, the character with blond hair, who encounters Norbert, a robot from outer space trying to fit into society, on the streets of a miniature New York.

To bring this to life, we needed to create a rig for Norbert and experiment with Mass AI to fill the bustling city.

Since the project is still under a strict NDA, I’m unable to provide specific details and the final render of Episode 01.