Camera, Ray, Actor - A Technique for Collaborative 3D Manipulation

This was an entry for the 3DUI 2016 Contest. Among the requirements, the submission had 1) allow two or more users to apply translation, rotation, and resizing transformations and 2) benefit from the collaboration of two or more users applying a single manipulation operation (translation OR rotation OR resizing) on one object together.
Our approach was based on two or more users jointly specifying the parameters of each transformation using a point, a ray, and a scalar value. The director decides which operations should be performed. He is responsible for investigating the scene and finding the best way to act. This director's camera can be rotated freely but is always targeted at the current selected object. Once satisfied, the director updates the actor’s camera based on his own. In the parallel (asynchronous) mode, the director sends a snapshot of his camera to the actors. In this case both cameras remain decoupled and the director can proceed looking for the next operation. In the synchronous mode, the cameras are coupled and the changes to director’s camera are seen live by the actor. In our implementation, cameras are coupled as long as a button is pressed. The actors use their currently assigned cameras to complete the operation defined for that “shot”. For translation operation, for example, the actor can choose how to move in the plane, drag or just use his pointer as a target. 

Our implementation used a RealSense camera for finger tracking (actors) and the Capsule for camera control and selection of operations.
Team: Wallace Lages