Projector, inferred cameras, retroreflective markers, Unity, Procreate, and Adobe Illustrator

Dec 15, 2023

Welcome to the enchanted garden, where each blossom embodies a unique dream! The mission is to carefully pick up the flowers on the ground and place them on the table. The illustrated space is being projected from the ceiling and the illustrations will shift and change according to the player’s interaction and movement. It is an interactive curated space inspired by Sip 2 Slumber (refer back to my previous projects).

As shown in the storyboard, the plate that the player holds has three retroreflective markers attached. The *OptiTrack* cameras set up in the room will be able to track the position of the markers as the player moves around the room. Through *Motive* software real-time motion data is sent through internal PC routing into *Unity*. Using this data I have coded different interactions the player can do with the projections and the location of the plate.
The last two interactions above got combined into one in the final project. Here are the demonstrations:

1. Pick up

When the player kneels down to pick up the flower projection with the plate, the flower transformation animation will be triggered. When the player walks away, the picked up flower projection will travel with the player and it will disappear from the ground.

2. Set down

When the player places the plate on the table, the flower will fall off the plate onto the table and player can walk away to continue their exploration.

Full Interaction Demonstration

Code

using System.Collections;

using System.Collections.Generic;

using UnityEngine;

public class Slide_off : MonoBehaviour

{

public GameObject Object1; // Flower animation

public GameObject Object2; // Player

public GameObject Object3; // Home/ table

public float dist; // Float storing the distance between two objects

public float disty;

public float homeDist;

Renderer obj1Rend;

Animator anim;

// Start is called before the first frame update

void Start()

{

obj1Rend = Object1.GetComponent<Renderer>();

anim = gameObject.GetComponent<Animator>();

}

// Update is called once per frame

void Update()

{

Ray cameraToObj = new Ray(Camera.main.transform.position, Object1.transform.position - Camera.main.transform.position);

Vector3 obj1Y = new Vector3(0, Object1.transform.position.y, 0);

Vector3 obj2Y = new Vector3(0, Object2.transform.position.y, 0);

Vector3 obj2XZ = new Vector3(Object2.transform.position.x, 0, Object2.transform.position.z);

Vector3 obj3XZ = new Vector3(Object3.transform.position.x, 0, Object3.transform.position.z);

disty = Vector3.Distance(obj1Y, obj2Y); //the vertical (Y-axis) distance so it knows if the player kneels to the ground or not

dist = DistanceToLine(cameraToObj, Object2.transform.position); //the x.z position of the player

homeDist = Vector3.Distance(obj2XZ, obj3XZ); // the distance between the player and the table

// If statement check

if (dist < 0.1f && disty < 0.1f)

{

print("Player positioned");

anim.SetTrigger("Playani"); // The animation is stopped on the first frame until this trigger is called, so the animation only plays when the player is collecting the flower

Object1.transform.parent = Object2.transform; //the collected flower is now following the player

if (homeDist < 0.4f) {

print("Player home");

Object1.transform.parent = Object3.transform; // now the flower is set on the table

}

public static float DistanceToLine(Ray ray, Vector3 point) { return Vector3.Cross(ray.direction, point - ray.origin).magnitude; }