Flow

Interactive Immersive Experience | 2024

Flow serves as a demo for one of the signature interactive rooms at CMU ETC, Cavern. It contains a 270-degree curved screen and multiple tracking devices. Inspired by Chinese folklores, Team Flow was able to create an interactive immersive experience that takes the guests on a spiritual journey through a world filled with dance, music, light and art. Over the course of spring 2024, Flow hosted more than 70 demo sessions for more than 100 guests and clients visiting ETC.

Tech

Role

Unreal Engine 5, Cavern, Kinects, Vive Trackers, Motion Capture

Visual Effects

Shader Creation

Set Dressing

Lighting

Art Directing

Time

Team

14 weeks

Linaixuan Wang - Producer, Creative Director, Sound Designer, Dancer

Christine Jung - Technical Environment Artist

Laura Yang - VFX Artist, Technical Artist

Ashling Tu - 2D & 3D Artist, Level Designer

Shih-Hung Liu - Programmer, Lead Gameplay Engineer

Jiajian Zhang - Programmer, Technical Animator

Demo Video:

Dancer VFX

I created a VFX to show the magical moment when the prayer of the deer revives its companion, the dancer, with the help of the guests. I wanted to make the moment touching, with a powerful yet coherent sensory experience, so I decided on the following key features:

Bright blue, coherent with the color of the dancer
Converging energy to show the process bringing back the dancer
Fairy dust, echoing the magic power the dancer has
Dancer dissolving in to existence

Interactive VFX

I was responsible for making some of the VFX related to the interactions, including boosting fire, growing trees, lighting fireworks. These are the effects that the guests can trigger by doing specific body movements. Corresponding to guests different movements and timing:

Tree grows taller, using a geometry cache animation made in Speedtree
Embers burst out and fire gets bigger
Fireworks shoot into the sky from corresponding locations

Guiding/Ambient VFX

During playtests, we received feedbacks from guests that reflects some major challenges of our design:

Guests sometimes didn’t know where to look

The scene felt too static

To improve upon those feedbacks, I proposed to use VFX as visual guidance to better engage the guests and I developed the following VFX:

Guiding particles to guide the focus across the screen

Hand Particles to reflect guests’ movements

Hand Particles on the dancer to reflect her power and signal guests’ interaction

Ambient wind to make the scene more dynamic

Shaders

I created stylized shader inspired by Chinese traditional ink paintings. I studied the tradition ink paintings and extracted the following visual elements for the main shaders for this project:

Gufa or “Bone Manner”: Capturing the skeletal structure with outline
Cunfa or “Texture Stroke”: Adding shadow or texture to the subject
Xuanran or “Wash painting”: Filling in the color with colorful ink wash, sometimes with gradient

The first shader used the Fresnel effect and object height to create outline, shadow, noise texture and color gradient. It works the best for organic objects that has no flat surface, such as mountain, rocks, trees. Therefore it is widely used for landscape in the project.

The second shader is a Cel shader implemented on the post-processing level. It also incorporated custom depth buffer, so I was able to apply it on only selected objects. I also overlaid multiple noise to achieve the brushstroke texture. It is complimentary to the first ink shader and is widely used on objects with flat surfaces such as buildings.