The Song of Drifters
“The Song of Drifters” traces the diasporic struggle between departure and return. Haunted by memories of a homeland far beyond reach while drifting through unfamiliar cities, the wanderers grasp onto fleeting senses of belonging in a foreign land. What lies ahead is a cross-dimensional voyage of unexpected surprises and revelation.
Behind the Scene
This project extensively pushed the boundary of making images in various technologies. To explore how to extend my creativity through AI, rather than letting it take my creativity, I customized my generative workflow in ComfyUI and used my own illustrations as the style guide. The base video materials are live-action shooting and 3D animation in Blender by myself. The hybrid pipeline boosted the efficiency of rendering the final look and saved more time for ideation.
Lora Training from the city walk
Capturing video data has become a vital part of the documentary process. Wandering through the streets where I once lived allows me to reconnect with these cities. Although most of the footage doesn't appear in the final cut, it becomes part of the visuals in the form of training data.
Rough 3D animation + AI morphing
This workflow experiments with the combination of 3D animation and AI-generated morphing. It started with more abstract expression in 3D to grasp the feeling of the interviewee’s dream, then guided with more representative illustrations and prompts. The morphing between the abstract and concrete images mimicked the experience of a half-awake dream.
Process Breakdown
The base video is a 3D animation rendered from Blender Cycles.
I converted it to three different passes in After Effects: fade in&out, mask, and depth. Each controls the timing of the morph, divides the areas between abstract and concrete visuals, and maintains the shape of the base animation separately.
I created a series of illustrations to express my impression of the dream. They landed on floral elements and ocean waves. Making these illustrations was a conversation with the big model. I tested different images, even groups of images, in the workflow. Throughout this process, when I optimized the final output, I also created a series of illustrations with the same theme and expanded my style guide archive.
