Creating crowds is a very consuming task in visual effects. It often requires 3D scanning extras on set, building the virtual characters, recording performances to be used as motion capture to animate the characters, and once we have a good amount of characters and variations, finally we do the crowd simulation.
This will always be the case, but what if we could do that only for the most important characters of the movie and use a hybrid pipeline to create many other virtual extras that are not as important for the story?
This hybrid pipeline does that. I’m able to generate multiple views of a character with a specific design, based on time period, or type of costume, or whatever it might be.
These images will be the input to create a 3D model of the characters that later I can use in a 3D software, in this case Houdini, to create a complex crowd of hundreds or thousands of characters. I would still have full control over everything that makes the final shot, including the number of characters, the number of variations, camera moves, lighting, environment, etc., but I will get there in a fraction of the time.
The same Comfyui workflow can generate props and other objects that the character need to wear, hold or carry.
The 3D models generated by AI are not the best when it comes to topology. I setup a quick procedural workflow in Houdini that makes better topology and LODs that work better for crowd simulations.
The images below showcase the crowd simulation stage where I can easily configure the amount of characters, the type of characters, the performance of the characters and any other imaginable thing.
For motion capture I’m using machine learning and AI assisted techniques, where I can create motion clips based on videos that I shot of myself or my colleagues. Most of this clips don’t need extra cleaning, as any inconsistencies will not be visible in a crowd with hundreds of characters.
Here you can see my army of 5 different types of characters. I’m using different motion clips, like walking, running or idling. This will be the base of my crowd simulation.
This is a test shot where the camera has been established and I can quickly test if the crowd works for the shot.
Sometimes you might need to present a key concept of what you are trying to do for the shot. But instead of painting everything from scratch, I can just use a frame of my animation to do the environment work. Maybe the feedback is that the director would want to see twice the number of soldiers. It would be very easy to accommodate the feedback, because I have full control in 3D.
Maybe the feedback is to have a completely different environment with the same look and amount of soldiers, again easy to accomodate and deliver.
Below the same technique applied to create a different type of crowd, different style, different universe.