Don't Move

Leonardo da Vinci understood perspective and projective geometry back in 1480. After that, all humanity needed was a little time and a few geniuses to go from plane table, to analog, to analytical, to digital photogrammetry.

But this isn’t a history lesson, no matter how fascinating the fact that Dr. Carl Pulfrich researched stereoscopic instrumentation while he himself had no depth perception (told you, fascinating!); this is about what we can do today with our knowledge of photogrammetry.

That word is a mouthful, but we won’t replace it with “3D scanning” because while they both aim at creating a model of an object as accurately as possible, they differ in how they get this done. Photogrammetry consists of taking many photographs of an object from different angles and then having specific software put them together to create a 3D model of that object. And photogrammetry is what we use at Dephion to create our digital coaches.

coach_image.jpg

Having a photogrammetry rig (how many times have we used that word now, five?) on site has served us well. One hundred and three cameras–Canon 6Ds with 85mm lenses for detail and a range of Canon 2000Ds with 50mm and 15-55 mm lenses for overlap, silhouette, and filler pixel information–can take a scan in just the amount of time that it takes to take a snapshot of its intended victim.

Content image (4).jpg

Eser stands in front of the rig.

That’s a lot of cameras, why not just use one? In fact, why not just use a phone? We talked to Kenny from the Talos team about how we use our own rig at Dephion, and he was happy to give us all the juicy details. “Today’s phones can have cameras that have enough resolution to make detailed pictures, but it's the time it takes to make each picture individually and the subject moving that usually throws it off. Humans move constantly and even the slightest movement can already cause major artifacts and tears in the scan.”

Which is an excellent point. That means that the team must be very selective of who it chooses as a model…

“Scott and I scanned ourselves numerous times when designing and calculating the scanner set-up. As well as the newspaper-covered mannequin. Very nice as it never protests or moves,” says Kenny.

scanner_mannequin.jpg

Ah, but the mannequin can’t smile at you, can it?

Photogrammetry is another way to capture the kind of facial expressions that ensure the coaches pass the uncanny valley test. While a single scan takes but a second, an entire session for full body and 30 facial expressions can take between one and three hours. Thirty facial expressions is a substantial number, but we want MORE. “We're currently looking at new methods and theories which would expand the list by roughly 23,” Kenny says.

Also, the newspaper-covered mannequin has no hair. “Hair is scanned as a reference for shape and volumes. Due to the nature of hair existing out of thousands of individual strands, we recreated it in Maya using xGen.”

coach_working_on_hair.jpg

So we have our model, we have a million cameras to take shots from all angles simultaneously, but we also need the light to be just right. “Lighting is very important for the diffuse texture extraction later on. Shadows and such are all baked into it and would require a lot of cleanup in a later stage if done poorly during the initial scan. That is why we have 4 flash boxes aimed at the walls to bounce an even light across the scan target.”

Once the data is collected from the cameras, software called Reality Capture is used to process it. Thanks to the creation of dense point clouds, which, as the name implies, are data points that represent the 3D shape and kind of look like clouds, great detail of detail can be captured before the time for clean-up and enhancement comes. “Processing the pictures into a point cloud can take anywhere from 2 minutes for a preview medium-quality mesh to 15-20 minutes for a high-detail calculation.”

coach_pointcloud.jpg

A point cloud.

If you think getting scanned sounds like fun, there is a good chance you could get yourself immortalized in 3D. “Team Talos has almost everyone scanned, some even multiple times as we keep testing new techniques. There are some other people in the building we'd love to scan as well, but they very conveniently keep getting stuck in meetings and our schedules never line up.”

How suspicious.

While Kenny does most of the scanning, Murtaza comes in when it’s time to clean up and make animation-ready models. He is also largely in charge of the character designs and looks. Yordan’s expertise lies in doing the rigging, that is, constructing the skeleton. Later, Eser and Anastasiia work on the animation of the models.

coach_cleaned_up_scan.jpg

Thanks to photogrammetry and the formidable work of the Talos team, Dephion’s coaches look and move like the digital avatars we envisioned for Habtic. We wonder if Leonardo saw that coming…

Topics

Talos

Share this post