Projection map a 3D printed face

Summary

It's a 3D printed head of my friend that is being projection mapped with photoshopped pictures of himself. The head is about 2.5-inches tall, and the projections are 2 photoshopped images of his face: one with mouth open and one with mouth closed. I used jQuery to bind the mouseDown and mouseUp events to the mouth open/closed images. The script is hosted on a web server and the Raspberry Pi running Raspbian Wheezy ( http://www.raspberrypi.org/downloads ) is used to drive the projector.

Why? Because I hope to find more and more ways to validate my 3D printer purchase.

Apparatus

Procedure

  1. Scan your face
  2. Clean up the scan data
  3. 3D print and projection map the head

Scan your face

Duct tape a large piece of cardboard to the back of the office chair. This gives the camera more tracking points. The Kinect Fusion documentation suggests to:

Add clutter at different depths to scenes when environment scanning to improve problematic tracking.

source: http://msdn.microsoft.com/en-us/library/dn188670.aspx

 

Setup the Kinect on a tripod, and have your subject sit in the office chair so that their head is roughly level with the sensor. This is what my setup looks like:

 

Open "KinectFusionExplorer" which for some reason was really hard to find. Here was the path it installed to on my computer:

C:\Program Files\Microsoft SDKs\Kinect\Developer Toolkit v1.7.0\Samples\bin\KinectFusionExplorer-WPF.exe

 

Move the subject's face as close to the sensor as it can pickup in order to maximize resolution. Set the max distance in KinectFusionExplorer to only capture the subject and office chair. In other words:

Try to isolate the object from the environment using the depth thresholds...

source: http://msdn.microsoft.com/en-us/library/dn188670.aspx

 

Slowly swivel the subject while instructing them to be as motionless as possible. Since I only have one projector, I only needed a ~180 degree scan of the person which is easily done in one sweep. If you need a 360 degree scan, aligning several scans in software may be easier to accomplish than one perfect scan.

source: http://burtonwould.tumblr.com/post/20917421134/creating-a-3d-printed-model-with-kinecttostl-meshlab

Clean up the model

Use Mesh Lab to clean up the scan mesh data and export to your 3D printer software/slicer.

Depending on the 3D printer and slicer, more or less work must be done on the 3D scan data to make it printable. My printer's software is robust in my experience so most of my scans just need to be cropped; the software handles the incomplete scan data, holes, and invalid geometry. The topic of 3D-scan cleanup is larger than is appropriate for this post so I suggest googling and/or following this instructable: http://www.instructables.com/id/Using-Meshlab-to-Clean-and-Assemble-Laser-Scan-Dat/

3D print and projection map the head

 

After printing the head and fastening it to a stand, if it can't stand on it's own, align the projector and the head so that it fills the area appropriately.

This is pretty straightforward but a little bit of fiddling will be necessary for good results. Depending on how closely your projector can focus, you should print the head as large as possible that remains practical. My projector allowed me to print the heads around 2.5-inches tall sitting 8-inches away from projector.

The alignment was done manually by stacking various boxes and looseleaf papers until it looked about right:

Summary and Discussion

Todo: Improve projection quality by painting the 3D print with a reflective paint used for projection screens.

Potential uses:

  • Preview merchandise, makeup, fashion accessories
  • "Virtual" avatar to video chat with
  • Halloween build
  • I remember seeing something like this at a museum to narrate an exhibit.