This summer we took the kids to the Crayola Experience in Dallas. It’s a bit of a racket – many of the activities were broken – but you can create your own crayon label. Way in the back, they’ve got a somewhat hidden area where you can color a dress or a race-car, then see your drawing featured on the “big screen”. The race-car was broken, but the kids had a blast drawing characters and seeing them pose on screen.
A few months later, the elementary school was looking for an indoor halloween party idea. This seemed like a perfect concept – so I hacked together some NodeJS and ThreeJS to build my own version.
In short, the kids color the front and back of a character. Then I scan it with my phone and it automatically updates one of the dancers on screen with the new outfit:
The kids ended up coloring over 100 characters and loved seeing them dance on screen.
It was such a big hit, that another grade had us do it for their Valentines day party. Then another school had me set it up for their STEM party. It’s really fun to see how creative the kids can be with their drawings.
Technology
Here are the highlights of how it works:
- NodeJS/ExpressJS server (for serving web pages, saving scans, etc)
- ThreeJS for client side animation
- Client-side QR Code Scanning (zxing-wasm) to identify the position of the QR codes
- perspective-transform – to fix the orientation/perspective of the scan & align it with the character texture file
- fiverr – I don’t know how to map a JPG character to an FBX file, so I hired someone on Fiverr for this
- Adobe mixamo – offers the “thriller” dance animation
- Camera Controls – to move the camera between different positions randomly
- ngrok – for making the website available on a public url
Code
The codebase is fully open-sourced (MIT License) at:
