arguments for estimating person segmentation.Ĭonst personSegmentation = await net.estimatePersonSegmentation(imageElement, outputStride, segmentationThreshold) Īn example output person segmentation looks like:ĭata: Uint8Array(307200) load the BodyPix model from a checkpoint
We call the API method estimatePersonSegmentation to perform person segmentation on an image or video this short code block shows how to use it: const imageElement = document.getElementById('image') For all these reasons, we think BodyPix is easily accessible as a tool for artists, creative coders, and those new to programming.īefore we do a deep dive of BodyPix, we want to thank Tyler Zhu from Google Research who’s behind the model and working on human body pose estimation, Nikhil Thorat and Daniel Smilkov, the engineers on the Google Brain team behind the TensorFlow.js library, and Daniel Shiffman and the Google Faculty Research Award for helping to fund Dan Oved’s work on this project, and Per Karlsson from Google Research for rendering the simulation dataset. Since all computing is done on device, the data stays private. And finally users can access these applications by just opening a url. You don’t need any specialized lenses to use these models - they work with any basic webcam or mobile camera. Instead both BodyPix and PoseNet can be used without installation and just a few lines of code.
Why would you want to do this in the browser? Similar to the case of PoseNet, real-time person segmentation was only possible to do before with specialized hardware or hard-to-install software with steep system requirements. We hope the same kind of creative experimentation happens with BodyPix. It’s up to you! When we released PoseNet last year (the first model ever that allowed for body estimation (what a Kinect does) in the browser using simple webcams), people came up with a wide variety of use cases for the technology. What can be done with person segmentation? This technology can be a tool for a wide range of applications from augmented reality to photography editing to artistic effects on images or videos. This might all make more sense if you try a live demo here. It can further classify pixels representing a person into any one of twenty-four body parts. In other words, BodyPix can classify the pixels of an image into two categories: 1) pixels that represent a person and 2) pixels that represent background. The BodyPix model is trained to do this for a person and twenty-four body parts (parts such as the left hand, front right lower leg, or back torso). What exactly is person segmentation? In computer vision, image segmentation refers to the technique of grouping pixels in an image into semantic areas typically to locate objects and boundaries.