How to do pose detection in browser? Let’s learn the Posenet

Hello everybody. I have not written article for a long time. I was not avaible due to my work and school situations. But this weekend, I have oppurtunity to write an article. I don’t want to miss this and this article is on a interesting topic.

First, we will take a capture from the camera and display it on the screen. Then we will do realtime pose detection with posenet library. Let’s get started right away.

What is the Posenet ?

Posenet is an artificial intelligence model created to recognize the human body. With Posenet, you can recognize many parts of the human body and operate with the help of them. We can give examples such as playing games or doing exercises with a webcam.

Step-1: Create a new project

First of all, let’s create the folder where our project will be located. I’m going to call it ‘project’ here. But you are completely free in naming. And I will create two files in this folder. These files are ‘index.html’ and ‘sketch.js’.

Project Structure

Step-2: Import ml5.js and Index.html

At this stage, we will import the library we will use into our project. Ml5.js is an amazing library that allows us to use artificial intelligence with javascript. It contains many different features (Pose detector, face api, object detector, sound classification etc.).

If you want to more discover this amazing library, you can click the link:

https://ml5js.org/

And we will use the posenet system of this library.

https://learn.ml5js.org/#/reference/posenet

After importing the library, the index.html will be as follows.

[embed]Index.html

Step-3: Write a Sketch.js

At this stage, we will recognize the poses on the image and draw them on the screen. we print each recognized pose to the console.

I would like to give some information about the below functions.

setup() : This function creates a canvas on the screen and recognizes poses using the library.

modelReady() : It is the function that the model triggers when it is ready.

draw() : This function draws on the screen depending on the position through the poses.

The Sketch.js file is as follows:

[embed]Sketch.js

Step-4: When we want to do real-time control

This step is optional. If we’ve come this step, we’ll see that everything is perfect when the project works. But what do we do if we want to run code realtime processing on the image?

Let me explain this with examples. If we want to find the angle between two keypoints or we want to control the height of the keypoints. Sometimes, we may want to check whether there is a pose on the screen or not. In these cases, find the following line in sketch.js and make edits in that part.

[embed]Realtime check keypoints

What keypoints can Posenet recognize?

Posenet can successfully recognize the following keypoints and gives us this output.

Posenet Output

Conclusion

In this article, we learned about the posenet library. We also learned how to run this library on the browser. Now you can easily use this library on your websites and applications.

See you in another article, take care of yourself.