Teach the computer: your own image classifier

Introduction

Every model you've used so far was trained by someone else. The face model knows faces, the object model knows cups and books. But what if you want the computer to recognise something you choose: your pet, your school badge, or the hand signs for rock, paper and scissors?

You train your own model! In this lesson you'll use PictoBlox's Machine Learning Environment to teach the computer three hand signs. In the next lesson you'll turn it into a game.

Working principle

• An image classifier looks at a whole picture and says which class it belongs to. (Object detection finds where things are; a classifier just says what the picture is.)
• You need at least two classes, with plenty of samples in each. Aim for at least 30 photos per class.
• Variety matters: different distances, angles, both hands, light and background. A model trained only on your left hand in your bedroom may fail with your right hand in the kitchen.
• Always add a "nothing" class (an empty background) so the model has an answer when no hand is showing.
• After training, test the model with new pictures it hasn't seen. Then export it so its blocks appear in your project.

What you'll need

• PictoBlox (signed in, if it asks) with a webcam
• Your hands, and a plain wall or background if you have one
• About 30 minutes for taking samples and training

Working in real life

1. In PictoBlox, open the Machine Learning Environment (in newer versions it's in the Files menu or on the home screen).
2. Create a new project, name it RPS, and choose Image Classifier.
3. Make four classes: rock (a fist), paper (a flat hand), scissors (two fingers) and nothing (no hand at all).
4. For each class, click Webcam and hold the button to take samples. Move your hand around while you do: closer, further, tilted, the other hand. Take 30 to 50 samples per class.
5. Click Train Model and wait. The bar shows the model studying your samples.
6. When training finishes, use the Testing panel: hold up each sign. The bars show how sure the model is about each class.
7. If a sign keeps getting mixed up, go back, add more varied samples to that class and train again.
8. Click Export Model. You're taken back to block coding, and new blocks for your model appear in the palette.

Code

Test your exported model with a quick script:

when green flag clicked
turn [on] video on stage with (0) % transparency
forever
  analyse image from [camera]
  say (get identified class)
end

If the block names in your palette are a little different, look for the one that analyses the camera image and the one that reports the identified class.

Try it

• Look at the confidence of each class while testing. Which sign is the model least sure about? Why might that be?
• Train a second model of your own: your toys, your school stationery, thumbs-up vs thumbs-down.
• Ask a friend to test your model. Does it work on their hands too? If not, add samples of their hands (with permission) and retrain.

Conclusion

• You can train your own image classifier by giving it labelled samples of each class.
• Lots of varied samples, and a "nothing" class, make the model more reliable.
• Test with new pictures, improve the samples, retrain, then export to use it in blocks.

What next?

Put your model to work: Rock, paper, scissors against the AI.