Emotion mirror: is that a happy face?

Introduction

When your friend smiles, you know they're happy. When they frown, something is wrong. Can a computer read faces like that?

PictoBlox's face detection can guess the expression on a face: happy, sad, angry, surprised and more. In this lesson you will build an emoji mirror: make a face at the camera and an emoji copies your mood.

Working principle

• Get expression of face (1) reports the model's best guess, as a word such as happy, sad, angry, surprised or neutral (the exact list is in the block's dropdown).
• Is expression of face (1) (happy)? is a true/false block. It is handy inside an if block.
• If no face is found, the expression block reports NULL, meaning "nothing".
• The model only sees the outside of your face. It is guessing, not reading your mind.

What you'll need

• PictoBlox with the Face Detection extension
• A webcam and good light on your face
• An emoji sprite with several costumes: paint your own in the Costumes tab (a happy, sad, angry and surprised face), or use costumes of any sprite you like

Working in real life

1. Make a new sprite called Emoji and paint four costumes named happy, sad, angry and surprised. Naming each costume after an expression is the trick that makes the code short.
2. Add the Face Detection extension.
3. Build the script and click the green flag.
4. Smile, frown, look surprised. Does the emoji copy you?

Code

when green flag clicked
turn [on] video on stage with (50) % transparency
go to x: (160) y: (100)
forever
  analyse image from [camera]
  if <(get # faces) > (0)> then
    switch costume to (get expression of face (1))
    say (get expression of face (1))
  else
    say [Where did you go?]
  end
end

Because each costume has the same name as an expression, switch costume to (get expression of face 1) picks the right one by itself. If the model says an expression you didn't draw (like neutral), the costume just stays the same.

Try it

• Add a neutral costume.
• Make Tobi tell a joke whenever your face is sad: if <is expression of face (1) (sad)?> then say [Why did the robot go on holiday? To recharge!].
• Test the model fairly: try it in bright light and in a dim room, with and without glasses. When does it make mistakes? Write down what you notice.

Conclusion

• Face detection can also guess the expression on a face.
• A clever naming trick (costumes named after expressions) keeps your code short.
• AI only guesses from what it sees. It doesn't know how you really feel.

What next?

Next, teach PictoBlox to recognise who a face belongs to: Who's at the door? Face recognition.