Read it out loud: text recognition

Introduction

Some phone apps can point at a menu in another language and translate it instantly. Others read books aloud for people who can't see well. Both start by reading the words in a picture. That's called text recognition, or OCR (optical character recognition).

In this lesson you'll build a reading robot: hold up a page, and Tobi reads it out loud.

Working principle

• Recognize text in image after (2) seconds gives you time to hold up the page, takes a picture and sends it to be read.
• Printed text result reports the words it found in printed text, such as a book or a sign.
• Handwritten text result reports words it found in handwriting.
• This model runs on the internet, not on your computer, so you need to be online and signed in to PictoBlox. That also means the picture leaves your computer, so only show it things you'd be happy to share.
• Speak () from the Text to Speech extension says any text out loud.

What you'll need

• PictoBlox with the Text Recognition and Text to Speech extensions
• A webcam and an internet connection
• A grown-up to help sign in to PictoBlox if it asks
• Some big, clear printed words: a book cover, a cereal box, or a sign you write in thick marker

Working in real life

1. Add the Text Recognition and Text to Speech extensions.
2. Build the script below on Tobi.
3. Click the green flag, then hold a page up to the camera, nice and still, before the countdown ends.
4. Listen to Tobi read it.

Code

when green flag clicked
turn [on] video on stage with (0) % transparency
say [Show me something to read!] for (2) seconds
recognize text in image after (3) seconds
if <(printed text result) = []> then
  say [I couldn't read that. Hold it closer and keep it still.]
else
  say (printed text result)
  speak (printed text result)
end

Try it

• Write a word in your best handwriting and use handwritten text result. Then write it messily. When does it stop working?
• Make a "secret password" door: if the text it reads is OPEN SESAME, Tobi says "Welcome!"
• Add the Translate extension and make Tobi say the words in Hindi, Marathi or another language from the Translate block's list.

Conclusion

• Text recognition (OCR) finds and reads the words in a picture.
• PictoBlox can read printed and handwritten text, and Text to Speech can say it aloud.
• Online AI models send your picture over the internet, so think before you show them something.

What next?

So far you've used models that someone else trained. Now train your own: Teach the computer: your own image classifier.