macOS Monterey: How to use Live Text and Visual Look Up

Live Text is a new feature in macOS Monterey that allows you to use the text in an image. It’s a feature that’s quite helpful—for example, if you’ve ever been in a meeting or lecture, and snapped a picture of a whiteboard filled with written information, you can now select the text in the photo, copy it, and then paste it into a document.

There are a lot of ways you can use the Live Text feature, and it’s a huge time-saving production tool. Live Text is also in iOS 15 and iPad OS 15, and because you probably use the cameras on those devices often, you might use Live Text on your iPhone and iPad more than you would your Mac. But it’s coming to macOS Monterey this fall and works in the Photos app, Safari, Quick Look, and the Screenshot function.

Here’s a look at how Live Text works on the Mac on macOS Monterey. Live Text works on both M1 and Intel Macs. You can watch the video below to see it in action, or you can read the article below to get instructions on how to use it.

Using Live Text in Photos, Quick Look, and Safari

When you are looking at an image, move your cursor over the text, and the text selection tool appears. (Some apps, such as Preview, require you to switch to the text tool first before you can select text.) You can then select the text, and you can copy it and then paste it into a document. You also have to option to grab the text in the image and drag it over to another app, and then drop the text, as you can see in the gif below.

Live Text does a very good job with its character recognition. In the above gif, you can see that Live Text was able to recognize the accent over the letter “O”. You may also notice that there are a couple of marks in the corner of the sign before the “S” in SOY; Live Text interpreted that as a bullet. Also, Live Text translated the “O” in BASURERO as a “D”, but it’s hard to fault Live Text because it does resemble a “D” in the image.

Macworld’s Glenn Fleishman took a look at Live Text and compared it to other character recognition tools, such as PDFpen, Google Docs, and Microsoft OneNote, and found that Live Text produced the best results.

Using Look Up

With text selected, you can right-click, bring up the contextual menu, and you have the option to Look Up the text. (If you’re using Safari, the Look Up selection in the third section of the contextual menu, under the Save Image and Copy Image sections.) 

IDG

Three Look Up options are offered:

Siri Knowledge, which displays information based on the context of the selected text.

IDG

Maps, which does a search in Maps based on the text and displays a location.

IDG

Siri Suggested Websites, which does a web search using the selected text and offers a website you can peruse.

IDG