Show and Tell is an Alexa feature designed to help blind and low vision customers identify grocery items.
After losing her sight suddenly at age 41, Stacie Grijalva faced a new reality, one that inspired a new mission. A mechanical engineer by background, Grijalva was committed to exploring how assistive technology could make life a little easier for the blind and visually impaired, and sharing that information with others. “My job is to help people with visual impairments see how technology can affect people’s lives and make them feel better about what they do on a day-to-day basis.”
Alexa-enabled devices are already a big help around the house for Grijalva, from changing the channel on her Fire TV to using her voice to turn off lights and appliances. Now, an Echo Show feature makes it easier to identify household pantry items.
With Show and Tell, blind and low vision customers can hold up an item to the Echo Show camera and ask, “Alexa, what am I holding,” and Alexa helps identify the item through advanced computer vision and machine learning technologies for object recognition.
“It’s a tremendous help and a huge time saver because the Echo Show just sits on my counter, and I don’t have to go and find another tool or person to help me identify something. I can do it on my own by just asking Alexa,” Grijalva said.
Amazon believes in starting from the customer and working backwards, a philosophy also applied to inventing and using our technology resources for good. That means paying attention to what all of our customers are telling us.
“The whole idea for Show and Tell came about from feedback from blind and low vision customers,” said Sarah Caplener, head of Amazon’s Alexa for Everyone team. “We heard that product identification can be a challenge and something customers wanted Alexa’s help with. Whether a customer is sorting through a bag of groceries, or trying to determine what item was left out on the counter, we want to make those moments simpler by helping identify these items and giving customers the information they need in that moment.”
From early research and exploration through product development and testing, Caplener’s team collaborated with the Vista Center for the Blind and Visually Impaired in Santa Cruz, California, where Grijalva is the assistive technology manager. For Show and Tell, Grijalva enlisted other blind and low vision customers for user studies, providing feedback to the Alexa for Everyone team. “By working together, we create an amazing product for everyone,” said Grijalva.
https://www.youtube.com/watch?v=NDYSXvsnl4Y
Vista Center community member Brett Fowler lost his vision at age 10. The stay-at-home dad loves to cook, but he has trouble identifying things like spices. Fowler says the Show and Tell feature on the Echo Show is a game changer. “All of these devices that are acting as your eyes, it’s revolutionary. For me, the less stress I have to put on somebody else is less stress on me. And it makes me feel good.”
The Alexa for Everyone team also collaborated closely with blind Amazonians, including principal accessibility researcher, Josh Miele. His decades of experience designing and evaluating the effectiveness of products for people with visual disabilities provided valuable insight into product and research design. “I get to help create products like Show and Tell, make our Fire tablets and TVs accessible and delightful, and constantly help to imagine new ways for our products and services to improve the lives of our customers, including those with disabilities,” said Miele. Globally, an estimated 1.3 billion people live with some form of vision impairment, according to the World Health Organization, and 15% of the world’s population experience some form of disability.
“It’s essential to work with our customers, not just for them, to create something that’s truly helpful. It’s just Day 1 and we’ll continue to invent on behalf of customers every day,” said Caplener.
Show and Tell is now available to Alexa customers in the U.S. on first and second-generation Echo Show devices. Customers can simply say, “Alexa, what am I holding” to get started.
Source:Amazon Blog