Nov 30, 2022 12:30:00

Amazon announces AI ``Create with Alexa'' that creates animation from voice, so that children can create stories via voice prompts

Image generation AIs such as Stable Diffusion and Midjourney are being talked about as being able to automatically generate images according to the sentences entered as prompts. Under such circumstances, Amazon will install an animation automatic generation AI ' Create with Alexa ' that creates stories, images, and BGM on the spot by inputting voice as a prompt to the smart speaker '

Amazon Echo Show ' equipped with a liquid crystal display. Announced.

'Alexa, let's make a story': Amazon introduces a new feature that turns your Echo device into a storytelling companion
https://www.aboutamazon.com/news/devices/what-is-create-with-alexa

The science behind Alexa's new interactive story-creation experience - Amazon Science
https://www.amazon.science/blog/the-science-behind-alexas-new-interactive-story-creation-experience

When children talk to the Amazon Echo Show equipped with Create with Alexa, 'Alexa, create a story', it will be activated, and after that AI will automatically generate a story by giving voice instructions such as characters and settings . Not only the scenario, but also the images, BGM, and sound effects of each scene are all generated by AI.

To tell the story, first select a theme such as 'space exploration,' 'underwater,' or 'magical forest.'

Then select a character and give it a name.

Finally, choose adjectives that describe the mood of the story, such as 'silly,' 'happy,' and 'mysterious.' Then, AI will create a short story of about 5 to 10 lines consisting of 5 scenes.

Once created, characters and stories can be saved in your personal media gallery. Children can also look back on their favorite stories later and have fun with their parents.

Create with Alexa consists of a 'story generator' that creates a story scenario, a model that generates a screen that matches the scenario, and a model that generates BGM. The story generator consists of a 'planner' that receives input prompts and creates a story name, and a 'text generator' that generates the story text from this name.

The story generator learns from a data set of stories written by human writers. This dataset is labeled with tags such as 'space exploration', 'underwater' and 'magic forest' for each story.

Then, the generated story is supplemented with pronouns and referents by two natural language processing modules, and adjusted so that even children can understand it. At the same time, a graph showing the relationships between objects in the text, such as characters and items, is automatically generated, and scenes suitable for the story are generated based on the automatically generated images and this graph. Many of the generated images are animated, for example, fish placed in the underwater background move their tail fins.

In addition, the background music that accompanies the story is automatically generated from a library containing various chord progressions, rhythms, and instrumentation. Moreover, the time it takes for the child to read the text is calculated, and the length and impression of the BGM are determined according to the atmosphere of the story. In addition, since it screens and excludes offensive content, content that is overly stimulating for children is not generated, and there is also a safety that requires parental consent to use Create with Alexa. It is said that there is.

Eshan Bhatnagar, head of product for Alexa AI, revealed that he and his 6-year-old son were doing beta testing. “From the beginning, we used carefully curated data sources to train our AI model,” said Bhatnagar. It inspires me not only to do more, but more broadly, to bring the power of AI to unlock the possibilities of the future.”