Amazon’s all-new Alexa voice assistant will be powered by the new Alexa LLM

Amazon’s Alexa is about to come out of its shell, and what emerges will be very interesting. At its fall hardware event on Wednesday, the company unveiled an all-new Alexa voice assistant powered by its new Alexa Big Language model. According to Dave Limb, Amazon’s SVP of current devices and services, this new Alexa can understand conversational phrases and respond appropriately, interpret context more effectively and complete multiple requests from a single command.

Voice assistants need a shake-up. Instead of the amazing technological advances we expected when they came on the scene a decade ago, a general lack of innovation and incomprehensible improvements have reduced them to rudimentary tools.

Generative AI seems to have a better shot at survival for a while. But while these digital assistants will always have elements of AI, they lack complex processing capabilities, and AI is capable of making human-like interactions. This is a big moment for the smart home, as it could take home automation to the next level, moving from a remote control experience to a truly smart home.

In an interview on the edge Ahead of the event, Limp explained that the new Alexa LLM “is a truly generalizable large language model that is highly optimized for Alexa applications; This is not what you find with Bart or ChatGPT or any of these.

However, this new Alexa isn’t unleashed everywhere, for everyone, at once. The company is rolling it out slowly through a preview program “over the coming months” — only in the US. Clearly, lessons have been learned from the missteps of Microsoft and Google, and Amazon is proceeding cautiously.

“When you connect an LLM to the real world, you want to minimize hallucinations — and while we think we have the right systems in place … there’s no substitute for putting it out there in the real world,” Limb says. If you want to be notified when you can join a preview, say “Alexa, let’s chat” to your Echo device and your interest will be registered.

Unsurprisingly, this super-powerful Alexa isn’t always independent. While Alexa is free as it is today, Limp said, “the idea of ​​a humanoid assistant that can supercharge your smart home, and perform complex tasks on your behalf, might provide enough utility that we’d charge something. Below that.”

The first big change in the new Alexa will be a more conversational assistant, one that understands more of what you say and requires less specific naming to do what you ask. This is one of the most common causes of frustration with voice assistants — when you ask the thermostat to turn it down or it responds with, “Some things share the name ‘Lights.’ want

With the new Alexa, you can say a phrase like, “Alexa, I’m cold,” and the Assistant will turn up the temperature on your connected thermostat. Or, as Limb explained, “Say, ‘Alexa, make this room feel like Seahawk colors,’ and it’ll know what room I’m in and what Seahawk’s colors are, and it’ll do those translations between the APIs.”

APIs are key, says Limp. “We’ve added over 200 smart home APIs to our LLM.” This data, combined with Alexa’s knowledge of what devices are in your home and what room you’re in based on the Echo speaker you’re talking to, gives Alexa the context it needs to manage your smart home more proactively and seamlessly.

This contextual understanding extends to other connected devices you might want to control, such as anticipating changes in your home. “If you add a new device to your home, you can say, ‘Alexa, turn on a new light,’ and it will know what the new light is. It makes things clearer, so if you put in a new smart plug or light, it’s easier to control,” explains Limb.

Another new capability is to respond to multiple requests at once. Not just basic things like “Alexa turn off the lights and lock the door” (to some extent), it’s already doable. It’s even more advanced. “You can say, ‘Alexa, turn on the sprinklers, open my garage door, turn off the outside lights,’ and everything will be fine,” says Limp.

This capability extends to creating routines on the fly – entirely by voice – in the Alexa app without any manual programming. “I set one up for my kid this morning by saying, ‘Alexa, every morning at 8 a.m., turn on the light, play my kid’s wake-up music in his bedroom, start the coffeemaker,'” says Limb. It’s going to pop up as usual.”

Initially, Limp says, the multi-command feature will only work with a subset of device types — including lights, smart plugs, and others. But the team is working towards including everything.

Developers can leverage Alexa’s new cognitive functions and integrate their products and services into this conversational format. Amazon is introducing two tools that allow the new Alexa to control some unique features of third-party manufacturer products that aren’t included in Amazon’s smart home ecosystem toolset. These are called dynamic controller and action controller.

Dynamic Controller makes features such as pre-built scenes for lighting control appear more natural. So, if you have GE Cync colored light bulbs and say, “Alexa, scare it here,” Alexa will know what to do without you having to program a regular program or import scenes into the Alexa app.

Similarly, Action Controller allows developers to add simple actions that Alexa can perform. For example, if you say, “Alexa, the floor is dirty,” Assistant will know that you want the robot vacuum to work.

Amazon says it is already working with GE Cync, Philips, GE Appliances, iRobot, Roborock and Xiaomi on these features. Submit your participation request now. Amazon’s developer blog has more details on the new capabilities and tools.

Limb says this is just the beginning of a new journey for Alexa. “We’ve developed a new generative AI LLM that will — over time — power many areas of Alexa, including new smart home experiences,” he says. “The first bucket is trying to make these everyday tasks easier.” Where it goes next will be an interesting journey.

The new Alexa LLM-powered voice assistant will launch first in preview in the US and will be available to anyone with an Echo device. Amazon hasn’t announced a date for the preview, and the new Alexa LLM-powered smart home features will be part of an additional invite-only preview. You can request a one-time invitation as part of the preview. Amazon says they will be available at a later date.

Updated, Thursday, September 21 at 9:20 am: Added details on how developers can register their devices and services to integrate with Alexa LLM.