I just tested the Rabbit R1’s next-gen LAM. Is this really what the company is promising?

The Rabbit R1 has had a bumpy ride this year – it got off to a strong start with a lot of interest from somewhere CES2024and ended up getting a lot of negative reviews (including from us) for not doing what it promised. On top of that, our tests had the major issue that many of the features we were told didn’t work as intended (bugs galore).

To be fair to the Rabbit team, they’ve been hard at work behind the scenes, releasing 16 over-the-air updates to bring new features, fix bugs, and speed up existing features, but the real heartbreaking thing about the launch was that the promise of this Large Action Model (LAM) AI tool never really came to fruition.

As founder and CEO Jesse Lyu told me in a two-hour interview (I have a lot more to share this weekend), the pre-existing system relied on a smaller set of “human records.” That means there were some things it could do, but it was quite limited in scope compared to the big promise it made at the start.

Well, Rabbit is back with a new generation of LAM — it’s launching in beta as the LAM playground on October 1 — and I got a chance to try it out and see what it’s like. I can’t show you what it looks like, but I can talk about it.

How does the new Rabbit LAM playground work?

(Image credit: Future)

That’s exactly what Rabbit calls a Generic Website Agent; something that can do things for you, either through a text prompt inside the Rabbit Hole or (the one you’ve all been waiting for) making natural language requests to your Rabbit R1.

So initially, everything runs through the virtual machine-like system that Rabbit users are probably accustomed to interacting with when logging into their accounts. From there, if you make a request for something online—Jesse’s example was adding a 12-pack of Diet Coke to an Amazon shopping cart—LAM goes to work.

You can watch it happen in real time because a vision model observes what is happening by taking screenshots, analyzing them, and directing a script to interact in a certain way by understanding what is happening in the browser window. This includes turning off cookie prompts and can be resilient to future UI changes.

Things I like

(Image credit: Future)

In fact, there’s only one big thing I really like about it, and that’s that Rabbit delivers on the promise it made months ago. From visiting a property website and finding all the houses in South London for under £500,000, to a Word-In this game that lasts three rounds, you watch your wish being fulfilled in real time.

No one mentioned buying things on stage at CES as claimed, but it is possible if you give the Rabbit R1 the right instructions and log into the retailer’s account. Another big theme we got a response to was the agent approach to travel – giving you a plan and actually taking action to book flights and experiences.

“We didn’t know we needed a travel agent license to book in the US. That was our mistake,” Jesse quickly commented when I asked him about the feature. But now, with LAM built the way it is—where you can see it working and interact with it directly and interrupt what it’s doing—it frees them from needing that license.

(Image source: Tom’s Guide)

Another big feature of this playground is that LAM is also trained on a desktop operating system – namely Linux. These are still very early days, but what Lyu managed to show me was a prompt to open up the operating system’s equivalent of Microsoft Word and write a poem. What this means is that with further development, it could essentially take over and create a task that you wouldn’t bother doing yourself, such as creating a presentation deck.

Watching LAM do this work in the background once again is a sign that Rabbit is heading in the right direction. The tool that was basically built here and shown to work was what was talked about months ago. It was quite the makeshift effort, but it was the first sign of the AI tool I was looking for. Better late than never, as the old adage goes.

Areas for improvement

(Image credit: Future)

There was currently clear transparency about the issues with this beta (emphasis on beta) because I encountered a bunch of issues while working. The first thing I noticed was that it was slow. On average, a new instruction is issued every 5-6 seconds after a screenshot is taken and analyzed.

And when I say task, I mean drill down to each instruction – opening a website, then accepting the cookie prompt, clicking in the text box, entering text in the text box, and pressing enter. All of these take about that amount of time for each one. Rabbit understands that there’s a lot of work to be done here in terms of making this faster and reducing latency.

Second, as you would expect from any beta, there are bugs. For example, in that poem by LAM, when he opened a Word document to write, the model hallucinated and gave us about four pages of mixed-up letters and numbers. Again, Lyu made it clear that these were going to happen and that the beta test was specifically to find these bugs.

(Image credit: Future)

Finally, let’s take a second look into the future here. One hurdle that every AI company faces is whether the companies their models interact with actually agree to play games.

In Rabbit’s case, the user-friendliness of a general AI tool that works across the entire web may be huge, but it’s equally reliant on those websites allowing the bot to visit and navigate the website. What happens if the big players say “no”?

And Lyu is aware of the task ahead to secure those permissions. He talked about using Linux for the operating system portion of LAM and how it could work with Windows or macOS, but that would require working out a comprehensive licensing agreement.

Just like that Open AII can imagine a scenario where money would need to change hands to get permission to visit certain big brand sites. It’s an interesting mountain that’s not here for Rabbit to climb yet, but it’s definitely on the horizon.

Don’t call this a comeback?

(Image credit: Future)

It’s been a while since the May launch, but from what I’ve heard and tested, I’m quietly confident that Rabbit is on the right track to becoming the LAM we’ve all been dreaming of after Jesse made a big splash with his CES announcement.

Will it clear the bad taste in the mouths of those who were torn between the expectations and reality of buying the R1 without these alleged features in the first place? That remains to be seen. But taking an action-over-words approach to the initial response is definitely the smarter approach, and I’d like to see it accelerate and evolve into something cross-platform and actually quite clever.

Other things that Rabbit brings include more natural language interactions with the R1 (this was last announced publicly), and an Alpha test of Teach Mode is currently running. From the time I’ve used it, it feels almost like a more streamlined version of Apple’s Shortcuts app — instead relying on natural language prompts rather than manually inputting each step of the process.

The biggest question that Rabbit doesn’t answer satisfactorily is whether, in an age of powerful AI phones, a smarter version of the AI gadget would make sense. iPhone 16, Galaxy S24 And Pixel 9.

Lyu told me his vision for the future goes beyond these phones, with the ability to be cross-platform — instead of being limited to Android or iOS like those phones, it brings in intermediary AI to fulfill any request, regardless of the operating system required to do so. But the average user will actually care What do you think about this? This is arguably the most important topic — especially since you can see that while the Rabbit is the first to go with its beta, phones could soon have similar features.

According to Lyu, there is still a lot of work to be done for this game to be released publicly, which is expected to be released in about six months, but if an OTA update is released every week to fix bugs and improve the experience, there is a possibility of achieving it.