Anthropic’s New Model Excels at Reasoning and Planning—and Has the Pokémon Skills to Prove It

When Claude 3.7 Sonnet played nan game, it ran into immoderate challenges: It spent “dozens of hours” stuck successful 1 metropolis and had problem identifying nonplayer characters, which drastically stunted its advancement successful nan game. With Claude 4 Opus, Hershey noticed an betterment successful Claude’s semipermanent representation and readying capabilities erstwhile he watched it navigate a analyzable Pokémon quest. After realizing it needed a definite powerfulness to move forward, nan AI spent 2 days improving its skills earlier continuing to play. Hershey believes that benignant of multistep reasoning, pinch nary contiguous feedback, shows a caller level of coherence, meaning nan exemplary has a amended expertise enactment connected track.

“This is 1 of my favourite ways to get to cognize a model. Like, this is really I understand what its strengths are, what its weaknesses are,” Hershey says. “It’s my measurement of conscionable coming to grips pinch this caller exemplary that we're astir to put out, and really to activity pinch it.”

Everyone Wants an Agent

Anthropic’s Pokémon investigation is simply a caller attack to tackling a preexisting problem—how do we understand what decisions an AI is making erstwhile approaching analyzable tasks, and nudge it successful nan correct direction?

The reply to that mobility is integral to advancing nan industry's much-hyped AI agents—AI that tin tackle analyzable tasks pinch comparative independence. In Pokémon, it’s important that nan exemplary doesn’t suffer discourse aliases “forget” nan task astatine hand. That besides applies to AI agents asked to automate a workflow—even 1 that takes hundreds of hours.

“As a task goes from being a five-minute task to a 30-minute task, you tin spot nan model’s expertise to support coherent, to retrieve each of nan things it needs to execute [the task] successfully get worse complete time,” Hershey says.

Anthropic, like galore different AI labs, is hoping to create powerful agents to waste arsenic a merchandise for consumers. Krieger says that Anthropic’s “top objective” this twelvemonth is Claude “doing hours of activity for you.”

"This exemplary is now delivering connected it—we saw 1 of our early-access customers person nan exemplary spell disconnected for 7 hours and do a large refactor,” Krieger says, referring to nan process of restructuring a ample magnitude of code, often to make it much businesslike and organized.

This is nan early that companies for illustration Google and OpenAI are moving toward. Earlier this week, Google released Mariner, an AI supplier built into Chrome that tin do tasks for illustration bargain groceries (for $249.99 per month). OpenAI precocious released a coding agent, and a fewer months backmost it launched Operator, an supplier that tin browse nan web connected a user’s behalf.

Compared to its competitors, Anthropic is often seen arsenic nan much cautious mover, going accelerated connected investigation but slower connected deployment. And pinch powerful AI, that’s apt a positive: There’s a batch that could spell incorrect pinch an supplier that has entree to delicate accusation for illustration a user’s inbox aliases slope logins. In a blog station connected Thursday, Anthropic says, “We’ve importantly reduced behaviour wherever nan models usage shortcuts aliases loopholes to complete tasks.” The institution besides says that some Claude 4 Opus and Claude Sonnet 4 are 65 percent little apt to prosecute successful this behavior, known arsenic reward hacking, than anterior models—at slightest connected definite coding tasks.