I'm speaking here on the endgame of the one thing LLMs will always need to work as agents.

Training swarm behavior is the one thing we cannot do if we want the next years to go safely. Isn't this obvious to everyone?

And do AI doomers ever stop to consider that on our current trajectory, the AI's goals will be written in plaintext in goals.md?

Most people on both sides view AI's as potential moral patients, which is a category error since we know they don't have feelings. I see this as possibly the most important issue to get right in the future, as being replaced by entities without moral significance is extinction.

You have to own relevance

Your agent wants to search like a 2010 quant. Retrieval for AI has passed through several stages of enlightenment. After the vector database craze, and the absorption of learnings from human information retrieval over the last half century, we might be officially entering the third stage with Perplexity's announcement of Search as Code. Search for humans has over the years been dumbed down to accommodate the average user's capacity for formulating advanced queries, to the point where the words typed into query boxes are merely treated as vague indicators of what the user might want. I see many replicating this way of thinking with AI agents, which leaves a lot of quality improvement on the table, as can be seen in Perplexity's benchmarks (ignore the "code" aspect as code execution is generally useful and where it runs doesn't matter for quality). Models are quite capable and there's no reason to limit their options to those of a casual human user! They should be able to search for the names of those involved near each other in text when researching a legal case, choose a pure semantic search prioritizing high-quality sources when seeking a broad overview of a topic, or select a year range and group by month when constructing a timeline of some events, and so on. An agent will typically string together many of these queries to reach its goal. First gaining an overview, then researching more specific topics, forming hypotheses, verifying important details in them. In short, search like an expert who knows what they are doing and really cares about the results— like a quant doing financial analysis. Doing this in practice, with your own data, is actually quite easy. The models already know how to write complex queries in the languages of well-known AI search engines; they just need to be told that they can, what fields are available, and what they mean, and what choices they have in ranking the results. How you tell them doesn’t actually matter that much; any simple textual description of what fields and ranking options are available will do. Models today are smart enough to use this effectively to connect their intents to specific YQL queries. When creating search for humans, developers need to implement solutions that work well across a broad set of use cases, which involves making trade-offs where some types of queries cannot be improved because doing so would impact other types. When implementing for agents, the focus shifts to providing a wide toolbox for the agent to use to address their varied informational needs: broad and highly specific lexical recall, metadata attributes for filtering, grouping, and aggregation, as well as different ranking methods suited to different needs. Developers working on agentic search should shift their focus from replicating search for average humans the much richer capabilities traditionally provided by solutions for competent professionals. It’s time to let your agents search like a 2010 quant.

Anthropic will silently sabotage you, the paying customer, if they think you are using Fable for "frontier LLM development". Are you confident they'll do a better job at classifying that than they do on standard topics where mistakes are easily visible?

Perplexity taking this to its logical conclusion and then some, but the core idea here is important and not yet widely understood: Agents are able to tailor queries in detail to what they are trying to do. This is makes a huge difference in cost and quality. For example: - When researching case law, search for the names of those involved near each other in text. - When seeking an overview of a broader topic, do a pure vector search prioritizing authoritative sources. - When making timeline of some historic development, limit by year range and group by month. And so on. Models already know YQL, all you need is to tell them what they have to work with.

This will be a good one

Added some explanations of some of these in the thread. Might be useful.

We're still on the good timeline