Computer scientist building the infra AI needs to scale around search engines, databases, and distributed systems.
Earth
Joined August 2012
- Tweets11.7K
- Following384
- Followers780
- Likes63.4K
Searching your documents by meaning alone, as vector search does, can leave your AI agent without the page that holds a product code or acronym from the customer's question.
A search by meaning finds pages that say the same thing in other words, while a keyword search finds pages with the exact term, so each one can miss what the other finds.
Ask your coding agent whether your AI agent's search tool merges a keyword ranking with a vector ranking.
This guide shows how you can merge the two rankings and how adding your customers' own words to each document helps both searches: narsil.sondelali.com/docs/ag…
Is an app that wraps another company's AI model a business, or a feature that the company can add next?
ALT On a dark square card, the sentence 'Your customers pay you to solve their problem, so a company that copies your feature will usually have to win them one at a time' appears in large type, with 'will usually have to win them one at a time' in blue.
Every fake account that a bot creates through your sign-up form can add to what you pay your database and email providers. If you deploy your app as serverless functions that share no memory between requests, which of these will you use to rate-limit that form?
→ Upstash Rate Limit
→ Vercel WAF
→ Cloudflare
→ Something else
Upstash Rate Limit is a rate-limiting library that works over HTTP, which suits serverless functions: upstash.com/docs/redis/sdks/…
Attackers are exploiting a flaw that lets them crash your Node.js server, read its memory, or run code on it with compressed data that your app unpacks.
Ask your coding agent to move the app to Node.js 22.23.4, 24.22.1, 26.12.1, or a later release, then rebuild and redeploy it.
CISA added the flaw to its list of exploited vulnerabilities on 27 October and told organisations to follow the vendor's instructions, which for Node.js are in this advisory: nodejs.org/en/blog/vulnerabi…
Your sign-in link can expire before a customer with a work email address clicks it.
Some companies' email security, such as Safe Links in Microsoft Defender, can open the links in an email before the customer does. Supabase's documentation says that this visit uses up the sign-in link at once.
Before you release your sign-in emails:
→ Put a one-time code in the email, or link to a page with a button that signs the customer in.
→ Request each link in the email once from a script, as a scanner does, then sign in from that email yourself.
ALT A three-step diagram on a dark square card shows your app emailing a sign-in link to a work address, the company's email security opening the link first, and your customer finding that the link has expired when they click it. The second step is in blue.
Supabase shows the email template for each of those two fixes in its documentation: supabase.com/docs/guides/aut…
Which of these will you use to find the step where new users give up on your sign-up, while you stay on a free plan?
→ PostHog
→ Mixpanel
→ Amplitude
→ Something else
When one customer reports that they can see another customer's data, the same flaw can let many more people see it.
At the South African bank FNB, for example, 696 customers whom it had wrongly marked as staff could open 88 people's home loan applications. After FNB treated two such reports in 2022 as isolated cases, the regulator fined the bank R1 million this week.
Before you close a report like that:
→ Find every account that holds the same access, then keep it only on the accounts that your team granted it to.
→ Sign in as an ordinary customer, then check that your server or database returns an error or an empty result for every request for staff-only data.
TechCentral describes the regulator's findings, including the dates on which two customers alerted FNB: techcentral.co.za/fnb-fine-i…
A CUSTOMER WHO PAYS YOU FROM ABROAD CAN COST YOU AN EXTRA 2.5% OF THE PAYMENT.
Stripe writes on its pricing page that it charges '2.9% + 30¢ per successful transaction for domestic cards', plus '1.5% for international cards' and '1% if currency conversion is required'.
Check what your payment provider charges for a card from another country, then set your prices abroad so that they cover it.
Will you charge customers abroad in their own currency, or in yours?
After you rename a tool or change its arguments on your server, your AI agent's calls to that tool can start failing.
That can happen when the tool list that your agent sends to the model still contains the old name or the old arguments.
Ask your coding agent whether your app builds the model's tool list from the same definitions that your server checks each call against.
The guide in the first comment shows how you can make your agent stop at startup when its tool list differs from your server's.
In the guide, the server returns a code with each refusal, so you can tell whether the model can repair the call by rewriting it: sirannon.sondelali.com/docs/…
Will your first hire be a junior developer who learns from your coding agent's mistakes, or a senior who can catch them?
ALT On a dark square card, the sentence 'Make your first hire a developer who can catch the mistakes in the code that your coding agent writes' appears in large type, with 'a developer who can catch the mistakes' in blue.
Anyone on the internet can try passwords on a Postgres database or an admin dashboard that accepts connections from any address. Which of these will you use so that only your team and your app's server can reach yours?
→ Tailscale
→ Cloudflare Zero Trust
→ WireGuard
→ Something else
An AI support agent obeyed an instruction hidden in a web page and showed another customer's address and orders in 28 of 100 tests.
Arize stopped every leak in its tests once its code limited each customer tool to the signed-in customer, so give your agent's tools that limit.
New users can sign up and then never find your sign-in email in their inbox.
Gmail, for example, can mark an email as spam or reject it when the sending domain has no SPF or DKIM record. If you use Supabase, its built-in email service sends only to the members of your project's team.
To get those emails into your users' inboxes:
→ Send the app's emails through a provider such as Resend or Postmark, from your own domain.
→ Add the SPF, DKIM, and DMARC records that the provider gives you to your domain's DNS.
→ Sign up on a preview build with a new Gmail address, then release only once you find the sign-in email in that inbox.
When your AI agent has to search the web for your users while you stay on a free plan, which of these will you use?
→ Tavily
→ Exa
→ Brave Search API
→ Something else
Customer data that you promised to delete can still leak in a breach, if your app never deletes it.
For example, Keystone, which sells crypto hardware wallets, promised to delete shipping details 180 days after each order. After a breach of its online store, however, a customer who ordered more than two years earlier says that they received Keystone's breach email.
Before you put a deletion date on your privacy page:
→ Ask your coding agent for a scheduled job that deletes or anonymises customer data once that date passes.
→ Run the job on a test record that is past the date, and check that the customer's details are gone.