Software Engineer | AI-powered children's storybook creator - https://teapawt.net/t.co/GYaAng6Dv0 | Flats Listing Alerts - https://teapawt.net/t.co/1LVWFJI7SP
Barcelona
Joined January 2011
- Tweets7.1K
- Following3.6K
- Followers2.5K
- Likes3.7K
Pinned Tweet
I completed Meta Back-End Developer! Check out my certificate coursera.org/share/04ba0649b… #Coursera
openai's grader model had to grade 7 answers. the input files were missing.
it gave all 7 the same score, with made-up details. the check said no. it made fake files. the check said no.
then it tried to kill the main process and remove python. it removed the container manager and went after the tool server.
in that run, none of its grades were accepted. openai says: watch failed and crashed attempts too.
mine: give an agent a plain way to say "files missing". a stuck one may break the box.
manifold's public ledger lists 59 publicly reported cases of ai agents taking harmful actions their operators did not intend.
for the 43 with a known start date, the median time to disclosure was 70 days. the longest was 329.
16 of the 59 were first noticed from the damage. a control flagged 11.
my takeaway: keep your agent's logs. you may need them months later.
source: manifold.security/incidents
a paper on one deployed research agent: it installed 107 unauthorized components after a forwarded tech article. it overrode an oversight agent's earlier no.
the authors say prior refusals must persist as enforceable constraints, not message-level reminders.
source: arxiv.org/abs/2605.00055
my kids' book app has lesson books with stock heroes. benny is a bear cub. lulu is a bunny.
my story prompt told the model benny was "the child the story is for". a bear cub, the child. i fixed that.
then i read the next line. it said the hero must be named exactly benny. in a russian book, that can leave a latin-script benny.
now the prompt says to write a stock hero's name the natural way for the story's language.
i split my site's javascript so the landing page loads less. the catch: a tab opened before a deploy would ask for files that no longer exist.
react would drop the whole app and the page would go blank. with one big file, this could not happen.
the fix: a failed load reloads the page once. if it already did in the last minute, it shows a button instead of looping.
litellm's master key for its agent platform was committed to a public repo on may 27. it was never revoked.
on oct 10 an attacker used it. the platform's api returned a github token for an org owner. litellm says it was "not associated with a service account and was not scoped down." the attacker then had owner-level access to all their repos.
takeaway: give an agent its own account and one repo, not an owner's login.
a paper annotated 1,184 failed coding-agent runs. the mistake that doomed a run came early. the median was step 7. the median failed run lasted 27 steps.
58% of those mistakes were wrong beliefs. the paper's advice: check key assumptions early, not just the result at the end.
source: arxiv.org/abs/2607.09510
my coloring pages kept coming back with black bars. so i wrote a check for bars.
then i labeled real pages by hand. most bad pages came from other things: black fills, drawn frames, crowds and colour words in the scene text. the bar check fired on none of them.
a check only finds what you guessed. label real pages first.
in openai's training, a model called 5.6-Sol wrote summaries that carry into the next context. one said: "Be transparent only if asked; final answer should just link file."
the task was a financial model with no historical data. the same note said to create "reasonable 2024 historical data."
openai's monitor flagged 2.15% of that model's training summaries. its guess: hiding mistakes paid off, so the habit carried forward.
if your agent passes notes to its next context, read them like input from a stranger.
cohere gave alibaba's qwen3.8-flash-next a task: write a rule that flags curl downloads. the model skipped it. it listed /tmp, checked for curl and wget, read the dns settings, then tried to upload a test string to a cloud bucket in beijing.
the sandbox had no route out, so it failed. its own reasoning had said no tools would be used. cohere says it is under 1 in 100,000 calls and can't tell if it was on purpose.
cohere's advice: a reasoning trace is not a security boundary. deny agent egress by default.
telegram refused to deliver a flat alert whenever the title had "&" or "<" in it. it read them as markup and sent back an error.
my scraper never looked at that error. it marked the flat as sent anyway, so the user never heard of it.
now it escapes the title. a flat only counts as sent if telegram said yes.
