20 • AI Engineer @Lenskart_com • 15x Hack Winner 🏆
🐲/acc
Joined March 2020
- Tweets6.5K
- Following1.9K
- Followers2.7K
- Likes32.4K
Mohit Goyal (Harness arc) retweeted
OpenAI and I clearly have a thing going on 🙈. Walked into DevDay Exchange Hackathon New Delhi with my homies @HarshitaBa31 and @Raghuvansyn walked out 2nd position!!!
Not bad for a Sunday. Thank you for hosting us, we will be back for 1st. 💗
Mohit Goyal (Harness arc) retweeted
End of October, end of #KestraHacktober 🎃 22 PRs merged into @kestra_io, from core fixes to the MongoDB, AWS, Serdes and ServiceNow plugins. I spent the month hunting race conditions, lost edits and sneaky data bugs. And I'm not done yet, more PRs on the way ..
@fdelbrayelle :)
3.35B in 7 days..
too low
the highest alpha play right now is so simple most people will never do it...
make a harness that works for YOU
it'll 10x the quality and speed of your AI outputs.
paste this ONE prompt into your agent:
---------start of the prompt----------------
Build me a Langfuse-style observability dashboard from my own AI coding sessions for the last 7 days.
SOURCES (read locally, read-only):
- Claude Code: ~/.claude/projects/**/*.jsonl
- Codex: ~/.codex/sessions/**/*.jsonl (and ~/.codex/history.jsonl)
Never print secrets, keys or full prompt text. Snippets max 10 words, with keys, paths, URLs and long ids stripped.
STEP 1. Write a stdlib-only Python script that computes, per tool:
- sessions per day, user prompts, tool calls by tool name, failed tool calls by tool and error type
- models used, tokens (input, output, cache read/write) per model per day
- tool-call latency p50/p95, overall and per tool; session length p50/p95
- top sessions by tokens, longest sessions, retries (same tool call repeated after an error)
- prompts clustered into task types (build, debug, deploy, research, writing, UI, review, other)
- sessions grouped by project (working directory), with subagent runs flagged
If a log doesn't record something, write "not logged". Never invent numbers.
Save the result as harness-summary.json.
STEP 2. Render a single-file dashboard.html (dark, animated numbers and charts):
- Summary strip: total tool calls, prompts, error rate, tokens, p95 latency
- Traces table: one row per session (tool, project, start, duration, prompts, tool calls, errors, tokens, model); search, filter by tool, sort, toggle to hide subagent runs
- Trace detail: click a session to see a waterfall of its turns and tool calls with ok/error and duration
- Sessions by project, tokens over time by model, a model table (tokens, error rate)
- Latency view per tool, and an error explorer grouped by tool and error type
- Heuristic scores (tool success rate, retries, per-session health), clearly labeled "heuristic"
Cap embedded traces at the top 200 sessions by tokens if the file gets too big.
STEP 3. From the data, write HARNESS.md proposing changes to CLAUDE.md / AGENTS.md:
- Model routing: which model for which task type, based on what I used and what failed
- Tools: keep (high use, low failure), fix (high failure), cut (rarely used)
- Skills: one for every task type I repeated 3+ times, with a trigger line and steps
- Guardrails: my top 3 failure patterns and a check that prevents each
- Budget: my top 5 token sinks and one change that would cut each
Cite the sessions behind each suggestion.
STEP 4. Show me the 5 headline numbers and the proposed diff before changing any config. I'll review weekly and change only what the data shows.
--------end of the prompt-------------
Mohit Goyal (Harness arc) retweeted
Today we introduce Heckler by @MercityAI
Heckler stress tests your AI agents with simulated users and shows you exactly where your agent breaks the rules you gave it.
We ran 3 agents (DeepSeek V4.1 Flash, GLM 5.3 and GPT-6 Luna) on Heckler and over 80% of chats broke a policy.
Constrained decoding already means one thing.
Keep generating, and mask any token the grammar rejects.
ScaleDown's CDMs are a different runtime.
Classify does not write the label. It scores the labels you supplied, softmax-normalizes them so they sum to 1, and returns top_label.
Use that when the schema is closed. Routing, tagging, fields you already named.
Do not treat top_label as abstention. The distribution always has a winner. Refusal is a threshold you set, or an "other" label you added. The model will not invent either.
The useful split is not how fast the loop is.
It is whether you still need a decoder.
You already gave the LLM the answer. So why is it writing it out from scratch?
Introducing a new AI model architecture: Constrained Decoding Models (CDMs)
LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable.
CDMs are purpose-built for 1 task each. They read your input once and skip the token-by-token generative loop that makes LLMs slow and expensive.
Not just a decision model, but a family of 3 CDMs live today:
• Classify: define your labels and get back a calibrated probability + reason for each. Used for routing, tagging, and intent detection.
• Extract: describe the entities in natural language and get back well-formed JSON with confidence scores + citations. Used for web scraping, document processing, and PII redaction.
• Summarize: pass in long text, images, or PDFs and get back a consistent summary under 1,000 tokens. Used for context compaction, metadata generation, and transcript digests.
Built for production:
• 1M tokens of multimodal (text + image) input, 31x larger than Jev
• 77.2% on DecisionBench vs Jev's 72%
• 200x faster, 467x cheaper than frontier LLM APIs
• $0.05/M input tokens. Output is free (yes, even for summarize)
Live on your favorite platforms today:
@OpenRouter @vercel_dev @LiteLLM @LangChain @TrustedRouter @itsArthurAI @useRouterPlus
Get 50M input tokens for free (for the first 50 hours) at link below 👀
Mohit Goyal (Harness arc) retweeted
Our #neatHack submission
Everyone goes on about tokenmaxxxing. But what about when you are on a budget? There have been countless incidents where devs have gone at par with their expected credits budget.
That's why we introduce: Tally, the CFO for your AI Agents.
Just do: npm install -g tally-agents, and it runs in your terminal. You can do it yourself or just let your agent handle the hassle 👀
Built with:
@arithecfo: helped frame Tally’s business around cost per verified task, making agent spending and potential savings measurable. Tally reads the monthly budget, records actual spend by model tier, and sends attempt reports to Ari for cost analysis and hypothetical model comparisons.
@EntireHQ: to track our work while building Tally and save checkpoints of important changes. Tally also uses Entire’s code graph to understand which files a task may affect. If a worker hits its budget limit, checkpoint information helps the next worker continue from where it stopped. When the graph finds no matches, Tally searches the repository instead.
@neatlogs: to track which model Tally chose, how much it spent, and whether each task succeeded. Tally uses those past results to help choose models for future tasks. The traces also helped us debug workers starting or stopping incorrectly. Our latest parallel run automatically recorded both Tally’s decisions and the workers’ activity.
@useOrchestrator: to coordinate the coding agents building Tally, each in its own branch and workspace. Tally also uses it to launch workers on selected models, monitor their token usage, and stop them at budget limits.
@crustdata: We used CrustData to research potential customers for Tally, looking for software companies with 20–500 employees. We reviewed 10 returned companies and kept six as possible prospects for the business plan.
links in comments :)
teapawt.net/bettersayAJ/status/210…
building agents is easy
getting them to production is hard
today we’re launching @neatlogs to fix that
