I did it. I found first "real critical issue" on @HackenProof . I am looking forward to presenting story behind it on @ETHWarsaw 😄 By the way, you can meet me today (30.06) in Cannes on lu.ma/app26j6x event and later on EthCC

I'm building an AI startup called ResearchTech, we automate risk assessment and due diligence processes. During last 30 days for coding alone I used 13.1B input tokens and 77M output tokens. We move lighting fast, soon some of our products will be finally publicly available, they will be great. We are hiring, if you are using AI coding tools and can build workflows which can take hours and they work well in practice, you are perfect match. We are no longer building code with AI, we are building coding frameworks for AI to build entire products for us, that's the only way to go, because during out analysis we process 100M+ tokens, it's no longer manageable to do it manually, so the only option is create AI agents to do it for us. Can't wait to share more, stay tuned.

Most posts/videos about AI are out of touch. Today I found an exception to that: a video about "The AI Scaling Problem" - youtube.com/watch?v=COOAssGk… I highly recommend watching it if you prefer to have a more realistic view of AI instead of following "hype". Props to @ejmejm1

I'm looking for a person experienced with using LLMs and creating prompts. If you can do it better and faster than me then I won't have a problem to pay you $100/h. I created recruitment task with an example problem I'm dealing with, if you know how to deal with those problem and want to work on very interesting projects just contact me and prove it. Task is on github: github.com/bbarwik/ai-summar… I don't care what is your background, current experience or if you know how LLMs work. I need someone who can work with the latest LLMs, long context, understands how cache/structured output works and can write prompts.

See you tomorrow at EthWarsaw! Just one thing, there was a change in plan because I was not yet allowed to publish story about my biggest issue which was allowing to steal $6M+. So I'll be doing presentation about "AI Audits - case study & more - 2,800 AI-Generated Tests, 22 Findings".

My personal review of executing a hard, real-world programming task with different models. TL;DR: only gemini 2.5 pro with deep thinking is somehow useful. I’m working on a few AI projects that use Prefect, Laminar, and multiple LLMs. To simplify development, I recently merged the core pieces into a single open-source package called ai-pipeline-core, available on GitHub. I have access to Gemini 2.5 Pro, GPT-5, Grok-4, and Claude Opus, and I primarily use Claude Code (MAX) for implementation. I’m generally frustrated using AI for coding: it often generates low-quality, hard-to-maintain code that needs major refactors. It works only when the instructions are extremely precise; otherwise it gets overly verbose, turning 100 lines into 300+. To mitigate this, I use one model to produce a detailed plan, then feed that to Claude Code for implementation. I was using GPT-5 for planning, but after some issues I decided to try Gemini 2.5 Pro with Deepthink. While migrating more features to ai-pipeline-core, I set up a comparative prompt for the LLMs: I’m working on three projects: ai-pipeline-core, ai-documentation-writer, and research-pipeline. Initially there was only research-pipeline, but I wanted to reuse its approach elsewhere, so I migrated the core code into ai-pipeline-core, which several projects now use. I want to keep improving ai-pipeline-core by moving more common functions there.Goals: • Have ai-pipeline-core own core deps: documents (JSON/YAML), Prefect, Laminar, and OpenAI (AI interactions), so other projects don’t import them directly. • Instead of importing Prefect in other projects, use: from ai_pipeline_core import task, flow. • Prohibit direct imports of Prefect and Laminar in other packages (similar to how I already prohibit importing logging). I’ve included some files from the Prefect library. • Move more shared components into ai-pipeline-core (e.g., things living in __main__.py across packages). • Create a custom decorator for flows so they behave consistently. Call it documents_flow(project_name, documents: DocumentList, flow_options) -> DocumentList. • Add tracing by default to my flow, task, and documents_flow. Add trace: Literal["always", "debug", "off"] = "always". • Add function args trace_ignore_input, trace_ignore_output, trace_ignore_inputs, trace_input_formatter, trace_output_formatter (for the tracing decorator).Tests: • Write tests to validate that my wrappers’ signatures stay compatible with Prefect/Laminar, so updates upstream get caught if I need to adjust my wrappers.Deliverables: • A detailed plan to achieve the above, with comparisons of approaches, plus suggestions for what else to move into ai-pipeline-core. Keep the core principle: make everything simpler, less code is better. The end goal is quickly spinning up new projects (like ai-documentation-writer and research-pipeline) using a ready-to-use ai-pipeline-core. • Code should target a principal-level Python engineer. No unnecessary comments/explainers/logging—just clear, self-explanatory code.ai-pipeline-core is open source: github.com/bbarwik/ai-pipeli…. ai-documentation-writer will be open sourced; other projects will not. I provided an extensive context prompt ~600k characters long (~100–150k tokens): the full source of ai-pipeline-core, ai-documentation-writer, key parts of Prefect (src/prefect), and ~10k LOC from private repos. I tested this prompt on every major model I have access to: - gemini-2.5-pro - gemini-2.5-pro-deepthink - gpt-5 (with “thinking”) - gpt-5 with deep research - claude-code with Opus 4.1 - opus-4.1 on claude.ai - grok-4 Then I fed the seven anonymized results back to each model and asked them to analyze/compare the outputs. A meta-consensus emerged: most models said GPT-5’s plan was best, with the Gemini variants usually 2nd/3rd. Here’s my manual review: Claude Code with Opus 4.1 — 4/10 Disappointing. It started rewriting the entire codebase, ignored my style, and generated lots of useless code. Even with my strict CLAUDE.md, the output quality was poor. Opus 4.1 on claude.ai — 6/10 Much better at planning than the dedicated claude-code model. It didn’t follow all instructions and used anti-patterns I dislike (e.g., imports inside functions), but the snippets were elegant. The implementation could’ve been ~50% more concise, but it was a solid improvement. Gemini 2.5 Pro with Deepthink — 8/10 The winner. It followed instructions almost perfectly. A few questionable choices (e.g., wrapping stdlib/Prefect/Laminar imports in try/except), but overall correct, tight, and free of unrequested features. I’ll use this plan for the final implementation. Gemini 2.5 Pro — 5/10 Good plan, weaker implementation. It optimized for brevity, sometimes leaving placeholders like # ... other prefect args, and didn’t complete everything requested. GPT-5 — 3/10 Overly complex, bloated with features I didn’t ask for. Hard to follow and stylistically rough (e.g., caller = str(f.f_back.f_back.f_globals.get("__name__", ""))) plus unnecessary import try/except blocks. GPT-5 with Deep Research — 6/10 Surprisingly good high-level plan. Less step-by-step, more strategic overview—useful as a starting point if I write detailed steps myself. Grok-4 — 3/10 Failed to understand the task. It felt like a downgraded model: after ~10 seconds it produced a very short, mostly irrelevant plan. Decision: I’m going with Gemini 2.5 Pro with Deepthink—it fits best. Only real downside: generation time. It might have been faster to write a detailed prompt for Claude Code than to wait for Gemini to generate. Takeaway: Current LLMs still struggle with high-quality, maintainable code on large, existing codebases. Senior developers and especially cybersecurity experts are safe for now 😄

At 15:00 CET / 13:00 UTC I'm hosting a webinar about "Top 10 Attack Paths Your Devs Are Blind To". This webinar will be about places where I search for issues in the first place when I'm doing audits or bug bounties. I found over 70 critical issues during last 3 years while doing audits and I'm top 3 bug hunter on @HackenProof . If you are interested in cybersecurity I recommend to join and watch it ;) lu.ma/5p4c8ost

I’ve built a tool that will critique and challenge your project to help you uncover hidden threats. Made for investors, VCs and startup founders. From my experience, it’s very hard to get constructive criticism when you’re deep in a project. It’s much easier to hear that what you’re doing is great and that you should follow your dreams, plans, and ideas. That’s one of the main reasons some of my own projects failed. They had a lot of flaws and issues but no one told me about them, all I heard was a positive feedback. To address this, I built a tool that acts as a “risk analyst” and does a deep review of projects. I spent a month on it, and the first version is ready. It uses top AI models like Gemini 2.5 Pro, o3, Grok-4, and Claude 4. It’s integrated with 10 data providers. In a single analysis, it visits around 2,000 pages, consumes ~100M input tokens, and generates ~5M output tokens. It’s already huge—and it’s just the first version. I’m currently testing it in practice before releasing it to everyone. It’s a great tool for investors, VCs, and startup founders, you can get a comprehensive due diligence report within a few hours. All it needs as input is a pitch deck. If you’re a startup founder or investor and would like a free analysis of a project, just send me a message. In return, I ask for detailed, constructive feedback 😄. In the future, reports will cost around $100–$200 USD (TBD). I’m attaching a report from my previous startup idea. It takes about 4 hours to produce a report like this for your project — just send me a pitch deck or post it in the comment 😉 Here's the report - s.bbarwik.com/privateai_repo…

Today @HackenProof finally updated leaderboard so it includes my bugs from February. Top 3 secured 🥉. The goal is the same - #1 place, so lets wait for issues from March and April to be finally resolved 📷 I'll be posting about them soon, two criticals are waiting. Stay tuned!

I was recommending to do it during SuiSummit in Denver, I am glad they finally did it! I hope that other protocol will do the same.