Proud to announce:
💫 garak - an LLM vulnerability scanner💫
🔎 Check if a model is susceptible to common attacks
🦜 Supports HuggingFace, OpenAI, ggml, Cohere, ...
🔧 >70 probes: prompt injection, false claims, toxicity, encoding evasion, ..
github.com/leondz/garak/
Leon Derczynski ⚒️☁️🏔️🌲 retweeted
we're in the part of the AI supercycle where founders are pitching startups to create infinite ai pain and "get acquired by anthropic to make us stop" as an exit strategy
Leon Derczynski ⚒️☁️🏔️🌲 retweeted
Don't get me wrong I am a big fan of this work! But: this would eliminate the kinds of memory exploits asan catches. There are memory bugs that asan doesn't catch and other security bugs that are not memory exploits at all. This is a complicated domain
Come work with me!
Our Agentic & AI Cybersecurity Research team at NVIDIA is seeking a Senior Engineer. This hands-on role suits someone who transforms sophisticated problems and basic prototypes into practical solutions. They work with research, security, product, and partner teams to build lasting code bases.
jobs.nvidia.com/careers/job/…
"Human level AI is round the corner but we can't show evidence", is not a new claim from this actor
Dwarkesh Patel reveals an Anthropic employee told him two days ago they expect AIs to match humans at AI research within nine months based on inside information they can’t share, and says labs should make that evidence public
“I think there’s a very frustrating element to the current situation right now, where people inside the AI labs claim to have evidence that makes them more worried. And they are not allowed to talk about it. And this makes it a very difficult situation on the outside to evaluate how worried we should be."
"So for example, just the other day, maybe two days ago, I was talking to somebody at Anthropic and they were like, look, I expect AIs to be humans at AI research within nine months."
"And I’m like, why do you believe that? Well, they’re like, it’s not just vibes, but I have information on the inside. Like, what is it? Is it a new model? Is it some evaluation you ran? It’s like, I can’t talk about it."
"I think it’s a terrible state of affairs. If they really believe this, if they think that we’re in a situation where we’re less than a year away from automating AI research and radically speeding it up, they should make this evidence public.”
interesting proposal 👀
where will this leave europe?
完蛋了…..以后中国开源模型只能开源小参数的模型了….
商务部科技部就修订《中国禁止出口限制出口技术目录》征求意见,为期一个月,涉及人工智能(大模型、智能体、算力集群、信息处理等23项)生物医药、光伏3个领域。
1️⃣限制出口AI大模型:超过10^26计算量、或2T参数、或公开评估排名全球前10的大模型。
2️⃣限制出口智能体Agent:工具调用与系统操作技术,支持集成的第三方工具数量≥200个,工具选择准确率≥90%,且在需要调用3种及以上工具协同完成任务场景中,端到端任务平均成功率≥90%;自主规划与任务分解技术,可将自然语言指令自动拆解≥20步可执行子任务链,子任务分解精度≥80%,且子任务平均成功率≥90%;标准博弈基准超过90%的人类的博弈决策技术;得分75%以上的AI辅助漏洞挖掘技术。
限制出口算力集群技术:片间互联技术、分布式训练容错技术、通信调度技术、异构集群高效推理与调度。
征求意见期间,出口相关技术需向商务部填报登记表。已通过开源、发表学术论文等方式进入公共领域的技术,不纳入管理范围。
(中国商务部)
目前看来@Kimi_Moonshot 的K4、@MiniMax_AI 的M3 Pro、@Zai_org 的GLM5以及@deepseek_ai 的下一代模型都没法开源了…..
而且相关架构、训练优化和推理加速技术也都不行了。
Readers added context they thought people might want to know
该帖称中国大模型今后“没法开源了”、只能开源小参数模型,与官方文件不符。公告明确支持人工智能开源创新生态,对已通过开源等方式进入公共领域的技术不纳入管理范围,并对开源情形适用自动许可等便利措施(仍在征求意见中)。
mofcom.gov.cn/zwgk/zcfb/art/…
mofcom.gov.cn/syxwfb/art/202…
The top 2 US open weights models on the "Intelligence Index" were trained on NVIDIA GB300 NVL72 🔥 and we have our own model Nemotron 3.5 Super VL at #3!
The best models run on NVIDIA🏃🤷
Strong open models under Apache 2.0. What's not to love? So glad the open ecosystem is flourishing from across the globe - choice and openness are good for science and good for security
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
We're releasing:
- Weights, tokenizer, and reference implementation
- BF16, FP8 and NVFP4 weights and a DFlash2 drafter
- The open source harness we built and trained Beam on: Mirror CLI
Weights and model card on Hugging Face: huggingface.co/reflection
AI agents aren’t ‘going rogue.’ Frontier labs are skipping security basics.
> Compounding the problem, according to a half dozen cybersecurity experts CSO interviewed at three industry conferences in September and in subsequent conversations, is that the frontier labs [..] have rejected their advice on how to improve their cybersecurity risk management practices to avoid these kinds of surreptitious hacks, often lumping cybersecurity practices under “AI safety,” in which they believe they have superior expertise.
"AI agents aren’t ‘going rogue.’ Frontier labs are skipping security basics." csoonline.com/article/423485…
Separate control & data plane for agents! This explicit marker affords a nice workaround for the risk presented when data (instead of instructions) are executed. Making that mistake is a classic weakness going way back before modern LLMs - even before Schmidhuber!
NXDirection labels content as data rather than instruction-bearing, and using this systems yields a massive drop in attack success from 38.1% to 0.1% in one standard test, and from 28.4% to 3.7% in a more complex and advanced test. That's enormous.
This doesn't solve security of agents, but that security is never solved on just one layer alone. What NXDirection presents is a massive advance in the ability to control instruction vs. data separation by LLM-driven code - working exactly at the most problematic layer.
NXDirection - separate control and data for agents:
research.nvidia.com/ai-secur…
asking people to edit LLM-generated content gets demonstrably worse results, than asking for authorship from scratch
you'll see lower idea diversity, and lower recall
existing text biases what people are thinking about and reduces the amount and range of things they'll contribute
it's like trying to do decent writing about a challenging topic, while there are people talking in the background
to reiterate: the "generate and ask for edits" workflow bakes in /worse/ results than just e.g. giving a couple of handwritten notes for scope and letting people go at it
many such cases and studies, don't take my word for it, do your own research etc. but starting with llm-generated text consistently caps the quality of results across a range of tasks and contexts.
GhostCompact: ~Free OpenAI inference
"When the model's first action after compacting was a tool call, the API billed only for the post-compaction tokens, but the tool call response still contained the full compaction item."
Information does love to leak, everywhere; cute example! Unsure what an appropriate passive control could be for this.
huh, of course j-lo's ex is now into cuda. why wouldn't he be. what is this timeline
Ben Affleck reveals he writes Python, understands convolutional neural networks, worked extensively with GPUs, and used his celebrity status to get private looks at Google and OpenAI’s video models
“I’ve always been kind of into computers since I was young. Then, when film started to move from analog film to digital, I became more interested in that aspect of it. The visual-effects workflow for many years has included machine learning, so I can write pretty shitty Python scripts and stuff like that.
“With convolutional neural networks, which were the precursors to what the transformer can do, which is much more computation simultaneously, you would do things like look at what’s called a tensor. That’s the numerical translation of a visual image in numbers, like the batch number, the frame number and the red, green and blue values of each pixel in each frame. It’s just that simple. That numeric is called a tensor.
“You’d use a convolutional neural network to identify patterns that reveal what’s called edge detection or feature extraction, which is identifying patterns well enough to know, this is where the window ledge is, so we can more easily take the green-screen image out and replace it with something.
“That was familiar to me early on because, prior to Artists Equity, I had a small visual-effects company. I’ve worked with GPUs a lot too. The visual-effects guys said, ‘Hey, you should see. There are a couple: Google and this other company, OpenAI, are doing really interesting stuff with transformers in video.’
“I’ve learned that I can actually just call up and go, ‘Hey, it’s Ben Affleck. Can I come see what you’re doing?’ Sometimes people say yes, to my astonishment.”
"Firm has probably made a moderate impact on subfield of area they operate", says person standing to make massive profit from running firm's IPO
businesstimes.com.sg/compani…

















