Moe Khalil (@mokhalil01) header
Moe Khalil (@mokhalil01) profile picture

Product @LiteLLM | Ex-Founder @WebhoundAI (YC S23) | Stanford CS ‘23 | San Francisco & Beirut

San Francisco, CA
Joined September 2011
Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)Photo by Moe Khalil (@mokhalil01)
Moe Khalil retweeted
Fully self hosted, sovereign inference stack with @_llm_d_ @vllm_project @LiteLLM
We’re launching Inference Stack, one-click deploy with everything you need to own your inference It ships with: @vllm_project for deploying models @huggingface for finding models @_llm_d_ for deployment-level routing @LiteLLM for inference @grafana for metrics Try it out here: inferencestack.io
3
21
1,549
We’re launching Inference Stack, one-click deploy with everything you need to own your inference It ships with: @vllm_project for deploying models @huggingface for finding models @_llm_d_ for deployment-level routing @LiteLLM for inference @grafana for metrics Try it out here: inferencestack.io
11
24
7
168
10,311
Question to CISOs: How do you deal with agent sprawl in your company? Do you have any visibility into what your employees' bots are doing?
5
1
15
1,775
Besides saving 5 minutes of setup time, why would you send your requests to anyone who isn’t the inference provider?
3
278
Self host your own gateway and use providers you trust. You should know where your requests are going.
the OpenRouter situation for anyone curious: -someone managed to access outputs sent to MiniMax -researcher discovered this by sending a secret links -40 to 130 hours later someone opened the secret links -oop, this means that someone accessed your outputs!! -AltasCloud/Novita/StreamLake were forwarding requests to MiniMax???? what the helly? ugh.
4
3
1
18
1,981
Does AI usage impact your company’s top line? If not, how do you justify spending so much on it?
1
3
186
Please listen to a podcast about CI
Yesterday I was a listening to a podcast about Riemann hypothesis. Today it is resolved. Back then, I was listening to a podcast about Navier Stokes just the day before it was resolved. Which one should I listen to next?
1
2
296
If you've built your own agent tracing platform, drop it below. I'll retweet the good ones.
1
1
147
For the past few months I’ve been having my agent /goal n hours of research on a topic then build me a personalized course on it It’s incredible, you should try it
Most students just use AI to cheat. But it is a first-rate learning tool for anyone who actually wants to learn.
1
194
If you use a tracing product, why? Isn’t the ideal experience here OTEL in Clickhouse with an MCP? Then you can have your agent run automations and maybe build a custom UI on top of it What am I missing here?
3
9
497
you'll never hear about APIs without an API Gateway. if you're building AI products without an AI gateway, why?
1
5
187
As a developer, how do you define "improving your agent"? Presumably at some point your agent harness has all the capabilities it needs to accomplish its goal. What comes next?
8
5
13
753
Can it file PRs?
JUST IN: Apple $AAPL is developing AI agent to rival OpenAI and Anthropic that can control devices, browse the web, and write code.
3
106
LiteLLM has evolved a lot since its first commit, and nowhere is that more evident than in its database Since joining LiteLLM, I've come across so many different tables and columns that didn't make sense at first. When I dug deeper it became clear they were crucial to making the product work. So, I built the LiteLLM database course. It goes through every table we have, what purpose it serves, why it was added (according to PR bodies), and the overall evolution of our DB and why it is the way it is today. Check it out ↓
4
2
15
851
How do you handle it when each one of your users wants a different set of abstractions? We want to support different versions of the same use case but keep the product simple Do you just build it all but keep the happy path simple?
3
7
388
So we have evals for evals now, but how are we supposed to eval those evals?
Introducing the Eval Optimizer. Not every eval needs a frontier model. Judgment now learns your agent trajectories and picks the right compute per judge: code, a decision model, an LLM or an agent. For one customer, it cut the cost per eval by up to 50×.
2
175
AI-native companies have two types of engineers One makes sure the agent doesn’t burn $50k overnight The other makes sure it’s getting smarter every day Which one are you?
3
7
302