Je-rod retweeted
First you have to understand that modern LLM inference already disaggregates weights as models outgrew single chips years ago. You shard either by layer (pipeline parallelism) or by slicing every layer (tensor parallelism), and the two do very different things.
As an example, let’s look at Llama 3.3. It has 70B of weights and at FP8 that’s 70 GB of memory which is enough to fit on a single H100. Now that H100 has 3.35 TB/s of HBM, so the fastest it can ever decode for one user is 70/3.35 ≈ 21 ms/token or ~48 tok/s while using under 1% of its FLOPs. Now if we pipeline it across 8 chips: each chip holds ~8.75 GB, which means it only needs 1/8th the bandwidth and 1/8th the FLOPs to sustain the same aggregate throughput. Now crucially the token/sec a user gets is limited by the amount of data that crosses the link. In current LLMs all that is a small amount of activations for LLama 3.3 it’s ~8 KB per token….
Yes, you read that right it’s 8 KILOBYTES we are sending over a <900 GB/s link. That’s only 9 ns of serialization time but the overhead of 224G PAM4 SerDes adds ~100 ns per link traversal with RS-FEC which is 11x longer than the payload itself. And then you have the NVSwitch adding ~300 ns per hop and you need to pay twice. That’s ~600 ns of just hardware latency wrapped around 9 ns of data making a 98% tax before software even shows up. Then NCCL’s collective stack turns 600 ns into 10-20+ us… all to move 8 kilobytes lol. For comparison 8 KB serializes over 10 Gigabit Ethernet NRZ, in just 6.6 us. Pipeline parallelism however doesn’t make a single user faster as the token still needs to visit every layer in the sequence, so per-user speed is still weights / per-chip bandwidth.
To get more speed per user token you need to use tensor parallelism and have all the chips work on the same layer simultaneously. TP costs you 2 all reduce OPs per layer, 160 per token on llama 3, that’s still kilobytes of traffic but with NVLink overhead it’s a massive tax and why pipeline parallelism on most models still gives more interactivity per user. However, this gives you a huge latency lever to pull that scales tokens per second with interconnect speed instead of memory BW.
The clever amongst you might have also realized that sharding doesn’t just cut memory bandwidth per chip it also cuts FLOPs per chip and is why we have such bad MFU on decode. So once you’ve sized the link for the memory, you need to size the compute for it too. This is called “balancing the pipeline”, and currently no shipping chip does it because they were all designed as standalone monsters. Remember Tokens/sec = ~aggregate memory BW / bytes touched per token. At batch 64 in FP4 you need ~250 FLOPs per byte, and Blackwell ships 1,250. Provisioned 5x more than the narrow pipe of HBM. Nobody saturates shit cause they are all building around HBM.
So now it all comes full circle. Parallelism reduces memory bw pressure and thus FLOPs but increases interconnect latency pressure. Despite having HBM and GigaSERDES we aren’t actually doing more work lol. But if you really wanted to balance the pipeline you need to match the memory bandwidth, the flops, and most importantly the interconnect.
So what does that look like ? Well if you build around LPDDR’s lower bandwidth, lower your interconnect latency, you actually can beat Nvidia on decode with a fraction of the silicon.
Je-rod retweeted
Which he WAS NOT SUPPOSED TO DO AND WAS ORDERED NOT TO DO!
(This is about the company commander who got killed in Grenada.)
Story straight from someone in the company. Lemme get a drink first, this one always makes me want to drink..
B/2/325 was the Division Reaction Company. DRC was different than Division Reaction Force. DRF was either an infantry battalion or a task force team. (Grenada they were limited Teams.) Teams had all the attachments and were to be 'wheels up in 18 hours.' Two hour recall and God help you if you missed formation.
(For those non-military, battalions/teams are commanded by lieutenant colonels, people with around 18 years of experience. Companies are captains, about 3-4 years. Platoons are guys fresh out of college. more or less zero.)
DRC was ONE HOUR recall and NINE HOURS wheels up. Just one company who was supposed to do 'something' on the other end until the full TF got there.
You've got to move fast and you don't take vehicles or anything else. Just rucks and whatever you can carry on your body.
Because there is much to do to get the rest of the company following, the XO and the First Sergeant stay behind and follow on with the Battalion.
Nine hours of a company commander on a more or less independent command. And in this case, that was not a good thing. (Foreshadowing.)
Before you go on DRF cycle, you go through Training cycle then ARTEP. (The Army's 'final exam.')
During Company Prepared Assault ARTEP the CO of B/2/325 put on his RANGER CAP, chose his senior platoon leader and an E5 RTO and went out on a leader's recon.
And walked right into enemy fire and all three were graded as killed.
The Executive Officer and First Sergeant took over, sent out a five man fire team, did a classic one-up distraction, two side flanking maneuver and won.
'Company ARTEP passed.'
Unit passed Battalion ARTEP and was certified Good To Go for DRF.
During the 'hotwash' after action for the whole battalion, the company commander was asked 'Why did you do a leader's recon?'
According to 'The Book', a COMPANY COMMANDER is supposed to chose a 5 man fire team to do recon. Company Commanders are too valuable to be lost doing a recon mission.
He stood up and proclaimed:
'THAT IS WHAT THEY TAUGHT ME IN RANGER SCHOOOOL!'
(I'd digress on my rant about Ranger School which I call 'Active Stupid School.' You have to be active all the time and most of what you do is stupid. It's active stupid school.)
His battalion commander then, in front of the entire battalion, replied:
'You're a company commander now, not a Ranger candidate. Send out a five man fire team next time. That's an order.' (Note: That wasn't just 'it's the standard.' The battalion commander gave him a direct legal order.)
Fast forward to Grenada. Same company commander is the Division Reaction Company Commander.
The company (after alot of other stuff) is tasked with protecting the flank of the Rangers (what's left of them) as they get the 'hostages' out of the medical school.
(Another thing to keep in mind in this story is they'd landed at Salinas when they were still pulling out the Ranger dead. They knew they were in a real war and it's unsettling when there is no real ramp up or special training. They were green as grass by even modern 'green' standards. Hell, we were less well trained than modern NG infantry. Seriously.)
The company is taking the main road to St. Georges.
Right at dusk, the point team spots 'movement on a hill' and 'multiple enemy.'
The company goes into perimeter.
The company commander PUTS ON HIS RANGER CAP, CHOOSES HIS SENIOR PLATOON LEADER AND AN E5 RTO AND GOES OUT ON A LEADER'S RECON!
EXACTLY WHAT HE'D DONE IN TRAINING AND BEEN TOLD NOT TO DO!
Some notes here from the well remembered conversation:
Me: Was he super good at it?
Him: He could not sneak up on a troop of cub scouts on sand. The guy never heard of hand and arm signals. He was the worst guy at recon you've ever seen.
Me; Fuuuuck.
From his POV:
'So, there we were, spread out around the road. The sun set quick then it was just silent. The sound of the tropic breeze was all you could hear. Then, about fifteen minutes after they went out, the sky filled with green tracers.
'You could hear everything. The RTO was on the radio, talking real quiet. 'Romeo 35, Romeo 35, Sierra 26, over... Romeo 35, Romeo 35, Sierra 26, over...'
'We only had two lieutenants and the one that was senior had gone with him.
(Mortars, 1LT, and XO 1LT, along with FSG would have been on birds headed there but they might as well have been on the moon.)
'So, the lieutenant is saying 'Sergeant, what do I do?' (to his platoon sergeant.) And the platoon sergeant is saying 'I dunno, sir. I think you're the CO now. Gimme some orders and we'll follow them.' 'I'm right out of basic! I was told to listen to my platoon sergeant. Wadda I do?' 'I dunno, sir. I think you're the CO...'
Now, imagine for just a second that you're in that situation. We were TOLD we were going to be fighting a BRIGADE of Cuban Infantry. You're a company, your CO (a captain) is apparently dead as well as the one half trained LT, your XO and first sergeant might as well be on the moon, and, oh yeah, your short ranged radios right now CANNOT REACH ANYONE! You are entirely on your own, in the dark AND YOUR LEADERSHIP IS HAVING THAT CONVERSATION.
(Which is WHY you DO NOT do a LEADER'S RECON if you are a COMPANY COMMANDER!)
'So, all of a sudden we hear this 'HUFF' and one of the staff sergeants gets up. Nobody knew too much about him. Old guy. He'd said he was in the Marines in Vietnam early on. (BTW: The 82nd in those days had several of those Vietnam Era marines who came back in and were basically hanging out for the rest of their 20 to get retirement. Good NCOs.)
'This guy walks over and says 'Sir, I'm going to give you some instructions. You are going to make those orders. And, sir, if you do not, I am going to shoot you in the head.'
😁
Gotta love the vietnam guys.
First, send out a five man fire team. Enemy machine gun company in the open on a ridge. 8 PPKs, 12 DhSKas, 1 37mm recoilless. No losses on the recon.
All we had were Michelin road maps. Plan was developed to leave the machine guns in a covered position facing the enemy and the rest of the company would swing around by road to the rear and attack at dawn.
At dawn, the 60s opened up and distracted them from the front. They were in hard cover, no casualties.
'So, what happened?'
'We walked up the back slope and took them from the rear. They didn't even know we were there til we were putting guns in their faces.'
Entire PRA company captured.
E5 was found wounded. One PPK round through his chest laterally, punctured both lungs but he survived.
The LT had 'dove in some bushes' and hid all night. His career was over but he was alive.
The company commander was found at the top of the ridge with his hands tied with commo wire and shot multiple times in the head.
The official US Army History has it as 'Company commander was killed in an ambush.' Nothing about the leader's recon. Nothing about him being executed. Nothing about it being his collossal fuck-up.
Which means future company commanders, even if they read about it, learn none of the lessons.
There are alot of morals to the story but I won't bother. They should be obvious. The 82nds entire response was to get rid of DRC.
The LT that took command got a medal. The former Marine did not and was never going to get promoted beyond Staff (for threatening an officer) but there would be waivers for him to stick around as long as he wanted. None of this 'up or out.' He was 'break glass in the event of war.'
And that's the true story.
PS: 'The Marine'. Went to Vietnam on one tour back in '64. Decided he liked war but not the Marines.
Joined the French Foreign Legion. Fought in Bush Wars all over Africa, notably in Rhodesia. After the Bush Wars died down and the State Department started getting weanie about 'mercenaries' he came back to the US, enlisted in the Army and was just hanging out to get retirement.
Yeah. Break Glass in Event of War.
I was later my battalion commander's driver. He'd been the S3 on the jump and maybe someday I'll write his story about it.
That leader's recon was anything BUT textbook.
Replying to @Jringo1508
I saw LTC Wesley Taylor's talk on 1/75's jump. Amazing story. I knew, in passing, the guy who got killed doing a textbook leader's recon.
Je-rod retweeted
🤔No, actually it doesn't.
Don't get me wrong, I don't like this.
But we provided Ukraine with intelligence for its strikes against Russia.
This is not about Russian accountability, it's about our own. Especially that of the Biden administration who so freely offered up that intelligence knowing full-well it would increase tensions. Not to mention the other war assets and funding to the tune of untold billions.
So the question is, how far do we want to take this?...
If we wanted to escalate this war we would've by now. We could have taken out Iranian military assets so fast that any intelligence from Russia would've been futile.
So if we aren't interested in escalation, then sometimes you just have to accept some underhanded business.
Russia swallowed its pride to avoid escalation with the US and NATO. We both did the same during the Cold War in Vietnam and Afghanistan.
Now it's time to do it again.
Yes, nuance matters. This is chess, not hungry hungry hippos.
Je-rod retweeted
Where Eric is wrong:
1. Pretti was waving for cars to drive thru the BP operation while BP Officers were in the street.
2. The woman in the white jacket who was pushed did not have her back to the officer. As she approached the snow she turned around to face the officer, and it was at that point he shoved her to get her out of the street.
3. Pretti putting himself between the BP Agent and the woman, no matter how much it might be instinctual, is a crime.
4. "Pretti did not start it" – Pretti had a clear path to the sidewalk which is where he was headed, and he reversed and went back to engage the BP Officer. THAT is where Pretti “started it.”
5. What you really want to argue but you can’t bring yourself to say it is “He didn’t resist that much.” “His motives for interfering with the Officer’s efforts were well intentioned.” That’s not the law. The “degree” of resistant nor the goodness in the heart of the lawbreaker don’t turn crimes into non-crimes.
6. BP didn’t pursue “them” and shove “them.” One of the three turned around and faced the BP Officer, stopping in her tracks. The BP Officer shoved her. Pretti could have kept going – Pretti did not. That was PRETTI’S CHOICE.
7. The “Agents” didn’t remove the gun. One Agent removed the gun and immediately turned away and left. It is UNKNOWN if any other the others were aware. This came after one or more Agents yelled “GUN” – communicating to all they were engaged with an ARMED individual who was resisting their efforts to subdue him. When an agent yells "GUN" he doesn't yell "One Sig P320 9 mm, silver with white grips, in a holster in he small of his back". "GUN" simply means armed with a handugn.
8. It is not “speculation” what gun he had – it was a Sig P320 AXG COMBAT, and the P320 DOES have a problem with unintended discharges.
9. Law enforcement agents were not obligated to assume he had only one gun. They knew nothing about him. What they knew was 1) he was resisting, and 2) he had a firearm. That means he was “armed”. Taking one gun does not, by itself, mean he was thereafter “unarmed.” If you don’t understand that, then you don’t understand the law.
10. Your continued reference to “they” in describing the actions of BP Officers is sloppy and inaccurate. "They" did many different things all at the same time.
I went into this ready to blame Alex Pretti.
After watching hours of footage from multiple angles… I can’t.
Here’s why 👇
youtu.be/T1r60gRauB0
Je-rod retweeted
Every Boomer-style solution is just older people forgetting problems that have been pointed out to them many times.
Boomer: “I don’t see why kids complain so much these days, they should just do <thing that I just thought of that worked 30 years ago>”
Zoomers: “We would do that except there are <manifestly obvious barriers to success> and even if we overcame those there are <bad circumstances everyone can see> preventing this approach from being a worthwhile proposition.
Boomer: “STOP WHINING. Kids today are so entitled. They want solutions but then when we give them solutions they just make excuses. Put on your big boy pants and solve your own problems as opposed to complaining ONLINE!”
Zoomers: “Ok, Boomer”
<Time passes>
Boomer: So I just noticed <manifestly obvious barriers to success> and <bad circumstances everyone can see> why isn’t anyone else talking about this?
Zoomers: “This is literally what we’ve been trying to tell you!”
Boomer: “Whoah now, let’s not get too upset. This is all very complicated. Maybe the problem will just fix itself. If not your generation can figure it out…”
Zoomers: “We know!”
Rinse repeat
Je-rod retweeted
the biggest victim of transgenders has been women with masculine jawlines. In the past, women with manjaws could marry a Tom Brady husband and breed eugenic handsome athletic warrior sons. But in an era of "transgender for everyone", she doesn't get asked out on the first date.
Je-rod retweeted
I genuinely wish companies knew how exhausting it was attach a resume, just to be asked to manually fill out a skills and work experience section that is literally in the resume.
Je-rod retweeted
I held off on saying anything for a day or two after PNW Guerilla’s post telling people not to join the military. I wanted to collect my thoughts not as a civilian, but as someone who did serve.
I was in the Marine Corps Reserve from 2009 to 2015 as an 0331 machine gunner. I joined my unit as they were getting back from Iraq. The unit hadn’t even fully demobilized yet when I checked in, with a relatively easy first two years I then got swept up into a full deployment work-up. As a reservist, I ended up doing training usually reserved for active duty infantry.
And let me be blunt:
I was a terrible Marine.
If there’s a “warrior gene,” I’ve got whatever the opposite of that is. I got promoted to corporal, but I was terrified of leadership.
I assure you, if you want the definition of failure, my picture is in there somewhere. I had bad leadership, sure but even with good leadership, I’d have been, at best, below average.
Did I learn some things? Definitely.
Did I have to unlearn a lot afterward? Also yes.
Am I glad I did it? …Maybe.
What I can offer here is just nuance. Many are shouting “don’t join” or “absolutely join,” and half the time it feels like it’s just branding. Some folks genuinely mean it. Some are just selling a narrative.
Here’s what I’ll say:
I don’t regret joining the Military Overall.
But, and I’m sure this comes as a surprise to no one, I also wasn’t cut out for standard infantry.
Maybe Air Force TACP would’ve suited me better? Combat-adjacent but more technical. Less “general issue,” more specialized. Or maybe I would’ve fit in much better as a veterinarian tech stocking piles of dog food, or maybe I’m just dreaming and would’ve been shit at any of them.
In any case, would I ever tell you I was not proud to serve? No.
Would I tell you I loved the Marine Corps? Hell no.
Would I tell you it was a waste of time?
Ultimately, no.
But life isn’t a clean spreadsheet of good or bad choices. Most of the decisions you make won’t matter in the end because you’re going to die anyway. So whether your path involves the military or doesn’t… make your choices, get the cool photos, be hot, and find some meaning in the middle of it all.
That’s about the best any of us can do.
Je-rod retweeted
Found three drawing tablets that I’m not going to be using. I don’t want to just throw them away so decided to do a little giveaway!
From left to right: Wacom CTL-460, Wacom Intuos small, HUION Kamvas Pro 13
Just retweet this post and I will pick 3 winners. See below for info.
Je-rod retweeted
Creepy coincidence of the day - my father is reading a fictional thriller by @BradThor , Full Black , written in 2010. Which talks about how the nonprofit industry collapsing America.
Page 62-65 of Full Black by Brad Thor
"Not until you understand how much money is involved and what’s at stake," Salomon replied. "One hundred years ago, there were only eighteen American tax-exempt private foundations. Today, there are more than sixteen thousand.
"The U.S. not-for-profit sector is the world's seventh-largest economy. The foundations sit on over five hundred billion untaxed and largely unregulated dollars. Some of the biggest foundations give away more in a year than some nations’ GDPs. The power of a few of these foundations rival that of our own federal government, as well as the power of countries like Russia, France, and Great Britain. That was the crux of our film—how, where, and why that money and power is being spent.
"We were looking at the foundation world in general, but more specifically at a disturbing ideological agenda shared by many of them. We wanted to know how many of these large foundations, started by successful pro-business Americans, had turned so anti-business and in some cases downright anti-American. Why were environmental organizations lobbying Washington on issues that had nothing to do with the environment? Why were labor organizations lobbying on issues that had nothing to do with workers? Why were foundations funding pro-socialist and pro-communist textbooks and lessons in schools? Why were others supporting the eugenics movement and the works of Josef Mengele from Auschwitz, masquerading under the banner of human genetics? The list went on and on.
"The key in each instance was in following the money, and the more we followed it, the further down the rabbit hole we went.
"What we discovered was that beginning in the 1940s, radical elements inside the United States had recognized that there were these huge piles of money just sitting inside multiple large foundations and endowments all across the country. These big government collectivists, globalists, socialists, and communists realized that if they could get into positions of power, say on the boards of directors at the foundations or the endowments, they could steer the money any way they wanted. And that was exactly what they did."
"So what you're saying," replied Ralston, "is that they used the money to buy influence."
"Not only to buy influence," the producer continued, "but to develop and push entire agendas. It was like having a tray of financial syringes. Any cause that met their radical agenda received huge injections of cash. Any causes that ran counter to their agenda received huge injections of poison and found themselves beset by opposition groups with bottomless wells of support. They used their money to cozy up to politicians, influence public policy, and elect their own candidates."
"But how could they get away with that?" asked Ralston. "How would nobody raise a fuss or try to expose them?"
Salomon shook his head. "This isn’t just a mountain of hush money we're talking about, it’s a whole range of mountains. This kind of power and influence can purchase a lot of silence."
"And their boards just rubber-stamp anything they do?"
"They don’t need a rubber stamp," replied the producer, "when the boards have all been stacked with members who see the world through exactly the same prism. If the people who had started many of these foundations were alive today, they’d be stunned to discover what was going on."
Ralston didn’t doubt it, but so what? "The more you talk, Larry, the less I think this has anything to do with your documentary. I think we ought to be looking at other possibilities."
"You’re wrong," replied Salomon.
"Am I?" asked Ralston. "So lots of foundations and endowments have drifted from their original intent. Big deal. Your documentary might bring some unwanted attention to some in that industry, but as long as they’re not breaking any laws, I don’t see how anybody is really going to care."
"That’s what I thought, too, until I saw what Chip and Jeremy had begun to dig up."
"Which was what?"
"The foundations and the endowments need to make a return on their principal, so they invest in different vehicles. Often times, those vehicles are hedge funds. We discovered that a small group, the most radical, invested with one hedge fund in particular. It’s called the Standing Fund and is managed by James Standing."
"The billionaire?"
"The vehemently anti-American billionaire," Salomon clarified.
"So what?"
"So we discovered that there were some things even the most radical foundations and endowments were afraid to be tied to. What they weren’t afraid to do, though, was to use Standing as a cutout. Each of them agreed to allow Standing to retain part of their investment return in order to fund something they referred to as Project Green Ramp."
"And what’s Project Green Ramp?"
Salomon looked Ralston directly in the eyes and stated, "An intricate plan to completely collapse the United States of America."
Je-rod retweeted
⚡️😥You know it's bad when even Kiev Post reports it with real casualty figures... even confirming that 30 NATO instructors were vaporized
(link in tweet below)
Je-rod retweeted
Last week, it was "how dare you be upset that the good jobs with a bright future go to immigrants."
Today, it's "how dare you not accept McDonalds with a smile!"
Republicans aren't black pilled by society. They're black pilled by other Republicans.
Je-rod retweeted
I hope mass deportations really do hurt the economy.
I hope many people whose lives depend on 10+ illegal aliens living in a one-bedroom sharing a stolen SSN and e-scooter to deliver their groceries and make their food suddenly find out they are not that rich at all, actually.
Je-rod retweeted
This is probably the best diet video for aesthetics I've seen, I get so much info about cutting, gyming, etc, no one talks about eating windows and fasting, love it
If you want an aesthetic body I highly recommend.
youtube.com/watch?v=-ynxfvtG…


































