Principal Animation Programmer @epicgames on @unrealengine. Previously anim tech lead at @amazon and other places. I like coding, investing and gaming.
Joined May 2015
- Tweets2.7K
- Following246
- Followers2K
- Likes5.9K
Pinned Tweet
Here's a screengrab of the sections of the ML Deformer presentation in the State of Unreal stream that shows it in action.
It shows an approximation to a full muscle sim that takes 1 minute per frame normally. Also does cloth approximation. Running live on PS5. #mldeformer #ue5
UE 5.9 preview 1 now ships with a new plugin I worked on called AnimGenOffline (name might change) which allows you to guide animations at high level and use AI to generate the actual frames. This integrates that functionality into Sequencer.
As initial plugin into this system we integrated NVidia's Kimodo. But the system itself is not limited to this. Basically any algo that generates pose data over time can integrate with it.
You can draw splines and make characters follow them, type a style in text, set target key poses at certain frames, etc. You can pose your character directly with Control Rig and retargeting is all handled automatically.
You can also type: "Use Kimodo to generate an animation of a drunk person walking forward and save it as an Animation Sequence asset in /Game/Anims/DrunkWalk".
Weights still have to be downloaded from Huggingface (goes automatically after creating an account and read-only token etc). We're working on making that easier too.
It can run on basically any PC, but for fastest generation you need a Geforce 3xxx series or better. And probably Windows only for now, although I haven't tried on other platforms yet.
This is just the start of course!
Worked on this with @FredNilsson
Will show some videos soonish, or share some community videos.
The Unreal Engine 5.9 MegaJam Preview is here!
It’s the first Unreal Engine release built primarily with agentic workflows and opens up a few new Experimental features: check out Unreal MCP’s expanded toolsets, work with AI coding agents directly in Editor, explore AI-assisted content creation, and run your projects in a browser.
This isn’t your typical Unreal Engine Preview, but it’s ready just in time for the Epic MegaJam.
Invite your agents and start experimenting: epic.gm/ue-5-9-preview
Just a little headsup: there are some python issues in the preview 1. You can work around it by pasting the errors in an AI agent. But will fix that for preview 2 of course. Also if you have a Geforce 5xxx series generation on GPU will fail becuase the engine ships a too old version of PyTorch which isn't compatible with that. We're also working on that.
Bit rough around the edges still lol :) But will be fixed!
1
4
578
Some strong tents! Now coupled simulation, so cloth interacts with cloth and other objects.
This ain't realtime btw. I guess next would be tearing/damage on the tent. But I first need to optimize it some more.
First subtle cloth sim results with self collisions and collision with objects, and some wind simulation and ropes.
13
712
Working on some more physics stuff. Building a little tower of London inspired scene that will involve some trebuchets firing projectiles.
The simulation will be pretty advanced on this one. Aiming to include fire, smoke, destruction, fluids, realistic material bending, mud simulation, cloth sim and damage and more, all in a coupled simulation. That's the goal at least, and not realtime. Just the goal, not fully realized yet :)
2
6
563
Some initial destruction with Pinata. All is dynamic, so no pre-fracturing. There is also some stress calculations to make stuff collapse when it should.
Some more improvements. This runs over 24000 cubes at over 100 fps, none of them are sleeping. It's running on the GPU now, but with sync back to CPU/RAM, which is of course a bottleneck. But this is now basically running instanced static meshes in UE, but simulation done on the GPU in compute shaders.
I can do a 100% GPU sim too which will allow way more objects. But with read-back it feels more 'real'. More experiments coming up soon :))
Need moar cubes! Tens of thousands of them! :)
Sorry for the blurry mess, I think something with the motion vectors not updating. Every cube is actively simulated and generating contacts. Trying to take another approach and got some first results.
Much faster performance. Data oriented design with full SIMD now. I think its like 4x faster than before.
Still need to get my multithreading faster and some more optims. But from my profiles in some tests it was now on par with Box3D and PhysX and Jolt performance wise (need to try different tests and Pinata has way less features of course). It is about 2x as fast as Chaos physics in the test I ran (using UE 5.6).
I noticed some parts of Chaos are very slow. I might try optimize that as side quest sometime at work lol. Already got several side quests though :S
I guess I'll check multithreading performance next and try to optimize that.
Further stability improvements. Added some initial optimizations. Still have to do bigger optimizations though. This is also a bit worst case, all in one big pile :)
Have to test in a shipping build also to see if that makes any difference. Will do some design changes soon to make it data oriented for better perf.
12
819
Spent a bit more time on my hobby physics system. Now the contacts are a lot more stable. Still a bit swingy with big stacks of boxes though, so have to see if I can do more there :)
Also some more constraints so that i can make a swinging wrecking ball kinda thing.
Contact generation is now improved and caching some things etc for more stability.
Will probably do some optimizations soonish to make it more data oriented.
Bonus crash at the end as well :)
2
16
1,175
Predicted this would happen.
Experienced engineers are higher in demand. And there will be less experienced engineers over time as many switch to management or retire.
Maybe the new seniors will just be ones that shipped things using Ai but don't really have the lower level experience.
I guess that will be the new reality later on.
Nothing wrong with that. Basically just the end of a generation, just like we have less and less people now that know how to write high perf code. Just another step above that.
The AI boom is creating a generational divide in software jobs:
US software developer employment for those aged 22-25 has declined -23% since November 2022, when ChatGPT was launched.
At the same time, the 26-30 age group has seen a decrease of -5%.
By comparison, employment among software developers aged 41-49 has risen +18% over the same period.
This creates a 41 percentage point gap between the youngest and one of the most experienced groups within the same profession.
Meanwhile, early-career workers in the most AI-exposed jobs have experienced a -16% decline in employment relative to less-exposed occupations and more experienced peers since November 2022, even after adjusting for firm-specific shocks.
By contrast, employment among workers in less AI-exposed fields, as well as more experienced workers within the same occupations, has remained steady or continued to grow.
The AI revolution is wiping out entry-level tech jobs.
7
578
People really have a wrong view of AI in terms of game development it seems. They seem too afraid and overly negative and might not really understand what it's really used for.
Also it's not going to replace jobs in games I think. It's just an accelerator.
I think people really are too afraid of AI. It is going to make development easier. Less time needs to be spent on tedious repetitive tasks. This is just natural progression of software.
It's not replacing jobs, it is an accelerator.
You need to hook up spacebar to some jump action being triggered? You can do it yourself, or look up how to do that again if you don't remember all the details, or you just tell the AI to do it in a few seconds. You verify what it did, and assuming it's all fine, you just saved some time. There is no real difference in the game in the end.
Of course there are more uses, like generate some art with it. This is what most people seem afraid of. But I can tell you artists are not going to be replaced. Their workflows can adopt to a more AI centric workflow later though, where they are still in control on the art direction and create it in the way they want things to look. Also you don't HAVE to use such workflows.
This isn't going to replace anybody. And the recent layoffs had nothing to do with AI either.
If you are using AI like ChatGpt or Gemini in your life now, did your life get worse or better? The same thing is happening in game development now. It can make it better for game devs. Of course it doesn't directly mean games will be better too, but if used correctly time can be saved and that can reduce costs or give more time for polish or adding more content.
Just my personal opinion.
Vibe coded a personal retirement calculator that shows different tax scenarios. Simulating what the government in the Netherlands is planning to do, and alternatives, using the current system, moving to Spain, etc.
It takes into account Dutch state pension, pension, investments, future windfalls and purchases and tells you when you can retire, and what the maximum safe payout per month would be.
It runs 10k montecarlo stock market simulations as well to calculate the success rate of the plan, and what the allowed payout is with 90+% success rate etc.
Not really game dev related, but fun and useful (for myself) :) Just entered some made up values in the image below to show the graphs etc.
It shows how bad the new tax on unrealized gains really is. It makes quite a big impact. You can see a massive difference with taxing realized gains, and with less crazy tax rates.
1
9
700
My personal views on Genie. And Ai usage in game development in the future.
The fact that it can create this is very impressive and very exciting to see. There is still a long way to go though, and this is not going to be the exact way games will be made.
For games you need more control, like when you cross a specific point some events have to happen. There is hit detection, multiplayer, sounds and effects that are carefully designed, environments are carefully crafted and lighting is done to set the mood, specific animations to play at specific times, game state changes, and complex gameplay logic as well as precise movement model control that is constantly tweaked to make things feel good and fun.
For that you need some game engine like unreal. What you will see (and are seeing) happen is that Ai is going to be integrated in these workflows first of all. So just like if you go to chatgpt and ask it a question now you can ask it for help to explain things and create things.
This helps setting up those previously mentioned things faster. Maybe you want to create animation setup that moves the character. You just ask the assistant to do it and it would create it.
Also you will see more ai/ml being used in algorithms that help visuals. Think of things like a machine learning mesh deformer for better deformations. Or improved lighting and textures using neural networks as well.
I think we will slowly go to something where we will render a base pass using traditional methods and thr Ai will fill in the details using something similar than the video kind of models.
So there the model would create all the cloth fold details and muscle movements like in real and understands if you step through snow there has to be footprints, or water ripples if you step into it. Next to that it could make it look very realistic rendering wise.
So in the end it will generate a lot using Ai, but with full control.
There will likely be some things where you feed the engine videos like output of these systems and it tries to extract a 3d world to use in engine and maybe basic animations and control. But this is only gonna be good for prototypes and isn't going to give you a full game that you can ship.
1
1
829
Just playing around with Codex now. It's also very good.
However! I asked it to implement SSE powered vector/math classes etc. All good and it works, but then when diving deeper I see it is not doing things in an optimal way. So asked it if it's sure about this and if it shouldn't use my approach after suggesting that. It said it doesn't matter and what it did was good.
So asked it to implement a version with what I suggested instead and to perf benchmark the difference.
Result is that what I suggested was way faster (multiple times) than what the AI model produced, even after giving it several chances to improve and giving it hints.
Still shows that having domain expertise really has a lot of value. This has been the main weakness I seen so far in the models. The performance of the code they generate is typically not great. This is probably not a big deal in 99% of the code though. Real-time stuff is kind of special here, but still!
1
7
644
I must say the vibe coding is really fun. This week I've been building some new SaaS website. I'm super noob with web development, but this really allows me to pretty much build any idea I have.
As it has seen so much training data on this topic, it also does better than say using it in building a physics system.
I really enjoy directing at higher level and getting quick results, rather than having to implement everything myself. At least for web development it makes the barrier to entry a lot lower.
7
498
Some more playing around with some constraints. Trying to make a pin constraint next.
Been trying to use AI on a few things here to see how far that gets me. It messes up a lot, but it definitely helps. A lot of prompts are needed to get something that kinda works :)
13
969
Physics system update:
Capsules now work against boxes as well. So now supporting spheres, capsules, and boxes. Had to add GJK with closest point calculation.
Box implemented as generic hull. Should make it relatively easy to add support for convex hulls later on.
Next step will be support for joints.