Uli Hesse

Ok, so after playing with text-to-video more, I’ve decided it’s actually a semi-shittyy workflow that isn’t very practical for storytellers and is pretty terrible (and expensive) from a production standpoint. The real solution is likely a combo of image-to-video with text prompting, where the style/look/feel comes mostly from the image, which can be generated anywhere, and the text prompt is used mostly to direct the camera movement. The issues with going straight from text to video are that it’s expensive, slow, and difficult to consistently control the overall style and vibe of the shot. The ideal workflow for me would be to use Midjourney for pre-production (casting, production design, storyboards, and shot lists) and then bring it into image-to-video for the cinematography, using text to prompt the camera (and perhaps some VFX) This would save a ton of time and money, and allow for greater control of things like lighting, coloring, style, look/feel, atmosphere, set design, etc. Plus you get the advantages of features like image blending and permutations. More control + faster/cheaper = win. Obviously it's still VERY early and text-to-video will get cheaper, faster, and more coherent to the prompt in the future, but I still don't see why you would wanna skip the pre-production process and jump straight to a tool that is far slower and WAY more expensive than image generation. What do you think?