From 4GB VRAM to an RTX 5080 Returning after 2 years & ComfyUI is breaking my brain...help me catch up with current state of the things

I originally got into AI image generation 2-3 years ago, back when SD1.5 was the main model and SDXL, and SD3 was just popping up. I only had a 4GB VRAM laptop and used Automatic1111/Forge only.

Despite the hardware limits, I got to a point that I was creative enough with generating images with custom faces, ControlNet, inpainting, etc. I eventually had to take a break because my setup became too outdated, and I couldn't afford to upgrade my hardware at the time.

I still lingered around this sub to keep up with the news & new model releases. Over a month ago, I finally got a new PC with an RTX 5080 and thought, "Let's get back into this"....

This time, I decided to use ComfyUI.....I had avoided it before because I hated overcomplicating my setup—I just wanted to create, not fiddle with tech. But now, I figured, I'm an engineer, I shouldn't be afraid of this and I can handle it.

Honestly, I can't even express how overwhelming everything feels compared to a few years ago.

There are so many models now, and every single one requires a different workflow and specific nodes. I spend most of my time just fixing environment issues. Every operation needs a unique workflow, taking hours to set up & then suddenly ComfyUI updates, and everything breaks.....on top of that, the models have gotten massive, with so many versions that it's confusing to know what to actually use for my specific goals.

Back in the day, it was just Stable Diffusion and every guide on the internet was for SD, and tools like Roop, ReActor, or IPAdapter etc just worked.

I'm not saying the space shouldn't have evolved. I know this is largely a "skill issue" on my part, and it's super exciting to see such massive advancements....but right now, it's killing my creative drive as I spend all my time fixing issues, testing models, and troubleshooting techniques rather than actually generating something......also don't even get me started on video generation.....I haven't even dared to touch that yet.

Rant over. Based on my current setup and goals, I’m hoping you guys can point me in the right direction

My Requirements and goals

Goal: Hyper-realistic images (and eventually videos) with a consistent custom face & by "hyper-realistic," I don't mean airbrushed and polished.....I mean real skin textures, real imperfections, and authentic lighting.

Hardware: Needs to run reasonably well on an RTX 5080 (for both photos and video)

Video: Needs to support Image-to-Video generation...

My Questions is

Which base model should I "main" for image generation right now? What is currently the most reliable method/workflow for applying a custom face and maintaining absolute consistency?

What model should I main for video generation that won't take ages just to generate a 5-second clip?

I don't need crazy cinematic quality right now since I'm just trying to learn.

Thanks in advance for the help!