How are video elements extracted without background?

How did they do that? [SOLVED]

Hello.

So I've been watching videos such as The Glorious Octagon of Destiny, Kitchen Gun - 3 Shots Of Derek Bum and Muscular Wonders - Terry Crews' Greatest Hits with great amazement not only towards the musical feat which was achieved in these collaborations but also to the large editing effort which went into creating the visuals.

From what I understand (with my rather basic knowledge of video editing and 3D animation) these videos are most often a blend of 3D or 2D animation followed by tons of work in a program such as Adobe After effects.

My question regarding these, however, is much simpler (I hope). The videos all base on a/a few proceeding video(s) which provide some key visuals used throughout the entire project. For example, in the case of The Glorious Octagon of Destiny this video would be Sesame Street: Jack Black defines Octagon. Now the element which I would specifically like trying my hand at (and can't seem to figure out on my own) is how to extract the video elements without background from the original video(s).

Is it done by "cutting out" the person or object from the original video by hand: frame-by-frame or is there an AI that is able to automatically do the work, or is it done in an entirely different way?

While I am primarily looking for the way to do so, I am also wondering if anyone knows if this is possible with the things I have directly available which would be Sony Vegas Pro 14 for video editing and a few programs such as Magix Photo & Graphics designer for image-based editing (as well as free alternatives such as Gimp and Paint.net).

I'd like to thank you for reading through my question and prematurely thank anyone who can solve this mystery.

I wish you a good day.

Footnote: I tried looking through the wiki for a solution, however as it would seem just exactly the page for rotoscoping/cutting stuff out of a video seems to be missing: the wiki page.

Footnote 2: I hope I didn't butcher any technical terms in my submission.

Posted Apr 13, 2020, 1:24 AM UTC ยท edited

1116
enineciโ€ข

Basically, to cut people out of video, what you're looking for is called rotoscoping. That is where they mask out the object of person frame by frame. However, if the background is a relatively solid color - while the same color is not present on what you're cutting out - you can use a process called chroma key. That is what using a green screen is called (but you can easemtially do it with any color.

You would need a compositing software like After Effects to pull that off though.

6
DaDeceptive0neโ€ข

Not OP but I have to thank you for this comment. This was something I was searching for a longer time. Anyways, I always ended up finding a mask and never thought about keyframes+mask. Rotoscoping is a good term to find some more tutorials - thanks for that kind stranger <3

2
enineciโ€ข

No problem. I'm glad I could help. FYI, rotoscoping is the worst. Haha

2
SomeProtagonistOPโ€ข

Hello, thanks a lot for your response. I've looked further into rotoscoping and found a tutorial on how to do it with Sony Vegas.

For now, I've only done 1 frame https://imgur.com/Tk0MsCC, but I am really happy with the result for doing it for the first time (please forgive the masking mess, I accidentally created 3 masks instead of 1 and couldn't figure out of to connect them).

Thanks again for your advice!

1
enineciโ€ข

Hey, that's pretty great for your first try.

Actually, using multiple masks is the best way to do it. So, instead of having to adjust all the points for the whole body throughout the duration of the clip, you would have 1 mask for the head, 1 for each arm, 1 for the body and so on.

Great job! Thanks for the silver, by the way. ๐Ÿ˜Š

1
V3Qn117x0UFQโ€ข

automatic rotoscoping. davinci resolve does this. not perfect, but you can tweak it slowly and save time instead of doing frame by frame - https://www.youtube.com/watch?v=P4xCHkFqg-4

ai/ML is far but it's there

3
SomeProtagonistOPโ€ข

Thanks a lot for your response. For now, I've found a way to do it manually in what I already have, but if I ever need to do it automatically I'll take a deeper look into DaVinci resolve!

1
montypython2000โ€ข

There are several ways. Yo don't really need AE to do that you can do that in Premiere Pro with so called opacity masks. Basically you can remove some parts of screen depending on their characteristics. Opacitiy masks also called mates. I did that even in Premiere elements. Yo can remove uniform colour with Chroma keys, you can also remove static bacground from moving objects. You can cut out pieces you don't want with garbage mattes etc. Here is an example https://www.youtube.com/watch?v=rX-Xmr-v57k&list=LLrwwR3bvrEFDadoTUBFJbUQ&index=5&t=170s

or this one https://www.youtube.com/watch?v=2_RllynGJO4&list=LLrwwR3bvrEFDadoTUBFJbUQ&index=3&t=142s

2
SomeProtagonistOPโ€ข

Thanks a lot for your response! I've managed to find a way to use masks in the program I already have. The videos you've send did help me understand the procedure a little better though, so thanks for your help!

2
[deleted]โ€ข

[deleted]

1
[deleted]โ€ข

[deleted]

1
[deleted]โ€ข

[deleted]

1
dells16โ€ข

You want a green screen and use chroma key via AE.

1
SomeProtagonistOPโ€ข

Hello, thanks for your response.

Sadly since I am not looking to edit self-made video, but just internet clips for fun I can't count on them being filmed in front of a green screen. I've found what I am looking for to be rotoscoping, but thanks for your response anyway!

1
cozmianโ€ข

I think you're talking about rotoscoping. Automatic Rotoscoping is available in most high-end softwares. Manual Rotoscoping (If you're decent at it) looks better imo. After Effects, Premiere, DaVinci Resolve, etc etc...

1
SomeProtagonistOPโ€ข

Thanks for your response.

After doing some more research on rotoscoping, yes that's what I've been looking for. I've found a way to manually do it in the program I already own (resulting in this), so thanks for showing me the right direction!

1
TheMMakerโ€ข

Heyo, glad you enjoyed the collabs! I helped work on all the videos / collabs you've mentioned, so I can provide a little more insight. I can say enineci's comment is pretty much correct -- either the footage is keyed or manually rotoscoped, and just about everyone doing visual production on these uses After Effects, and a few cases of Fusion, Nuke, etc.

Usually we aren't terribly concerned about being 100% accurate and try to figure out a way to "key" things before rotoscoping, because rotoscoping is going to be the most time consuming. Most collabs try to alleviate this by keying and rotoscoping as much material beforehand and then sharing it among participants, so they can focus on actual visual production.

As some examples: The Jack Black video is against a blue background, and nothing else in the video is blue, so that can be easily keyed. Some tweaking and garbage masks have to be created to account for clouds and the edges of the frame but there isn't too much manual work here.

For the Old Spice videos, most are commercials that have a static background composited in, and some of those commercials have at least one frame where it's only the background, so you can use that background as a difference matte to get a pretty good cutout that only needs minimal cleanup. Brief example of that here: https://twitter.com/MowtenDoo/status/627989497611825152

There's multiple other techniques to get a good key, another example would be keying with HSL, example of that here: https://www.youtube.com/watch?v=r_A7kmfKex8 (After Effects' equivalent to this is Lumetri Color, which has a HSL key tool)

Worst case scenario, it's rotoscope time. And for best results I wouldn't recommend the rotobrush tool in After Effects, but if speed is more of a concern then it's fine.

Apologies if this is a little too wordy. Hope my rambling helps in some way at least!

1
SomeProtagonistOPโ€ข

Hello, thanks a lot for your in-depth response!

Really interesting hearing about the work that went into the collaborations and how they are managed.

I've just tried out masking for now resulting in this (please excuse the masking mess, I accidentally created 3 different masks and couldn't find a way to connect them together), but if I ever find a project that keying would be possible in, I'll definitely give that a try as well!

1
TheMMakerโ€ข

No problem, seems you're on the right track! ๐Ÿ‘

1