Using consoles with different connections on one RGB input?

My BVM has one RGB input, and I use it with several consoles outputting over SCART and component. Right now, if I want to switch from using SCART consoles to component ones, I have to unplug the SCART-BNC adapter and switch manually. What can I use to have SCART and component connected via BNC at the same time?

Is an Extron Crosspoint my only option? Is there anything that's not so picky about sync?