Would a police officer have probably cause if they pulled you over lack of registration while you were driving to get an inspection?

As I understand it, it's legal to drive a vehicle without registration as long as you're driving it to/from getting it inspected for the purpose of obtaining registration. It has me curious, if a police pulled you over for lack of registration, I assume they'd normally have probable cause. But what happens when they find out you're driving it legally for the purpose of obtaining an inspection? Does the probable cause still stand or does it go away?