Changing Gears without Clucth

Today Ride on the highway I accidentally change Gear 3 to 4 without Pulling the Clutch..
should i be worried?