[Question] How to avoid game breaking

So, I'm a level 40 warrior, who has finished Main Quest, Companions and done some other stuff. I've powerleveld smithing to 100, and am a proud owner of full legendary daedric armor.

After I had created my fine armor, I upped the difficulty once from adept to expert. The thing is, my enchanting is pretty high as well, and I intend to get it to 100.

I've heard it should be a very game breaking to combine Smithing and Enchanting, yet with only smithing and on expert difficulty, I still feel challenged when fighting.

My Question: Is there a way to avoid breaking the difficulty using this combination, or is there maybe still challenge to the game on Master difficulty using it?