Freyr Breton and Apprentice - Dragonheart

Hey all, for my next playthrough I want to build a character who utilizes Dragonheart.

• Apprentice - Dragonheart: Power attacks, bashes and sneak attacks deal random elemental damage equal to half your Destruction level.

I think this is cool for a melee character with a little extra splash of damage. My problem is leveling up destruction is antithetical from the bash and slash gameplay I want this character to have. I have to use destruction magic to level up destruction so I'm either bashing and power attacking and reaping the benefits of the buff but not making it any stronger or I'm not using the buff at all and leveling up destruction to make it stronger for later. I tried this character once before and I ended up just playing as a destruction mage and never bothering to switch back to melee.

I run full suite Enairim, Valravn on Expert, Survival, and for this character no crafting.

For those of you who have tried building into dragonheart, how did you manage your destruction skill level or what limits did you give yourself to keep it fun?