Shadow thief

If i got attacked and I rolled my defense and gained shadows and then my opponent used bah bye to get ride of my shadows after I gained it do I still take dmg ?