Noise in Render to Texture

Currently have an issue where I can get a model using VRay to render beatifully with little to no noise. However, when I go to bake the material using Render to Texture I get some serious noise in the item. Any ideas of what is going on? I am using VRay Adv for my rendering.

Thanks.