Vendor offered me a job

How ethical is for a current vendor of my company to try to "steal me" so I can work for his company? I feel like I need to be quiet about this otherwise it'll hurt the vendor, but if I'm quiet about it it'll bite me in the future. Note: I politely rejected the offer right away but the person is nice and I feel I need to be nice also.