Asked about relocation at offer stage. Should I backtrack and just accept it?

Hello, got a job offer last Thursday and was very happy with the pay, but it would involve a large cross-country move from Oregon to New York.

The details about relocation were murky, but I had stated that I was "open to relocation". I asked about it when I got a verbal offer last week, but haven't received a reply yet. I'm freaking out. I realize this is an employer's market, but is just simply asking enough to rescind you?

I'm considering just flat-out accepting the verbal offer today and backtracking. Any advice?