Origins of my brother's family: a lost cause?

My brother is adopted, and has no remaining connection to his biological family. I have been trying to trace his bio family's ancestry for him as a way to help him feel connected to his roots. We have run into a wall, though, and I am not sure there is any way around it.

My brother's biological father was born in the US, but spoke Normán French in the home. From what my brother remembers, his father said that they family has been in the US since Virginia first became a colony, but always spoke Normán because the family was descended from Huguenots and wanted to preserve that ancestry. This heritage is really important to my brother, so I want to try and find where in France the family came from.

I have been able to confirm from primary sources that he IS descended from a family that is listed in the early colonial documents. There is just one problem: I can't find any evidence of a Huguenot connection beyond an oral history written by a distant cousin. The first member of that family I have found any record of is James Sebree, listed as being born in Virginia around 1675. No one on any site I have found has even SUGGESTED parentage for James, and while I have found primary sources mentioning him, many sites doubt James' existence and record his son Robert Sebree (born in Virginia in 1700) as the first member of the family.

I have checked every extant Virginia colonial records I can get my hands on, including reading the headright records. I have looked for any Sebrees, or anyone with a name that could possibly have become Sebree (Sebre, Seabry, Sabre, Sabree, etc) in other US colonial records and records in Quebec, in case the family got to North America via another colonyy. I have checked ship passenger lists, and Huguenot society resources. I am coming up absolutely empty.

So, at this point I am wondering - is it just hopeless? Or is there somewhere I should be looking that I haven't tried yet?