I am a writer who spent ten years putting together a historical fiction novel based on John Storms original book on the people he called "Jackson Whites." At first I believed his research but soon found it to be more folk story than history.
However his story triggered an idea for a really galloping adventure yarn and so I have a 620 page novel awaiting a publisher's nod.
John Storms idea was that a contract was let to a British Army officer, Jackson, to bring Comfort Women to the British occupied New York City.
Historians kept trying to find Officer Jackson. They never did.
The label, Jackson Whites, comes from a newspaper of the day. They teased about soldiers visiting Jackson's Whites at Lispenard Meadow where they were housed. The joke was that Officer Jackson's women were not all white. He lost one shipload of women when their ship was lost. He replaced the women with a shipload of West Indian blacks.
At the close of the American Revolution when the British abandoned NYC they left their Comfort Women locked in their compound without means of acquiring food or fuel. The women escaped--now it gets murky--Storm believed that they treked across NJ in November with little food and children in tow, seeking a refuge for the winter. They found it in the Ramapoughs with the mountain community--mostly men. These women may be a root of the isolated the clan you are talking about. That's the path of my novel. I lived in Westwood, NJ.