I, too, have read about Harriet's surname being "Green." There were some blacks on the lower Eastern Shore of Maryland, in Somerset County, named Green too, who later took their master's name of James. I wonder if there's a family connection?