I looked at the family in 1870, and noticed that the ages of the children placed the oldest as being born in 1863. They also seemed to have all been born in Tennessee, with the exception of the 2nd child who was born in VA, which means they family most likely lived first in TN, moved to Virginia for a short time in 1864,then moved back to TN again by 1866 where they remained. I also noticed that Charles was born in VA and was a blacksmith. I don't know if you checked or not, but I found him in 1860 as follows:
1860 Hancock Co.,TN,Briar Creek Dist.(Sneedsville), Image 29, #248/232:
(Living in the household of RHEA STERLING, white, Blacksmith,TN)
CHARLES MASON - 22 - MU - SMITH HAND - VA
So, we know for sure that Charles was a Free Person of Color, and the ages of the children means that he and Athy were married prior to the end of the Civil War, and it is most likely that Athy was also a Free Person of Color. The exception might be if Charles Mason married a her as a slave, which occasionally happened.
The ages also suggest that Athy must have married Charles shortly after the 1860 Census and she perhaps was already living in TN in 1860.
I searched high and low, and can't find a NANCY or ATHY MILLS living in TN who would fit their descriptions. I even tried a search of just the first name of NANCY or ATHEY/ATHY for Hancock Co.,TN, but no luck. I find no one fitting their names & ages living together in Halifax Co.,NC, either. Although it is a good bet that if they were Free, that they could have come from that area, it's difficult to say where they came from without more clues. Do you have something else that would lead you to believe that Nancy came from Halifax Co.NC?