Not prior to 1880, but prior to 1870, that is the 1860 Census and any Census before that. Unless they were in those Census years, they were not likely to have been free persons.
Deloris