I haven't seen anywhere that they were called "Free Born People of Color," or perhaps I missed that? I agree with E.T. The census before 1870 listed people who were "Free" as of the census date, regardless of how they came to be free, by birth or by manumission, and regardless of what color they were.
toot