I did not find Charles Allen or any other member of an Allen family in the "list of free Negroes over the age of 16" in the Personal Property Tax Lists for Bedford County in 1813. You might try the later tax lists and the Register of Free Negroes for Bedford, both available on interlibrary loan from the Library of Virginia.
Paul