In 1870 Eugene was a farm laborer and his wife was a domestic servant. They were both in Allen's Fresh.
In 1860 he was living with his parents John W. and Ann L. Thomas in the same area. However, in 1860 they are free. You may be able to find their freedom papers (some do exist for Charles County in the MD State Archives).
If you have access, try to find the land records
which may list previous slave owners or how he was able to acquire the land.