Try searching Micheal Plunkett's publication on African Americans in Virginia. At the home page double click on "search text" then input "Tayloe" and it will bring up several entries. The University of Virginia and the Virginia Historical Society have thousands of Tayloe family papers. I've not looked at these, just am simply aware that that family existed. Good luck and happy hunting!

http://www.upress.virginia.edu/plunkett/mfp.html