Try Charleston newspapers as they are likely to have carried a maritime columns noting ship arrivals and departures. Some of these newspapers may have bee digitized, so I would suggest that you contact a public library in Charleston, to ask them to steer you in the right direction.

Also consider that he may have been on a domestic slave ship. What was the earliest date that you found him in Baltimore records? If he was on a slave ship, but not a slave, is it possible that he was employed on the ship?