The Frederick Douglass home, Cedar Hill, is definitely a must-see if you are in D.C. on a visit. Unfortunately, it is in Anacostia (Uniontown in 1865) - an area that is desperately in need of revitalization. Cedar Hill is operated by the National Park Service, but it is advisable to travel there in a group or take a taxi to its door.
There is a visitors' center with a nice selection of books. And, the house itself appears as if Mr. Douglass had just stepped out the door. The lucky NPS has many of the original furnishings.