You said someone was on the right track in an earlier clue. If it house is unrelated to Presidents Lincoln or Garfield; then either it's somehow connected to Grant or a marriage of a President (or the marriage of one of his family members) took place there? I've looked at the images of a bunch of different Grant houses in different places and none seem to match - so I'm guessing a marriage place associated with a President?