Did the Grants even know that Mary was in that close proximity? Yes, I agree with Linda, that Mary did nothing to endear herself to them when her husband was President, but then that was simply Mary! She had strong likes and dislikes - and it appears that it did not stand well for her in later years.
"The Past is a foreign country...they do things differently there" - L. P. Hartley