Was there an assassin on Grant's train?

[RJNorton]

In her memoirs Julia Grant wrote:

"At about midday, a rap at my door was followed, in answer to my "come in," by the entrance of a man dressed in light-colored corduroy coat and trousers and with rather a shabby hat of the same color. I remarked his dress; as he came in, I started up and said: "I thought it was the bellboy with cards. What do you want?" He reddened and, bowing, said: "This is Mrs. Grant?" I bowed assent. "Mrs. Lincoln sends me, Madam, with her compliments, to say she will call for you at exactly eight o'clock to go to the theater." To this, I replied with some feeling (not liking either the looks of the messenger or the message, thinking the former savored of discourtesy and the latter seemed like a command), "You may return with my compliments to Mrs. Lincoln and say I regret that as General Grant and I intend leaving the city this afternoon, we will not, therefore, be here to accompany the President and Mrs. Lincoln to the theater." He hesitated a moment, then urged: "Madam, the papers announce that General Grant will be with the President tonight at the theater." I said to this: "You deliver my message to Mrs. Lincoln as I have given it to you. You may go now." He smiled as he turned to leave. I have thought since that this man was one of the band of conspirators in that night's sad tragedy, and that he was not sent by Mrs. Lincoln at all. I am perfectly sure that he, with three others, one of them [John Wilkes] Booth himself, sat opposite me and my party at luncheon that
day."

In his book John Fazio speculates that Mary Lincoln's "messenger" might have been David Herold. If this be true then Booth could have known the Grants were leaving town a lot earlier than most books indicate (although, at that time of day, he would not know where they were going). Still this knowledge would have given Booth more time to arrange a potential Grant assassination. Most books indicate Booth didn't find out the Grants were leaving until late in the afternoon when he saw their carriage go by while talking to John Mathews.

If the man were not David Herold who could it have been? Did Mary Lincoln send a messenger to the Willard Hotel at midday?