Thomas Wentworth Higginson, Army Life in a Black Regiment

With heavy hearts their officers floated down the lovely river, which we had ascended with hopes so buoyant; and from that day to this, the reasons for our recall have never been made public. It was commonly attributed to proslavery advisers, acting on the rather impulsive nature of Major-General Hunter, with a view to cut short the career of the colored troops, and stop their recruiting. But it may have been simply the scarcity of troops in the Department, and the renewed conviction at Headquarters that we were too few to hold the post alone. The latter theory was strengthened by the fact that, when General Seymour reoccupied Jacksonville, the following year, he took with him twenty thousand men instead of one thousand,--and the sanguinary battle of Olustee found him with too few.