McClellan continued to bring up his heavy siege guns to the Yorktown line, and on the night of May 3, 1862, Johnston withdrew toward Richmond rather than risk a battle. Blackford wrote his mother:
At 8 o’clock the whole Army moved quickly out of the works. I, with my company, was left to support the pickets when they should come in. I was there left behind, our whole Army keeping perfect silence. I soon became aware that the enemy were apprised of our movements, and were already deployed in the field opposite our works, waiting to come in. After an interval of about 2½ hours our advanced pickets fell back & formed on me, and cautioning the men as to silence we moved off with rear, each man holding his canteen to prevent its rattling. After moving slowly about ½ a mile our aid met me with orders to double quick, whereupon we struck the run and soon left Yorktown far in the rear.
The retreat, as was so often case in those early days, was marred by poor planning and staff work.
No halt was made until 12 o’clk the next day when we bivouacked 2 miles beyond Williamsburg. The Artillery and wagon trains delayed us constantly, some times we would not move ½ mile an hour tho’ on our feet the whole time. At 2 AM Monday morning we were on the road and after standing under arms until 8 o’clock in a hard rain we were ordered to stack and await orders.
Finally, just outside of Williamsburg, the Confederates turned to fight a rear guard action.
So we stood until 2 pm—when the battle of Wmsburg became general, and we were ordered up. Just before entering the town we all piled knapsacks & overcoats, and thus lightly equipped we passed thro’ the town into the battle. Have you ever heard the expression ‘the light of battle,’ this was plainly visible upon the face of every man as the ladies would wave their handkerchiefs & address encouraging words to the men. I never saw such a scene & never expect to see such another. Some women were upon their knees, some waving handkerchiefs, while most of them were giving water to the poor wounded fellows who were coming from the fight.
Much to their chagrin, however, the 5th Alabama and Rodes’ brigade were once more simply spectators to the battle. To add to their woes they were simply left where they stood, then marched off without an rest whatsoever.
There we were forced to stand in ankle deep mud until 3 AM Tuesday morning, being unable to sit at all and exposed all the while to a cold wind & rain. You can scarcely realize how we suffered. I can’t now recall it—tis strange that some things should be in the midst of civilization, if we could have lain or sat down it would have been a great relief but it was impossible owing to the depth of the mud. At 3 we moved off & continued the march without any thing to eat until 5 in the evening-we were off again at day and marched until 2 o’clock the next morning, again up at day and marched until after dark. The extra ordinary depth of the mud caused us to move so slowly that 2 miles an hour was our best speed. No true account of the battle has reached you—tis a far great loss on both sides than at Manassas, and will not fall short of 2,000, the enemy’s 4000. Our men never fought so well—tis the great regret of my life that I was unable to go in. I have always been willing but was never so disappointed in my life before. The scene in Wmsburg stirred up strange & novel emotions within me. I had 90 men with me, and I would not have hesitated to charge & rout any one Yankee regiment.