On July 7, Brig. Gen.John D. Imboden stopped Brig. Gen. John Buford's Union cavalry from occupying Williamsport and destroying Confederate trains. Brig. Gen. Judson Kilpatrick's cavalry division drove two Confederate cavalry brigades through Hagerstown before being forced to retire by the arrival of the rest of Stuart's command. Lee's infantry reached the rain-swollen Potomac River but could not cross, the pontoon bridge having been destroyed by a cavalry raid.
On July 11, Lee entrenched a line, protecting the river crossings at Williamsport and waited for Maj. Gen. George G. Meade's Army of the Potomac to advance. On July 12, Meade reached the vicinity and probed the Confederate line. On July 13, skirmishing was heavy along the lines as Meade positioned his forces for an attack. In the meantime, the river fell enough to allow the construction of a new bridge, and Lee's army began crossing the river after dark on the 13th. On the morning of July 14, Kilpatrick's and Buford's cavalry divisions attacked the rearguard division of Maj. Gen. Henry Heth still on the north bank, taking more than 500 prisoners. Confederate Brig. Gen. J. Johnston Pettigrew was mortally wounded in the fight. On July 16, Brig. Gen. David McM. Gregg's cavalry approached Shepherdstown where the brigades of Brig. Gens. Fitzhugh Lee and John R. Chambliss, supported by Col. Milton J. Ferguson's brigade, held the Potomac River fords against the Union infantry. Fitzhugh Lee and Chambliss attacked Gregg, who held out against several attacks and sorties, fighting sporadically until nightfall when he withdrew.