Born in Culpepper, Virginia, Wharton stood second in the Virginia Military Institute's class of 1847. A civil engineer in Arizona before the Civil War, Wharton became the major of the 45th Virginia in July 1861 and the next month the colonel of the 51st Virginia Infantry. During the summer and fall of 1861, Wharton fought in western Virginia. He then headed west at the beginning of 1862 and helped garrison Fort Donelson, Tennessee. Commanding a brigade, Wharton fought his way out of Donelson on February 15, which earned him the accolades of his superiors.

Wharton's brigade returned to Virginia in the spring of 1862 and occupied the Kanawha Valley the following September. Though a promotion for Wharton was in order, he still remained a colonel. His slow promotions througout the war resulted from a stormy relationship with Jefferson Davis. Not until Wharton commanded the Shenandoah Valley District in July 1863 did he receive a brigadier generalship on the eighth of the month. Wharton's brigade was transferred to Gen. James Longstreet's command in eastern Tennessee that fall. He reported to Gen. John C. Breckenridge in April 1864, and fought at New Market (May 15) and Cold Harbor (June 3). Wharton also participated in Jubal Early's raid on Washington, D.C. (June 27-August 7). He led a division in the 1864 Shenandoah Valley campaign, but never received the appropriate rank of major general. Though official recognition rarely came from Richmond authorities, his superiors in the army demonstrated their confidence in Wharton by assigning him responsibilities above his rank.

After the war he devoted himself to mining enterprises in southwestern Virginia, and served two terms in the Virginia State Senate. He died on May 12, 1906. His mile-long funeral procession included veterans of the 51st Virginia serving as honor guard. He was buried wrapped in the battle flag of the Regiment.