Gray was born to William and Mary (Stone) Gray in Fredericksburg, Virginia. He moved with his parents and siblings to Houston, Texas in 1838. He read law with his father and was admitted to the bar. After his father died, Gray was appointed Houston’s District Attorney on April 24, 1841, remaining in the job until Texas became a state in 1845.
Gray was elected to the House of Representatives in the first Texas state legislature in 1846. In 1848 he founded the Houston Lyceum, which later became the Houston Public Library. He was elected to the state Senate in 1854, then served as a State District Court Judge from 1856 to 1861.
In 1861, Gray attended the Texas State Secession Convention, and voted to leave the union. In November that year, he was elected to the Confederate House of Representatives. After the war he returned to his law practice in Houston, Gray, Botts & Baker. In 1874 he quit his practice upon being appointed as an Associate Justice of the Texas Supreme Court, but served only a few months before resigning due to declining health.