Lydia R. Baldwin was born in Southampton, Massachusetts. Her father was Ezekiel Baldwin. She entered Mount Holyoke Female Seminary in 1843 and graduated in 1845. She worked as a teacher in Easthampton, Northampton, Southampton, and Westfield, Massachusetts, and then married David B. Phelps in 1849. They had three sons and a daughter. She died in Southampton in 1872.