Ludlow, Fitz Hugh, 1836-1870