Burton Norvell Harrison had been the private secretary to Confederate president Jefferson Davis. Burton married Constance Cary on November 26, 1867 and settled in New York City.
In 1871, the Harrisons first visited Bar Harbor, Mount Desert Island, Maine, staying at the cottage of Captain Royal George Higgins. Sometime in the 1880s, they commissioned Arthur Rotch of the architectural firm Rotch & Tilden to build a seaside cottage called “Sea Urchins,” with a garden designed by Beatrix Farrand. The property now is owned by the College of the Atlantic, transformed into the Deering Common student center. “Sea Urchins” was a center of hospitality during the "Gilded Age" in Bar Harbor and she entertained many noted visitors there.