One of the early ferries to operate across the Choptank River was started in 1690 which ran between Castle Haven on the Dorchester County side and Chlora Point on the Talbot County side. The salary for the ferryman was four thousand pounds of tobacco. Anthony Le Compte, a Huguenot refugee, patented the land named “Antonine” in 1659. Castle Haven is named for this early house which dates from at least 1730 (see also LeCompte Bay).
Reverend James Kemp, bishop of Maryland, lived here as well as Thomas King Carroll, a Maryland governor, who used it as a summer house. A wharf is clearly indicated on the Geologic Atlas of the United States, Choptank Folio No. 182, dated 1908, reprinted 1912, mapped by B.L. Miller (see also Chlora Point).
This site, “commands the most extensive and unobstructed view of any house on the river, eight miles up the Choptank and down the river to its mouth and on a clear day to the Chesapeake beyond.”
During the War of 1812 a British raiding party comprising of eighteen barges and a schooner entered the Choptank River and at Castle Haven on October 19, 1814, made a landing where they took poultry and cattle from a tenant at the farm of Dr. Kemp.