Parish Church of St. Helena is at 505 Church St. between King and North sts. and was founded in 1712. The wooden altar was carved by the crew of the USS New Hampshire, which was stationed nearby during Reconstruction. In the churchyard are tombstones used as operating tables during the Civil War and a brick sepulcher built for a doctor who feared being buried alive. His friends promised to place bread, water and an ax in his tomb to ensure his escape.
The church and cemetery grounds are open to visitors.