After the British evacuated New York, Governor's Island was owned by Columbia University. Columbia was still known as King's College and was located in lower Manhattan at the time. Students and Faculty of Columbia volunteered to build the fort to protect the growing city from future invasions.