Castle Williams became obsolete after twenty years or so. It was used to hold Confederate prisoners of war during the Civil War. In 1912 it became a military prison. The windows were recently added to protect the interior of the building from the elements.