After the purchase of Gibraltar Island in 1864, Jay Cooke asked his brother Pitt Cooke to oversee the construction of his summer home. The fifteen-room structure was constructed from native limestone and encompassed nearly 9,000 square feet. Italianate in design, the retreat featured an octagonal tower and dominant eaves supports. The tower provided a vantage point for scanning the island, its surroundings, and Lake Erie. French doors led to two porches - one on the Lake Erie side and the other on the South Bass Island side. The porches brought Lake Erie's cool breezes directly into the castle.