In 1647 all bishops and bishoprics were abolished and church land confiscated. Then, in 1660 the monarchy was restored and so were the bishops, with Bishop Duppa beginning work on restoring Wolvesey Castle. Duppa's successor, Bishop George Morley, who became Bishop in 1662, decided in 1680 to rebuild the Palace next door to the Norman Castle ( despite the work undertaken on Wolvesey Castle), the remains of which he used as a source of stone. The original medieval chapel was retained but the moat was filled in and stone from Wolvesey Castle was used to build the new Baroque-style palace.

However, Morley died in 1684 and as his succeeding Bishops preferred to stay in Farnham Castle, Surrey, the Palace he had planned was not finished until forty years later. Even then it continued to be neglected. In 1785, most of the new Baroque Palace was demolished, except the West Wing, which was used as a training college until 1926. In 1927, the Bishopric of Winchester was divided into three, and with Farnham Castle becoming the property of the new Bishop of Guildford, the neglected surviving Wolvesey Palace wing became the official residence of the Bishop of Winchester.

Wolvesey Castle is