Hasted mentions the remains of an Archbishop's Palace that formerly stood south of the churchyard. There is little firm evidence of the complex but it is said that the old vicarage that was demolished in the 1960s contained medieval walling, complete with arched windows. Ireland, writing in 1830, said that there was a large stone building with a chimney at each end which may imply a great hall or kitchen. There is no evidence that the site was a 'Benedictine Monastery' as indicated on the 2nd edition Ordnance Survey map (1897-1900).