Compton House stands on Castle Street with its notable shop-front, red sandstone steps, fanlight doorcase and cobbled paving in front. William Henry Cooper's account of the town dates the building to probably the early 19th century, though its architect is unrecorded, and describes it as originally housing a milliner and tailor's business run by George Bradshaw; Brown's 1894 Directory likewise lists a Mrs Mary Rebecca Bradshaw, general draper, trading as Compton House. The building later held the Bon Vivant restaurant in its cellar and has more recently been used as offices, including by an estate agency.