Grade 1 listed building being overrun by monkeys