Mail (13.4) on iMac displays all old messages, including deleted ones

Hi, I have Mail 13.4 on my 2012 iMac every time I check for new mail all my old messages come back even the deleted ones. The bin icon has gone on the left and replaced by All Mail . . . Whats going on? In other words I can't delete my old mail.

[Re-Titled by Moderator]

Original Title: Mail going mad.