Mail fails to download attachments

Mac OS 12.6.3

Mail 16.0

I tried to open an attachment (an .xlsx file) in Mail on my Mac, but it fails to open, saying file format is invalid. I checked in webmail that the file is not corrupted, and it wasn't. I also noticed that the attachment is originally well over 1 MB in size, but in Mail it's only 785 KB. I suspect that Mail failed to download the whole document.

I also checked the same message on my iPhone, and I could open the attachment just fine. I'm using the same IMAP settings in all my devices, so it shouldn't be a server issue.

However this is not a constantly recurring issue, because most of the time I can open every attached file I download via Mail.

Can I make Mail deliver attached files reliably?