slow boot from external drive

Hi all, I have Mac mini M4 with 16GB RAM and 256GB drive, so I installed the OS on external drive after fresh install it takes 30 sec to boot even after I have few apps enabled at startup , but as and when I update the OS , with each update it adds 30 to 50 seconds to boot time, after few updates it takes up to 5 mins , but if I erase the disk and install the latest OS again and restore the data from backup it's back to 30 seconds boot. anyone else experienced this ?

btw this is on both 26 and 15.7 version of macOS.