items in my downloads file

If it's a compressed file other than a disk image, you can.

If it's a disk image and you've dragged the program somewhere else, you can.

If you're running it straight off a disk image or from the Downloads folder, you can't.

(240578)

You can ATTEMPT to delete anything in the downloads folder.

Any items "in use" for any reason will simply refuse to be deleted, and produce a message to that effect. Your Mac will not crash because you tried to delete something, items 'in use' are all locked against deletion when MacOS opens them.

Are you talking about stuff that you downloaded from your browser? Once you're done installing the app or whatever, you can clear out your downloads.