Is it okay to leave a new MacBook Air M4 plugged in

Yes, it's fine.

If you turn on Optimize Charging in System Settings ➜ Battery ➜ Battery Health (i) ➜ Optimize Charging, the Mac will manage its charging and lean your routine. It will keep the charge at 80% if it knows it's not being used on battery, and will be ready at 100% when it knows you usually disconnect it from power.

The Mac cannot be over charged whether you use Optimize charging or not.

click here ➜ Change Battery settings on a Mac laptop - Apple Support

Here is some of Apple's guidance regarding battery health and management:

Batteries - Maximizing Performance - Apple Support

About battery health management in Mac laptops - Apple Support 

In a nutshell, to get the most from your battery:

• Let the OS and charging circuits manage the charging.

• Plug in when using the computer near available power.

• Use on battery when ac is unavailable or when you must be mobile.

• Don’t subject the computer and battery to extreme temperatures.

• Don’t allow the battery to regularly discharge deeply before plugging in. Yes, it may happen from time to time, just don't make it a habit.

• Don’t obsess over battery life and/or health.  

Just to add...I have my 2022 MacBook M2 Air plugged into mains 99% of the time. It is shut off maybe once a month but never for longer than a minute or so, so it's in sleep mode when not being used. Since purchased in 2022, I have 9 battery cycles on it and battery percentage is at 100%.