Can I use an Intel iMac in Target Disk or Screen mode on a Mac Studio computer?

That hasn't been possible for many years now. Review the document: Use your iMac as a display with target display mode - Apple Support

Target display mode is a feature available with late 2009-Mid 2014 iMacs. The display Mac has to be running macOS High Sierra or older and the signal producing Mac has to be running macOS Catalina (ca. 2019) or earlier.

There is

Share the screen of another Mac - Apple Support

but that is sharing, not as a main display.

You cannot use that iMac (Retina 5K, 27-inch, 2020) in Target Display Mode at all. That mode went away for good when the first 27" 5K Retina iMac came out in Late 2014. Your iMac doesn't support it.

Apple also retroactively added restrictions on Target Display Mode that mean that no recent Mac supports the use of any iMac (even an old one) as a Target Display.

There is a product called Luna Display, by Astropad, that could let you use the screen of that iMac as a second display.

The connection will not be as good as with a hardware monitor. There may be lags and ajtifacting, and even the Astropad people say that Luna Display is not suitable for high motion applications like video editing and gaming.

I believe that as a practical matter, you will also need to have at least one real hardware monitor.

If you get a Mac Studio, you are going to need an external display to set it up.

Connect a display to Mac Studio - Apple Support

You can access the iMac to some degree as mentioned by my fellow users, but they can not be fully integrated into a single system.

You can use your iMac as a Target Disk for a Mac Studio - with a rather significant limitation. The cable connecting the two Macs must be a Thunderbolt cable – not a "plain" USB-C cable.

Transfer files between two Mac computers using target disk mode - Apple Support

This is due to the rule

"Note: If either of the computers has macOS 11 or later installed, you must connect the two computers using a Thunderbolt cable."