Sierra

I doubt running Big Sur will make it any cooler. It isn't the OS that's causing the heat from what you say, it's the graphics and video.

We can't advise you as to the upgrade problem without knowing the exact model and YEAR computer.

As already noted, some applications use a lot of power with heat a a by-product. Read this document:

Apps can affect Mac performance, battery runtime, temperature, and fan activity --> https://support.apple.com/HT203184

You don't say what model computer you have.

See this document about the operating temperature of Mac notebooks 'Keep your Mac notebook within acceptable operating temperatures' - https://support.apple.com/HT201640

That can definitely upgrade to Big Sur, and even Monterey when it is released. It would help to know the exact error message.

How to upgrade to macOS Big Sur, click here --> https://support.apple.com/HT201475

For the document 'macOS Big Sur is compatible with these computers' click here --> https://support.apple.com/HT211238

These models or newer or each type:

2015 MacBook

Late 2013 MacBook Pro

Mid 2014 iMac

2013 Mac Pro

2013 MacBook Air

2014 Mac mini

2017 iMac Pro

From Catalina to Sierra is a downgrade, not an upgrade.

Those apps use a lot of processing power, so you'll experience the same issue and will have a lot of work because a downgrade requires a clean install of the older system.

Incidentally, it can help if you have a fan running across your hardware. I have an external drive and I have noticed a 10°C (18°F) drop in temperature since running a small desk fan directed at it. Jut make sure it isn't working against the exhaust direction of any vents.