Greetings jasontimothyarnold,
The Volume button in the Apple TV Remote in Control Center won't be accessible, unless you're using Apple TV with a HomePod, a sound bar, or other AirPlay 2-compatible speakers. Check out that information outlined here:
Set up the Apple TV Remote on your iPhone or iPad - Apple Support
The Volume button on your Apple TV Remote in Control Center works only when your Apple TV is set up with a HomePod, soundbar, or other AirPlay 2-compatible speakers. If the Volume button is dimmed, use the volume button on the remote that came with your television.
If the issue persists, see if restarting your router and modem helps, as that can clear up communication issues the devices may be experiencing with AirPlay on the network. To restart your router and modem you can simply unplug the devices for one minute.
Additionally, ensure tvOS and your iPhone are updated, as that can help resolve issues related to software:
Update your Apple TV manually
1. Go to Settings > System > Software Updates and select Update Software.
2. If there's an update, select Download and Install.
3. Wait for your Apple TV to download the update. Keep your Apple TV connected and plugged in to power until the update is complete.
After the update downloads, your Apple TV will restart, prepare the update, then install it. When the update is complete, your Apple TV will automatically restart again.
If the issue continues, then reach out to Apple Support directly to help investigate the behavior. You can use the following link to reach out: Contact - Official Apple Support
Kind regards.