Apple TV on Ethernet

Then it sounds like the ethernet portion of your network is seggrated from Wifi as mentioned by Rudegar.

The Home App, views the devices over the internet rather than through the network directly which is why it can see the other Apple TV.

You'll need to take a look at your router settings, to see if you can bridge both networks.

What model of Apple TV is the downstairs Apple TV that is connected via ethernet? Identify your Apple TV model - Apple Support

it should be transparent for the wifi connected devices how the appletv is connected

unless the router haven't set the 2 networks as bridged some also keep 2.4Ghz wifi and 5Ghz wifi as separate networks

so it's a network configuration thing and how one resolve it change from router to router