Apple TV is connected to Wi-Fi, but not internet

I don't want to restart the modem/router because it's not mine (I'm housesitting for the weekend). I don't want to reset the Apple TV either for the same reasons.

Restarting (not resetting) the Apple TV box shouldn’t change anything about this device, except for a short outage of service, and a clean slate from starting up freshly without network information that may have become invalid. It can only get better. Apple TV may do so on its own anyway (e.g. from automatic updates). I would suggest to try the Apple TV box first anyway, as that wouldn’t interrupt anything else. Choose Settings﹥System﹥Restart for what I think is the safest option, where Apple TV can gently end any processes that are better not interrupted abruptly. But if you have instructions to not touch anything, or are uncertain about what else depends on continuous internet connectivity (HomeKit perhaps?), then I suppose you better not do it without contacting the owner first.

Is there any way to fix this without having to mess with the router or do a reset?

Restarting devices would normally be the recommended first step, if the issue appears to be for all apps. Further steps may be «do this then restart» anyway.

Else try this for any misbehaving app (if some other app does get internet connectivity): Force-quit the app, then relaunch and try again.