Apple TV Remote intermittently turns on the TV (Samsung)

My TV has started turning on intermittently, but only when the Apple TV remote is out in the room somewhere. When I put it in its "normal" place in a drawer, the TV never comes on. I thought at first it was sending a bluetooth signal to the Apple TV since Apple TV turns on too, but only after the TV, but given it doesn't happen when it is in the drawer, it must be infrared. It cannot be sending infrared to the Apple TV since this is in a different cupboard from the drawer. I assume it is waking up the Apple TV that is connected via HDMI. The TV is Samsung. I've reset the remote multiple times, but it hasn't made any difference. It's a 4th generation Apple TV with the latest software updates. If I turn off the waking via HDMI, I won't be able to leave the TV remote in the drawer like I normally do since the Apple TV won't be able to turn on the regular TV.