My Apple TV 4K (2nd Gen) is only showing a selection of 4 photos for the portrait screensaver

I recently purchased a Apple TV 4K (2nd Gen) secondhand and wanted to try out the new screensavers. The Photos & Memories one works with shared albums and functions as expected, as does the Aerials. However when I switch to the Portraits option it only ever picks a very small selection of photos to display and seemingly never updates them. I mainly wanted some specific people to show up so I disabled the other categories (cities, nature, etc.) and selected the people I wanted as favorites. However, for testing I re-enabled Nature to see if I got new photos. Photos of nature were selected from my library but the photos of people from before have disappeared despite not being deselected. I have a large number of photos of the people I selected it can pick from in my library so I found this peculiar behavior.

I'm running 18.6 Beta [22M5065b] and wondered if this is a beta problem or if there is something I can do to get a wider range of pictures to show in the Portraits screensaver mode?