How can I get the App Store back on Apple TV?

I recently purchased some Apple TV 4K units

New or previously used units? Do you know anything about its previous owner or previous use?

… and when i set them up, they don't have the app store on them.

• Apple TV boxes could have been configured with a device profile that restricts certain functions. Only the administrator that added that profile can remove it. E.g. for units in enterprise use or school use, or otherwise deployed for a very specific use (hotels, bars, etc.).
• Apple TV boxes may also lack an App Store if the Apple Account with which it is being set up, is for a country where Apple doesn’t offer an Apple TV box App Store. I expect this to align with countries where Apple doesn’t sell Apple TV boxes.

… set them up

Setting up an Apple TV box for a new user should involve a complete reset (although a device profile might remain). Is that what you did? Reset or restore your Apple TV to its factory settings - Apple Support