Why won't Preview open new files on High Sierra, even after resetting preferences?

The most recent macOS Preview version does not impose a limit on the number of files that can be simultaneously open. I did not happen to test High Sierra's version but I can if need be.

Quit the Preview app if it is open. Press and hold a Shift key while opening the Preview app from Applications or the Dock.

• That action will open Preview without any previously opened documents.

Release the Shift key when Preview opens.

In that manner you can find the problematic document causing it to crash. Drag it to the Trash, or attempt to open it with some other app.

18 documents? It might be you have run into the natural limit based on how much RAM you have of how many documents it can open. Keep in mind, the Dock (The translucent toolbar on the bottom of the screen) merely shows the recently opened documents. Preview's sidebar will show you all the current opened documents in Preview. Apple menu (Apple logo in upper left corner) -> System Preferences -> General lets you pick number of recent documents it will created shortcuts for at a time, but these documents aren't actually open until they show in the Window menu or the sidebar of Preview.

If Preview won't open any documents even when it is the only application, and select "Show all windows" picking its icon from the Dock, it is possible the document is corrupted, or the drive is unhealthy. I would have a tech look at exactly what isn't appearing to behave correctly to verify if it is a systems issue or not.