YouTube will use chapter markers that are placed in the video Description section in the format of:
0:00 Chapter One must always start at 0:00
1:23 Chapter Two
2:34 Etc...
You can just write the list and paste it into the video description and that's all you need to do. YT automatically "reads" that chapter list and applies it to the video.
I made a tool to help simplify the process — it will be free today only:
https://fcpxtemplates.com/sc-chapter-maker/
Set it up with a basic title or an adjustment layer spanning the entire video.
If you already have markers in place, you can use keyboard shortcuts to Go To > Next Marker to speed things up.
The effect tracks the chapter time in the HH:MM:SS format — you will have to type it in with your title text though.
You *won't* have to re-render the video - just apply the effect and extract the text, then you can delete title/adjustment layer with the effect after.
Watch the video for how to use it. (Turn on CC for the captioning — there's no actual speaking in the video.)
HTH