iPhone SE screen turns darker at times while in a bright space, regardless of screen brightness at maximum...

My 2020 iPhone SE has an issue where the screen turns darker in bright / sunny places such as outside or in my car (living in LA so lots of sunlight). It's not related to the auto-brightness option since I turned that off, i.e. the brightness slider doesn't change until I change it. When it happens the screen brightness seems to be around 50% intensity, to the point that I can't properly see my screen. At these moments the slider is on maximum brightness, and often (and ironically) when walking into a darker space I can see the screen brightness increase to full brightness. It's not that it looks brighter, it just jumps to being very bright where it was dimmed before. This seems counters intuitive as i'm sure a screen is supposed to be darker in dark places and brighter in bright places. Sometimes in the car with the slider on max brightness the screen brightness will drop to 50%, again the slider still being at max when it happens, and simply making the screen too dark to really see anything on it. Weird.

This current 2021 iPhone SE was a replacement under Apple warranty since my previous 2020 iPhone SE had the same issue (along with a few other issues). But now it turns out this iPhone has the same issue. Also, when mentioning this to a friend who has the iPhone 13 she mentioned she has this issue aswell from time to time and confirmed that just like I had initially experienced when going to the Apple store they couldn't really help or even acknowledge this issue properly, and simply recommended a software update or hard restore. Which I have done but to no avail.

I'm wondering if anybody else has had experience with their iPhone. If so i'd love to hear it, since it's very frustrating and seems to be a general thing, not device or software related...