Virtual Display brightness changes dynamically

I've been using the Mac Virtual Display feature for hours and hours and I noticed that the brightness of the virtual screen changes slightly. If I look at a bright white page (a blank webpage for example) in the middle of my virtual monitor, and I look at it dead on centered with my AVP, brightness lowers a tad. If I move my head a bit up, so the white page is now at the bottom edge of my AVP, the whole virtual monitor gets much brighter.

I would love to turn this automatic feature off and control this myself. Is there a setting for this somewhere?