Hi Richard,
Weirdly, the issues you're seeing sound a lot like an issue with the backlight, but the problem with that theory is that your phone doesn't come with a backlight; on an OLED display the pixels each generate their own light so no need for a backlight.
However, you mentioned that the display is kinda new - I assume that means it's been replaced recently. By any chance did you opt for the cheaper LCD version of the screen rather than an OLED replacement?
If that's the case, I'd suggest you may need to replace the screen again. The problem I see with the cheap LCD replacements is that the OLED screens don't have any need for control signals to run the backlight, so they don't have any. The aftermarket manufacturers had to come up with some weird kind of kludge to get the backlight to work and be adjustable, so I'm suspicious that circuit has gone haywire on your phone.
Having two grandchildren with iPhone X phones, I have had to replace a ***lot*** of screens, so I'm kind of an expert on them by now, lol. If you go on AliExpress you can find OLED screens for the iPhone X for as low as $14 USD, although generally the cheapest ones are running around $20 right now. That's barely more than the cost of an LCD and is worthwhile for the power savings alone (backlights take a lot more power than OLED displays so your battery is going to run down sooner).