Getting PAL profiles for OSSC fixed the sawtooth/stairs problem. SNES and SMD worked without any issues on the standard OSSC profile but NESRGB needed a separate pal profile that I got from:
https://videogameperfection.com/forums/topic/ossc-profiles-for-pal-users/
screenshot comparison Standard vs PAL profiles: https://i.postimg.cc/8zh6XCcw/NESRGB-OSSC-STD-vs-PAL.png
Now, I need to figure out the yellow shadows on ''Duck Tales'' and especially the flashing background on ''Blaster Master''.