sounds like it could be one of two things. Either the climate control switches are bad, or your blend door actuator is.
https://www.vwvortex.com/threads/diy-heater-air-conditiong-fix-blend-door-issue.4445527/ has what looks like the way to replace the actuator. Dont use the part number the poster supplied he got it wrong. A commenter gave the right one. As this is the more common component that fails, it is where I would look first. Especially since you said it takes so long for it to change from hot to cold. Makes me think it is having trouble or is seizing up, and driving around (vibrations) are helping free it up to move. Its also the cheaper component so it is where I would start. The control setup im seeing is about $40, as opposed to ~$15 for the actuator. From what I can tell, open it to where you can see the actuator itself (there are youtube videos if needed) and select from hot to cold. Does it move? Take the plug off and use a meter or voltage tester to see if the actuator is getting power. No power means bad switch. Power means bad actuator. Remember to check to ground.