Generally isopropyl alcohol isn't damaging to rubbers or electronics but could affect glues inside the phone. You might consider removing the little side button circuit board and try to clean it by itself. If worse comes to worse it wouldn't be expensive to replace. iFixit is short on guides for your phone, but there's a Spanish site called Nadie Me Llama Gallina that has just what you need to open up the phone and remove the side buttons.
[link|https://www.nadiemellamagallina.com/en/manuales/smartphones-1/samsung-4/samsung-galaxy-s23-ultra-1415/pulsadores-laterales-12888|Manuals / Samsung Galaxy S23 Ultra / Side buttons|new_window=true]
Doesn't look like too difficult an operation; if it was me I'd go ahead and order a replacement board anyway but you will want to have some precut adhesive to glue the rear panel back on when you're done.
Good luck; let us know how it goes!