The 3 larger pins on the outside are mainly to hold the USB port in place. There’s no electrical signal connection in them, and they just provide a ground shield to the outer USB port. As long as the 4 smaller pins are making good connection with the circuit board then you should be fine for power and signal. Just solder the 3 larger pins in place to keep the USB port tight and stable. If the USB port is loose due to loose connections on those 3 larger pins, then the port could wiggle the smaller pins loose and break a signal connection. If necessary, gently bend the larger outer pins to help hold the USB port in place without it moving around. Be careful to not have solder spread across different connection or you could short something out. And you can use a small dab of glue on the front side of the board (where the USB port is, not where the solder connections are) to help keep the USB port in place.