There was just not enough room to get a good grip on the driver.

Luckily this iFixit driver has a hole on the top part. I used a tool to gain some mechanical advantage. While applying downward pressure on the driver, turn the tool to the left to loosen the screw

Once loosened, the iFixit extension that came with my set, works perfectly to remove the screw,