This is a classic headache in STAR-CCM+ (and CFD in general). The behavior you are seeing—where the "Gas Only" model is rock-solid but the "VOF (even with 100% Gas)" model falls apart—is due to how the VOF solver handles numerical dissipation and pressure-velocity coupling compared to a standard single-phase solver.
Here is the breakdown of why VOF is killing your stability and how to fix it: