Some time ago there was a recommendation in Cadence's training material and manuals that suggested setting maxsideband=0 for oscillators. This was a big mistake that ended up causing virtually everyone to get overly optimistic results.  Ever since they have been telling everyone that they can find that you should never use maxsidebands=0 for any circuit. In addition, in the latest version of SpectreRF it is illegal to set maxsidebands=0.

Even setting maxsidebands to 5-6 seem low. The maxsidebands parameter indicates how much noise folding the simulator includes. Since internally, oscillators tend to have sharply changing waveforms (particularly the current waveforms), you should err on setting maxsidebands high.

To understand maxsidebands, consider a 1GHz oscillator and assume that I am interested in the phase noise at a 1MHz offset. If I set maxsidebands=0, then SpectreRF only includes in its output noise that originates from sources at 1.001GHz (1.001GHz +/- 0*1GHz).  In this case the effect of  flicker noise will not be included, since it orginates at low frequencies. With maxsidebands=1 the effect of noise orginating at 1MHz and 2.001Gz is included in the output noise (1.001GHz +/- 1*1GHz). For maxsideband=2, noise orginating at -999MHz and 3.001GHz is included (1.001GHz +/- 2*1GHz). Etc.

One issue that has been noticed with high-Q oscillators is that in order to achieve convergence, tstab must be set very large. This results in the interval used for the steady-state computation being far from zero. The fact that the distance between the start and stop time for the shooting interval is very small relative to the offset from 0 causes numerical errors that ends up reducing the accuracy of the noise simulation. However, there is a work around, start the transient interval in negative time.

Consider a 1GHz oscillator that has a Q of 10,000. This will require a shooting interval of 1ns and a tstab of 100us. To  make the start time of the shooting interval to be small, set both the start time of the PSS analysis (tstart) to be -100us while setting the stabilization time (tstab) to 100us.  That should avoid much of the ill-conditioning in the problem and result in much better noise results.

Alternatively, the approach you took of running transient first, writing the final condition into a file, then running a PSS analysis that uses this condition as a starting point should work well as long as you do not specify a long tstab in the PSS analysis. High-Q oscillators inherently need more accuracy from the simulator, so you should probably consider tightening reltol. Finally, in all noise analyses where you get more noise than you expect, you should see if it comes from the flicker noise sources. The flicker noise model parameters are often not set correctly.

Jitter Man