I believe that the jitter you are computing is effectively the edge-to-edge jitter, J_ee. I don't talk about it in the original paper, though I do in in the most recent one. It only is valid in driven systems (i.e. for PM jitter sources). Consider a system driven with a clean or noise free square wave, and assume each transition at the input creates a transition on the output, but that the system is noisy so that the output signal exhibits jitter. Edge-to-edge jitter is then the variation in the time between an input transition and the resulting output transition. If the system is exhibiting simple PM jitter (the noise is white and so the jitter in the transitions is uncorrelated) then the k-cycle jitter J_k = sqrt(2)*J_ee for all k because J_k contains the jitter from two transitions where J_ee contains the jitter from only one transition.

To see that the jitter you compute is J_ee, realize that the instantaneous jitter j is related to the instantaneous phase noise phi by
   j(t) = phi(t)*T/(2*pi)
When you divide L by 2 you are converting L to Sphi using eqn 30 (and so implicitly assuming that L contains only phase noise), and then when you integrate it over frequency you are computing the total noise power in Sphi, which equals the variance of phi. Applying the square root converts the variance to the standard deviation, which gives you a number in radians (rather than radians², which is the units for phase noise power). You can then either convert to degrees by multiplying by 360/(2*pi) or to jitter in seconds by multiplying by T/(2*pi), which is J_ee.

You did not integrate over all frequency, you integrated from f_a to f_b where f_a > 0 and f_b < f₀. For your result to be reasonable it must be that the phase noise below f_a and above f_b is negligible. This is only true with driven systems. With autonomous systems, the phase noise always goes to infinity as f->0. You are "okay" because you are looking at a frequency synthesizer that is assumed to be locked to a clean source and so is a driven system. It also helps to explain why J_ee is only defined for driven systems.

Concerning your comments about the correlation in the lower and upper sidebands, we are both assuming that the noise in the signal is completely in the phase (there is no amplitude noise) and so, in fact, the sidebands are completely correlated. I discuss this phenomenon in both my papers on RF simulation (designers-guide.com/Analysis->Introduction to RF simulation and its application (also in JSSC Sep99) and my paper and presentation on cyclostationary noise (designers-guide.com/Theory->Noise in mixers, oscillators, samplers, and logic: an introduction to cyclostationary noise). However, the factor of two is valid, as you can see by examining the derivation of eqn 30.

If they are integrating the noise over 12kHz to 20MHz then the noise outside that range must be negligible. Clearly the noise below 12kHz in an oscillator is not small, so it must be that the application is such that it is insensitive to noise below this frequency. It probably means that whatever follows the oscillator is able to track its phase up to about 12kHz. So while I believe the measurement makes sense, I do not know what to call the result. That seems to be a general problem with jitter. The terminology is often not well defined and inconsistent definitions exist.