I am unfamiliar with Cadence's apnote or their procedure for extracting the PSD in situations like this, so I cannot really comment on that. But I do have a few things to offer you.
In my original paper I coupled that Matlab script with a Verilog-A model of the combined VCO and FD. It carefully controlled the sampling of the zero phase point so I was accurately measuring the period of the signal. This is really not possible in a post-processing approach, which is what the Cadence calculator implements. I rarely use the Cadence waveform calculator because for the things I tend to measure it often exhibits poor accuracy. There is a fundamental flaw in the architecture of virtually all simulator/simulation environments available today: that is the simulator runs open loop as it is given no information about what is being measured, and then the calculator picks through the resulting waveforms and uses interpolation to compute its results. In many of the measurements that are interesting to me, such those that involve jitter, noise and spectral analysis, the thing I want to measure is often smaller than the interpolation error. My approach is to supplement my circuit with Verilog-A instruments that I used to control the simulator time point placement and to extract data. Then I do the processing in Python. That way I am in control of everything.
In other words, I really don't spend much of my time wondering why the Cadence measurements are wrong. I just always assume that they will be.
Concerning using the Welch transform, it might be contributing to the problem you are seeing. If I remember correctly, the Welch transform tries to reduce its noise floor by splitting the interval into many adjacent intervals, performing the transform on them individually, and then averaging the result. This would have the effect of reducing the length of the time interval, which would make the method insensitive to low frequencies. I would not put much weight on my comments, I am responding after giving this only a few minutes of thought and after not having worked with these kind of results for many years.