matteo,
In absence of information exactly describing your issue. Here are some thoughts on what to check.
you are using a mos varactor where the source/drain is tied to your VCO core and the gate is tied to the vtune voltage.
Does the device sit inside a well?
What voltage is the well tied to?
If the well is modeled and has a pin is current entering/leaving this pin if so what is the magnitude?
Is the signal swing large enough to turn on a parasitic diode?
If this is occurring is this related to the varactor Q at some voltages?
you are experiencing problems when the loop filter is tied to the varactor describe what you are experiencing.
How balanced is your VCO tank?
Is there a fair amount of common mode voltage on the loop filter?
If so why?
Varactors shouldn't leak to a level where the performance is poor or they wouldn't be good for most of the foundries customers.
In addition:
Check to make sure your varactor is connected correctly, a lot of times checking the model and confirming the physical structure is modeled correctly is essential. At least a couple times in my carrier I have witnessed a colleagues VCO's being built where a parasitic diode was not in the model, but was there in the real application.
http://rfcooltools.com