In addition to the spectrum analyszer, you generally need a couple of other things.

1. A low noise amplifier suitable for amplifying the DC voltage noise up to something measurable.  (Frequently this is AC coupled)

2. A methodolgy where you can do the measurements and math to remove the additive noise of the LNA, spectrum analyzer and associated circuits. Also, if you need to revert to AC coupling, you need to be able to measure the spectral profile of the coupling and remove the high pass filter characteristics of that.

A good sanity check when you have all of this set up is to measure the noise of a resistor. You know what the theoretical of that should be, and it serves as a good test case to validate both the additive noise and spectral weighting of your test setup.

There are low frequency spectrum analyzers out there that go down to fractions of a hertz, they get used in control system work and other "near DC" applications.

Jerry