[Edit: Change the formula for the normalized squared distance slightly: the change is to divide the squared distance by the sum of the variances, instead of dividing by the mean variance or by the larger variance. Rename this measure the "sum of normalized squared distance" (SNSD, instead of NSD).]
I like the answer from @Mathieu NOE. I always like and learn from his answers. Here is a diferent approach. A quantitative measure of similarity (or actually, difference) is the sum of the normalized squared distances (in the complex plane) between the two FRFs, at corresponding frequencies. Let's assume the two FRFs are measured at the same frequencies. (If the FRFs are not measured at the same frequncies, then interpolate to a common set of frequencies, and ignore frequency ranges where the FRFs don't overlap.)
Define the sum normalized squared distance (SNSD) between two spectra as follows:
In the equation above, X and Y are the complex FRF estimates, and 
and 
are the estimated variance of the X and Y, at frequency 
. If the spectra X and Y are the same, except for random measurement noise, then SNSD has a chi-squared distribution with 2N degrees of freedom, where N is the number of points compared. Of course, that assumes that certain conditions are true (independence, normality of the variability of the real and imaginary parts). But even if those conditions are not true, it's still a reasonable measure of difference. The low-coherence parts of the spectrum will have high variance, which means they will not count as much in the measure of difference. That is good, because it gives less weight to the noisy parts of the frequency response estimates.
The tricky part is estimating the variance of each FRF at each frequency.
Notes on transfer function estimation are attached.
I am also attaching a script, spectrumDifferenceTest.m, which demonstrates (by Monte Carlo simulation) that SNSD does in fact have a chi-squared distribution with 2N degrees of freedom, when the two spectra are the same except for random variation. It computes the SNSD for 1000 independent cases, using 100 points per spectrum in every case. The script shows that the mean and variance of SNSD have the expected values (2N and 4N respectively), and that the probability histogram of SNSD values is very similar to the probability density function of the chi squared distribution with 2N degrees of freedom (figure below). In this script, the variance of each FRF at each frequency is known. I should provide for you a formula to estimate the variance of the FRF estimate, at each frequency.
