This report has a good write up on using FFTs to make signal amplitude measurements: http://www.schmid-werren.ch/hanspeter/publications/2012fftnoise.pdf
Some example MATLAB code. Three signals are generated at levels of -20 dB, -40 dB & -60 dB:
clear
clf
%sampling frequency
Fs=48000;
%test frequencies
f1=11300;
f2=1510;
f3=17750;
%time
t=(0:1/Fs:1);
%generate signals
% 0.1 amplitude = -20 dB
% 0.01 = -40 dB
% 0.001 = -60 dB
y=0.1*cos(2*pi*f1*t)+0.01*cos(2*pi*f2*t)+0.001*cos(2*pi*f3*t);
%add some random background noise
noise = randn(size(y));
y=y+1e-4*noise;
%wavwrite(y,Fs,'test.wav');
N=1024;
w=blackman(N);
Y=fft(y(1:N).*w');
%scaling factors
%e.g. see
%http://www.schmid-werren.ch/hanspeter/publications/2012fftnoise.pdf
scaling = 2/(mean(w)*N);
fx=Fs/N*(0:N/2-1);
Y=Y(1:N/2);
figure(1)
plot(fx,20*log10(abs(Y) * scaling))
xlabel('Frequency/Hz')
ylabel('Amplitude/dB')
grid