What is It?
When a periodic time dependent data set, such as NDVI, is decomposed into sum of sinusoidal functions, the procedure is called Harmonic ANalysis of Time series (HANTS) or Fourier Analysis. At the end, one can obtain phase and amplitudes, which can be used for further processing and analyzing of the data set.
Why HANTS?
Moving data from time or space into frequency domain may be done for various reasons. The main advantage of this is that it becomes much easier to study how the data is varying based on different frequencies. However, my focus here is on the three applications:
- Smoothing the data and removing the outliers,
- Gap filling or obtaining the missing data,
- Data compression.
How to perform HANTS?
The goal in this step is to find a set of coefficients which shows the contribution of each sine or cosine function in reconstructing the original data set. There are various method to obtain this. Here, the coefficients are obtained through least square curve fitting method, based on the algorithm/software provided on:
http://gdsc.nlr.nl/gdsc/en/tools/hants.
The NLR HANTS, freely available to download from above link, calculates the Fourier series coefficients while filtering out the outliers. This software originally was developed to remove the cloud effect in various remote sensing applications and perform temporal interpolation to obtain a gapless data set when there is no satellite image or product available. However, it can be also used to perform data compression.
The software is about 2MB and freely available to download. However, the software is available in executable (binary) format. As more and more students, scientists, and researchers are using MATLAB these days, it was decided to also provide the MATLAB version of the main function, i.e. the HATNS itself.
MATLAB version of HANTS?
The MATLAB version of HANTS is the exact translation of the original FORTRAN code developed by prof. Wout Verhoef to MATLAB m-file. The only difference is that the sort and matrix inversion available in MATLAB are used instead of the ranking and the inversion algorithm used in the FORTRAN code. A sample output of the MATLAB version is shown below.
Smoothing a data sets and removing the outliers
There are four MATLAB files available:
- HANTS.m: which calculates amplitude, phase, and the smoothed data for a single time series.
- ApplyHANTS.m: which calls HANTS for each pixel of a data set assuming the input data set has the dimension of [time,lat,lon].
- ReconHANTSData.m: receives the amplitude and phase and reconstructs the smoothed time series.
- ReconstructImage.m: receives the amplitude and phase and reconstructs the spatial data set assuming that both phase and amplitude have the dimension of [nf+1,lat,lon], where nf is the number of frequencies above zero frequency, one of the inputs of HANTS.m.
The figure below shows smoothed and gap filled land surface temperature provided through MODIS products.
Data Compression:
Once it is decided to use the smoothed data set, it is possible to perform data compression. The entire data sets can be reconstructed by knowing only the phase and the amplitudes. In above figure only three frequencies above zero frequency was used. This means to reconstruct the red line, i.e. the data that we want to use, only 4 amplitudes and 3 phase are needed, i.e. 7 numbers, while the entire time series consist of 366 numbers for one year. This means a compression rate of 52.3. This can be very beneficial for spatial data. As an example a MODIS tile with 1200X1200 pixels and one year of daily data consists of 527.04 data entry. If this many elements are about to be stored as floating points one needs 2011MB, whereas storing only the phases and amplitudes requires only 38 MB. This is a huge reduction of data volume. Yet, it is still possible to use one of the compression programs (gzip, rar) to further compress the file.
NOTE: In order to obtain the MATLAB version of HANTS algorithm, you can either download it from here or send me an e-mail.
Welcome to the website of Mohammad Abouali 
Dear Sir, I am Md basha from India, i want to apply Harmonics analysis for my time series temperature data to find out the amplitude and phase for Diurnal semi diurnal variation. please send me the program for Harmonic analysis.
Thanks for your interest in HANTS implementation on MATLAB. I have attached the program to the separate e-mail. Please feel free to contact me if there was any questions.
Dear Sirs I’m interested in matlab code of HANTS algoritmh. I’m working on daily time series of Modis land surface temperature. Thank you in advance. Silvia Alfieri
Hi，could you send me the program (matlab version )please？I want to analyze the diurnal cycles of rainfall in my research. Thanks !
Dear Mohammad aziz
I am very glad find you again in the same related topic to my work. I also used HANTS a lot during my research, I am wondering if it possible extend the number of image which HANTS can handles to for example 3*12*744 or not, i mean is it any restriction about number of images. Another hints for my experiment is that in binary version of HANTS, we select NOF (no of ferequency) and based on that more relevant amplitude calculate and used to reconstruct smooth image. I think if it is possible to do it for each pixel separately it would be give more accurate results.
Thanks In advance
Hamid Ghafarian
Dear Hamid,
It is nice to hear from you again. I am happy that you find MATLAB implementation of HANTS interesting. This MATLAB version, which I will send you in separate e-mail, does not have any restriction on the number of images that can be used. If you can hold the entire data set in the memory, it shouldn’t be any problem. Though, I have not test it really for longer than 366 images per year.
I do agree that if you provide more frequencies, i.e. bigger NoF, the time series is reconstructed much more exactly. This means that the reconstructed harmonic functions will resemble the original time series better. In fact if you have 366 data entry in one time series, and you choose NoF to be 366 then the reconstruction would be exact; hence, no smoothing at all.
However, the point of using this implementation of HANTS is that your data set has some outliers. Therefore, you do not want to completely reconstruct the data sets. And you want to get rid of those outliers. At the end, it might take couple of try and error to find out the optimized number for NoF. (Since this is MATLAB and looping in MATLAB is slow; therefore, it is suggested to apply it first on small sub set of the image).
Current implementation of HANTS on MATLAB, accepts only one NoF for the entire image. So, no changes in that aspect. But including the varying NoF across the image isn’t that hard. Feel free to make changes to the code. And If you wanted we can include that modified version also in the distribution.
Once again, I am so happy to hear from you again. Keep in touch my friend.
regards,
Hi, could you send the matlab files ?
(HANTS.m)
Thank you for your work
Just sent you the files to this e-mail that you have provided. Let me know if there was any questions.
hi Hamid, I am looking for the matlab algorithm of HANSTS. May I get this. thank you in advance.
Hi, could you send the matlab files
(HANTS.m;ApplyHANTS.m;ReconHANTSData.m;ReconstructImage.m)?
Thank you in advance. Salvatore Falanga Bolognesi
Hi, I am working with MODIS NDVI 8-day composite, can you suggest how to select parameters in HANTS
In HANTS.m all the input variables are defined. I copy paste it here for further information
% Inputs:
% ni = nr. of images (total number of actual samples of the time
% series)
% nb = length of the base period, measured in virtual samples
% (days, dekads, months, etc.)
% nf = number of frequencies to be considered above the zero frequency
% y = array of input sample values (e.g. NDVI values)
% ts = array of size ni of time sample indicators
% (indicates virtual sample number relative to the base period);
% numbers in array ts maybe greater than nb
% If no aux file is used (no time samples), we assume ts(i)= i,
% where i=1, …, ni
% HiLo = 2-character string indicating rejection of high or low outliers
% low = valid range minimum
% high = valid range maximum (values outside the valid range are rejeced
% right away)
% fet = fit error tolerance (points deviating more than fet from curve
% fit are rejected)
% dod = degree of overdeterminedness (iteration stops if number of
% points reaches the minimum required for curve fitting, plus
% dod). This is a safety measure
% delta = small positive number (e.g. 0.1) to suppress high amplitudes
%
Dear Mohammad aziz,
Hi, I am working on the time series processing and notice this HANTS page. May I have the matlab files you mentioned? Thanks in advance!
I think one critical issue in HANTS is the selection of the harmonic periods. For 10-day NDVI time series, the pre-defined harmonic periods (36, 18 and 12) may be optimal, but for other time series, it may not be. I am thinking to use FFT to identify some “characteristic” frequency of my time series and use those periods rather than the priori set periods to perform HANTS. Do you think there will be any problem doing this? Thanks for your time reading and thinking about it.
Regards,
In the MATLAB version there is no priori defined periods. You have to choose the values yourself.
Hi! thanx for your reply….
whenever i use hants for smoothing the NDVI curve, it makes the NDVI curve of crops smooth but it deteriorate the curve of forest area…why it is so????
Could you please send me a sample so I can see how it deteriorates the curve?
hi,
HANTS creates the amplitude and phase images…..can u please explain what does this phase images reprsents?? and what is the meaning of phase offset….plz explain in detail..
Hi Neha,
If you want more information on the theory and how to interpret the results I suggest going on http://gdsc.nlr.nl/gdsc/en/tools/hants. The report available on http://gdsc.nlr.nl/gdsc/sites/default/files/publications/2005-474-cr.pdf and the references there are also a great source of information on HANTS algorithm.
regards
hi,
thank you so much
Hi,
I work actually on ndvi time series from SPOT-VGT.
Could you send me the matlab files please ?
Thanks for your work
You must have received MATLAB version of HANTS algorithm in your e-mail by now. Let me know if there was any questions.
Dear Sirs,
I am working on harmonic analysis of precipitation over Southwest. I am interested in in your matlab code (HANTS). Can you please send me this code on my email.
Thank you in advance.
Suru
Dear sir,
Could you please send me the MATLAB version of HANTS algorithm?
Regads
Mouhamad
Dear Sir,
I am a student learning to process the SPOT-VGT NDVI recently. I am wondering if you could kindly share with me the MATLAB version of the HANTS algorithm? I think it will greatly help me learn about the time series analyses. Thanks in advance for your kind help!
Dear Sir,
I am a student learning to calculate the Harmonic Analysis of MODIS 8-Day Best data using Vegetation Indices recently. I am wondering if you could kindly share with me the MATLAB version of the HANTS algorithm? I think it will greatly help me learn about the time series analysis. Thanks in advance for your kind help!
Dear Sir, I would also like to receive a copy of the HANTS matlab file if possible, for use in NDVI timeseries analysis. Also, thank you very much for sharing this post and for such a useful contribution! All the best.
Dear Sir,
I am a Civil Engineering graduate student. I would like to use the MATLAB codes to perform the HANTS algorithm on daily MODIS LST data. Could you please email me the MATLAB codes for the HANTS algorithm? Your help will be greatly appreciated.
Thanks,
Ryne C. Phillips
Hi,
How can i obtain predictions from this library. For instance, say i run the harmonics analysis for 1:n to obtain the amplitude and phase parameters. But now, i want to get the reconstructed values for n+1:N. Is it possible?
Well this approach assumes periodic behavior. So, if you applied hants on n points and n is your base period, then the value at n+1 is equal to the value at point 1.
In general if your period is T then the value at point i is equal to the value at point i+T
Can I get your harmonic analysis tool? I want to apply for my research about diurnal cycle. Thank you in advance.
Of course, the entire code along with some sample data are uploaded on MATHWORKS and can be downloaded free of charge. Here is the link:
http://www.mathworks.com/matlabcentral/fileexchange/38841-matlab-implementation-of-harmonic-analysis-of-time-series-hants
Start with Test*.m to get acquainted with the code. Let me know if you had any other questions.
Dear mabouali,
Thank you very much for your matlab codes to perform the HANTS algorithm. Now I use these codes and meet with some difficulties. I calculate the Harmonic Analysis of GIMMS 15-Day NDVI data recently. I would like to reconstruct the daily NDVI based on 24 NDVI values in one year at each grid. First I applied the ApplyHants.m to get the smoothed data which is based on 24 NDVI values. But I do not know how to transfer the 24 smoothed NDVI data (2 data in each month, so 24 NDVI data in one year) to 365 smoothed data (365 days in one year). Could you please explain it to me? Your help will be greatly appreciated.
Best wishes
liugh
Was there an answer to this question? I am having a similar problem.
Vincent
Dear mabouali,
I’m using HANTS(http://gdsc.nlr.nl/gdsc/en/tools/hants) now.. But output file has no extension(aaa_pif, aaaA001_bin, aaa_psf, aaa_pof…). How can I read this file..?
Are there some programs to read this file? then, Could you send me the program by e-mail..please.
Your help will be greatly appreciated.
I responded to your email. hope that solved the problem.
Dear Sir,
Greetings,
Got a useful information regarding HANTS from your post and has helped me in my work. I m working with MODIS 16 bit signed data with 8 day temporal interval. I amusing a Hants software for windows, in which it shows error phase file does not exist.Please tell me how to rectify it. I have tried many time n have cross checked my inputs parameters.
Thanking You
Revati M
Is there an R option to implement this?
It might be. I don’t know
Hello,
I would like to analysis of diurnal cycle of rainfall.Could i get first harmonic analysis code?
You can download it from
http://www.mathworks.com/matlabcentral/fileexchange/38841-matlab-implementation-of-harmonic-analysis-of-time-series–hants-/content//HANTS/HANTS.m
Dear M. Abouali,
Thank you for sharing your program, it seem to be a very powerful program as I try it on my own data, and it can plot a function very close to my data. Right now, I am learning how to use it to determine phase and amplitudes of a time series data because I need that for my work. From your examples (Hi, Lo and none), there are arrays representing amplitudes and phases given by ‘amp_??’ and ‘phi_??’, I try to understand what it is about and please correct me if I am wrong.
1. Are the amplitudes given by the sum of the amp_?? for example, sum(amp_Hi) = 107.828, which is roughly equal to the max(yr_Hi).
2. Can you tell me how to select the value of nf?
3. I cannot understand how to use the phase array, for example, phi_Hi =
0
180.924
319.48
157.083
But from the graph the phase of this time series seem to be within 40 (scale of the x-axis), and I don’t understand how this number related to the phase.
Hi there,
I am interested to know how to obtain phase and amplitudes from your program, based on your
example, which array represent the phase and amplitudes?
Regards,
Calvin
If you look at the function definition:
function [amp,phi,yr]=HANTS(ni,nb,nf,y,ts,HiLo,low,high,fet,dod,delta)
the first one is amplitudes and the second one is phase.
Dear Mabouali,
Would you kind enough to send me a Matlab version of HANTS which is used to determine high frequency signal with reasonalbe amplitude of magnetometer data? (140 data points of 1 day at a given longitude: Totally I have 72 longitude and 365 days)
You can download it from http://www.mathworks.com/matlabcentral/fileexchange/38841-matlab-implementation-of-harmonic-analysis-of-time-series–hants-
Hi Abouali,
Please provide an example of how you did the gap filling using HANTS similar to the second graph above. I have data for a years period with gaps (NaNs). I need to estimate the phi and amp from the same data then gap fill it. Will appreciate.
Thanks
Vincent
Hi Abouali,
How can I cite this package?
Cheers,
Dongdong