Using Moving Averages to Decrease Data Variability

Averaging is a computationally simple way to smooth out data fluctuations in a time series. By removing noise you can expose the true underlying signal. Depending on the span chosen, either high or low periodicity is revealed by the smoothed series. This is extremely helpful when dealing with seasonal data for example.

The following function generates a simple moving average. Specify either "mean" or "med" in the type parameter depending on which statistical moment you decide to use to generate the smoothed series. Note that the first "span-1" elements in the resulting smoothed vector are set to zero, since the moving average requires exactly "span" elements to create one smoothed point.

Now generate two smoothed series, each over a span of 5 days:

Plot the two smoothed series against the original raw data and notice the decrease in day-to-day variability of the smoothed series:

A slightly more complex moving average involves a user-defined weighting scheme to increase the importance of more recent observations relative to older data. Numerous weighting schemes can be defined. As an example, the following method weights the oldest data point fractionally less than the most recent point in the series:

Detrending a Time Series

Forecasting algorithms tend to be sensitive to linear trends in your data. At worst, this can lead to blatantly biased forecasts, at best it lowers the accuracy of your forecasts. Differencing your time series data is a simple but very effective tool for removing trend from your data. Like most data-processing tools, detrending must be used judiciously. Practitioners of time series analysis recommend that at most two detrending iterations are employed. Overuse of this powerful technique can easily render your data meaningless. The following function creates a matrix containing the results of successive differencing iterations:

Compare this with the plot of the raw data at the beginning of this article. Notice that the differenced series bear little resemblance to the original time series and that the strong upward trend in the data has been removed.

This article introduced some mathematically simple yet very effective techniques for removing noise and trends from your time series data using Mathcad. In the September article you will learn about additional transformations that can be deployed within Mathcad for optimally exposing time series' information content to modeling and forecasting algorithms.