The article is based on research done by Qi-Qiao He, Patrick Cheong-Iao Pang, and Yain-Whar Si on Transfer Learning for Financial Time Series Forecasting and Multi-source Transfer Learning with Ensemble for Financial Time Series Forecasting.

Time Series forecasting is the process of analyzing time-series data and making predictions based on it. It is one of the most challenging tasks in data analytics and machine learning. Time-series prediction plays a crucial role in a variety of applications including weather forecasting, economic forecasting, business forecasting, production planning, etc. Over the past decades, a lot of effort has been contributed to the development of time series prediction, such as:

1. Traditional linear models: AutoRegressive model (AR), Moving Average model (MA), AutoRegressive Moving Average model (ARMA), and AutoRegressive Integrated Moving Average (ARIMA).
2. Machine learning techniques: Neural Networks (NN, DNN, CNN, RNN), Long Short-Term Memory(LSTM), Extreme Learning Machines (ELM), Support Vector Machines (SVM).

Issues with deep learning models

Empirical results demonstrate that these nonlinear machine learning methods usually have preferable and more reliable performance in comparison with linear methods. In recent years, deep learning models have been highly successful in image processing and Natural Language Processing (NLP) applications thus becoming a driving force behind the adoption of deep learning models for time-series forecasting. But deep learning have there own problems in time-series data:

• The biggest constraint in training a model is obtaining a sufficient amount of training data. The deeper (greater number of layers) your model, the more weights/parameters need to be estimated which increases the need for a larger quantity of training data.
• Deep learning algorithms designed for time series prediction are based on the assumption that training samples and testing samples are drawn from the same distribution. But in most practical situations, time-series data are most likely to vary over time, resulting in a wide variability between old data and new data.

Transfer Learning: how to learn from little data

Transfer learning is a machine learning method where a model developed for a task is reused as the starting point for a model on a second task. Transfer learning is about leveraging feature representations from a pre-trained model, so you don't have to train a new model from scratch.

Transfer Learning process:

1. Create a Neural Network for the new task by using some number of layers that are closest to the input of a trained model for some source task. Must select a related predictive modeling problem with an abundance of data.
2. The model fit on the source task can then be used as the starting point for a model on the second task of interest. Append new untrained layers to the target task.
3. Optionally, the model may need to be adapted or refined on the input-output pair data available for the task of interest.
4. Finally use regression or classification to get the output.