Classification of stackers

In machine learning, stacking is an ensemble learning technique that involves combining multiple models to improve the overall predictive performance. There are different types of stackers based on the way the base models are combined and the type of meta-model used for the final prediction. Here are some common classification of stackers:

1. Linear Stackers: Linear stacking involves training a meta-model on the predictions of the base models. The meta-model is typically a linear regression or logistic regression model that learns to combine the base model predictions by assigning weights to them. 

2. Non-Linear Stackers: Non-linear stacking involves training a meta-model on the non-linear transformations of the base model predictions. The meta-model can be a decision tree, a neural network or any other non-linear model that can learn complex relationships between the features and the target variable.

3. Hierarchical Stackers: Hierarchical stacking involves stacking multiple layers of models to make the final prediction. In this approach, the base models in the lower layers make predictions on the input data, and the predictions are passed on to the next layer of models. The final prediction is made by the meta-model in the top layer.

4. Feature Stacking: Feature stacking involves using the predictions of the base models as additional features in the training data for the meta-model. The meta-model then learns to predict the target variable using both the original features and the predictions of the base models.

5. Blending: Blending involves training the base models on a subset of the training data, and using the remaining data to train the meta-model. The predictions of the base models on the test data are then averaged or weighted to make the final prediction.

Overall, the choice of stacker depends on the nature of the problem, the size and quality of the data, and the computational resources available.