Spaceship Titanic Kaggle Prediction

Abstract

A Kaggle classification experiment on the Spaceship Titanic dataset. The pipeline engineers cabin, spending, and group features from raw tabular data, handles missing values, and combines LightGBM and CatBoost gradient-boosting classifiers into a single ensemble for competition submission. The interesting part is the feature-engineering layer and how two different boosting algorithms complement each other.

1. What This Is

A 2025 Kaggle competition entry. The task is classification of passenger outcomes from a sci-fi "Titanic" dataset. Rather than a single scikit-learn pipeline, the repo provides a repeatable preprocessing path plus a two-model gradient-boosting ensemble. The project is marked closed in the portfolio; the public repository documents the actual modeling workflow.

2. How It Works

The workflow is a linear five-stage pipeline from raw CSV to a competition-format submission file.

3. Implementation Notes

3.1 Feature Engineering

The raw dataset contains cabin strings, spending columns, and group identifiers that are not directly usable as model inputs. I derived cabin-level aggregates (deck, number, side), spending ratios, and group-size features. Missing values are imputed before encoding so that the boosting trees see a consistent numeric schema.

3.2 Two-Model Ensemble

LightGBM and CatBoost build trees differently—LightGBM uses leaf-wise growth with histogram binning, CatBoost uses ordered boosting with its own categorical handling. Combining their predictions reduces the variance of either single model. The ensemble here is a simple combination of the two output sets before writing the submission.

4. Constraints

• Single-dataset scope

  The pipeline is tuned to one Kaggle competition dataset. There is no cross-validation loop documented, so generalisation beyond the Spaceship Titanic split is unverified.

• No hyperparameter search

  Both boosters run with default or lightly adjusted parameters. Without a tuning pass (e.g. Optuna), the ensemble ceiling is lower than what the data might support.

• Simple ensemble blend

  The combination is a direct merge of two prediction sets, not a stacking meta-learner. Weighted or learned blending could capture complementary error patterns better.

• Competition artifact only

  The output is a submission CSV. There is no API, no model serialisation, and no deployment path—this is a research/competition script, not a service.

5. Next

1. a. Add a stratified k-fold cross-validation loop to get a stable out-of-fold score before the final submission.
2. b. Run an Optuna hyperparameter search on both LightGBM and CatBoost to lift the individual model ceilings.
3. c. Replace the simple blend with a logistic-regression stacking meta-learner trained on out-of-fold predictions.