Chapter workbooks
Predict a result, inspect the worked notebook, then apply a source-backed lesson to your own modeling decision. 65 chapter workbooks, 561 conditional lessons.
Each page keeps your answers on the page until you download them.
01: What Actually Wins on Kaggle
When the model gets more capacity on the same columns, which assessment rewards the upgrade, and which one tracks the score on subjects the model never saw?
0 lessons · Interactive chapter · Notebook on GitHub
02: The Process Mindset
How much of the winning version's public gain is still there on rows the choice never used, as the number of logged versions grows?
0 lessons · Interactive chapter · Notebook on GitHub
03: Building a CV That Matches the Test Set
When the relationship between features and outcomes changes from season to season, how far does each validation scheme's Brier score sit from the score on the next season?
20 lessons · Interactive chapter · Notebook on GitHub
04: Tracking the CV-LB Gap
With no shift and no leak, how far does the CV-LB gap move from sampling alone, and can a public sample of a given size reveal a candidate that is truly better by a small amount?
3 lessons · Interactive chapter · Notebook on GitHub
05: Adversarial Validation
When train and test differ, does the domain AUC tell you whether the target model's score has changed?
0 lessons · Interactive chapter · Notebook on GitHub
06: Running Case: March Mania 2026
Does an Elo decay chosen on past seasons find how much team strength truly carries over, and does it beat the chapter's 0.65?
1 lessons · Interactive chapter · Notebook on GitHub
07: Domain Features as Testable Hypotheses
How much does an entity-history feature add over the raw amount when the key merges several customers, and how much of its apparent value comes from rows the prediction could not see?
15 lessons · Interactive chapter · Notebook on GitHub
08: The Five-Stage Feature Engineering Process
When the target is multiplicative and skewed, does fitting log1p(y) help, and does the answer depend on the metric and on how the prediction is brought back to the raw scale?
22 lessons · Interactive chapter · Notebook on GitHub
09: Target Encoding Without Leakage
How far does each target-encoding pipeline's cross-validation estimate drift from its score on new rows?
0 lessons · Interactive chapter · Notebook on GitHub
10: Interactions, Groups, and Text
As more candidate pairs are screened, does the search score keep describing what the selected features earn on new rows?
6 lessons · Interactive chapter · Notebook on GitHub
11: When Feature Engineering Fails
How many principal components does a reduction need before it keeps the label, and what does the variance kept say about that?
2 lessons · Interactive chapter · Notebook on GitHub
12: The GBM Trio
How much does it matter whether a category is fed as ordered integer codes, as native categorical splits or as one-hot columns?
9 lessons · Interactive chapter · Notebook on GitHub
13: Hyperparameter Tuning with Optuna
As the trial budget grows, how much of the search's reported improvement shows up on rows the search never used?
7 lessons · Interactive chapter · Notebook on GitHub
14: What Doesn't Work
When the hidden grouping matters to the label, how much of that value does a K-means column recover for a boosted-tree model?
2 lessons · Interactive chapter · Notebook on GitHub
15: When the Problem Isn't Tabular
How good must a reviewer be before reviewing every answer beats the base model, and does routing change that?
0 lessons · Interactive chapter · Notebook on GitHub
16: The Fourth-Root Blend
How alike can three models' predictions be before a blend stops beating the best single model, and do the weights matter?
6 lessons · Interactive chapter · Notebook on GitHub
17: Stacking and Hill-Climbing
As the candidate library fills with near-duplicate models, how much of the hill-climbing improvement shows up on assessment rows?
4 lessons · Interactive chapter · Notebook on GitHub
18: Final Submission Selection
How much public evidence does it take before the public leaderboard picks as well as a local validation set?
25 lessons · Interactive chapter · Notebook on GitHub
19: March Mania: Prediction at Scale
If the team history quietly includes later seasons, would the development score tell you, and would the label-flip test?
0 lessons · Interactive chapter · Notebook on GitHub
20: Horse Health: Small Data, Big Discipline
How much of the accuracy a tuned class factor adds on its own OOF rows survives on new rows?
0 lessons · Interactive chapter · Notebook on GitHub
21: Financial Competitions: Time, Regime, and Leakage
How far does each validation scheme's estimate drift from the model's score on the future as the label horizon grows?
0 lessons · Interactive chapter · Notebook on GitHub
22: Vision and Multimodal: When You Need a VLM
If the label signal sits low in the embedding's variance ranking, what does a PCA that keeps 90% of the variance leave for the second stage?
13 lessons · Interactive chapter · Notebook on GitHub
23: Sensors and HAR: The Cross-Subject Problem
How far above the score on new people does a random stratified split sit, and does leave-one-subject-out close the gap?
3 lessons · Interactive chapter · Notebook on GitHub
24: Building a Kaggle Portfolio
How much of a blend's gain is lost, and what does it look like, when a teammate's OOF file is joined by position but is partly out of order?
3 lessons · Interactive chapter · Notebook on GitHub
25: The Continuous Learning System
How much of the noise behind a logged gain does averaging more fold seeds remove, and what does a seed-based threshold then keep?
20 lessons · Interactive chapter · Notebook on GitHub
26: Advanced Target Encoding
How much smoothing does a target encoding need, and does the answer change with how strongly categories really differ?
1 lessons · Interactive chapter · Notebook on GitHub
27: SHAP as a Feature Engineering Signal
When two columns carry the same information, does the one with low attribution deserve to be dropped, and what does dropping both cost?
1 lessons · Interactive chapter · Notebook on GitHub
28: Nearest-Neighbor Features
How many neighbours does it take before including the row itself stops distorting the estimate, and what does the legitimate feature gain?
4 lessons · Interactive chapter · Notebook on GitHub
29: Advanced Feature Selection
Which columns does each screen keep, and at what null percentile does a strict screen start discarding real signal?
7 lessons · Interactive chapter · Notebook on GitHub
30: External Data and Leakage
How far does a validation score built on revised data drift from deployment, and when does the training join start to cost accuracy too?
27 lessons · Interactive chapter · Notebook on GitHub
31: Deep Learning for Tabular Data
Does a neural model that is worse on its own lower the blend's error, and does that survive as the tree model gets more data?
5 lessons · Interactive chapter · Notebook on GitHub
32: Denoising Autoencoders for Tabular Pretraining
Does a lower reconstruction error mean the pretrained features help a small labeled model more than an untrained encoder would?
3 lessons · Interactive chapter · Notebook on GitHub
33: Data Augmentation for Tabular Data
How much does resampling before the split inflate cross-validation, and does resampling inside the fold beat weighting or doing nothing?
1 lessons · Interactive chapter · Notebook on GitHub
34: Pseudo-Labeling
How much does the shortcut teacher inflate the student's cross-validation score, and what does honest pseudo-labeling add?
8 lessons · Interactive chapter · Notebook on GitHub
35: Memory Optimization for Large Datasets
At what ID size does a blanket float32 downcast start merging customers, and how much memory does it save compared with a checked downcast?
15 lessons · Interactive chapter · Notebook on GitHub
36: Categorical Embeddings
How large can the share of unseen categories get before an integer label code is worse than dropping the column, and what does the unknown rule cost a target encoding?
4 lessons · Interactive chapter · Notebook on GitHub
37: Stacking Deep
How far does a meta-model's score on its own OOF rows drift from an honest assessment as the library of base models grows, and when does a regularized meta-model overtake the unregularized one?
43 lessons · Interactive chapter · Notebook on GitHub
38: Calibration and Post-Processing
How large must the development set be before isotonic calibration beats the simpler Platt calibrator on new rows?
6 lessons · Interactive chapter · Notebook on GitHub
39: Imbalanced Data
How much log loss does negative downsampling cost, and at what keep rate does the analytic correction stop being as good as calibrating on development rows?
8 lessons · Interactive chapter · Notebook on GitHub
40: Knowledge Distillation
Do soft teacher targets beat hard labels for a small neural student, or does that depend on how well the hard-label baseline is tuned?
0 lessons · Interactive chapter · Notebook on GitHub
41: Online Learning
How much of an adaptive model's advantage over a frozen model survives as labels arrive later, and how wrong is a replay that ignores the delay?
1 lessons · Interactive chapter · Notebook on GitHub
42: Image Augmentation
When does adding transformed copies of the training images raise held-out accuracy, and when does it lower it?
13 lessons · Interactive chapter · Notebook on GitHub
43: Image Classification Tricks
How much does a random split of crops overstate accuracy compared with splitting by source image, and how does that grow with crops per image?
30 lessons · Interactive chapter · Notebook on GitHub
44: Loss Functions for CV
Does focal loss beat cross-entropy on the metric that is scored, and what does it cost elsewhere?
3 lessons · Interactive chapter · Notebook on GitHub
45: Metric Learning and ArcFace
How much does the angular margin improve retrieval of unseen identities over a cross-entropy descriptor, and where does a larger margin stop helping?
15 lessons · Interactive chapter · Notebook on GitHub
46: Segmentation Architectures
Which threshold and post-processing rule is best depends on how Dice is averaged: how large is that effect, and when does it appear?
15 lessons · Interactive chapter · Notebook on GitHub
47: Medical Imaging Patterns
How far does image-level cross-validation overstate the score on new patients, and does grouping by patient remove the error?
17 lessons · Interactive chapter · Notebook on GitHub
48: Pseudo-Labeling for CV
Does a stricter confidence threshold make pseudo-labeling pay, and does it remove the leak from a teacher that saw the validation labels?
2 lessons · Interactive chapter · Notebook on GitHub
49: Text Feature Engineering
At what typo rate, and at what term coverage, does a character TF-IDF model overtake a word model?
22 lessons · Interactive chapter · Notebook on GitHub
50: LLM Fine-Tuning for Kaggle
How does the rank of a LoRA-style update change held-out error against the original checkpoint and a full update when target data is small?
24 lessons · Interactive chapter · Notebook on GitHub
51: Time Series Cross-Validation
How much does a backtest that ignores label availability flatter each recipe, and does it still rank the recipes correctly?
14 lessons · Interactive chapter · Notebook on GitHub
52: Time Series Feature Engineering
How much does a backtest flatter a model whose lag features read days that would not yet exist at the forecast origin?
36 lessons · Interactive chapter · Notebook on GitHub
53: Financial Competition Patterns
When does removing factor exposure from the features help the score, and when does it cost correlation?
1 lessons · Interactive chapter · Notebook on GitHub
54: Combinatorial Optimization
With a fixed number of evaluated moves, does a hotter starting temperature or a better starting tour give the shorter final tour?
6 lessons · Interactive chapter · Notebook on GitHub
55: Pancake Sorting: A Constructed Search Example
Under the same beam width, how many instances does each ranking actually solve, and how far from optimal are its solutions?
0 lessons · Interactive chapter · Notebook on GitHub
56: Reinforcement Learning for Game Competitions
At what size of intermediate reward does a learner stop trying to win, and does the shaped return reveal it?
4 lessons · Interactive chapter · Notebook on GitHub
57: PatchTST for Time Series and HAR
Does patching beat a feature baseline at the same forward origins, and how does the patch length change the answer and the token count?
6 lessons · Interactive chapter · Notebook on GitHub
58: Sensor Embeddings and Augmentation
When does a left-right swap help a wearable activity classifier, and what does it cost when the label is not updated with the swap?
11 lessons · Interactive chapter · Notebook on GitHub
59: Threshold Optimization
How far does the development F1 of a tuned threshold overstate its F1 on fresh labels, and how does that depend on the size of the development set?
5 lessons · Interactive chapter · Notebook on GitHub
60: Multi-Target Learning
When does a dense auxiliary target improve a scarce main target on new rows, and how much does the loss weight matter?
4 lessons · Interactive chapter · Notebook on GitHub
61: Playground Series Strategy
How much of the public leader's visible advantage is still there on private rows as more near-equal candidates are screened?
2 lessons · Interactive chapter · Notebook on GitHub
62: The Kaggle Landscape, 2024 to 2026
At what deadline does a stronger model stop being the better pipeline, and what does a retained feasible reference recover?
5 lessons · Interactive chapter · Notebook on GitHub
63: Lessons from the Expanded Winning-Solution Corpus
When winners use a technique more often than the field does, how much does that say about the technique's effect?
0 lessons · Interactive chapter · Notebook on GitHub
64: Audio and Signal Competitions
How far does chunk-level cross-validation overstate accuracy on new recordings, and does it distort the comparison between models?
18 lessons · Interactive chapter · Notebook on GitHub
65: Recommendation Systems
As the candidate cap grows, how much of the rising oracle score does a trained reranker actually capture?
13 lessons · Interactive chapter · Notebook on GitHub