v7 Save Metrics JSON¶
View code on GitHub Get the Video Course
Progress¶
✅ STEP 1: Load Data
✅ STEP 2: Split Train/Test
✅ STEP 3: Create Pipeline
✅ STEP 4: Train Pipeline
✅ STEP 5: Evaluate & Metrics
✅ STEP 6: Save Pipeline
🟡 STEP 7: Save Metrics JSON ← NEW IN THIS VERSION
What's New¶
| File | Change |
|---|---|
utils/ | Implemented save_ |
utils/ | Exports save_ |
p3_ | Added STEP 7: save metrics + training summary |
New Function¶
# utils/metrics_utils.py
def save_metrics_json(metrics, algorithm_name, model_params, dataset_info):
"""Save detailed metrics to results/ as a timestamped JSON file."""
What Gets Saved¶
Contains: all technical metrics, business metrics, model parameters, dataset info (train/test sizes, fraud rates, feature names).
Key Concept: Manual Experiment Tracking¶
In v6 we saved the model. Now we also save metrics separately for tracking experiments:
This is the manual way to track experiments. You manage files, naming, timestamps yourself.
Good to know: tools like MLflow (not part of this course) replace both v6 + v7 with:
Three lines instead of two separate save functions. That's the power of MLflow.
Not part of this course
MLflow, DVC, Kubernetes, CI/CD and Kubeflow belong to MLOps, the next layer. This course covers the machine learning side, end to end, and stops at the API. See Important Notes, note 3.
How to Run¶
cd v7_save_metrics_ccfd-project/
# PRE-REQUISITE: Generate data first
python p1_01_generate_initial_dataset.py
# Run training script
python p3_01_train_model_baseline.py
Expected Output (full run):
STEP 1: Loading Data
Loaded 10,000 transactions from data/credit_card_transactions_latest.csv
STEP 2: Splitting Data
Split: 8,000 train | 2,000 test
STEP 3: Creating Pipeline
STEP 4: Training Pipeline
Training complete!
STEP 5: Evaluating Performance
F1 Score: 0.2329 | Recall: 0.7667 | Net Benefit: $39,625
STEP 6: Saving Pipeline
Model saved to: models/p3_01_baseline/
STEP 7: Saving Metrics
Metrics saved to: results/
======================================================================
TRAINING SUMMARY
======================================================================
Model: Logistic Regression (Baseline)
Features: 8 (base features, no FE)
F1 Score: 0.2329
Net Benefit: $39,625
======================================================================
BASELINE TRAINING COMPLETE!
======================================================================
Files Created¶
| File | Description |
|---|---|
data/ | Generated by p1_01 |
models/ | Trained sklearn Pipeline |
models/ | Model metadata + metrics |
results/ | Detailed metrics JSON |
All Steps Complete!¶
The training pipeline is fully built. See ccfd-project/ for the complete reference with full docstrings.
Next Section¶
| Next | Topic | What You'll Do |
|---|---|---|
| 04_02_Inference_API | Serve Predictions | Build FastAPI server, test via browser + scripts |
Prefer to learn by watching?
The video course builds this whole project with you on screen, step by step.
Back to 04 A Train Baseline Model v6 Save Pipeline Next: Complete Reference