v7 - Save Metrics JSON¶

Progress¶

  ✅ STEP 1: Load Data
  ✅ STEP 2: Split Train/Test
  ✅ STEP 3: Create Pipeline
  ✅ STEP 4: Train Pipeline
  ✅ STEP 5: Evaluate & Metrics
  ✅ STEP 6: Save Pipeline
  🟡 STEP 7: Save Metrics JSON       ← NEW IN THIS VERSION

What's New¶

File	Change
`utils/metrics_utils.py`	Implemented `save_metrics_json()`
`utils/__init__.py`	Exports `save_metrics_json`
`p3_01_train_model_baseline.py`	Added STEP 7: save metrics + training summary

New Function¶

# utils/metrics_utils.py
def save_metrics_json(metrics, algorithm_name, model_params, dataset_info):
    """Save detailed metrics to results/ as a timestamped JSON file."""

What Gets Saved¶

results/
└── logistic_regression_p3_01_baseline_20260323_143052.json

Contains: all technical metrics, business metrics, model parameters, dataset info (train/test sizes, fraud rates, feature names).

Key Concept: Manual Experiment Tracking¶

In v6 we saved the model. Now we also save metrics separately for tracking experiments:

v6 saves to models/:     Pipeline pickle + basic metadata
v7 saves to results/:    Detailed metrics + params + dataset info

This is the manual way to track experiments. You manage files, naming, timestamps yourself.

Coming later: MLflow replaces both v6 + v7 with:
mlflow.log_metrics(metrics)
mlflow.log_params(model_params)
mlflow.sklearn.log_model(pipeline)
Three lines instead of two separate save functions. That's the power of MLflow.

How to Run¶

cd v7_save_metrics_ccfd-project/

# PRE-REQUISITE: Generate data first
python p1_01_generate_initial_dataset.py

# Run training script
python p3_01_train_model_baseline.py

Expected Output (full run):

STEP 1: Loading Data
Loaded 10,000 transactions from data/credit_card_transactions_latest.csv

STEP 2: Splitting Data
Split: 8,000 train | 2,000 test

STEP 3: Creating Pipeline
STEP 4: Training Pipeline
Training complete!

STEP 5: Evaluating Performance
   F1 Score: 0.2329 | Recall: 0.7667 | Net Benefit: $39,625

STEP 6: Saving Pipeline
   Model saved to: models/p3_01_baseline/

STEP 7: Saving Metrics
   Metrics saved to: results/

======================================================================
TRAINING SUMMARY
======================================================================
   Model: Logistic Regression (Baseline)
   Features: 8 (base features, no FE)
   F1 Score: 0.2329
   Net Benefit: $39,625
======================================================================
BASELINE TRAINING COMPLETE!
======================================================================

Files Created¶

File	Description
`data/credit_card_transactions_latest.csv`	Generated by p1_01
`models/p3_01_baseline/model_latest.pkl`	Trained sklearn Pipeline
`models/p3_01_baseline/metadata_latest.json`	Model metadata + metrics
`results/logistic_regression_*.json`	Detailed metrics JSON

All Steps Complete!¶

The training pipeline is fully built. See ccfd-project/ for the complete reference with full docstrings.

Next Section¶

Next	Topic	What You'll Do
04_02_Inference_API	Serve Predictions	Build FastAPI server, test via browser + scripts

Prefer to learn by watching?

The video course builds this whole project with you on screen, step by step.

Get the Video Course

Back to 04 A - Train Baseline Model