Skip to content

v7 Save Metrics JSON

View code on GitHub Get the Video Course

Progress

  ✅ STEP 1: Load Data
  ✅ STEP 2: Split Train/Test
  ✅ STEP 3: Create Pipeline
  ✅ STEP 4: Train Pipeline
  ✅ STEP 5: Evaluate & Metrics
  ✅ STEP 6: Save Pipeline
  🟡 STEP 7: Save Metrics JSON       ← NEW IN THIS VERSION

What's New

File Change
utils/metrics_utils.py Implemented save_metrics_json()
utils/__init__.py Exports save_metrics_json
p3_01_train_model_baseline.py Added STEP 7: save metrics + training summary

New Function

# utils/metrics_utils.py
def save_metrics_json(metrics, algorithm_name, model_params, dataset_info):
    """Save detailed metrics to results/ as a timestamped JSON file."""

What Gets Saved

results/
└── logistic_regression_p3_01_baseline_20260323_143052.json

Contains: all technical metrics, business metrics, model parameters, dataset info (train/test sizes, fraud rates, feature names).

Key Concept: Manual Experiment Tracking

In v6 we saved the model. Now we also save metrics separately for tracking experiments:

v6 saves to models/:     Pipeline pickle + basic metadata
v7 saves to results/:    Detailed metrics + params + dataset info

This is the manual way to track experiments. You manage files, naming, timestamps yourself.

Good to know: tools like MLflow (not part of this course) replace both v6 + v7 with:

mlflow.log_metrics(metrics)
mlflow.log_params(model_params)
mlflow.sklearn.log_model(pipeline)
Three lines instead of two separate save functions. That's the power of MLflow.

Not part of this course

MLflow, DVC, Kubernetes, CI/CD and Kubeflow belong to MLOps, the next layer. This course covers the machine learning side, end to end, and stops at the API. See Important Notes, note 3.

How to Run

Terminal
cd v7_save_metrics_ccfd-project/

# PRE-REQUISITE: Generate data first
python p1_01_generate_initial_dataset.py

# Run training script
python p3_01_train_model_baseline.py

Expected Output (full run):

STEP 1: Loading Data
Loaded 10,000 transactions from data/credit_card_transactions_latest.csv

STEP 2: Splitting Data
Split: 8,000 train | 2,000 test

STEP 3: Creating Pipeline
STEP 4: Training Pipeline
Training complete!

STEP 5: Evaluating Performance
   F1 Score: 0.2329 | Recall: 0.7667 | Net Benefit: $39,625

STEP 6: Saving Pipeline
   Model saved to: models/p3_01_baseline/

STEP 7: Saving Metrics
   Metrics saved to: results/

======================================================================
TRAINING SUMMARY
======================================================================
   Model: Logistic Regression (Baseline)
   Features: 8 (base features, no FE)
   F1 Score: 0.2329
   Net Benefit: $39,625
======================================================================
BASELINE TRAINING COMPLETE!
======================================================================

Files Created

File Description
data/credit_card_transactions_latest.csv Generated by p1_01
models/p3_01_baseline/model_latest.pkl Trained sklearn Pipeline
models/p3_01_baseline/metadata_latest.json Model metadata + metrics
results/logistic_regression_*.json Detailed metrics JSON

All Steps Complete!

The training pipeline is fully built. See ccfd-project/ for the complete reference with full docstrings.

Next Section

Next Topic What You'll Do
04_02_Inference_API Serve Predictions Build FastAPI server, test via browser + scripts

Prefer to learn by watching?

The video course builds this whole project with you on screen, step by step.

Get the Video Course


Back to 04 A Train Baseline Model v6 Save Pipeline Next: Complete Reference

🎉 New Course
Machine Learning End-to-End: Credit Card Fraud Detection
$15.99 $84.99 81% OFF
Get the Video Course