52 lines
2.6 KiB
Markdown
52 lines
2.6 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Project Overview
|
||
|
||
A computer vision + LLM pipeline that extracts rowing machine workout data from photos of Concept 2 PM5 displays. Photos go through screen detection, classification, and OCR-via-LLM to produce structured workout metrics.
|
||
|
||
## Pipeline
|
||
|
||
```
|
||
photos/ → crop_to_screen.py → screen_classifier.py → extract_rowing_data.py → rowing_results.csv
|
||
```
|
||
|
||
1. **crop_to_screen.py** — Detects and perspective-corrects the LCD screen region using OpenCV edge detection, contour filtering, and morphological operations. Scores candidates by `edge_density × area × rectangularity`.
|
||
2. **screen_classifier.py** — Binary classifier (rowing display vs. not). Two modes: a rule-based feature scorer (no training needed) and a 4-layer CNN with batch norm (requires training on `train/` data).
|
||
3. **extract_rowing_data.py** — Sends cropped images to Claude Haiku vision API, extracts time/distance, validates against sanity bounds (pace, distance, duration), computes derived metrics (pace/500m, calories), and reads EXIF date.
|
||
4. **optimize_crop.py** — Optuna-based hyperparameter tuner for crop_to_screen.py detection parameters (12 params). Evaluates trials by counting CNN-classified rowing displays.
|
||
|
||
## Commands
|
||
|
||
```bash
|
||
# Crop screens from photos
|
||
python crop_to_screen.py photos/ cropped/
|
||
|
||
# Classify images (feature-based or CNN)
|
||
python screen_classifier.py predict --dir cropped/
|
||
python screen_classifier.py predict --image path/to/img.jpg --mode cnn
|
||
|
||
# Train the CNN classifier
|
||
python screen_classifier.py train --data-dir train/
|
||
|
||
# Extract rowing data (requires ANTHROPIC_API_KEY)
|
||
python extract_rowing_data.py --dir photos/
|
||
python extract_rowing_data.py --image path/to/img.jpg
|
||
|
||
# Optimize crop parameters
|
||
python optimize_crop.py --n-trials 300 --photos-dir photos/
|
||
```
|
||
|
||
## Dependencies
|
||
|
||
No requirements.txt exists. Key packages: `anthropic`, `torch`, `torchvision`, `opencv-python` (cv2), `Pillow`, `numpy`, `optuna`.
|
||
|
||
## Key Details
|
||
|
||
- The Claude API call in extract_rowing_data.py uses `claude-haiku-4-5-20251001` for cost efficiency on a structured extraction task.
|
||
- Validation bounds in extract_rowing_data.py mirror those from a Go handler (`handle_rowing.go`): distance 100–100k meters, time 30s–2hrs, pace 1:20–2:30 /500m.
|
||
- Training data lives in `train/0/` (non-rowing, 80 samples) and `train/1/` (rowing displays, 48 samples).
|
||
- The trained CNN model is saved as `screen_classifier_model.pth`.
|
||
- `.env` contains API keys — it is gitignored and must not be committed.
|