Append-only role handoff log. Each role adds one entry when its step is complete.
Append-only role handoff log. Each role adds one entry when its step is complete.
### T-003,T-004 — plan — 2026-09-07T10:14:33Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Wrote Priority 2 implementation plan: T-003 (noplaylist option, title-based sanitized filenames, fix for the UUID-looking download name caused by `response.blob()` dropping `Content-Disposition`) and T-004 (job-based `/download` + `/progress/{job_id}` + `/download/{job_id}/file` with an in-memory job store and yt-dlp `progress_hooks`, plus a polling-driven progress bar UI); both tasks moved to ready_for_implement |
| Validation | `python3 -m py_compile app.py` passed (`python` is unavailable); mocked container route check for `noplaylist=True` and `Song Title (Live).mp3` passed; `docker build -t yt-dl .`, container homepage smoke test, and `git diff --check` passed |
| Commit | Pending reviewer approval |
| Next Role | review |
---
### T-003 — review — 2026-09-07T12:45:00Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Reviewed `noplaylist`/filename-sanitization changes against `.ai/PLAN.md`; re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download plus a plain video download through the built Docker image against real YouTube URLs — all passed. |
Goal: implement a containerised YouTube downloader web app per `ROADMAP.md`.
Goal: implement Priority 2 of `ROADMAP.md` — single-video-only downloads from playlist links, correctly-named output files, and a live progress bar in the browser UI.
## Scope
## Scope
- Single Docker container running a FastAPI server (Python 3.12-slim + ffmpeg + yt-dlp).
- **T-003 — Single video from playlist links + correct filenames** (no architecture change; still one synchronous `POST /download` request/response):
- Minimal browser UI: one URL text field + **Download MP3** and **Download Video** buttons.
- Pass yt-dlp's `noplaylist: True` so a playlist URL or a `watch?v=...&list=...` URL only ever downloads the one referenced video.
- MP3 download: yt-dlp extracts best audio, ffmpeg re-encodes to MP3 at highest quality (`-q:a 0`), file streamed back to browser.
- Add a `sanitize_filename(title, fallback_id)` helper: strip characters outside letters (any language)/digits/spaces/`- _ . ( ) , '`, collapse resulting whitespace, trim, and fall back to the video id if nothing usable remains.
- Video download: yt-dlp fetches `bestvideo+bestaudio`, merges to MP4 via ffmpeg, file streamed back to browser.
- Extract `title`/`id` from yt-dlp's result info dict and pass the sanitized name explicitly as `FileResponse(..., filename=...)`, decoupled from whatever yt-dlp happens to name the file on disk.
- Invalid/empty URL returns a JSON error; the UI displays it without crashing.
- Fix the real UUID-looking-filename bug: `response.blob()` in the frontend JS strips the `Content-Disposition` header, so `link.download = ''` makes the browser invent a blob-id-like name. Read the filename from the `Content-Disposition` response header in JS and set `link.download` to it explicitly (fallback to a generic name only if the header is somehow missing).
- No persistent storage — temp dir per request, cleaned up after response.
- **T-004 — Background job + live progress bar** (architecture change, builds on T-003):
- Split `POST /download` into three endpoints:
- `POST /download` — validates the URL, creates an in-memory job record (`job_id = uuid4()`, status `pending`), schedules the actual yt-dlp run on a background thread (`asyncio.get_running_loop().run_in_executor`), and returns `{"job_id": ...}` immediately (no file streaming here anymore).
- `GET /progress/{job_id}` — returns the job's current `{status, percent, speed, downloaded_bytes, total_bytes, filename, error}` from the in-memory store. `status` is one of `downloading`, `finished`, `error`.
- `GET /download/{job_id}/file` — once `status == finished`, streams the prepared file via `FileResponse` (reusing T-003's sanitized filename) with `BackgroundTask` cleanup of the job's temp dir and job record; returns 409/404 if called before completion or after cleanup.
- Progress capture: a yt-dlp `progress_hooks` callback updates the job's dict on each tick using `downloaded_bytes`, `total_bytes` (or `total_bytes_estimate`), `speed`, and computed percent; a `postprocessor_hooks` callback (or the `finished` hook state) marks the job `finished` once the final file is ready.
- Job store: a plain in-memory `dict[str, JobState]` guarded by a `threading.Lock` (single-process app, matches "no persistent storage" constraint). Entries are removed after the file is served, and a periodic/best-effort cleanup drops stale jobs (e.g. no progress endpoint hit within N minutes) so a browser tab closed mid-download doesn't leak a temp dir forever.
- Frontend JS rework:
- `downloadFile(mode)` now: POSTs to `/download` to obtain `job_id`, shows the progress bar area, then polls `GET /progress/{job_id}` every 500ms.
- On each poll: update a `<progress>` bar (or styled div) with percent, and a text line with speed (human-readable, e.g. `1.2 MB/s`) and `downloaded / total` size (human-readable bytes).
- On `status == finished`: stop polling, fetch `GET /download/{job_id}/file`, read filename from `Content-Disposition` (per T-003's fix), trigger the save via the existing blob+anchor pattern, then hide the progress bar and clear status text.
- On `status == error`: stop polling, show the error message in the existing `#status` div, hide the progress bar.
- HTML/CSS: add a progress bar element (native `<progress>` plus a small text line for speed/size) directly below the two buttons, hidden by default, shown only while a job is active.
## Acceptance Criteria
## Acceptance Criteria
- `docker build -t yt-dl .` exits 0.
(from `ROADMAP.md` Priority 2)
- `docker run --rm -p 8080:8080 yt-dl` starts the server and `curl http://localhost:8080/` returns 200.
- A valid YouTube URL + MP3 button delivers a `.mp3` file to the browser.
- A valid YouTube URL + Video button delivers a `.mp4` file to the browser.
- An empty URL or garbage URL returns an error visible in the UI.
## Implementation Phases
- Pasting a playlist URL (or a watch URL with a `list=` param) and clicking MP3 or Video downloads only that one video, never the rest of the playlist.
- Starting a download shows a progress bar below the buttons that updates with percentage, speed, and downloaded/total size while the server is processing.
- When the download finishes, the file is automatically delivered to the browser (save dialog) without an extra click.
- The saved file's name matches the video's title (emoji/icons/symbols removed, normal characters kept) instead of a UUID or other opaque id, and still ends in the correct extension.
**Files to create:**
- If the download fails server-side, the UI shows a clear error in place of the progress bar (no server crash, no stuck spinner).
(`\w` under `re.UNICODE`, the Python 3 default, matches any-language letters/digits/underscore, so non-Latin titles are preserved; emoji/symbols/pictographs fall outside `\w`, whitespace, and the explicit punctuation set and are dropped.)
3. In `fetch_media`, call `downloader.extract_info(url, download=True)` (instead of `.download([url])`) to get the info dict back; read `info.get("title")` and `info.get("id")`, compute `display_name = sanitize_filename(title, id) + Path(media_path).suffix`.
4. Pass `filename=display_name` to the existing `FileResponse(...)` call instead of `media_path.name`.
5. In the frontend `<script>`, change the success branch of `downloadFile`:
const match = /filename\*?=(?:UTF-8''|")?([^\";]+)/i.exec(disposition);
const filename = match ? decodeURIComponent(match[1].replace(/"/g, '')) : 'download';
...
link.download = filename;
```
Keep the rest of the blob/anchor logic unchanged.
**Validation:**
- `python -m py_compile app.py`
- Unit-style check (via a small mocked test or manual `python -c`) that `sanitize_filename("Song 🔥 Title ✨ (Live)", "abc123") == "Song Title (Live)"` collapsed to single spaces → `"Song Title (Live)"`, and `sanitize_filename("🔥🔥🔥", "abc123") == "abc123"`.
- Manual/live check: paste a `watch?v=...&list=...` URL and confirm only the single video downloads; confirm the saved filename in the browser matches the (emoji-stripped) video title, not a UUID.
### Phase 2 — T-004: Background job + live progress bar
**Files to change:**
- `app.py`
- `README.md` (document the new `/download`, `/progress/{job_id}`, `/download/{job_id}/file` flow and the progress bar UI; note job state is in-memory/non-persistent)
**Backend changes in `app.py`:**
1. Add a `JobState` structure (dataclass or `TypedDict`) with fields: `status` (`"downloading" | "finished" | "error"`), `percent`, `speed`, `downloaded_bytes`, `total_bytes`, `temp_dir`, `file_path`, `filename`, `error_message`, `mode`, `created_at`.
2. Add a module-level `_jobs: dict[str, JobState]` and `_jobs_lock = threading.Lock()`.
3. `POST /download` (`url`, `mode` form fields, same validation as today):
- Validate URL as today (raise `HTTPException(400)` on invalid/empty).
- Create `job_id = str(uuid4())`, create the temp dir, seed a `JobState(status="downloading", ...)` in `_jobs` under the lock.
- Define a `progress_hook(d)` closure that, under the lock, updates the job's `percent`/`speed`/`downloaded_bytes`/`total_bytes` from `d["status"]`, `d.get("downloaded_bytes")`, `d.get("total_bytes") or d.get("total_bytes_estimate")`, `d.get("speed")`; on `d["status"] == "error"` marks the job `error`.
- Build `ydl_opts` same as T-003 (`noplaylist: True`, plus `progress_hooks: [progress_hook]`).
- Define `run_job()` that calls `extract_info(url, download=True)`, computes `display_name` via `sanitize_filename` (from T-003), locates the produced file, and updates the job to `status="finished"` with `file_path`/`filename` set; wraps in try/except to set `status="error"`/`error_message` on failure, and cleans up the temp dir on error (not on success — success cleanup happens after the file is served).
- Schedule `run_job` via `asyncio.get_running_loop().run_in_executor(None, run_job)`**without awaiting it** (fire-and-forget task kept referenced, e.g. appended to a module-level `set` of pending futures to avoid GC warnings).
- Return `JSONResponse({"job_id": job_id})`.
4. `GET /progress/{job_id}`:
- Look up the job under the lock; 404 (JSON error) if unknown.
- Return `{"status", "percent", "speed", "downloaded_bytes", "total_bytes", "error"}` (omit file path from the response).
5. `GET /download/{job_id}/file`:
- Look up the job; 404 if unknown, 409 (JSON error) if not yet `finished`.
- Return `FileResponse(job.file_path, filename=job.filename, background=BackgroundTask(cleanup_job, job_id))` where `cleanup_job` removes the temp dir and pops the job from `_jobs` under the lock.
6. Best-effort stale-job sweep: a small helper (called opportunistically, e.g. at the top of `POST /download`) that drops any job older than a fixed TTL (e.g. 30 minutes) whose temp dir hasn't been claimed, cleaning up its temp dir — keeps the in-memory store from growing unbounded if a browser tab is closed mid-download. No background scheduler/thread is introduced solely for this; it piggybacks on request handling to keep the app dependency-free.
**Frontend changes (inline `HTML`/JS in `app.py`):**
- POST to `/download`, parse `{job_id}` (existing error handling for non-2xx/JSON stays the same shape as today).
- Show `#progress-wrap`, start `setInterval` polling `GET /progress/{job_id}` every 500ms.
- On each tick: update `#progress-bar.value` from `percent`; update `#progress-text` with a human-readable line, e.g. `formatSpeed(speed) + ' · ' + formatBytes(downloaded_bytes) + ' / ' + formatBytes(total_bytes)`.
- Add small `formatBytes(n)` / `formatSpeed(n)` helpers (KB/MB/GB, `/s` suffix), handling `null`/unknown total gracefully (e.g. show downloaded size only if total is unknown).
- On `status === 'finished'`: `clearInterval`, fetch `GET /download/{job_id}/file`, apply the same `Content-Disposition`-based filename logic from T-003, trigger the blob/anchor save, then hide `#progress-wrap` and reset text/bar.
- On `status === 'error'`: `clearInterval`, hide `#progress-wrap`, show `error` (or a generic message) in `#status`.
- Guard against overlapping downloads from double-clicks (e.g. disable buttons while a job is active, re-enable on finish/error).
**Validation:**
- `python -m py_compile app.py`
- Manual/live check via `docker run` (or local uvicorn): start an MP3 download, observe the progress bar advancing with percent/speed/size, confirm the file auto-downloads on completion with the correct sanitized title-based filename; repeat for Video mode.
- Manual check: trigger a server-side failure path (e.g. malformed-but-http-prefixed URL) and confirm the progress bar disappears and the existing `#status` error text appears instead, with no server crash.
- Manual check: confirm a playlist URL still only fetches the single video (retained from T-003) end-to-end through the new job flow.
## Validation (both phases)
```bash
```bash
# Build
python -m py_compile app.py
docker build -t yt-dl .
docker build -t yt-dl .
# Smoke test — server starts and home page returns 200
docker run --rm -d -p 8080:8080 --name yt-dl-test yt-dl
docker run --rm -d -p 8080:8080 --name yt-dl-test yt-dl
No unit-test framework is introduced; the app is small enough that the docker smoke test is the validation gate. The reviewer will perform a live download test.
Live download checks (MP3, Video, playlist-link-single-video, progress bar behavior, filename correctness, error path) are performed manually/by the reviewer against a real YouTube URL, consistent with how T-001/T-002 were validated — no new automated test framework is introduced.
- Re-read `.ai/PLAN.md` (Phase 1 spec for T-003) and diffed it against `app.py` / `README.md`; implementation matches the plan (`noplaylist: True` on both modes, `sanitize_filename` helper with the specified regex, `extract_info(..., download=True)` used to recover `title`/`id`, `filename=display_name` passed to `FileResponse`, frontend `Content-Disposition`-based `link.download` fix).
- `python3 -m py_compile app.py` — succeeded.
- Unit-checked `sanitize_filename` directly (scratch venv, no mocks):
- `sanitize_filename("Song 🔥 Title ✨ (Live)", "abc123")` → `"Song Title (Live)"`.
- `sanitize_filename("🔥🔥🔥", "abc123")` → `"abc123"` (fallback to id).
- Verified Starlette's `FileResponse` encodes such filenames via `filename*=utf-8''...` (RFC 5987) when they contain non-ASCII/space/apostrophe characters, and confirmed the frontend regex (`/filename\*?=(?:UTF-8''|")?([^";]+)/i`) plus `decodeURIComponent` correctly recovers the original title from that header format.
- **Live end-to-end test**, built and ran the actual Docker image (`docker build -t yt-dl-review .`, `docker run -p 8081:8080`):
- `GET /` → 200, contains "Download MP3".
- `POST /download` with a playlist-context URL (`watch?v=dQw4w9WgXcQ&list=RDdQw4w9WgXcQ`), `mode=mp3` → 200; container logs show `Downloading just the video dQw4w9WgXcQ because of --no-playlist`, confirming only the referenced video is fetched, never the rest of the playlist/mix; response is a single valid MP3 (`file` confirms ID3 v2.4 MPEG audio) with `Content-Disposition: attachment; filename*=utf-8''Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster).mp3` — title-based, not a UUID.
- `mode=video` against a plain (non-playlist) URL → 200, valid MP4 (`file` confirms ISO Media MP4), filename `Me at the zoo.mp4` derived from the video title.
- `POST /download` with an invalid (non-`http`) URL → unchanged 400 JSON `{"message": ...}` behavior, confirming the existing validation path was not regressed.
- Cleaned up: stopped/removed the test container and scratch venv, deleted downloaded test files.
- Reviewed `README.md` changes; the new paragraph accurately describes the playlist-single-video behavior and the title-based/emoji-stripped filename rule, matching observed behavior.
**Findings from verification:** All acceptance criteria for T-003 hold:
- Pasting a playlist/`list=` URL downloads only the referenced video (confirmed via yt-dlp's own `--no-playlist` log line and a single file being produced).
- Saved filename matches the video title with emoji/icons stripped (normal chars, including non-Latin, kept) instead of a UUID, with the correct extension.
- `python -m py_compile app.py` passes.
**Risks:**
- None outstanding. `sanitize_filename` strips characters outside `\w`/whitespace/`- _ . ( ) , '`; this only affects the `Content-Disposition` header value used for the browser's save-as name, not the on-disk path used to read the file, so there is no path-traversal or injection concern from unsanitized titles.
| T-001 | FastAPI backend (`app.py`) + inline HTML/JS frontend (`requirements.txt`) | done | `GET /` returns 200 with HTML containing "Download MP3"; `POST /download` with valid URL + mode streams a file; invalid URL returns 400 JSON with `.message` | Reviewer re-ran mocked FastAPI route checks (GET /, valid download, invalid/empty/missing-field errors, yt-dlp failure, unicode filename) — all passed; see `.ai/REVIEW.md` | none |
| T-001 | FastAPI backend (`app.py`) + inline HTML/JS frontend (`requirements.txt`) | done | `GET /` returns 200 with HTML containing "Download MP3"; `POST /download` with valid URL + mode streams a file; invalid URL returns 400 JSON with `.message` | Reviewer re-ran mocked FastAPI route checks (GET /, valid download, invalid/empty/missing-field errors, yt-dlp failure, unicode filename) — all passed; see `.ai/REVIEW.md` | none |
| T-002 | Dockerfile + README.md (build/run docs) | done | `docker build -t yt-dl .` exits 0; `docker run --rm -p 8080:8080 yt-dl` starts server; `curl http://localhost:8080/` returns HTML with download buttons | Reviewer re-ran `docker build` + container smoke test, plus a live end-to-end MP3 and video download against a real YouTube URL through the container — all passed; see `.ai/REVIEW.md` | none |
| T-002 | Dockerfile + README.md (build/run docs) | done | `docker build -t yt-dl .` exits 0; `docker run --rm -p 8080:8080 yt-dl` starts server; `curl http://localhost:8080/` returns HTML with download buttons | Reviewer re-ran `docker build` + container smoke test, plus a live end-to-end MP3 and video download against a real YouTube URL through the container — all passed; see `.ai/REVIEW.md` | none |
| T-003 | `app.py` (`noplaylist` option, `sanitize_filename` helper, title-based `FileResponse` filename, frontend `Content-Disposition`-based `link.download` fix) + `README.md` update | done | Pasting a playlist/`list=` URL downloads only the referenced video; saved filename matches the video title with emoji/icons stripped (normal chars kept) instead of a UUID, with correct extension; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download + a plain video download through the built Docker image against real YouTube URLs — all passed; see `.ai/REVIEW.md` | none |
| T-004 | `app.py` (job-based `POST /download` + `GET /progress/{job_id}` + `GET /download/{job_id}/file`, in-memory job store, yt-dlp `progress_hooks`, frontend polling + progress bar UI) + `README.md` update | ready_for_implement | Progress bar appears below the buttons on download start and updates with percent/speed/downloaded-total size; file auto-delivers to the browser on completion with the correct sanitized filename; server-side failure shows an error in place of the bar with no crash; `python -m py_compile app.py` passes | | implement |
@ -26,6 +26,8 @@ Open [http://localhost:8080](http://localhost:8080) in a browser, paste a YouTub
Downloads are prepared on the server before they are sent to your browser. Large videos and higher-quality formats can take several minutes, depending on the source video and your network connection.
Downloads are prepared on the server before they are sent to your browser. Large videos and higher-quality formats can take several minutes, depending on the source video and your network connection.
If a pasted video link includes playlist context (such as a `list=` parameter), the app downloads only that referenced video. Downloaded files use the video title, with emoji and unsupported symbols removed while normal letters (including non-Latin characters), digits, and common punctuation are retained. An emoji-only title falls back to the video ID.
## AI Workflow
## AI Workflow
This project includes the persistent planner/implementer/reviewer workflow with file-based coordination, plus the PO orchestration layer.
This project includes the persistent planner/implementer/reviewer workflow with file-based coordination, plus the PO orchestration layer.
@ -22,6 +22,39 @@ Objective: Containerised YouTube downloader with a minimal browser UI.
## Out of Scope
## Out of Scope
- Playlist batch downloads.
- Playlist batch downloads (only the single video referenced by the pasted link is ever downloaded — see Priority 2).
- User accounts / history.
- User accounts / history.
- Progress bars / live download status (nice-to-have but not required for v1).
## Priority 2
Objective: single-video-only downloads from playlist links, plus live download progress in the UI.
- If the pasted URL is a playlist link or a video link that also carries a playlist context (e.g. `watch?v=...&list=...`), only the one referenced video is downloaded — the rest of the playlist is ignored. Implemented via yt-dlp's `noplaylist` option.
- Below the two download buttons, a progress bar area appears once a download starts, showing for the video currently being processed:
- percentage complete
- current download speed
- downloaded size / total size
- Progress transport: the server runs each download as a background job identified by a `job_id` and records live progress (from yt-dlp's `progress_hooks`) in server-side memory. The browser polls a `GET /progress/{job_id}` endpoint (~every 500ms) and updates the bar/text from the response.
- File delivery: when a job reaches 100%, the page automatically fetches `GET /download/{job_id}/file` and triggers the browser's normal save behavior — no extra click required, preserving today's one-click UX.
- Errors surfaced during the background job (invalid URL, yt-dlp failure) are reported through the same progress endpoint/UI status area used today, without a server crash.
- Still single-container, no external dependencies beyond Docker, no persistent storage — job state is in-memory only and is cleaned up once the file has been delivered (or after a reasonable failure/timeout window).
- The delivered filename is derived from the video's real title, not a random/opaque id (currently the browser sometimes ends up saving with a UUID-like name):
- Take the video title as reported by yt-dlp.
- Strip characters that aren't plain text — emoji, pictograms/icons, and other symbols outside normal letters/digits/punctuation — rather than replacing the whole title.
- Keep letters (any language), digits, spaces, and common punctuation (`- _ . ( ) , '`); collapse repeated whitespace left behind by stripped characters and trim the ends.
- If stripping leaves an empty name (e.g. an emoji-only title), fall back to the video id so a filename always exists.
- Keep the correct extension for the mode (`.mp3` for audio, `.mp4`/`.mkv` for video) after the cleaned title.
## Acceptance Criteria (Priority 2)
- Pasting a playlist URL (or a watch URL with a `list=` param) and clicking MP3 or Video downloads only that one video, never the rest of the playlist.
- Starting a download shows a progress bar below the buttons that updates with percentage, speed, and downloaded/total size while the server is processing.
- When the download finishes, the file is automatically delivered to the browser (save dialog) without an extra click.
- The saved file's name matches the video's title (with emoji/icons/symbols removed, normal characters kept) instead of a UUID or other opaque id, and still ends in the correct extension.
- If the download fails server-side, the UI shows a clear error in place of the progress bar (no server crash, no stuck spinner).
## Out of Scope
- Playlist batch downloads (downloading every video in a playlist in one action).
- User accounts / history.
- Persisting job/progress state across server restarts.
Domagoj Zecevic
2 weeks ago