Append-only role handoff log. Each role adds one entry when its step is complete.
### T-003,T-004 — plan — 2026-09-07T10:14:33Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Wrote Priority 2 implementation plan: T-003 (noplaylist option, title-based sanitized filenames, fix for the UUID-looking download name caused by `response.blob()` dropping `Content-Disposition`) and T-004 (job-based `/download` + `/progress/{job_id}` + `/download/{job_id}/file` with an in-memory job store and yt-dlp `progress_hooks`, plus a polling-driven progress bar UI); both tasks moved to ready_for_implement |
| Validation | `python3 -m py_compile app.py` passed (`python` is unavailable); mocked container route check for `noplaylist=True` and `Song Title (Live).mp3` passed; `docker build -t yt-dl .`, container homepage smoke test, and `git diff --check` passed |
| Commit | Pending reviewer approval |
| Next Role | review |
---
### T-003 — review — 2026-09-07T12:45:00Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Reviewed `noplaylist`/filename-sanitization changes against `.ai/PLAN.md`; re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download plus a plain video download through the built Docker image against real YouTube URLs — all passed. |
Goal: implement a containerised YouTube downloader web app per `ROADMAP.md`.
Goal: implement Priority 2 of `ROADMAP.md` — single-video-only downloads from playlist links, correctly-named output files, and a live progress bar in the browser UI.
## Scope
- Single Docker container running a FastAPI server (Python 3.12-slim + ffmpeg + yt-dlp).
- Minimal browser UI: one URL text field + **Download MP3** and **Download Video** buttons.
- MP3 download: yt-dlp extracts best audio, ffmpeg re-encodes to MP3 at highest quality (`-q:a 0`), file streamed back to browser.
- Video download: yt-dlp fetches `bestvideo+bestaudio`, merges to MP4 via ffmpeg, file streamed back to browser.
- Invalid/empty URL returns a JSON error; the UI displays it without crashing.
- No persistent storage — temp dir per request, cleaned up after response.
- **T-003 — Single video from playlist links + correct filenames** (no architecture change; still one synchronous `POST /download` request/response):
- Pass yt-dlp's `noplaylist: True` so a playlist URL or a `watch?v=...&list=...` URL only ever downloads the one referenced video.
- Add a `sanitize_filename(title, fallback_id)` helper: strip characters outside letters (any language)/digits/spaces/`- _ . ( ) , '`, collapse resulting whitespace, trim, and fall back to the video id if nothing usable remains.
- Extract `title`/`id` from yt-dlp's result info dict and pass the sanitized name explicitly as `FileResponse(..., filename=...)`, decoupled from whatever yt-dlp happens to name the file on disk.
- Fix the real UUID-looking-filename bug: `response.blob()` in the frontend JS strips the `Content-Disposition` header, so `link.download = ''` makes the browser invent a blob-id-like name. Read the filename from the `Content-Disposition` response header in JS and set `link.download` to it explicitly (fallback to a generic name only if the header is somehow missing).
- **T-004 — Background job + live progress bar** (architecture change, builds on T-003):
- Split `POST /download` into three endpoints:
- `POST /download` — validates the URL, creates an in-memory job record (`job_id = uuid4()`, status `pending`), schedules the actual yt-dlp run on a background thread (`asyncio.get_running_loop().run_in_executor`), and returns `{"job_id": ...}` immediately (no file streaming here anymore).
- `GET /progress/{job_id}` — returns the job's current `{status, percent, speed, downloaded_bytes, total_bytes, filename, error}` from the in-memory store. `status` is one of `downloading`, `finished`, `error`.
- `GET /download/{job_id}/file` — once `status == finished`, streams the prepared file via `FileResponse` (reusing T-003's sanitized filename) with `BackgroundTask` cleanup of the job's temp dir and job record; returns 409/404 if called before completion or after cleanup.
- Progress capture: a yt-dlp `progress_hooks` callback updates the job's dict on each tick using `downloaded_bytes`, `total_bytes` (or `total_bytes_estimate`), `speed`, and computed percent; a `postprocessor_hooks` callback (or the `finished` hook state) marks the job `finished` once the final file is ready.
- Job store: a plain in-memory `dict[str, JobState]` guarded by a `threading.Lock` (single-process app, matches "no persistent storage" constraint). Entries are removed after the file is served, and a periodic/best-effort cleanup drops stale jobs (e.g. no progress endpoint hit within N minutes) so a browser tab closed mid-download doesn't leak a temp dir forever.
- Frontend JS rework:
- `downloadFile(mode)` now: POSTs to `/download` to obtain `job_id`, shows the progress bar area, then polls `GET /progress/{job_id}` every 500ms.
- On each poll: update a `<progress>` bar (or styled div) with percent, and a text line with speed (human-readable, e.g. `1.2 MB/s`) and `downloaded / total` size (human-readable bytes).
- On `status == finished`: stop polling, fetch `GET /download/{job_id}/file`, read filename from `Content-Disposition` (per T-003's fix), trigger the save via the existing blob+anchor pattern, then hide the progress bar and clear status text.
- On `status == error`: stop polling, show the error message in the existing `#status` div, hide the progress bar.
- HTML/CSS: add a progress bar element (native `<progress>` plus a small text line for speed/size) directly below the two buttons, hidden by default, shown only while a job is active.
## Acceptance Criteria
- `docker build -t yt-dl .` exits 0.
- `docker run --rm -p 8080:8080 yt-dl` starts the server and `curl http://localhost:8080/` returns 200.
- A valid YouTube URL + MP3 button delivers a `.mp3` file to the browser.
- A valid YouTube URL + Video button delivers a `.mp4` file to the browser.
- An empty URL or garbage URL returns an error visible in the UI.
- Run `yt_dlp.YoutubeDL(ydl_opts).download([url])` inside `asyncio.get_event_loop().run_in_executor(None, ...)` to avoid blocking the event loop
- Find the downloaded file in tmpdir (glob the single file present)
- Return `FileResponse(path, filename=..., background=BackgroundTask(cleanup_tmpdir))` — cleanup runs after response is sent
- Wrap yt-dlp call in try/except; on any exception raise `HTTPException(500, detail=str(e))`
3. **`requirements.txt`**:
```
fastapi>=0.111.0
uvicorn[standard]>=0.29.0
yt-dlp>=2024.4.9
python-multipart>=0.0.9
```
**Key decisions:**
- Inline HTML keeps the project a two-file app (plus Dockerfile) — no template engine, no static asset serving needed.
- `run_in_executor` keeps uvicorn's event loop unblocked while yt-dlp runs (which can take 30–120 s).
- `FileResponse` with `BackgroundTask` cleanup is the standard FastAPI pattern for temp file responses.
- yt-dlp is called via its Python API (not subprocess) to avoid shell escaping issues with URLs.
---
### Phase 2 — T-002: Dockerfile + README
**Files to create/update:**
- `Dockerfile`
- `README.md` (update with build/run instructions)
**`Dockerfile`**:
```dockerfile
FROM python:3.12-slim
- Pasting a playlist URL (or a watch URL with a `list=` param) and clicking MP3 or Video downloads only that one video, never the rest of the playlist.
- Starting a download shows a progress bar below the buttons that updates with percentage, speed, and downloaded/total size while the server is processing.
- When the download finishes, the file is automatically delivered to the browser (save dialog) without an extra click.
- The saved file's name matches the video's title (emoji/icons/symbols removed, normal characters kept) instead of a UUID or other opaque id, and still ends in the correct extension.
- If the download fails server-side, the UI shows a clear error in place of the progress bar (no server crash, no stuck spinner).
(`\w` under `re.UNICODE`, the Python 3 default, matches any-language letters/digits/underscore, so non-Latin titles are preserved; emoji/symbols/pictographs fall outside `\w`, whitespace, and the explicit punctuation set and are dropped.)
3. In `fetch_media`, call `downloader.extract_info(url, download=True)` (instead of `.download([url])`) to get the info dict back; read `info.get("title")` and `info.get("id")`, compute `display_name = sanitize_filename(title, id) + Path(media_path).suffix`.
4. Pass `filename=display_name` to the existing `FileResponse(...)` call instead of `media_path.name`.
5. In the frontend `<script>`, change the success branch of `downloadFile`:
const match = /filename\*?=(?:UTF-8''|")?([^\";]+)/i.exec(disposition);
const filename = match ? decodeURIComponent(match[1].replace(/"/g, '')) : 'download';
...
link.download = filename;
```
Keep the rest of the blob/anchor logic unchanged.
**Validation:**
- `python -m py_compile app.py`
- Unit-style check (via a small mocked test or manual `python -c`) that `sanitize_filename("Song 🔥 Title ✨ (Live)", "abc123") == "Song Title (Live)"` collapsed to single spaces → `"Song Title (Live)"`, and `sanitize_filename("🔥🔥🔥", "abc123") == "abc123"`.
- Manual/live check: paste a `watch?v=...&list=...` URL and confirm only the single video downloads; confirm the saved filename in the browser matches the (emoji-stripped) video title, not a UUID.
### Phase 2 — T-004: Background job + live progress bar
**Files to change:**
- `app.py`
- `README.md` (document the new `/download`, `/progress/{job_id}`, `/download/{job_id}/file` flow and the progress bar UI; note job state is in-memory/non-persistent)
**Backend changes in `app.py`:**
1. Add a `JobState` structure (dataclass or `TypedDict`) with fields: `status` (`"downloading" | "finished" | "error"`), `percent`, `speed`, `downloaded_bytes`, `total_bytes`, `temp_dir`, `file_path`, `filename`, `error_message`, `mode`, `created_at`.
2. Add a module-level `_jobs: dict[str, JobState]` and `_jobs_lock = threading.Lock()`.
3. `POST /download` (`url`, `mode` form fields, same validation as today):
- Validate URL as today (raise `HTTPException(400)` on invalid/empty).
- Create `job_id = str(uuid4())`, create the temp dir, seed a `JobState(status="downloading", ...)` in `_jobs` under the lock.
- Define a `progress_hook(d)` closure that, under the lock, updates the job's `percent`/`speed`/`downloaded_bytes`/`total_bytes` from `d["status"]`, `d.get("downloaded_bytes")`, `d.get("total_bytes") or d.get("total_bytes_estimate")`, `d.get("speed")`; on `d["status"] == "error"` marks the job `error`.
- Build `ydl_opts` same as T-003 (`noplaylist: True`, plus `progress_hooks: [progress_hook]`).
- Define `run_job()` that calls `extract_info(url, download=True)`, computes `display_name` via `sanitize_filename` (from T-003), locates the produced file, and updates the job to `status="finished"` with `file_path`/`filename` set; wraps in try/except to set `status="error"`/`error_message` on failure, and cleans up the temp dir on error (not on success — success cleanup happens after the file is served).
- Schedule `run_job` via `asyncio.get_running_loop().run_in_executor(None, run_job)`**without awaiting it** (fire-and-forget task kept referenced, e.g. appended to a module-level `set` of pending futures to avoid GC warnings).
- Return `JSONResponse({"job_id": job_id})`.
4. `GET /progress/{job_id}`:
- Look up the job under the lock; 404 (JSON error) if unknown.
- Return `{"status", "percent", "speed", "downloaded_bytes", "total_bytes", "error"}` (omit file path from the response).
5. `GET /download/{job_id}/file`:
- Look up the job; 404 if unknown, 409 (JSON error) if not yet `finished`.
- Return `FileResponse(job.file_path, filename=job.filename, background=BackgroundTask(cleanup_job, job_id))` where `cleanup_job` removes the temp dir and pops the job from `_jobs` under the lock.
6. Best-effort stale-job sweep: a small helper (called opportunistically, e.g. at the top of `POST /download`) that drops any job older than a fixed TTL (e.g. 30 minutes) whose temp dir hasn't been claimed, cleaning up its temp dir — keeps the in-memory store from growing unbounded if a browser tab is closed mid-download. No background scheduler/thread is introduced solely for this; it piggybacks on request handling to keep the app dependency-free.
**Frontend changes (inline `HTML`/JS in `app.py`):**
- POST to `/download`, parse `{job_id}` (existing error handling for non-2xx/JSON stays the same shape as today).
- Show `#progress-wrap`, start `setInterval` polling `GET /progress/{job_id}` every 500ms.
- On each tick: update `#progress-bar.value` from `percent`; update `#progress-text` with a human-readable line, e.g. `formatSpeed(speed) + ' · ' + formatBytes(downloaded_bytes) + ' / ' + formatBytes(total_bytes)`.
- Add small `formatBytes(n)` / `formatSpeed(n)` helpers (KB/MB/GB, `/s` suffix), handling `null`/unknown total gracefully (e.g. show downloaded size only if total is unknown).
- On `status === 'finished'`: `clearInterval`, fetch `GET /download/{job_id}/file`, apply the same `Content-Disposition`-based filename logic from T-003, trigger the blob/anchor save, then hide `#progress-wrap` and reset text/bar.
- On `status === 'error'`: `clearInterval`, hide `#progress-wrap`, show `error` (or a generic message) in `#status`.
- Guard against overlapping downloads from double-clicks (e.g. disable buttons while a job is active, re-enable on finish/error).
**Validation:**
- `python -m py_compile app.py`
- Manual/live check via `docker run` (or local uvicorn): start an MP3 download, observe the progress bar advancing with percent/speed/size, confirm the file auto-downloads on completion with the correct sanitized title-based filename; repeat for Video mode.
- Manual check: trigger a server-side failure path (e.g. malformed-but-http-prefixed URL) and confirm the progress bar disappears and the existing `#status` error text appears instead, with no server crash.
- Manual check: confirm a playlist URL still only fetches the single video (retained from T-003) end-to-end through the new job flow.
## Validation (both phases)
```bash
# Build
python -m py_compile app.py
docker build -t yt-dl .
# Smoke test — server starts and home page returns 200
docker run --rm -d -p 8080:8080 --name yt-dl-test yt-dl
No unit-test framework is introduced; the app is small enough that the docker smoke test is the validation gate. The reviewer will perform a live download test.
Live download checks (MP3, Video, playlist-link-single-video, progress bar behavior, filename correctness, error path) are performed manually/by the reviewer against a real YouTube URL, consistent with how T-001/T-002 were validated — no new automated test framework is introduced.
- Re-read `.ai/PLAN.md` (Phase 1 spec for T-003) and diffed it against `app.py` / `README.md`; implementation matches the plan (`noplaylist: True` on both modes, `sanitize_filename` helper with the specified regex, `extract_info(..., download=True)` used to recover `title`/`id`, `filename=display_name` passed to `FileResponse`, frontend `Content-Disposition`-based `link.download` fix).
- `python3 -m py_compile app.py` — succeeded.
- Unit-checked `sanitize_filename` directly (scratch venv, no mocks):
- `sanitize_filename("Song 🔥 Title ✨ (Live)", "abc123")` → `"Song Title (Live)"`.
- `sanitize_filename("🔥🔥🔥", "abc123")` → `"abc123"` (fallback to id).
- Verified Starlette's `FileResponse` encodes such filenames via `filename*=utf-8''...` (RFC 5987) when they contain non-ASCII/space/apostrophe characters, and confirmed the frontend regex (`/filename\*?=(?:UTF-8''|")?([^";]+)/i`) plus `decodeURIComponent` correctly recovers the original title from that header format.
- **Live end-to-end test**, built and ran the actual Docker image (`docker build -t yt-dl-review .`, `docker run -p 8081:8080`):
- `GET /` → 200, contains "Download MP3".
- `POST /download` with a playlist-context URL (`watch?v=dQw4w9WgXcQ&list=RDdQw4w9WgXcQ`), `mode=mp3` → 200; container logs show `Downloading just the video dQw4w9WgXcQ because of --no-playlist`, confirming only the referenced video is fetched, never the rest of the playlist/mix; response is a single valid MP3 (`file` confirms ID3 v2.4 MPEG audio) with `Content-Disposition: attachment; filename*=utf-8''Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster).mp3` — title-based, not a UUID.
- `mode=video` against a plain (non-playlist) URL → 200, valid MP4 (`file` confirms ISO Media MP4), filename `Me at the zoo.mp4` derived from the video title.
- `POST /download` with an invalid (non-`http`) URL → unchanged 400 JSON `{"message": ...}` behavior, confirming the existing validation path was not regressed.
- Cleaned up: stopped/removed the test container and scratch venv, deleted downloaded test files.
- Reviewed `README.md` changes; the new paragraph accurately describes the playlist-single-video behavior and the title-based/emoji-stripped filename rule, matching observed behavior.
**Findings from verification:** All acceptance criteria for T-003 hold:
- Pasting a playlist/`list=` URL downloads only the referenced video (confirmed via yt-dlp's own `--no-playlist` log line and a single file being produced).
- Saved filename matches the video title with emoji/icons stripped (normal chars, including non-Latin, kept) instead of a UUID, with the correct extension.
- `python -m py_compile app.py` passes.
**Risks:**
- None outstanding. `sanitize_filename` strips characters outside `\w`/whitespace/`- _ . ( ) , '`; this only affects the `Content-Disposition` header value used for the browser's save-as name, not the on-disk path used to read the file, so there is no path-traversal or injection concern from unsanitized titles.
| T-001 | FastAPI backend (`app.py`) + inline HTML/JS frontend (`requirements.txt`) | done | `GET /` returns 200 with HTML containing "Download MP3"; `POST /download` with valid URL + mode streams a file; invalid URL returns 400 JSON with `.message` | Reviewer re-ran mocked FastAPI route checks (GET /, valid download, invalid/empty/missing-field errors, yt-dlp failure, unicode filename) — all passed; see `.ai/REVIEW.md` | none |
| T-002 | Dockerfile + README.md (build/run docs) | done | `docker build -t yt-dl .` exits 0; `docker run --rm -p 8080:8080 yt-dl` starts server; `curl http://localhost:8080/` returns HTML with download buttons | Reviewer re-ran `docker build` + container smoke test, plus a live end-to-end MP3 and video download against a real YouTube URL through the container — all passed; see `.ai/REVIEW.md` | none |
| T-003 | `app.py` (`noplaylist` option, `sanitize_filename` helper, title-based `FileResponse` filename, frontend `Content-Disposition`-based `link.download` fix) + `README.md` update | done | Pasting a playlist/`list=` URL downloads only the referenced video; saved filename matches the video title with emoji/icons stripped (normal chars kept) instead of a UUID, with correct extension; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download + a plain video download through the built Docker image against real YouTube URLs — all passed; see `.ai/REVIEW.md` | none |
| T-004 | `app.py` (job-based `POST /download` + `GET /progress/{job_id}` + `GET /download/{job_id}/file`, in-memory job store, yt-dlp `progress_hooks`, frontend polling + progress bar UI) + `README.md` update | ready_for_implement | Progress bar appears below the buttons on download start and updates with percent/speed/downloaded-total size; file auto-delivers to the browser on completion with the correct sanitized filename; server-side failure shows an error in place of the bar with no crash; `python -m py_compile app.py` passes | | implement |
@ -26,6 +26,8 @@ Open [http://localhost:8080](http://localhost:8080) in a browser, paste a YouTub
Downloads are prepared on the server before they are sent to your browser. Large videos and higher-quality formats can take several minutes, depending on the source video and your network connection.
If a pasted video link includes playlist context (such as a `list=` parameter), the app downloads only that referenced video. Downloaded files use the video title, with emoji and unsupported symbols removed while normal letters (including non-Latin characters), digits, and common punctuation are retained. An emoji-only title falls back to the video ID.
## AI Workflow
This project includes the persistent planner/implementer/reviewer workflow with file-based coordination, plus the PO orchestration layer.
@ -22,6 +22,39 @@ Objective: Containerised YouTube downloader with a minimal browser UI.
## Out of Scope
- Playlist batch downloads.
- Playlist batch downloads (only the single video referenced by the pasted link is ever downloaded — see Priority 2).
- User accounts / history.
- Progress bars / live download status (nice-to-have but not required for v1).
## Priority 2
Objective: single-video-only downloads from playlist links, plus live download progress in the UI.
- If the pasted URL is a playlist link or a video link that also carries a playlist context (e.g. `watch?v=...&list=...`), only the one referenced video is downloaded — the rest of the playlist is ignored. Implemented via yt-dlp's `noplaylist` option.
- Below the two download buttons, a progress bar area appears once a download starts, showing for the video currently being processed:
- percentage complete
- current download speed
- downloaded size / total size
- Progress transport: the server runs each download as a background job identified by a `job_id` and records live progress (from yt-dlp's `progress_hooks`) in server-side memory. The browser polls a `GET /progress/{job_id}` endpoint (~every 500ms) and updates the bar/text from the response.
- File delivery: when a job reaches 100%, the page automatically fetches `GET /download/{job_id}/file` and triggers the browser's normal save behavior — no extra click required, preserving today's one-click UX.
- Errors surfaced during the background job (invalid URL, yt-dlp failure) are reported through the same progress endpoint/UI status area used today, without a server crash.
- Still single-container, no external dependencies beyond Docker, no persistent storage — job state is in-memory only and is cleaned up once the file has been delivered (or after a reasonable failure/timeout window).
- The delivered filename is derived from the video's real title, not a random/opaque id (currently the browser sometimes ends up saving with a UUID-like name):
- Take the video title as reported by yt-dlp.
- Strip characters that aren't plain text — emoji, pictograms/icons, and other symbols outside normal letters/digits/punctuation — rather than replacing the whole title.
- Keep letters (any language), digits, spaces, and common punctuation (`- _ . ( ) , '`); collapse repeated whitespace left behind by stripped characters and trim the ends.
- If stripping leaves an empty name (e.g. an emoji-only title), fall back to the video id so a filename always exists.
- Keep the correct extension for the mode (`.mp3` for audio, `.mp4`/`.mkv` for video) after the cleaned title.
## Acceptance Criteria (Priority 2)
- Pasting a playlist URL (or a watch URL with a `list=` param) and clicking MP3 or Video downloads only that one video, never the rest of the playlist.
- Starting a download shows a progress bar below the buttons that updates with percentage, speed, and downloaded/total size while the server is processing.
- When the download finishes, the file is automatically delivered to the browser (save dialog) without an extra click.
- The saved file's name matches the video's title (with emoji/icons/symbols removed, normal characters kept) instead of a UUID or other opaque id, and still ends in the correct extension.
- If the download fails server-side, the UI shows a clear error in place of the progress bar (no server crash, no stuck spinner).
## Out of Scope
- Playlist batch downloads (downloading every video in a playlist in one action).
- User accounts / history.
- Persisting job/progress state across server restarts.
Domagoj Zecevic
2 weeks ago