diff --git a/.ai/HANDOFF.md b/.ai/HANDOFF.md index 34d1ef1..5e22246 100644 --- a/.ai/HANDOFF.md +++ b/.ai/HANDOFF.md @@ -2,6 +2,17 @@ Append-only role handoff log. Each role adds one entry when its step is complete. +### T-003,T-004 — plan — 2026-09-07T10:14:33Z + +| Field | Value | +|-------|-------| +| Agent | claude | +| Summary | Wrote Priority 2 implementation plan: T-003 (noplaylist option, title-based sanitized filenames, fix for the UUID-looking download name caused by `response.blob()` dropping `Content-Disposition`) and T-004 (job-based `/download` + `/progress/{job_id}` + `/download/{job_id}/file` with an in-memory job store and yt-dlp `progress_hooks`, plus a polling-driven progress bar UI); both tasks moved to ready_for_implement | +| Files Changed | ROADMAP.md, .ai/PLAN.md, .ai/TASKS.md, .ai/HANDOFF.md | +| Next Role | implement | + +--- + ### T-001,T-002 — plan — 2026-09-05T00:00:00Z | Field | Value | @@ -90,3 +101,42 @@ Append-only role handoff log. Each role adds one entry when its step is complete | Next Role | none | --- + +### T-003 — implement — 2026-09-07T11:27:56Z + +| Field | Value | +|-------|-------| +| Agent | codex | +| Summary | Limited playlist-context links to their referenced video and delivered sanitized title-based download filenames. | +| Files Changed | app.py, README.md, .ai/TASKS.md, .ai/HANDOFF.md | +| Validation | `python3 -m py_compile app.py` passed (`python` is unavailable); mocked container route check for `noplaylist=True` and `Song Title (Live).mp3` passed; `docker build -t yt-dl .`, container homepage smoke test, and `git diff --check` passed | +| Commit | Pending reviewer approval | +| Next Role | review | + +--- + +### T-003 — review — 2026-09-07T12:45:00Z + +| Field | Value | +|-------|-------| +| Agent | claude | +| Summary | Reviewed `noplaylist`/filename-sanitization changes against `.ai/PLAN.md`; re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download plus a plain video download through the built Docker image against real YouTube URLs — all passed. | +| Files Changed | .ai/REVIEW.md, .ai/TASKS.md, .ai/HANDOFF.md | +| Verdict | PASS | +| Blocking Findings | none | +| Next Role | implement | + +--- + +### T-003 — implement — 2026-09-07T12:48:26Z + +| Field | Value | +|-------|-------| +| Agent | codex | +| Summary | Squashed the reviewed playlist-safe, title-based download changes into the task commit. | +| Files Changed | app.py, README.md, ROADMAP.md, .ai/PLAN.md, .ai/REVIEW.md, .ai/TASKS.md, .ai/HANDOFF.md | +| Validation | Reviewer verification passed; `git diff --check` passed | +| Commit | Created by this handoff's task commit | +| Next Role | none | + +--- diff --git a/.ai/PLAN.md b/.ai/PLAN.md index 440ce78..726ea3b 100644 --- a/.ai/PLAN.md +++ b/.ai/PLAN.md @@ -2,119 +2,136 @@ Status: **ready_for_implement** -Goal: implement a containerised YouTube downloader web app per `ROADMAP.md`. +Goal: implement Priority 2 of `ROADMAP.md` — single-video-only downloads from playlist links, correctly-named output files, and a live progress bar in the browser UI. ## Scope -- Single Docker container running a FastAPI server (Python 3.12-slim + ffmpeg + yt-dlp). -- Minimal browser UI: one URL text field + **Download MP3** and **Download Video** buttons. -- MP3 download: yt-dlp extracts best audio, ffmpeg re-encodes to MP3 at highest quality (`-q:a 0`), file streamed back to browser. -- Video download: yt-dlp fetches `bestvideo+bestaudio`, merges to MP4 via ffmpeg, file streamed back to browser. -- Invalid/empty URL returns a JSON error; the UI displays it without crashing. -- No persistent storage — temp dir per request, cleaned up after response. +- **T-003 — Single video from playlist links + correct filenames** (no architecture change; still one synchronous `POST /download` request/response): + - Pass yt-dlp's `noplaylist: True` so a playlist URL or a `watch?v=...&list=...` URL only ever downloads the one referenced video. + - Add a `sanitize_filename(title, fallback_id)` helper: strip characters outside letters (any language)/digits/spaces/`- _ . ( ) , '`, collapse resulting whitespace, trim, and fall back to the video id if nothing usable remains. + - Extract `title`/`id` from yt-dlp's result info dict and pass the sanitized name explicitly as `FileResponse(..., filename=...)`, decoupled from whatever yt-dlp happens to name the file on disk. + - Fix the real UUID-looking-filename bug: `response.blob()` in the frontend JS strips the `Content-Disposition` header, so `link.download = ''` makes the browser invent a blob-id-like name. Read the filename from the `Content-Disposition` response header in JS and set `link.download` to it explicitly (fallback to a generic name only if the header is somehow missing). +- **T-004 — Background job + live progress bar** (architecture change, builds on T-003): + - Split `POST /download` into three endpoints: + - `POST /download` — validates the URL, creates an in-memory job record (`job_id = uuid4()`, status `pending`), schedules the actual yt-dlp run on a background thread (`asyncio.get_running_loop().run_in_executor`), and returns `{"job_id": ...}` immediately (no file streaming here anymore). + - `GET /progress/{job_id}` — returns the job's current `{status, percent, speed, downloaded_bytes, total_bytes, filename, error}` from the in-memory store. `status` is one of `downloading`, `finished`, `error`. + - `GET /download/{job_id}/file` — once `status == finished`, streams the prepared file via `FileResponse` (reusing T-003's sanitized filename) with `BackgroundTask` cleanup of the job's temp dir and job record; returns 409/404 if called before completion or after cleanup. + - Progress capture: a yt-dlp `progress_hooks` callback updates the job's dict on each tick using `downloaded_bytes`, `total_bytes` (or `total_bytes_estimate`), `speed`, and computed percent; a `postprocessor_hooks` callback (or the `finished` hook state) marks the job `finished` once the final file is ready. + - Job store: a plain in-memory `dict[str, JobState]` guarded by a `threading.Lock` (single-process app, matches "no persistent storage" constraint). Entries are removed after the file is served, and a periodic/best-effort cleanup drops stale jobs (e.g. no progress endpoint hit within N minutes) so a browser tab closed mid-download doesn't leak a temp dir forever. + - Frontend JS rework: + - `downloadFile(mode)` now: POSTs to `/download` to obtain `job_id`, shows the progress bar area, then polls `GET /progress/{job_id}` every 500ms. + - On each poll: update a `` bar (or styled div) with percent, and a text line with speed (human-readable, e.g. `1.2 MB/s`) and `downloaded / total` size (human-readable bytes). + - On `status == finished`: stop polling, fetch `GET /download/{job_id}/file`, read filename from `Content-Disposition` (per T-003's fix), trigger the save via the existing blob+anchor pattern, then hide the progress bar and clear status text. + - On `status == error`: stop polling, show the error message in the existing `#status` div, hide the progress bar. + - HTML/CSS: add a progress bar element (native `` plus a small text line for speed/size) directly below the two buttons, hidden by default, shown only while a job is active. ## Acceptance Criteria -- `docker build -t yt-dl .` exits 0. -- `docker run --rm -p 8080:8080 yt-dl` starts the server and `curl http://localhost:8080/` returns 200. -- A valid YouTube URL + MP3 button delivers a `.mp3` file to the browser. -- A valid YouTube URL + Video button delivers a `.mp4` file to the browser. -- An empty URL or garbage URL returns an error visible in the UI. +(from `ROADMAP.md` Priority 2) -## Implementation Phases - -### Phase 1 — T-001: FastAPI backend + HTML/JS frontend - -**Files to create:** -- `app.py` — entire server + inline HTML -- `requirements.txt` - -**`app.py` structure:** - -1. **HTML constant** (`HTML`) — inline single-page UI: - - Text input for URL, placeholder "Paste YouTube URL…" - - Two buttons: `Download MP3` / `Download Video` - - Status `
` for errors/progress text - - JS `downloadFile(mode)`: - - Reads URL from input, validates non-empty client-side - - `fetch('/download', { method:'POST', body: FormData{url, mode} })` - - On success (content-type not application/json): creates object URL → `` click → revoke URL - - On error or JSON response: parse JSON and show `.message` in status div - -2. **FastAPI app** (`app`): - - `GET /` — returns `HTMLResponse(HTML)` - - `POST /download` — `url: str = Form(...)`, `mode: Literal["mp3","video"] = Form(...)` - - Validate URL non-empty and starts with `http`; if not, raise `HTTPException(400)` - - Create `tempfile.TemporaryDirectory()` - - Build `ydl_opts`: - - MP3: `format='bestaudio/best'`, postprocessor `FFmpegExtractAudio` with `preferredcodec='mp3'`, `preferredquality='0'` - - Video: `format='bestvideo+bestaudio/best'`, `merge_output_format='mp4'` - - `outtmpl=tmpdir/%(title)s.%(ext)s` - - Run `yt_dlp.YoutubeDL(ydl_opts).download([url])` inside `asyncio.get_event_loop().run_in_executor(None, ...)` to avoid blocking the event loop - - Find the downloaded file in tmpdir (glob the single file present) - - Return `FileResponse(path, filename=..., background=BackgroundTask(cleanup_tmpdir))` — cleanup runs after response is sent - - Wrap yt-dlp call in try/except; on any exception raise `HTTPException(500, detail=str(e))` - -3. **`requirements.txt`**: - ``` - fastapi>=0.111.0 - uvicorn[standard]>=0.29.0 - yt-dlp>=2024.4.9 - python-multipart>=0.0.9 - ``` - -**Key decisions:** -- Inline HTML keeps the project a two-file app (plus Dockerfile) — no template engine, no static asset serving needed. -- `run_in_executor` keeps uvicorn's event loop unblocked while yt-dlp runs (which can take 30–120 s). -- `FileResponse` with `BackgroundTask` cleanup is the standard FastAPI pattern for temp file responses. -- yt-dlp is called via its Python API (not subprocess) to avoid shell escaping issues with URLs. - ---- - -### Phase 2 — T-002: Dockerfile + README - -**Files to create/update:** -- `Dockerfile` -- `README.md` (update with build/run instructions) - -**`Dockerfile`**: -```dockerfile -FROM python:3.12-slim +- Pasting a playlist URL (or a watch URL with a `list=` param) and clicking MP3 or Video downloads only that one video, never the rest of the playlist. +- Starting a download shows a progress bar below the buttons that updates with percentage, speed, and downloaded/total size while the server is processing. +- When the download finishes, the file is automatically delivered to the browser (save dialog) without an extra click. +- The saved file's name matches the video's title (emoji/icons/symbols removed, normal characters kept) instead of a UUID or other opaque id, and still ends in the correct extension. +- If the download fails server-side, the UI shows a clear error in place of the progress bar (no server crash, no stuck spinner). -RUN apt-get update \ - && apt-get install -y --no-install-recommends ffmpeg \ - && rm -rf /var/lib/apt/lists/* - -WORKDIR /app - -COPY requirements.txt . -RUN pip install --no-cache-dir -r requirements.txt +## Implementation Phases -COPY app.py . +### Phase 1 — T-003: Single video from playlists + correct filenames -EXPOSE 8080 +**Files to change:** +- `app.py` +- `README.md` (document: playlist links only download the referenced video; filenames follow the video title with icons/emoji stripped) -CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"] -``` +**Changes in `app.py`:** +1. Add `ydl_opts["noplaylist"] = True` for both `mp3` and `video` modes. +2. Add a module-level helper: + ```python + import re -**`README.md`** must document: -- Prerequisites: Docker -- Build: `docker build -t yt-dl .` -- Run: `docker run --rm -p 8080:8080 yt-dl` -- Usage: open `http://localhost:8080` in a browser -- Note on video size / download time expectations + _FILENAME_ALLOWED = re.compile(r"[^\w\s.,'()\-]", re.UNICODE) + _WHITESPACE = re.compile(r"\s+") -## Validation + def sanitize_filename(title: str, fallback_id: str) -> str: + cleaned = _FILENAME_ALLOWED.sub("", title or "") + cleaned = _WHITESPACE.sub(" ", cleaned).strip() + return cleaned or fallback_id + ``` + (`\w` under `re.UNICODE`, the Python 3 default, matches any-language letters/digits/underscore, so non-Latin titles are preserved; emoji/symbols/pictographs fall outside `\w`, whitespace, and the explicit punctuation set and are dropped.) +3. In `fetch_media`, call `downloader.extract_info(url, download=True)` (instead of `.download([url])`) to get the info dict back; read `info.get("title")` and `info.get("id")`, compute `display_name = sanitize_filename(title, id) + Path(media_path).suffix`. +4. Pass `filename=display_name` to the existing `FileResponse(...)` call instead of `media_path.name`. +5. In the frontend `