@ -229,3 +229,64 @@ Append-only role handoff log. Each role adds one entry when its step is complete
| Next Role | implement |
| Next Role | implement |
---
---
### T-006 — plan — 2026-09-11T13:33:14Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Planned Priority 4 fix for PO-Token/429 download failures: add `extractor_args.youtube.player_client` ordering (android/ios/tv before web) to the shared `ydl_opts` in `app.py` so extraction avoids the web client's PO Token requirement first. |
| Summary | Planned Priority 5 feature: add a third "Download Video 720p" mode/button, capped at 720p via `bestvideo[height<=720]+bestaudio/best[height<=720]` with graceful fallback to lower quality, reusing existing progress/filename/playlist behavior. |
| Validation | `python3 -m py_compile app.py`, `docker build -t yt-dl .`, and a live MP3 job for `https://www.youtube.com/watch?v=LV-NXucnyrc` passed; alternate clients were used after a web 429 and the file was delivered. |
| Commit | Pending reviewer approval |
| Next Role | review |
---
### T-006 — implement — 2026-09-11T13:42:04Z
| Field | Value |
|-------|-------|
| Agent | codex |
| Summary | Squashed the reviewed alternate YouTube player-client fallback into the task commit. |
| Commit | Created by this handoff's task commit |
| Next Role | none |
---
### T-006 — review — 2026-09-11T13:41:31Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Reviewed and verified the player-client ordering fix: rebuilt the Docker image, confirmed `player_client` is genuinely consumed by the installed yt-dlp extractor, and drove live downloads (previously-failing `LV-NXucnyrc`, plain MP3/Video, playlist-link, error-path) through the container — no PO-Token-blocking failure, alternate clients used after a web 429, all regressions passed. |
Goal: implement Priority 4 (player-client fallback for PO Token / 429 failures) and Priority 5 (720p video download option) of `ROADMAP.md`.
## Scope (Priority 4) — T-006: Prefer non-web player clients
- Problem: after T-005, some downloads now fail via a different path: `HTTP Error 429: Too Many Requests` fetching the webpage, then `Unable to fetch GVS PO Token for web client: Missing required Visitor Data`, ending in `This video is not available`. The `web` client increasingly requires a PO Token yt-dlp can't generate, and is more exposed to IP-based rate limiting.
- Fix: add `extractor_args: {"youtube": {"player_client": ["android", "ios", "tv", "web"]}}` (or the current yt-dlp-recommended ordering) to the shared `ydl_opts` in `app.py` (same dict built at line ~317, alongside `js_runtimes`/`remote_components`), so yt-dlp tries clients that don't strictly require a PO Token first, and only falls back to `web` last.
- This applies uniformly to both `mp3` and `video` (and the new `video720` from Priority 5) modes since it's set on the shared `ydl_opts` before the per-mode `format`/`postprocessors` update.
- No new error-handling paths are needed: if every client fails, the existing `run_job` try/except already marks the job `status="error"` and the UI already surfaces that.
### Files to change (T-006)
- `app.py` — add the `extractor_args``player_client` ordering to the shared `ydl_opts` dict.
- `README.md` — document that the app tries multiple YouTube player clients (`android`, `ios`, `tv`, then `web`) to avoid PO-Token/rate-limit failures on the `web` client, and that this is a best-effort mitigation that may need retuning as YouTube changes its anti-bot behavior.
### Validation (T-006)
- `python -m py_compile app.py`
- `docker build -t yt-dl .` — must exit 0.
- Live check: re-run a download of a video URL that previously failed with `Missing required Visitor Data` / `This video is not available` (e.g. the reported `LV-NXucnyrc` case) through the running container, and confirm it now succeeds.
- Inspect `docker logs` during that download and confirm an alternate client (`android`/`ios`/`tv`) is used and the job completes without the PO-Token warning blocking it.
- Regression spot-check: re-run one plain MP3 download and one plain Video download (non-playlist) and confirm they still succeed with progress bar + correct filename, unaffected by the client-ordering change.
## Scope (Priority 5) — T-007: Add a "Download Video 720p" option
- Add a third mode, `video720`, alongside the existing `mp3`/`video` modes, both server-side and in the UI.
- Server-side (`app.py`):
- Extend the `mode` type from `Literal["mp3", "video"]` to `Literal["mp3", "video", "video720"]` everywhere it's declared: the `/download` route's `Form(...)` parameter (line ~280) and `JobState.mode` (line ~216).
- In the `if mode == "mp3": ... else: ...` branch (line ~325 onward) that sets `format`/`merge_output_format`, add a branch for `video720`:
This caps the video stream at 720p while still falling back to the best available quality below 720p if that's all that exists (yt-dlp's format selector already does this — no extra fallback logic needed).
- No changes needed to `sanitize_filename`, the job store, `progress_hook`, or the `/progress`/`/download/{job_id}/file` endpoints — they're mode-agnostic already.
- Frontend (inline HTML/JS in `app.py`):
- Add a third button next to the existing two (line ~49): `<button type="button" onclick="downloadFile('video720')">Download Video 720p</button>`.
- `downloadFile(mode)` already takes `mode` as a parameter and POSTs it through unchanged — confirm no mode-specific branching exists in the JS that would need a new case (per the current implementation, the JS is generic over `mode`, so this should be a markup-only change).
### Files to change (T-007)
- `app.py` — extend the `mode``Literal` type (route param + `JobState`), add the `video720` format-selection branch, add the third HTML button.
- `README.md` — document the new **Download Video 720p** button: caps video quality at 720p, falls back to the best available quality if the source is below 720p, and behaves like the existing Video button otherwise (progress bar, auto-delivery, sanitized filename).
### Validation (T-007)
- `python -m py_compile app.py`
- `docker build -t yt-dl .` — must exit 0.
- Live check: click **Download Video 720p** on a video known to have >720p streams available; inspect the delivered file's video resolution (e.g. via `ffprobe`) and confirm it is ≤720p.
- Live check: click **Download Video 720p** on a video whose best quality is below 720p (or force via a low-quality test video) and confirm the download still succeeds with no error.
- Regression spot-check: confirm **Download MP3** and **Download Video** (unrestricted) still behave exactly as before — correct format/quality, progress bar, filename, playlist-single-video behavior.
## Acceptance Criteria
(from `ROADMAP.md` Priority 4 and Priority 5 — see those sections for full text)
- Priority 4: a previously PO-Token/429-failing video now downloads via an alternate player client; existing behavior (playlist/progress/filename/Deno) unaffected; failures still surface a clean UI error with no crash.
- Priority 5: a third "Download Video 720p" button exists; it caps video quality at 720p with graceful fallback when the source is lower quality; progress bar, auto-delivery, filename sanitization, and existing MP3/Video buttons are unaffected.
## Implementation Order
1. T-006 first (player-client fallback) — it's an independent, small `ydl_opts` change that other live-download validation (including T-007's) benefits from being in place first, since it reduces the chance of unrelated PO-Token failures muddying 720p testing.
2. T-007 second (720p mode) — builds on the same `ydl_opts` dict shape.
## Validation (both tasks, combined)
```bash
python -m py_compile app.py
docker build -t yt-dl .
docker run --rm -d -p 8080:8080 --name yt-dl-test yt-dl
sleep 3
curl -sf http://localhost:8080/ | grep -q "Download Video 720p"
docker stop yt-dl-test
```
Live download checks (PO-Token/429 recovery, 720p resolution cap + fallback, MP3/Video regressions) are performed manually/by the reviewer against real YouTube URLs, consistent with how prior tasks were validated — no new automated test framework is introduced.
---
# Previous Plan (Priority 3 — completed, kept for reference)
Goal: implement Priority 3 of `ROADMAP.md` — fix downloads failing with a false `This video is not available` error caused by yt-dlp having no JavaScript runtime available in the container.
Goal: implement Priority 3 of `ROADMAP.md` — fix downloads failing with a false `This video is not available` error caused by yt-dlp having no JavaScript runtime available in the container.
@ -193,3 +193,40 @@ No blocker, major, minor, or nit findings.
**Risks:**
**Risks:**
- The fix depends on outbound network access to `github.com` (for the EJS solver script) and `deno.land` (Dockerfile install-time only); if GitHub is unreachable from the container at runtime, the EJS-dependent extraction path could fail again — this is already called out in `README.md` as a requirement, so no action needed.
- The fix depends on outbound network access to `github.com` (for the EJS solver script) and `deno.land` (Dockerfile install-time only); if GitHub is unreachable from the container at runtime, the EJS-dependent extraction path could fail again — this is already called out in `README.md` as a requirement, so no action needed.
- `js_runtimes`/`remote_components` are relatively new, evolving yt-dlp options (confirmed only against the currently pinned `requirements.txt` range, `yt-dlp>=2024.4.9`, as resolved to `2026.08.19` in the built image); a future yt-dlp release changing this option's shape would need to be caught by re-running this same build+live-download validation, not by `py_compile` alone.
- `js_runtimes`/`remote_components` are relatively new, evolving yt-dlp options (confirmed only against the currently pinned `requirements.txt` range, `yt-dlp>=2024.4.9`, as resolved to `2026.08.19` in the built image); a future yt-dlp release changing this option's shape would need to be caught by re-running this same build+live-download validation, not by `py_compile` alone.
- Re-read `.ai/PLAN.md` (Priority 4 / T-006 spec) and diffed it against `app.py`, `README.md`, `ROADMAP.md`. The single-line addition — `"extractor_args": {"youtube": {"player_client": ["android", "ios", "tv", "web"]}}` in the shared `ydl_opts` dict, applied uniformly before the per-mode `format` update — matches the plan exactly. `README.md` documents the client-ordering mitigation and its best-effort nature; `ROADMAP.md` Priority 4 accurately describes the bug, fix, and acceptance criteria.
- Verified the `extractor_args`/`player_client` key is genuinely consumed by the installed yt-dlp (`2026.08.19`) inside the built image, not silently ignored: `grep player_client` in `yt_dlp/extractor/youtube/_video.py` shows `self._configuration_arg('player_client')` is read directly by the extractor, and constructing a real `yt_dlp.YoutubeDL(...)` with the exact `ydl_opts``app.py` builds raised no warnings/errors.
- Ran the container and drove live downloads through the real job flow (`POST /download` → poll `GET /progress/{job_id}` → `GET /download/{job_id}/file`):
- **Previously-failing video** (`LV-NXucnyrc`, mode `mp3`) — completed successfully. `docker logs` showed `WARNING: ... HTTP Error 429: Too Many Requests` on the initial webpage fetch (the originally-reported failure trigger), immediately followed by `Downloading android player API JSON`, `Downloading ios player API JSON`, `Downloading tv client config`/`tv player API JSON` — confirming the alternate clients were tried and used. No `Missing required Visitor Data` or `This video is not available` appeared; the job reached `status: "finished"` and the file was delivered via `Content-Disposition` with a correctly sanitized title-based filename (`Baby's 1st Space Adventure ....mp3`), verified as a valid MP3 with `file`.
- **Regression: plain Video** (`jNQXAC9IVRw`, mode `video`) — completed, delivered a valid MP4 (`file` confirms ISO Media MP4) with filename `Me at the zoo.mp4`.
- **Regression: playlist-link** (`jNQXAC9IVRw&list=RDMMjNQXAC9IVRw`, mode `mp3`) — `docker logs` showed `Downloading just the video jNQXAC9IVRw because of --no-playlist`, confirming T-003's single-video behavior is unaffected by the client-ordering change.
- **Error path**: posted a non-existent URL (`http://example.com/not-a-video`) — job transitioned to `status: "error"` with a clear `error` message (`HTTP Error 404: Not Found`), no server crash (`GET /` still returned 200 immediately after), confirming the existing error-handling path is unaffected.
- Progress bar percent/speed/downloaded/total advanced correctly for all live jobs.
- Stopped and removed the test container, deleted downloaded test files.
**Findings from verification:** All acceptance criteria for T-006 hold:
- A video URL previously failing with `Missing required Visitor Data` / `This video is not available` now downloads successfully via an alternate player client (confirmed live with `LV-NXucnyrc`).
- Server logs show the alternate player clients (`android`/`ios`/`tv`) being used, and no PO-Token-blocking failure occurred.
- Existing playlist-single-video/progress-bar/filename/Deno behavior (T-003/T-004/T-005) is unaffected — all regression checks passed.
- Failures still surface a clean UI error with no crash (verified with the error-path check).
- `python -m py_compile app.py` passes.
**Risks:**
- This is an explicitly best-effort mitigation (as documented in `README.md` and `ROADMAP.md`): YouTube's anti-bot posture changes over time, and the `android`/`ios`/`tv` clients could themselves become restricted later, requiring the client list to be retuned. No action needed now; this is a known, accepted risk per the plan's scope.
- The live test observed `WARNING: ... Some android/ios client https formats have been skipped ... SABR-only streaming experiment` — a yt-dlp/YouTube-side warning unrelated to this change (it did not block the download, format selection fell through to a working format) — noted for awareness, not a regression introduced by T-006.
| T-003 | `app.py` (`noplaylist` option, `sanitize_filename` helper, title-based `FileResponse` filename, frontend `Content-Disposition`-based `link.download` fix) + `README.md` update | done | Pasting a playlist/`list=` URL downloads only the referenced video; saved filename matches the video title with emoji/icons stripped (normal chars kept) instead of a UUID, with correct extension; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download + a plain video download through the built Docker image against real YouTube URLs — all passed; see `.ai/REVIEW.md` | none |
| T-003 | `app.py` (`noplaylist` option, `sanitize_filename` helper, title-based `FileResponse` filename, frontend `Content-Disposition`-based `link.download` fix) + `README.md` update | done | Pasting a playlist/`list=` URL downloads only the referenced video; saved filename matches the video title with emoji/icons stripped (normal chars kept) instead of a UUID, with correct extension; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download + a plain video download through the built Docker image against real YouTube URLs — all passed; see `.ai/REVIEW.md` | none |
| T-004 | `app.py` (job-based `POST /download` + `GET /progress/{job_id}` + `GET /download/{job_id}/file`, in-memory job store, yt-dlp `progress_hooks`, frontend polling + progress bar UI) + `README.md` update | done | Progress bar appears below the buttons on download start and updates with percent/speed/downloaded-total size; file auto-delivers to the browser on completion with the correct sanitized filename; server-side failure shows an error in place of the bar with no crash; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile` and drove the full job lifecycle live through the built Docker image against real YouTube URLs (progress polling, auto-delivered file, 404/409 semantics, server-side error path with no crash) — all passed; see `.ai/REVIEW.md` | none |
| T-004 | `app.py` (job-based `POST /download` + `GET /progress/{job_id}` + `GET /download/{job_id}/file`, in-memory job store, yt-dlp `progress_hooks`, frontend polling + progress bar UI) + `README.md` update | done | Progress bar appears below the buttons on download start and updates with percent/speed/downloaded-total size; file auto-delivers to the browser on completion with the correct sanitized filename; server-side failure shows an error in place of the bar with no crash; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile` and drove the full job lifecycle live through the built Docker image against real YouTube URLs (progress polling, auto-delivered file, 404/409 semantics, server-side error path with no crash) — all passed; see `.ai/REVIEW.md` | none |
| T-005 | `Dockerfile` (install Deno JS runtime) + `app.py` (point yt-dlp `ydl_opts` at the installed Deno binary and permit official EJS solver retrieval) + `README.md` update | done | `docker build` exits 0; `docker exec <container> deno --version` succeeds; a video URL previously failing with `This video is not available` now downloads successfully; server logs no longer show `No supported JavaScript runtime could be found` during a download; playlist-single-video/progress-bar/filename behavior from T-003/T-004 is unaffected; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, `docker build`, verified the `js_runtimes`/`remote_components` `ydl_opts` shape against the installed yt-dlp source, and drove live MP3 downloads (previously-failing URL + playlist-link regression) through the built Docker image — no missing-runtime warning, EJS solver fetched and used, single-video/progress/filename behavior intact; see `.ai/REVIEW.md` | none |
| T-005 | `Dockerfile` (install Deno JS runtime) + `app.py` (point yt-dlp `ydl_opts` at the installed Deno binary and permit official EJS solver retrieval) + `README.md` update | done | `docker build` exits 0; `docker exec <container> deno --version` succeeds; a video URL previously failing with `This video is not available` now downloads successfully; server logs no longer show `No supported JavaScript runtime could be found` during a download; playlist-single-video/progress-bar/filename behavior from T-003/T-004 is unaffected; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, `docker build`, verified the `js_runtimes`/`remote_components` `ydl_opts` shape against the installed yt-dlp source, and drove live MP3 downloads (previously-failing URL + playlist-link regression) through the built Docker image — no missing-runtime warning, EJS solver fetched and used, single-video/progress/filename behavior intact; see `.ai/REVIEW.md` | none |
| T-006 | `app.py` (`extractor_args.youtube.player_client` ordering in shared `ydl_opts`) + `README.md` update | done | A video URL previously failing with `Missing required Visitor Data` / `This video is not available` now downloads successfully via an alternate player client; existing playlist-single-video/progress-bar/filename/Deno behavior unaffected; failures still surface a clean UI error with no crash; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, `docker build`, confirmed the `player_client` key is read by the installed yt-dlp extractor, and drove live downloads (previously-failing `LV-NXucnyrc`, plain MP3/Video, playlist-link, error-path) through the built Docker image — all passed; see `.ai/REVIEW.md` | none |
| T-007 | `app.py` (`video720` mode: `Literal` type extension, `bestvideo[height<=720]+bestaudio/best[height<=720]` format branch, third HTML button) + `README.md` update | ready_for_implement | UI shows a third "Download Video 720p" button; downloads are capped at 720p with graceful fallback to lower quality when the source is below 720p; progress bar, auto-delivery, and sanitized filename behavior match existing modes; existing MP3/Video buttons unchanged; `python -m py_compile app.py` passes | pending | implement |
@ -79,3 +79,48 @@ Objective: fix downloads failing due to yt-dlp's missing JavaScript runtime.
- Fixing videos that are genuinely unavailable (age-restricted with no workaround, region-blocked, deleted, private) — this priority only addresses the missing-JS-runtime-induced false negative.
- Fixing videos that are genuinely unavailable (age-restricted with no workaround, region-blocked, deleted, private) — this priority only addresses the missing-JS-runtime-induced false negative.
- General yt-dlp version upgrade policy beyond what's needed to pair with the new JS runtime.
- General yt-dlp version upgrade policy beyond what's needed to pair with the new JS runtime.
## Priority 4
Objective: fix downloads failing due to YouTube's PO Token requirement and IP-based rate limiting on the `web` player client.
- Bug: after Priority 3's fix, some downloads now fail differently: `WARNING: [youtube] LV-...: Unable to download webpage: HTTP Error 429: Too Many Requests`, followed by `WARNING: [youtube] Unable to fetch GVS PO Token for web client: Missing required Visitor Data`, ending in `ERROR: [youtube] ...: This video is not available`. The `web` client increasingly requires a Proof-of-Origin (PO) Token yt-dlp cannot generate on its own, and is also more exposed to YouTube's IP-based rate limiting.
- Fix: configure yt-dlp's `extractor_args` (`player_client`) to prefer player clients that historically don't require a PO Token as strictly (e.g. `android`, `ios`, `tv`), falling back to `web` only if those fail, instead of relying on the `web` client first.
- This is a best-effort mitigation, not a permanent fix: YouTube's anti-bot requirements change over time, so the chosen client list may need retuning later if YouTube tightens restrictions on the alternate clients too.
## Acceptance Criteria (Priority 4)
- Re-running a download of a video URL that previously failed with `Missing required Visitor Data` / `This video is not available` under the `web` client now succeeds using an alternate player client.
- Server logs show the alternate player client(s) being used and no longer show `Unable to fetch GVS PO Token for web client` blocking the download.
- Existing Priority 1/2/3 behavior (single-video-from-playlist, progress bar, filename sanitization, Deno JS runtime) is unaffected.
- If all configured player clients fail for a given video, the existing error-handling path still surfaces a clear error in the UI with no server crash (no new failure mode introduced).
## Out of Scope
- Running a separate PO-Token-provider service/container.
- Retry/backoff handling for transient 429s (may be revisited later if switching clients doesn't sufficiently resolve rate limiting).
- Guaranteeing every video downloads successfully — YouTube's anti-bot measures are outside this app's control.
## Priority 5
Objective: add a capped-720p video download option alongside the existing best-quality video download.
- Add a third button, **Download Video 720p**, next to the existing **Download MP3** and **Download Video** buttons.
- **Download Video** keeps today's behavior unchanged: best available video+audio quality, no cap.
- **Download Video 720p** downloads the best available video+audio quality capped at 720p — i.e. yt-dlp format selection equivalent to `bestvideo[height<=720]+bestaudio/best[height<=720]`, merged the same way as today's video mode (`mp4` container via `merge_output_format`).
- If the video's best available quality is below 720p (e.g. only 480p exists), the 720p button downloads the best quality actually available — no error, no upscaling.
- All existing behavior applies unchanged to the new mode: single-video-from-playlist (`noplaylist`), background job + live progress bar, sanitized title-based filename with the correct extension, and error handling on failure.
## Acceptance Criteria (Priority 5)
- The UI shows three buttons: Download MP3, Download Video, Download Video 720p.
- Clicking **Download Video 720p** on a video that has streams above 720p delivers a file whose video stream is at most 720p (not the unrestricted best quality).
- Clicking **Download Video 720p** on a video whose best quality is below 720p still succeeds and delivers that lower-quality file (no error).
- The 720p mode shows the same live progress bar (percent/speed/downloaded-total size) and the same auto-delivery-on-completion behavior as the existing Video and MP3 modes.
- The delivered filename follows the same sanitized-title convention as the existing modes, with the correct video extension.
- Existing **Download MP3** and **Download Video** behavior is unchanged.
## Out of Scope
- Additional quality tiers beyond 720p (e.g. 480p, 1080p buttons) — only the existing best-quality option and the new 720p-capped option are in scope.
- A quality-selection dropdown or other UI beyond a third fixed button.
Domagoj Zecevic
1 week ago