@ -179,3 +179,53 @@ Append-only role handoff log. Each role adds one entry when its step is complete
| Next Role | none |
---
### T-005 — plan — 2026-09-11T12:45:39Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Planned Priority 3 fix for downloads failing with a false "This video is not available" error caused by yt-dlp having no JS runtime in the container: install Deno in the Dockerfile and point yt-dlp's `ydl_opts` at it explicitly, plus a README note on the new dependency. |
| Summary | Installed Deno, explicitly configured yt-dlp to use it, and enabled official GitHub EJS challenge-solver retrieval so the previously failing YouTube URL downloads successfully. |
| Validation | `python3 -m py_compile app.py`, `docker build -t yt-dl .`, Deno/EJS configuration checks, and a live MP3 job for `https://www.youtube.com/watch?v=7MrdyaSlOfI` passed; no missing-runtime or skipped-EJS warning appeared. |
| Commit | Pending reviewer approval |
| Next Role | review |
---
### T-005 — implement — 2026-09-11T13:26:25Z
| Field | Value |
|-------|-------|
| Agent | codex |
| Summary | Squashed the reviewed Deno and EJS solver configuration into the task commit. |
| Commit | Created by this handoff's task commit |
| Next Role | none |
---
### T-005 — review — 2026-09-11T13:25:24Z
| Field | Value |
|-------|-------|
| Agent | claude |
| Summary | Reviewed and verified the Deno JS runtime fix: rebuilt the Docker image, confirmed the `js_runtimes`/`remote_components` `ydl_opts` shape against the installed yt-dlp source, and drove live MP3 downloads (previously-failing URL + a playlist-link regression check) through the container with no missing-runtime warning and the EJS solver actively used. |
Goal: implement Priority 3 of `ROADMAP.md` — fix downloads failing with a false `This video is not available` error caused by yt-dlp having no JavaScript runtime available in the container.
## Scope (Priority 3)
- **T-005 — Install Deno as yt-dlp's JS runtime**:
- Add Deno (a single static binary, yt-dlp's documented lightweight JS runtime) to the `Dockerfile` so it's on `PATH` at container run time.
- Point yt-dlp at it explicitly via `ydl_opts` (rather than relying on autodetection) so behavior doesn't silently regress if `deno` ever isn't found on `PATH`.
- Allow yt-dlp to retrieve its official GitHub-hosted EJS challenge-solver scripts when current YouTube extraction requires them.
- Verify server logs stop showing `WARNING: ... No supported JavaScript runtime could be found` during a normal download, and that a video URL exhibiting the `This video is not available` failure now succeeds.
- Document the new build-time/runtime dependency and its purpose in `README.md`.
## Acceptance Criteria
(from `ROADMAP.md` Priority 3)
- `docker build` still completes without errors after adding Deno.
- Re-running a download of a video URL that previously failed with `This video is not available` (due to the missing JS runtime) succeeds and delivers the file.
- Server logs no longer show `No supported JavaScript runtime could be found` during a normal MP3 or Video download.
- Existing Priority 1/2 behavior (single-video-from-playlist, progress bar, filename sanitization) is unaffected.
## Implementation Phases
### Phase 1 — T-005: Add Deno and wire it into yt-dlp
**Files to change:**
- `Dockerfile`
- `app.py`
- `README.md` (document the Deno dependency and why it's needed)
**Changes in `Dockerfile`:**
1. Install Deno in the build stage. Simplest reliable path for a `python:3.12-slim` (Debian) base without adding curl/unzip as extra layers if avoidable — use the official install script, which only needs `curl` and `unzip`:
&& curl -fsSL https://deno.land/install.sh | DENO_INSTALL=/usr/local sh \
&& rm -rf /var/lib/apt/lists/*
```
This places the `deno` binary at `/usr/local/bin/deno`, already on `PATH` for subsequent `RUN`/`CMD` layers and for the app at runtime.
2. Keep the rest of the Dockerfile (`WORKDIR`, `COPY requirements.txt`, `pip install`, `COPY app.py`, `EXPOSE`, `CMD`) unchanged.
**Changes in `app.py`:**
1. In the shared `ydl_opts` construction (around line 317, alongside `outtmpl`/`noplaylist`/`progress_hooks`), explicitly point yt-dlp at the installed runtime instead of relying purely on `PATH` autodetection, so a missing/misconfigured binary fails loudly rather than silently degrading:
```python
ydl_opts: dict[str, object] = {
"outtmpl": output_template,
"noplaylist": True,
"progress_hooks": [progress_hook],
"js_runtimes": {"deno": {"path": shutil.which("deno") or "deno"}},
"remote_components": ["ejs:github"],
}
```
(Exact option name/shape to be confirmed against the installed yt-dlp version's `--js-runtimes` support at implementation time — yt-dlp exposes this as a CLI flag `--js-runtimes RUNTIME[:PATH]`; the implementer should check `yt_dlp.YoutubeDL` / `yt_dlp.options` for the corresponding `ydl_opts` key in the pinned `yt-dlp` version and use that key, falling back to relying on autodetection via `PATH` only if no explicit option key exists in that version, in which case the Dockerfile's `PATH` install alone satisfies the requirement.)
2. Add `remote_components: ["ejs:github"]` to allow the EJS scripts needed by current yt-dlp releases to be fetched from the official yt-dlp GitHub repository.
3. Add `import shutil` if not already imported and used only for this lookup.
**Validation:**
- `python -m py_compile app.py`
- `docker build -t yt-dl .` — must exit 0.
- `docker run --rm -d -p 8080:8080 --name yt-dl-test yt-dl`, then `docker exec yt-dl-test deno --version` to confirm the binary is present and runnable in the final image.
- Live check: paste the video URL that previously failed (`https://www.youtube.com/watch?v=7MrdyaSlOfI`, without playlist params) through the running container for MP3 or Video mode, and confirm it downloads successfully instead of failing with `This video is not available`.
- Inspect `docker logs yt-dl-test` during that download and confirm the `No supported JavaScript runtime could be found` warning is no longer present.
- Regression spot-check: re-run one playlist-link download (`&list=...`) and confirm only the single video downloads, progress bar still updates, and the delivered filename still matches the sanitized video title (Priority 1/2 behavior unaffected).
- `docker stop yt-dl-test`
**Documentation update (`README.md`):**
- Add a short note under "Getting Started" / near the Build section stating that the image installs Deno and permits official GitHub-hosted EJS challenge-solver retrieval so yt-dlp can perform JS-based signature/PO-token deciphering required by current YouTube extraction, and that GitHub access is needed for downloads requiring those scripts.
## Validation (Priority 3)
```bash
python -m py_compile app.py
docker build -t yt-dl .
docker run --rm -d -p 8080:8080 --name yt-dl-test yt-dl
Live download checks (previously-failing video URL now succeeds, log warning gone, playlist-link/progress-bar/filename regressions) are performed manually/by the reviewer against real YouTube URLs, consistent with how prior tasks were validated — no new automated test framework is introduced.
---
# Previous Plan (Priority 2 — completed, kept for reference)
Goal: implement Priority 2 of `ROADMAP.md` — single-video-only downloads from playlist links, correctly-named output files, and a live progress bar in the browser UI.
- Progress percent can remain at 0% for formats where yt-dlp cannot report a total size (noted above, minor, not required).
- The in-memory job store means an app restart mid-download loses all job state/progress for any in-flight browser sessions; this matches the explicitly stated "no persistent storage" design constraint and is now documented in `README.md`.
---
## T-005 — `Dockerfile` (install Deno JS runtime) + `app.py` (point yt-dlp `ydl_opts` at the installed Deno binary and permit official EJS solver retrieval) + `README.md` update
**Verdict:** PASS
### Findings
No blocker, major, minor, or nit findings.
### Verification
**Steps performed:**
- Re-read `.ai/PLAN.md` (Phase 1 spec for T-005) and diffed it against `Dockerfile`, `app.py`, `README.md`, `ROADMAP.md`. The Dockerfile change (install `curl`/`unzip`, run the official Deno install script into `/usr/local`), the `ydl_opts` additions (`js_runtimes`, `remote_components`), and the README note all match the plan.
- `python3 -m py_compile app.py` — succeeded.
- `docker build -t yt-dl .` — succeeded (cached from the implementer's build, layers unchanged).
- Confirmed `shutil` is already imported at the top of `app.py` (used elsewhere for `rmtree`), so no missing-import risk from the new `shutil.which("deno")` call.
- **Verified the `js_runtimes`/`remote_components` `ydl_opts` shape against the actual installed yt-dlp version inside the built image** (yt-dlp `2026.08.19`), since the plan flagged this as needing confirmation at implementation time:
- Inspected `yt_dlp/YoutubeDL.py` inside the container: `self.params['js_runtimes']` is validated by `_clean_js_runtimes`, which requires exactly `dict[str, dict|None]` (`{"deno": {"path": ...}}`) — matches `app.py`'s shape precisely, including that `config.get('path')` is what's read internally.
- Inspected `yt_dlp/options.py`: `dest='js_runtimes'` and `dest='remote_components'` confirm these are the correct `ydl_opts` keys for the installed version.
- Constructed a `yt_dlp.YoutubeDL(...)` instance inside the container with the exact `ydl_opts` dict `app.py` builds — no warnings/errors, confirming the options are accepted and not silently ignored.
- Ran the container (`docker run --rm -d -p 8081:8080 ...`) and:
- `docker exec ... deno --version` → `deno 2.9.6` — binary present and runnable, on `PATH`.
- Live MP3 download of the previously-failing URL (`https://www.youtube.com/watch?v=7MrdyaSlOfI`) via the real job flow (`POST /download` → poll `GET /progress/{job_id}` → `GET /download/{job_id}/file`) — completed successfully, file delivered with the correct sanitized title-based filename via `Content-Disposition`.
- `docker logs` during that download showed `[youtube] [jsc:deno] Solving JS challenges using deno` and `[youtube] [jsc:deno] Downloading challenge solver lib script from https://github.com/yt-dlp/ejs/releases/...` — confirms Deno and the GitHub-hosted EJS solver are actively used, not just installed-but-unused.
- Full container log for both live downloads showed **no**`No supported JavaScript runtime could be found` warning anywhere.
- Regression check: re-ran a playlist-context URL (`watch?v=7MrdyaSlOfI&list=RDMM7MrdyaSlOfI`) through the same job flow — logs show only the single video's extraction/download (no playlist enumeration), progress percent/speed/downloaded/total advanced correctly through completion, and the delivered filename matched the sanitized video title — confirms T-003/T-004 behavior is unaffected.
- Stopped and removed the test container.
- Reviewed the `README.md` addition: accurately explains the Deno runtime dependency, the GitHub-hosted EJS remote component, why they're needed (JS-based signature/PO-token deciphering), and the GitHub reachability requirement.
- Reviewed the `ROADMAP.md` Priority 3 entry: accurately reflects the bug, root cause, and fix, consistent with what was implemented and verified.
**Findings from verification:** All acceptance criteria for T-005 hold:
- `docker build` completes without errors after adding Deno.
- The previously-failing video URL now downloads successfully end-to-end.
- Server logs no longer show the missing-JS-runtime warning during a normal download.
- Priority 1/2 behavior (single-video-from-playlist, progress bar, filename sanitization) is unaffected.
- `python -m py_compile app.py` passes.
**Risks:**
- The fix depends on outbound network access to `github.com` (for the EJS solver script) and `deno.land` (Dockerfile install-time only); if GitHub is unreachable from the container at runtime, the EJS-dependent extraction path could fail again — this is already called out in `README.md` as a requirement, so no action needed.
- `js_runtimes`/`remote_components` are relatively new, evolving yt-dlp options (confirmed only against the currently pinned `requirements.txt` range, `yt-dlp>=2024.4.9`, as resolved to `2026.08.19` in the built image); a future yt-dlp release changing this option's shape would need to be caught by re-running this same build+live-download validation, not by `py_compile` alone.
| T-002 | Dockerfile + README.md (build/run docs) | done | `docker build -t yt-dl .` exits 0; `docker run --rm -p 8080:8080 yt-dl` starts server; `curl http://localhost:8080/` returns HTML with download buttons | Reviewer re-ran `docker build` + container smoke test, plus a live end-to-end MP3 and video download against a real YouTube URL through the container — all passed; see `.ai/REVIEW.md` | none |
| T-003 | `app.py` (`noplaylist` option, `sanitize_filename` helper, title-based `FileResponse` filename, frontend `Content-Disposition`-based `link.download` fix) + `README.md` update | done | Pasting a playlist/`list=` URL downloads only the referenced video; saved filename matches the video title with emoji/icons stripped (normal chars kept) instead of a UUID, with correct extension; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, unit-checked `sanitize_filename`, and ran a live end-to-end playlist-link MP3 download + a plain video download through the built Docker image against real YouTube URLs — all passed; see `.ai/REVIEW.md` | none |
| T-004 | `app.py` (job-based `POST /download` + `GET /progress/{job_id}` + `GET /download/{job_id}/file`, in-memory job store, yt-dlp `progress_hooks`, frontend polling + progress bar UI) + `README.md` update | done | Progress bar appears below the buttons on download start and updates with percent/speed/downloaded-total size; file auto-delivers to the browser on completion with the correct sanitized filename; server-side failure shows an error in place of the bar with no crash; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile` and drove the full job lifecycle live through the built Docker image against real YouTube URLs (progress polling, auto-delivered file, 404/409 semantics, server-side error path with no crash) — all passed; see `.ai/REVIEW.md` | none |
| T-005 | `Dockerfile` (install Deno JS runtime) + `app.py` (point yt-dlp `ydl_opts` at the installed Deno binary and permit official EJS solver retrieval) + `README.md` update | done | `docker build` exits 0; `docker exec <container> deno --version` succeeds; a video URL previously failing with `This video is not available` now downloads successfully; server logs no longer show `No supported JavaScript runtime could be found` during a download; playlist-single-video/progress-bar/filename behavior from T-003/T-004 is unaffected; `python -m py_compile app.py` passes | Reviewer re-ran `py_compile`, `docker build`, verified the `js_runtimes`/`remote_components` `ydl_opts` shape against the installed yt-dlp source, and drove live MP3 downloads (previously-failing URL + playlist-link regression) through the built Docker image — no missing-runtime warning, EJS solver fetched and used, single-video/progress/filename behavior intact; see `.ai/REVIEW.md` | none |
@ -58,3 +58,24 @@ Objective: single-video-only downloads from playlist links, plus live download p
- Playlist batch downloads (downloading every video in a playlist in one action).
- User accounts / history.
- Persisting job/progress state across server restarts.
## Priority 3
Objective: fix downloads failing due to yt-dlp's missing JavaScript runtime.
- Bug: some videos fail with `ERROR: [youtube] <id>: This video is not available`, preceded by `WARNING: [youtube] No supported JavaScript runtime could be found. Only deno is enabled by default...`. yt-dlp falls back to an alternate player client (e.g. `visionos`) when it can't run JS-based signature/PO-token deciphering, and that fallback path incorrectly reports some videos as unavailable.
- Root cause: the Docker image installs no JS runtime, so yt-dlp can't perform the JS-dependent extraction steps modern YouTube extraction increasingly requires.
- Fix: install Deno (yt-dlp's documented lightweight JS runtime) in the `Dockerfile`, configure yt-dlp (via `ydl_opts` and/or a `--js-runtimes` equivalent) to use it, and allow retrieval of the official GitHub-hosted EJS challenge-solver scripts required by current yt-dlp releases, so normal extraction succeeds without falling back to a degraded client.
- Confirm the `WARNING: No supported JavaScript runtime could be found` message no longer appears in server logs during a download.
## Acceptance Criteria (Priority 3)
- `docker build` still completes without errors after adding Deno.
- Re-running a download of a video URL that previously failed with `This video is not available` (due to the missing JS runtime) succeeds and delivers the file.
- Server logs no longer show `No supported JavaScript runtime could be found` during a normal MP3 or Video download.
- Existing Priority 1/2 behavior (single-video-from-playlist, progress bar, filename sanitization) is unaffected.
## Out of Scope
- Fixing videos that are genuinely unavailable (age-restricted with no workaround, region-blocked, deleted, private) — this priority only addresses the missing-JS-runtime-induced false negative.
- General yt-dlp version upgrade policy beyond what's needed to pair with the new JS runtime.
Domagoj Zecevic
1 week ago