12 KiB

Raw Blame History

Plan

Status: ready_for_implement

Goal: implement Priority 2 of ROADMAP.md — single-video-only downloads from playlist links, correctly-named output files, and a live progress bar in the browser UI.

Scope

T-003 — Single video from playlist links + correct filenames (no architecture change; still one synchronous POST /download request/response):
- Pass yt-dlp's noplaylist: True so a playlist URL or a watch?v=...&list=... URL only ever downloads the one referenced video.
- Add a sanitize_filename(title, fallback_id) helper: strip characters outside letters (any language)/digits/spaces/- _ . ( ) , ', collapse resulting whitespace, trim, and fall back to the video id if nothing usable remains.
- Extract title/id from yt-dlp's result info dict and pass the sanitized name explicitly as FileResponse(..., filename=...), decoupled from whatever yt-dlp happens to name the file on disk.
- Fix the real UUID-looking-filename bug: response.blob() in the frontend JS strips the Content-Disposition header, so link.download = '' makes the browser invent a blob-id-like name. Read the filename from the Content-Disposition response header in JS and set link.download to it explicitly (fallback to a generic name only if the header is somehow missing).
T-004 — Background job + live progress bar (architecture change, builds on T-003):
- Split POST /download into three endpoints:
  - POST /download — validates the URL, creates an in-memory job record (job_id = uuid4(), status pending), schedules the actual yt-dlp run on a background thread (asyncio.get_running_loop().run_in_executor), and returns {"job_id": ...} immediately (no file streaming here anymore).
  - GET /progress/{job_id} — returns the job's current {status, percent, speed, downloaded_bytes, total_bytes, filename, error} from the in-memory store. status is one of downloading, finished, error.
  - GET /download/{job_id}/file — once status == finished, streams the prepared file via FileResponse (reusing T-003's sanitized filename) with BackgroundTask cleanup of the job's temp dir and job record; returns 409/404 if called before completion or after cleanup.
- Progress capture: a yt-dlp progress_hooks callback updates the job's dict on each tick using downloaded_bytes, total_bytes (or total_bytes_estimate), speed, and computed percent; a postprocessor_hooks callback (or the finished hook state) marks the job finished once the final file is ready.
- Job store: a plain in-memory dict[str, JobState] guarded by a threading.Lock (single-process app, matches "no persistent storage" constraint). Entries are removed after the file is served, and a periodic/best-effort cleanup drops stale jobs (e.g. no progress endpoint hit within N minutes) so a browser tab closed mid-download doesn't leak a temp dir forever.
- Frontend JS rework:
  - downloadFile(mode) now: POSTs to /download to obtain job_id, shows the progress bar area, then polls GET /progress/{job_id} every 500ms.
  - On each poll: update a <progress> bar (or styled div) with percent, and a text line with speed (human-readable, e.g. 1.2 MB/s) and downloaded / total size (human-readable bytes).
  - On status == finished: stop polling, fetch GET /download/{job_id}/file, read filename from Content-Disposition (per T-003's fix), trigger the save via the existing blob+anchor pattern, then hide the progress bar and clear status text.
  - On status == error: stop polling, show the error message in the existing #status div, hide the progress bar.
- HTML/CSS: add a progress bar element (native <progress> plus a small text line for speed/size) directly below the two buttons, hidden by default, shown only while a job is active.

Acceptance Criteria

(from ROADMAP.md Priority 2)

Pasting a playlist URL (or a watch URL with a list= param) and clicking MP3 or Video downloads only that one video, never the rest of the playlist.
Starting a download shows a progress bar below the buttons that updates with percentage, speed, and downloaded/total size while the server is processing.
When the download finishes, the file is automatically delivered to the browser (save dialog) without an extra click.
The saved file's name matches the video's title (emoji/icons/symbols removed, normal characters kept) instead of a UUID or other opaque id, and still ends in the correct extension.
If the download fails server-side, the UI shows a clear error in place of the progress bar (no server crash, no stuck spinner).

Implementation Phases

Phase 1 — T-003: Single video from playlists + correct filenames

Files to change:

app.py
README.md (document: playlist links only download the referenced video; filenames follow the video title with icons/emoji stripped)

Changes in app.py:

Add ydl_opts["noplaylist"] = True for both mp3 and video modes.

Add a module-level helper:

import re

_FILENAME_ALLOWED = re.compile(r"[^\w\s.,'()\-]", re.UNICODE)
_WHITESPACE = re.compile(r"\s+")

def sanitize_filename(title: str, fallback_id: str) -> str:
    cleaned = _FILENAME_ALLOWED.sub("", title or "")
    cleaned = _WHITESPACE.sub(" ", cleaned).strip()
    return cleaned or fallback_id

(\w under re.UNICODE, the Python 3 default, matches any-language letters/digits/underscore, so non-Latin titles are preserved; emoji/symbols/pictographs fall outside \w, whitespace, and the explicit punctuation set and are dropped.)

In fetch_media, call downloader.extract_info(url, download=True) (instead of .download([url])) to get the info dict back; read info.get("title") and info.get("id"), compute display_name = sanitize_filename(title, id) + Path(media_path).suffix.
Pass filename=display_name to the existing FileResponse(...) call instead of media_path.name.

In the frontend <script>, change the success branch of downloadFile:

const disposition = response.headers.get('content-disposition') || '';
const match = /filename\*?=(?:UTF-8''|")?([^\";]+)/i.exec(disposition);
const filename = match ? decodeURIComponent(match[1].replace(/"/g, '')) : 'download';
...
link.download = filename;

Keep the rest of the blob/anchor logic unchanged.

Validation:

python -m py_compile app.py
Unit-style check (via a small mocked test or manual python -c) that sanitize_filename("Song 🔥 Title ✨ (Live)", "abc123") == "Song Title (Live)" collapsed to single spaces → "Song Title (Live)", and sanitize_filename("🔥🔥🔥", "abc123") == "abc123".
Manual/live check: paste a watch?v=...&list=... URL and confirm only the single video downloads; confirm the saved filename in the browser matches the (emoji-stripped) video title, not a UUID.

Phase 2 — T-004: Background job + live progress bar

Files to change:

app.py
README.md (document the new /download, /progress/{job_id}, /download/{job_id}/file flow and the progress bar UI; note job state is in-memory/non-persistent)

Backend changes in app.py:

Add a JobState structure (dataclass or TypedDict) with fields: status ("downloading" | "finished" | "error"), percent, speed, downloaded_bytes, total_bytes, temp_dir, file_path, filename, error_message, mode, created_at.
Add a module-level _jobs: dict[str, JobState] and _jobs_lock = threading.Lock().
POST /download (url, mode form fields, same validation as today):
- Validate URL as today (raise HTTPException(400) on invalid/empty).
- Create job_id = str(uuid4()), create the temp dir, seed a JobState(status="downloading", ...) in _jobs under the lock.
- Define a progress_hook(d) closure that, under the lock, updates the job's percent/speed/downloaded_bytes/total_bytes from d["status"], d.get("downloaded_bytes"), d.get("total_bytes") or d.get("total_bytes_estimate"), d.get("speed"); on d["status"] == "error" marks the job error.
- Build ydl_opts same as T-003 (noplaylist: True, plus progress_hooks: [progress_hook]).
- Define run_job() that calls extract_info(url, download=True), computes display_name via sanitize_filename (from T-003), locates the produced file, and updates the job to status="finished" with file_path/filename set; wraps in try/except to set status="error"/error_message on failure, and cleans up the temp dir on error (not on success — success cleanup happens after the file is served).
- Schedule run_job via asyncio.get_running_loop().run_in_executor(None, run_job) without awaiting it (fire-and-forget task kept referenced, e.g. appended to a module-level set of pending futures to avoid GC warnings).
- Return JSONResponse({"job_id": job_id}).
GET /progress/{job_id}:
- Look up the job under the lock; 404 (JSON error) if unknown.
- Return {"status", "percent", "speed", "downloaded_bytes", "total_bytes", "error"} (omit file path from the response).
GET /download/{job_id}/file:
- Look up the job; 404 if unknown, 409 (JSON error) if not yet finished.
- Return FileResponse(job.file_path, filename=job.filename, background=BackgroundTask(cleanup_job, job_id)) where cleanup_job removes the temp dir and pops the job from _jobs under the lock.
Best-effort stale-job sweep: a small helper (called opportunistically, e.g. at the top of POST /download) that drops any job older than a fixed TTL (e.g. 30 minutes) whose temp dir hasn't been claimed, cleaning up its temp dir — keeps the in-memory store from growing unbounded if a browser tab is closed mid-download. No background scheduler/thread is introduced solely for this; it piggybacks on request handling to keep the app dependency-free.

Frontend changes (inline HTML/JS in app.py):

Add markup below the buttons:

<div id="progress-wrap" hidden>
  <progress id="progress-bar" max="100" value="0"></progress>
  <div id="progress-text"></div>
</div>

Rework downloadFile(mode):
- POST to /download, parse {job_id} (existing error handling for non-2xx/JSON stays the same shape as today).
- Show #progress-wrap, start setInterval polling GET /progress/{job_id} every 500ms.
- On each tick: update #progress-bar.value from percent; update #progress-text with a human-readable line, e.g. formatSpeed(speed) + ' · ' + formatBytes(downloaded_bytes) + ' / ' + formatBytes(total_bytes).
- Add small formatBytes(n) / formatSpeed(n) helpers (KB/MB/GB, /s suffix), handling null/unknown total gracefully (e.g. show downloaded size only if total is unknown).
- On status === 'finished': clearInterval, fetch GET /download/{job_id}/file, apply the same Content-Disposition-based filename logic from T-003, trigger the blob/anchor save, then hide #progress-wrap and reset text/bar.
- On status === 'error': clearInterval, hide #progress-wrap, show error (or a generic message) in #status.
- Guard against overlapping downloads from double-clicks (e.g. disable buttons while a job is active, re-enable on finish/error).