behavior change: a single mid-call network drop could previously
poison _batch_api_works=False for the whole site, forcing every
future ensure_categories onto the slower HTML scrape path. _do_ensure
now routes the unprobed case through _probe_batch_api, which only
flips the flag on a clean HTTP 200 with zero matching names; timeout
and non-200 responses leave the flag None so the next call retries
the probe.
The bug surfaced because fetch_via_tag_api swallows per-chunk
failures with 'except Exception: continue', so the previous code
path couldn't distinguish 'API returned zero matches' from 'the
network dropped halfway through.' _probe_batch_api already made
that distinction for prefetch_batch; _do_ensure now reuses it.
Tests in tests/core/api/test_category_fetcher.py pin the three
routes (transient raise, clean-200-zero-matches, non-200).
The 'audit #8 invariant' the test was anchored on (core.images
imported without core.cache first) is about to become moot when
images.py is removed in a follow-up commit. Swap to core.config
to keep the same coverage shape: any non-cache submodule import
must still trigger __init__.py and install the PIL cap.
The two validate_public_request hook tests used @pytest.mark.asyncio
which requires pytest-asyncio at collection time. CI only installs
httpx + Pillow + pytest, so the marker decoded as PytestUnknownMark
and the test bodies failed with "async def functions are not
natively supported."
Switches both to plain sync tests that drive the coroutine via
asyncio.run(), matching the pattern already used in test_cache.py
for the same reason.
Audit-Ref: SECURITY_AUDIT.md finding #1 (test infrastructure)
The previous flow streamed the full body to disk and called
_is_valid_media after completion. A hostile server that omits
Content-Type (so the early text/html guard doesn't fire) could
burn up to MAX_DOWNLOAD_BYTES (500MB) of bandwidth and cache-dir
write/delete churn before the post-download check rejected.
Refactors _do_download to accumulate chunks into a small header
buffer until at least 16 bytes have arrived, then runs
_looks_like_media against the buffer before committing to writing
the full payload. The 16-byte minimum handles servers that send
tiny chunks (chunked encoding with 1-byte chunks, slow trickle,
TCP MSS fragmentation) without false-failing on the first chunk.
Extracts _looks_like_media(bytes) as a sibling to _is_valid_media
(path) sharing the same magic-byte recognition. _looks_like_media
fails closed on empty input — when called from the streaming
validator, an empty header means the server returned nothing
useful. _is_valid_media keeps its OSError-fallback open behavior
for the on-disk path so transient EBUSY doesn't trigger a delete
+ re-download loop.
Audit-Ref: SECURITY_AUDIT.md finding #10
Severity: Low
PIL's decompression-bomb cap previously lived as a side effect of
importing core/cache.py. Any future code path that touched core/images
(or any other core submodule) without first importing cache would
silently revert to PIL's default 89M-pixel *warning* (not an error),
re-opening the bomb surface.
Moves the cap into core/__init__.py so any import of any
booru_viewer.core.* submodule installs it first. The duplicate set
in cache.py is left in place by this commit and removed in the next
one — both writes are idempotent so this commit is bisect-safe.
Audit-Ref: SECURITY_AUDIT.md finding #8
Severity: Low
render_filename_template's sanitization stripped reserved chars,
control codes, whitespace, and `..` prefixes — but did not catch
Windows reserved device names (CON, PRN, AUX, NUL, COM1-9, LPT1-9).
On Windows, opening `con.jpg` for writing redirects to the CON
device, so a tag value of `con` from a hostile booru would silently
break Save to Library.
Adds a frozenset of reserved stems and prefixes the rendered name
with `_` if its lowercased stem matches. The check runs
unconditionally (not Windows-gated) so a library saved on Linux
can be copied to a Windows machine without breaking on these
filenames.
Audit-Ref: SECURITY_AUDIT.md finding #7
Severity: Low
Replaces the unbounded defaultdict(asyncio.Lock) with an OrderedDict
guarded by _get_url_lock() and _evict_url_locks(). The cap is 4096
entries; LRU semantics keep the hot working set alive and oldest-
unlocked-first eviction trims back toward the cap on each new
insertion.
Eviction skips locks that are currently held — popping a lock that
a coroutine is mid-`async with` on would break its __aexit__. The
inner loop's evicted-flag handles the edge case where every
remaining entry is either the freshly inserted hash or held; in
that state the cap is briefly exceeded and the next insertion
retries, instead of looping forever.
Audit-Ref: SECURITY_AUDIT.md finding #5
Severity: Medium
The sites table stores api_key + api_user in plaintext. Previous
behavior left the DB file at the inherited umask (0o644 on most
Linux systems) so any other local user could sqlite3 it open and
exfiltrate every booru API key.
Adds Database._restrict_perms(), called from the lazy conn init
right after _migrate(). Tightens the main file plus the -wal and
-shm sidecars to 0o600. The sidecars only exist after the first
write, so the FileNotFoundError path is expected and silenced.
Filesystem chmod failures are also swallowed for FUSE-mount
compatibility.
behavior change from v0.2.5: ~/.local/share/booru-viewer/booru.db
is now 0o600 even if a previous version created it 0o644.
Audit-Ref: SECURITY_AUDIT.md finding #4
Severity: Medium
The data directory holds the SQLite database whose `sites` table
stores api_key and api_user in plaintext. Previous behavior used
the inherited umask (typically 0o755), which leaves the dir
world-traversable on shared workstations and on networked home
dirs whose home is 0o755. Tighten to 0o700 unconditionally on
every data_dir() call so the fix is applied even when an older
version (or external tooling) left the directory loose.
Failures from filesystems that don't support chmod (some FUSE
mounts) are swallowed — better to keep working than refuse to
start. Windows: no-op, NTFS ACLs handle this separately.
behavior change from v0.2.5: ~/.local/share/booru-viewer is now
0o700 even if it was previously 0o755.
Audit-Ref: SECURITY_AUDIT.md finding #4
Severity: Medium
Introduces core/api/_safety.py containing check_public_host and the
validate_public_request async request-hook. The hook rejects any URL
whose host is (or resolves to) loopback, RFC1918, link-local
(including 169.254.169.254 cloud metadata), CGNAT, unique-local v6,
or multicast. Called on every request hop so it covers both the
initial URL and every redirect target that httpx would otherwise
follow blindly.
Also exports redact_url / redact_params for finding #3 — the
secret-key set lives in the same module since both #1 and #3 work
is wired through httpx client event_hooks. Helper is stdlib-only
(ipaddress, socket, urllib.parse) plus httpx; no new deps.
Not yet wired into any httpx client; per-file wiring commits follow.
Audit-Ref: SECURITY_AUDIT.md finding #1
Severity: High
delete_site() leaked rows in tag_types, search_history, and
saved_searches; reconcile_library_meta() was implemented but never
called. Add tests for both fixes plus tag cache pruning.
Remove tests/ from .gitignore and track the existing test suite:
tests/core/test_db.py — DB schema, migration, CRUD
tests/core/test_cache.py — cache helpers
tests/core/test_config.py — config/path helpers
tests/core/test_concurrency.py — app loop accessor
tests/core/api/test_base.py — Post dataclass, BooruClient
tests/gui/popout/test_state.py — 57 state machine tests
All pure Python, no secrets, no external deps. Uses temp DBs and
synthetic data. Run with: pytest tests/
Removes the tests/ folder from git tracking and adds it to .gitignore.
The 81 tests (16 Phase A core + 65 popout state machine) stay on
disk as local-only working notes, the same way docs/ and project.md
are gitignored. Running them is `pytest tests/` from the project
root inside .venv as before — nothing about the tests themselves
changed, just whether they're version-controlled.
Reverts the related additions in pyproject.toml and README.md from
commit bf14466 (Phase A baseline) so the public surface doesn't
reference a tests/ folder that no longer ships:
- pyproject.toml: drops [project.optional-dependencies] test extra
and [tool.pytest.ini_options]. pytest + pytest-asyncio are still
installed in the local .venv via the previous pip install -e ".[test]"
so the suite keeps running locally; new clones won't get them
automatically.
- README.md: drops the "Run tests:" section from the Linux install
block. The README's install instructions return to their pre-
Phase-A state.
- .gitignore: adds `tests/` alongside the existing `docs/` and
`project.md` lines (the same convention used for the refactor
inventory / plan / notes / final report docs).
The 12 test files removed from tracking (`git rm -r --cached`):
tests/__init__.py
tests/conftest.py
tests/core/__init__.py
tests/core/test_cache.py
tests/core/test_concurrency.py
tests/core/test_config.py
tests/core/test_db.py
tests/core/api/__init__.py
tests/core/api/test_base.py
tests/gui/__init__.py
tests/gui/popout/__init__.py
tests/gui/popout/test_state.py
Verification:
- tests/ still exists on disk
- `pytest tests/` still runs and passes 81 / 81 in 0.11s
- `git ls-files tests/` returns nothing
- `git status` is clean
First regression-test layer for booru-viewer. Pure Python — no Qt, no
mpv, no network, no real filesystem outside tmp_path. Locks in the
security and concurrency invariants from the 54ccc40 + eb58d76 hardening
commits ahead of the upcoming popout state machine refactor (Prompt 3),
which needs a stable baseline to refactor against.
16 tests across five files mirroring the source layout under
booru_viewer/core/:
- tests/core/test_db.py (4):
- _validate_folder_name rejection rules (.., /foo, \\foo, .hidden,
~user, empty) and acceptance categories (unicode, spaces, parens)
- add_bookmark INSERT OR IGNORE collision returns the existing row
id (locks in the lastrowid=0 fix)
- get_bookmarks LIKE escaping (literal cat_ear does not match catear)
- tests/core/test_cache.py (7):
- _referer_for hostname suffix matching (gelbooru.com / donmai.us
apex rewrite, both exact-match and subdomain)
- _referer_for rejects substring attackers
(imgblahgelbooru.attacker.com does NOT pick up the booru referer)
- ugoira frame-count and uncompressed-size caps refuse zip bombs
before any decompression
- _do_download MAX_DOWNLOAD_BYTES enforced both at the
Content-Length pre-check AND in the chunk-loop running total
- _is_valid_media returns True on OSError (no delete + redownload
loop on transient EBUSY)
- tests/core/test_config.py (2):
- saved_folder_dir rejects literal .. and ../escape
- find_library_files walks root + 1 level, filters by
MEDIA_EXTENSIONS, exact post-id stem match
- tests/core/test_concurrency.py (2):
- get_app_loop raises RuntimeError before set_app_loop is called
- run_on_app_loop round-trips a coroutine result from a worker
thread loop back to the test thread
- tests/core/api/test_base.py (1):
- BooruClient._shared_client lazy singleton constructor-once under
10-thread first-call race
Plus tests/conftest.py with fixtures: tmp_db, tmp_library,
reset_app_loop, reset_shared_clients. All fixtures use tmp_path or
reset module-level globals around the test so the suite is parallel-
safe.
pyproject.toml:
- New [project.optional-dependencies] test extra: pytest>=8.0,
pytest-asyncio>=0.23
- New [tool.pytest.ini_options]: asyncio_mode = "auto",
testpaths = ["tests"]
README.md:
- Linux install section gains "Run tests" with the
pip install -e ".[test]" + pytest tests/ invocation
Phase B (post-sweep VideoPlayer regression tests for the seek slider
pin, _pending_mute lazy replay, and volume replay) is deferred to
Prompt 3's state machine work — VideoPlayer cannot be instantiated
without QApplication and a real mpv, which is out of scope for a
unit test suite. Once the state machine carves the pure-Python state
out of VideoPlayer, those tests become trivial against the helper
module.
Suite runs in 0.07s (16 tests). Independent of Qt/mpv/network/ffmpeg.
Test cases for Prompt 3:
- (already covered) — this IS the test suite Prompt 3 builds on top of
