pax/booru-viewer
1
0
Fork 0
Code Issues Releases 18 Activity
booru-viewer/booru_viewer/gui/bookmarks.py
pax eb58d76bc0 Route async work through one persistent loop, lock shared httpx + DB writes 
Mixing `threading.Thread + asyncio.run` workers with the long-lived
asyncio loop in gui/app.py is a real loop-affinity bug: the first worker
thread to call `asyncio.run` constructs a throwaway loop, which the
shared httpx clients then attach to, and the next call from the
persistent loop fails with "Event loop is closed" / "attached to a
different loop". This commit eliminates the pattern across the GUI and
adds the locking + cleanup that should have been there from the start.

Persistent loop accessor (core/concurrency.py — new)
- set_app_loop / get_app_loop / run_on_app_loop. BooruApp registers the
  one persistent loop at startup; everything that wants to schedule
  async work calls run_on_app_loop instead of spawning a thread that
  builds its own loop. Three functions, ~30 lines, single source of
  truth for "the loop".

Lazy-init lock + cleanup on shared httpx clients (core/api/base.py,
core/api/e621.py, core/cache.py)
- Each shared singleton (BooruClient._shared_client, E621Client._e621_client,
  cache._shared_client) now uses fast-path / locked-slow-path lazy init.
  Concurrent first-callers from the same loop can no longer both build
  a client and leak one (verified: 10 racing callers => 1 httpx instance).
- Each module exposes an aclose helper that BooruApp.closeEvent runs via
  run_coroutine_threadsafe(...).result(timeout=5) BEFORE stopping the
  loop. The connection pool, keepalive sockets, and TLS state finally
  release cleanly instead of being abandoned at process exit.
- E621Client tracks UA-change leftovers in _e621_to_close so the old
  client doesn't leak when api_user changes — drained in aclose_shared.

GUI workers routed through the persistent loop (gui/sites.py,
gui/bookmarks.py)
- SiteDialog._on_detect / _on_test: replaced
  `threading.Thread(target=lambda: asyncio.run(...))` with
  run_on_app_loop. Results marshaled back through Qt Signals connected
  with QueuedConnection. Added _closed flag + _inflight futures list:
  closeEvent cancels pending coroutines and shorts out the result emit
  if the user closes the dialog mid-detect (no use-after-free on
  destroyed QObject).
- BookmarksView._load_thumb_async: same swap. The existing thumb_ready
  signal already used QueuedConnection so the marshaling side was
  already correct.

DB write serialization (core/db.py)
- Database._write_lock = threading.RLock() — RLock not Lock so a
  writing method can call another writing method on the same thread
  without self-deadlocking.
- New _write() context manager composes the lock + sqlite3's connection
  context manager (the latter handles BEGIN / COMMIT / ROLLBACK
  atomically). Every write method converted: add_site, update_site,
  delete_site, add_bookmark, add_bookmarks_batch, remove_bookmark,
  update_bookmark_cache_path, add_folder, remove_folder, rename_folder,
  move_bookmark_to_folder, add/remove_blacklisted_tag,
  add/remove_blacklisted_post, save_library_meta, remove_library_meta,
  set_setting, add_search_history, clear_search_history,
  remove_search_history, add_saved_search, remove_saved_search.
- _migrate keeps using the lock + raw _conn context manager because
  it runs from inside the conn property's lazy init (where _write()
  would re-enter conn).
- Reads stay lock-free and rely on WAL for reader concurrency. Verified
  under contention: 5 threads × 50 add_bookmark calls => 250 rows,
  zero corruption, zero "database is locked" errors.

Smoke-tested with seven scenarios: get_app_loop raises before set,
run_on_app_loop round-trips, lazy init creates exactly one client,
10 concurrent first-callers => 1 httpx, aclose_shared cleans up,
RLock allows nested re-acquire, multi-threaded write contention.
2026-04-07 17:24:23 -05:00

429 lines
17 KiB
Python
Raw Blame History

"""Bookmarks browser widget with folder support."""
from __future__ import annotations
import logging
from pathlib import Path
from PySide6.QtCore import Qt, Signal, QObject, QTimer
from PySide6.QtGui import QPixmap
from PySide6.QtWidgets import (
QWidget,
QVBoxLayout,
QHBoxLayout,
QLineEdit,
QPushButton,
QLabel,
QComboBox,
QMenu,
QApplication,
QInputDialog,
QMessageBox,
)
from ..core.db import Database, Bookmark
from ..core.cache import download_thumbnail
from ..core.concurrency import run_on_app_loop
from .grid import ThumbnailGrid
log = logging.getLogger("booru")
class BookmarkThumbSignals(QObject):
thumb_ready = Signal(int, str)
class BookmarksView(QWidget):
"""Browse and search local bookmarks with folder support."""
bookmark_selected = Signal(object)
bookmark_activated = Signal(object)
bookmarks_changed = Signal() # emitted after bookmark add/remove/unsave
open_in_browser_requested = Signal(int, int) # (site_id, post_id)
def __init__(self, db: Database, parent: QWidget | None = None) -> None:
super().__init__(parent)
self._db = db
self._bookmarks: list[Bookmark] = []
self._signals = BookmarkThumbSignals()
self._signals.thumb_ready.connect(self._on_thumb_ready, Qt.ConnectionType.QueuedConnection)
layout = QVBoxLayout(self)
layout.setContentsMargins(0, 0, 0, 0)
# Top bar: folder selector + search.
# 4px right margin so the rightmost button doesn't sit flush
# against the preview splitter handle.
top = QHBoxLayout()
top.setContentsMargins(0, 0, 4, 0)
# Compact horizontal padding matches the rest of the app's narrow
# toolbar buttons. Vertical padding (2px) and min-height (inherited
# from the global QPushButton rule = 16px) give a total height of
# 22px, lining up with the bundled themes' inputs/combos so the
# whole toolbar row sits at one consistent height — and matches
# what native Qt+Fusion produces with no QSS at all.
_btn_style = "padding: 2px 6px;"
self._folder_combo = QComboBox()
self._folder_combo.setMinimumWidth(120)
self._folder_combo.currentTextChanged.connect(lambda _: self.refresh())
top.addWidget(self._folder_combo)
manage_btn = QPushButton("+ Folder")
manage_btn.setToolTip("New folder")
manage_btn.setFixedWidth(75)
manage_btn.setStyleSheet(_btn_style)
manage_btn.clicked.connect(self._new_folder)
top.addWidget(manage_btn)
self._search_input = QLineEdit()
self._search_input.setPlaceholderText("Search bookmarks by tag")
# Enter still triggers an immediate search.
self._search_input.returnPressed.connect(self._do_search)
# Live search via debounced timer: every keystroke restarts a
# 150ms one-shot, when the user stops typing the search runs.
# Cheap enough since each search is just one SQLite query.
self._search_debounce = QTimer(self)
self._search_debounce.setSingleShot(True)
self._search_debounce.setInterval(150)
self._search_debounce.timeout.connect(self._do_search)
self._search_input.textChanged.connect(
lambda _: self._search_debounce.start()
)
top.addWidget(self._search_input, stretch=1)
layout.addLayout(top)
# Count label
self._count_label = QLabel()
layout.addWidget(self._count_label)
# Grid
self._grid = ThumbnailGrid()
self._grid.post_selected.connect(self._on_selected)
self._grid.post_activated.connect(self._on_activated)
self._grid.context_requested.connect(self._on_context_menu)
self._grid.multi_context_requested.connect(self._on_multi_context_menu)
layout.addWidget(self._grid)
def _refresh_folders(self) -> None:
current = self._folder_combo.currentText()
self._folder_combo.blockSignals(True)
self._folder_combo.clear()
self._folder_combo.addItem("All Bookmarks")
self._folder_combo.addItem("Unfiled")
for folder in self._db.get_folders():
self._folder_combo.addItem(folder)
# Restore selection
idx = self._folder_combo.findText(current)
if idx >= 0:
self._folder_combo.setCurrentIndex(idx)
self._folder_combo.blockSignals(False)
def refresh(self, search: str | None = None) -> None:
self._refresh_folders()
folder_text = self._folder_combo.currentText()
folder_filter = None
if folder_text == "Unfiled":
folder_filter = "" # sentinel for NULL folder
elif folder_text not in ("All Bookmarks", ""):
folder_filter = folder_text
if folder_filter == "":
# Get unfiled: folder IS NULL
self._bookmarks = [
f for f in self._db.get_bookmarks(search=search, limit=500)
if f.folder is None
]
elif folder_filter:
self._bookmarks = self._db.get_bookmarks(search=search, folder=folder_filter, limit=500)
else:
self._bookmarks = self._db.get_bookmarks(search=search, limit=500)
self._count_label.setText(f"{len(self._bookmarks)} bookmarks")
thumbs = self._grid.set_posts(len(self._bookmarks))
from ..core.config import saved_dir, saved_folder_dir, MEDIA_EXTENSIONS
for i, (fav, thumb) in enumerate(zip(self._bookmarks, thumbs)):
thumb.set_bookmarked(True)
# Check if saved to library
saved = False
if fav.folder:
saved = any(
(saved_folder_dir(fav.folder) / f"{fav.post_id}{ext}").exists()
for ext in MEDIA_EXTENSIONS
)
else:
saved = any(
(saved_dir() / f"{fav.post_id}{ext}").exists()
for ext in MEDIA_EXTENSIONS
)
thumb.set_saved_locally(saved)
# Set cached path for drag-and-drop and copy
if fav.cached_path and Path(fav.cached_path).exists():
thumb._cached_path = fav.cached_path
if fav.preview_url:
self._load_thumb_async(i, fav.preview_url)
elif fav.cached_path and Path(fav.cached_path).exists():
pix = QPixmap(fav.cached_path)
if not pix.isNull():
thumb.set_pixmap(pix)
def _load_thumb_async(self, index: int, url: str) -> None:
# Schedule the download on the persistent event loop instead of
# spawning a daemon thread that runs its own throwaway loop. This
# is the fix for the loop-affinity bug where the cache module's
# shared httpx client would get bound to the throwaway loop and
# then fail every subsequent use from the persistent loop.
async def _dl():
try:
path = await download_thumbnail(url)
self._signals.thumb_ready.emit(index, str(path))
except Exception as e:
log.warning(f"Bookmark thumb {index} failed: {e}")
run_on_app_loop(_dl())
def _on_thumb_ready(self, index: int, path: str) -> None:
thumbs = self._grid._thumbs
if 0 <= index < len(thumbs):
pix = QPixmap(path)
if not pix.isNull():
thumbs[index].set_pixmap(pix)
def _do_search(self) -> None:
text = self._search_input.text().strip()
self.refresh(search=text if text else None)
def _on_selected(self, index: int) -> None:
if 0 <= index < len(self._bookmarks):
self.bookmark_selected.emit(self._bookmarks[index])
def _on_activated(self, index: int) -> None:
if 0 <= index < len(self._bookmarks):
self.bookmark_activated.emit(self._bookmarks[index])
def _copy_to_library_unsorted(self, fav: Bookmark) -> None:
"""Copy a bookmarked image to the unsorted library folder."""
from ..core.config import saved_dir
if fav.cached_path and Path(fav.cached_path).exists():
import shutil
src = Path(fav.cached_path)
dest = saved_dir() / f"{fav.post_id}{src.suffix}"
if not dest.exists():
shutil.copy2(src, dest)
def _copy_to_library(self, fav: Bookmark, folder: str) -> None:
"""Copy a bookmarked image to the library folder on disk."""
from ..core.config import saved_folder_dir
if fav.cached_path and Path(fav.cached_path).exists():
import shutil
src = Path(fav.cached_path)
dest = saved_folder_dir(folder) / f"{fav.post_id}{src.suffix}"
if not dest.exists():
shutil.copy2(src, dest)
def _new_folder(self) -> None:
name, ok = QInputDialog.getText(self, "New Folder", "Folder name:")
if ok and name.strip():
try:
self._db.add_folder(name.strip())
except ValueError as e:
QMessageBox.warning(self, "Invalid Folder Name", str(e))
return
self._refresh_folders()
def _on_context_menu(self, index: int, pos) -> None:
if index < 0 or index >= len(self._bookmarks):
return
fav = self._bookmarks[index]
from PySide6.QtGui import QDesktopServices
from PySide6.QtCore import QUrl
from .dialogs import save_file
menu = QMenu(self)
open_browser = menu.addAction("Open in Browser")
open_default = menu.addAction("Open in Default App")
menu.addSeparator()
save_as = menu.addAction("Save As...")
# Save to Library submenu
save_lib_menu = menu.addMenu("Save to Library")
save_lib_unsorted = save_lib_menu.addAction("Unsorted")
save_lib_menu.addSeparator()
save_lib_folders = {}
for folder in self._db.get_folders():
a = save_lib_menu.addAction(folder)
save_lib_folders[id(a)] = folder
save_lib_menu.addSeparator()
save_lib_new = save_lib_menu.addAction("+ New Folder...")
unsave_lib = None
# Only show unsave if the post is saved locally
from ..core.config import saved_dir, saved_folder_dir, MEDIA_EXTENSIONS
_saved = False
_sd = saved_dir()
if _sd.exists():
_saved = any((_sd / f"{fav.post_id}{ext}").exists() for ext in MEDIA_EXTENSIONS)
if not _saved:
for folder in self._db.get_folders():
d = saved_folder_dir(folder)
if d.exists() and any((d / f"{fav.post_id}{ext}").exists() for ext in MEDIA_EXTENSIONS):
_saved = True
break
if _saved:
unsave_lib = menu.addAction("Unsave from Library")
copy_file = menu.addAction("Copy File to Clipboard")
copy_url = menu.addAction("Copy Image URL")
copy_tags = menu.addAction("Copy Tags")
# Move to folder submenu
menu.addSeparator()
move_menu = menu.addMenu("Move to Folder")
move_none = move_menu.addAction("Unfiled")
move_menu.addSeparator()
folder_actions = {}
for folder in self._db.get_folders():
a = move_menu.addAction(folder)
folder_actions[id(a)] = folder
move_menu.addSeparator()
move_new = move_menu.addAction("+ New Folder...")
menu.addSeparator()
remove_bookmark = menu.addAction("Remove Bookmark")
action = menu.exec(pos)
if not action:
return
if action == save_lib_unsorted:
self._copy_to_library_unsorted(fav)
self.refresh()
elif action == save_lib_new:
name, ok = QInputDialog.getText(self, "New Folder", "Folder name:")
if ok and name.strip():
try:
self._db.add_folder(name.strip())
except ValueError as e:
QMessageBox.warning(self, "Invalid Folder Name", str(e))
return
self._copy_to_library(fav, name.strip())
self._db.move_bookmark_to_folder(fav.id, name.strip())
self.refresh()
elif id(action) in save_lib_folders:
folder_name = save_lib_folders[id(action)]
self._copy_to_library(fav, folder_name)
self.refresh()
elif action == open_browser:
self.open_in_browser_requested.emit(fav.site_id, fav.post_id)
elif action == open_default:
if fav.cached_path and Path(fav.cached_path).exists():
QDesktopServices.openUrl(QUrl.fromLocalFile(fav.cached_path))
elif action == save_as:
if fav.cached_path and Path(fav.cached_path).exists():
src = Path(fav.cached_path)
dest = save_file(self, "Save Image", f"post_{fav.post_id}{src.suffix}", f"Images (*{src.suffix})")
if dest:
import shutil
shutil.copy2(src, dest)
elif action == unsave_lib:
from ..core.cache import delete_from_library
if delete_from_library(fav.post_id, fav.folder):
self.refresh()
self.bookmarks_changed.emit()
elif action == copy_file:
path = fav.cached_path
if path and Path(path).exists():
from PySide6.QtCore import QMimeData, QUrl
from PySide6.QtGui import QPixmap
mime = QMimeData()
mime.setUrls([QUrl.fromLocalFile(str(Path(path).resolve()))])
pix = QPixmap(path)
if not pix.isNull():
mime.setImageData(pix.toImage())
QApplication.clipboard().setMimeData(mime)
elif action == copy_url:
QApplication.clipboard().setText(fav.file_url)
elif action == copy_tags:
QApplication.clipboard().setText(fav.tags)
elif action == move_none:
self._db.move_bookmark_to_folder(fav.id, None)
self.refresh()
elif action == move_new:
name, ok = QInputDialog.getText(self, "New Folder", "Folder name:")
if ok and name.strip():
try:
self._db.add_folder(name.strip())
except ValueError as e:
QMessageBox.warning(self, "Invalid Folder Name", str(e))
return
self._db.move_bookmark_to_folder(fav.id, name.strip())
self._copy_to_library(fav, name.strip())
self.refresh()
elif id(action) in folder_actions:
folder_name = folder_actions[id(action)]
self._db.move_bookmark_to_folder(fav.id, folder_name)
self._copy_to_library(fav, folder_name)
self.refresh()
elif action == remove_bookmark:
self._db.remove_bookmark(fav.site_id, fav.post_id)
self.refresh()
self.bookmarks_changed.emit()
def _on_multi_context_menu(self, indices: list, pos) -> None:
favs = [self._bookmarks[i] for i in indices if 0 <= i < len(self._bookmarks)]
if not favs:
return
menu = QMenu(self)
save_all = menu.addAction(f"Save All ({len(favs)}) to Library")
unsave_all = menu.addAction(f"Unsave All ({len(favs)}) from Library")
menu.addSeparator()
move_menu = menu.addMenu(f"Move All ({len(favs)}) to Folder")
move_none = move_menu.addAction("Unfiled")
move_menu.addSeparator()
folder_actions = {}
for folder in self._db.get_folders():
a = move_menu.addAction(folder)
folder_actions[id(a)] = folder
menu.addSeparator()
remove_all = menu.addAction(f"Remove All Bookmarks ({len(favs)})")
action = menu.exec(pos)
if not action:
return
if action == save_all:
for fav in favs:
if fav.folder:
self._copy_to_library(fav, fav.folder)
else:
self._copy_to_library_unsorted(fav)
self.refresh()
elif action == unsave_all:
from ..core.cache import delete_from_library
for fav in favs:
delete_from_library(fav.post_id, fav.folder)
self.refresh()
self.bookmarks_changed.emit()
elif action == move_none:
for fav in favs:
self._db.move_bookmark_to_folder(fav.id, None)
self.refresh()
elif id(action) in folder_actions:
folder_name = folder_actions[id(action)]
for fav in favs:
self._db.move_bookmark_to_folder(fav.id, folder_name)
self._copy_to_library(fav, folder_name)
self.refresh()
elif action == remove_all:
for fav in favs:
self._db.remove_bookmark(fav.site_id, fav.post_id)
self.refresh()
self.bookmarks_changed.emit()
Reference in New Issue View Git Blame Copy Permalink