Build and send a book only when it changed

A key of the chapters, rules and metadata of a story tells if its book
changed. A check without a change no longer builds the book again, also
for a book that waits for an upload. A held book goes out without a new
build. The Rebuild button still builds and sends in each case.
This commit is contained in:
tim
2026-10-02 23:37:33 +02:00
parent f961e4781e
commit 7a36e26751
9 changed files with 367 additions and 55 deletions
+8 -4
View File
@@ -71,7 +71,7 @@ Environment variables set the configuration. **Restart the container after a cha
| `URL_BASE` | A path prefix, such as `/bindery`. See "Reverse proxy". | None | | `URL_BASE` | A path prefix, such as `/bindery`. See "Reverse proxy". | None |
| `INGEST_DIR` | The CWA ingest folder. If unset, all uploads stay paused. | None | | `INGEST_DIR` | The CWA ingest folder. If unset, all uploads stay paused. | None |
| `CONFIG_DIR` | The story database and `personal.ini` | `/config` | | `CONFIG_DIR` | The story database and `personal.ini` | `/config` |
| `TMP_DIR` | The folder for ePubs while Bindery builds them | `/tmp/bindery` | | `TMP_DIR` | The folder for the built ePubs. Bindery keeps the last build of each story. | `/tmp/bindery` |
| `LOG_LEVEL` | The log level | `INFO` | | `LOG_LEVEL` | The log level | `INFO` |
Separate the values in a list with commas. Separate the values in a list with commas.
@@ -318,7 +318,7 @@ for approval, library linking is on, and uploads are not paused. Bindery keeps t
Without either value, it stays off and disabled. Turn it off to skip matching with a configured library connection. Without either value, it stays off and disabled. Turn it off to skip matching with a configured library connection.
Uploads use saved CWA book IDs when available. Without an ID, every build goes to `INGEST_DIR` as a plain ePub. Uploads use saved CWA book IDs when available. Without an ID, every build goes to `INGEST_DIR` as a plain ePub.
If CWA imports these files, updates will create duplicate books. Settings and the story page show a highlighted warning. If CWA imports these files, updates will create duplicate books. Settings and the story page show a highlighted warning.
Changing this setting rebuilds unsent books unless uploads are paused. A change of this setting sends the unsent books, unless uploads are paused.
- **Hold new links for approval.** This setting applies only when library linking is on. - **Hold new links for approval.** This setting applies only when library linking is on.
Use this to catch a story that is linked to the wrong CWA book. Use this to catch a story that is linked to the wrong CWA book.
The first upload of a new story waits for your approval. The first upload after a change of the The first upload of a new story waits for your approval. The first upload after a change of the
@@ -326,7 +326,7 @@ for approval, library linking is on, and uploads are not paused. Bindery keeps t
the linked book is gone. The story page names the book that gets the update, or says that the the linked book is gone. The story page names the book that gets the update, or says that the
story goes to CWA as a new book. story goes to CWA as a new book.
- **Pause all uploads.** No book goes to CWA. This is always checked when `INGEST_DIR` is unset. - **Pause all uploads.** No book goes to CWA. This is always checked when `INGEST_DIR` is unset.
When you turn the pause off, Bindery rebuilds unsent books. The other upload settings still apply. When you turn the pause off, Bindery sends the unsent books. The other upload settings still apply.
While paused, use "Approve" or "Approve as a new book instead" to save a decision without sending the book. While paused, use "Approve" or "Approve as a new book instead" to save a decision without sending the book.
To approve a link, open the story page and do one of these: To approve a link, open the story page and do one of these:
@@ -380,7 +380,11 @@ downloaded before.
| Recheck all chapters | Downloads every chapter on the site again, and new chapters too. | | Recheck all chapters | Downloads every chapter on the site again, and new chapters too. |
| Download and replace entire book | Downloads every chapter again. Chapters that the site no longer lists leave the book. They stay stored, and you can put them back. | | Download and replace entire book | Downloads every chapter again. Chapters that the site no longer lists leave the book. They stay stored, and you can put them back. |
| Update metadata from site | Updates only the metadata in the ePub. | | Update metadata from site | Updates only the metadata in the ePub. |
| Rebuild book | Builds the book again from the stored chapters, for example after you change a rule. | | Rebuild book | Builds the book again from the stored chapters, and sends it. |
**Bindery builds and sends a book only if the book changed.** A change is a new, changed, moved or
renamed chapter, a changed rule, or changed metadata. A job that finds no change answers "No changes"
and sends nothing to CWA. Only "Rebuild book" builds and sends a book that did not change.
### Find edited chapters ### Find edited chapters
+9 -4
View File
@@ -238,16 +238,21 @@ def word_count(chapters: list[dict[str, Any]]) -> int:
return sum(len(block.split()) for c in chapters for block in html_to_blocks(c["html"])) return sum(len(block.split()) for c in chapters for block in html_to_blocks(c["html"]))
def effective_overrides(overrides: dict[str, Any]) -> dict[str, Any]:
"""The kept values that replace a site value. An empty title, author list or language does not."""
return {k: v for k, v in overrides.items() if v or k not in ("title", "authors", "languages")}
def apply_overrides(story, overrides: dict[str, Any]) -> None: def apply_overrides(story, overrides: dict[str, Any]) -> None:
"""Put the kept values of locked fields into FanFicFare's story metadata. """Put the kept values of locked fields into FanFicFare's story metadata.
The epub writer takes the OPF metadata from these entries, and CWA takes The epub writer takes the OPF metadata from these entries, and CWA takes
the book record from the OPF on the first import. the book record from the OPF on the first import.
""" """
for key, value in overrides.items(): for key, value in effective_overrides(overrides).items():
if key == "title" and value: if key == "title":
story.setMetadata("title", value) story.setMetadata("title", value)
elif key == "authors" and value: elif key == "authors":
for n, author in enumerate(value): for n, author in enumerate(value):
story.addToList("author", author, clear=(n == 0)) story.addToList("author", author, clear=(n == 0))
elif key == "comments": elif key == "comments":
@@ -263,7 +268,7 @@ def apply_overrides(story, overrides: dict[str, Any]) -> None:
story.setMetadata("datePublished", datetime.fromisoformat(value) if value else None) story.setMetadata("datePublished", datetime.fromisoformat(value) if value else None)
except ValueError: except ValueError:
logger.warning(f"Ignoring a kept publication date that is not a date: {value!r}") logger.warning(f"Ignoring a kept publication date that is not a date: {value!r}")
elif key == "languages" and value: elif key == "languages":
story.setMetadata("langcode", value) story.setMetadata("langcode", value)
elif key == "tags": elif key == "tags":
tags = list(value or []) tags = list(value or [])
+127 -42
View File
@@ -1,6 +1,7 @@
"""Download modes, metadata locks, delivery to CWA, and the job threads.""" """Download modes, metadata locks, delivery to CWA, and the job threads."""
import hashlib import hashlib
import json
import logging import logging
import re import re
import threading import threading
@@ -94,8 +95,13 @@ def queue_urls(urls: list[str]) -> tuple[list[dict[str, Any]], list[str]]:
def queue_story( def queue_story(
story_id: int, mode: str, chapter_ids: list[int] | None = None, force_fields: list[str] | None = None story_id: int,
mode: str,
chapter_ids: list[int] | None = None,
force_fields: list[str] | None = None,
force: bool = False,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Queue a job for a story. With `force`, a rebuild also builds and sends a book that did not change."""
story = _require_story(story_id) story = _require_story(story_id)
mode_enum = DownloadMode(mode) mode_enum = DownloadMode(mode)
if mode_enum == DownloadMode.IMPORT: if mode_enum == DownloadMode.IMPORT:
@@ -111,6 +117,7 @@ def queue_story(
title=story["title"], title=story["title"],
chapter_ids=sorted(int(c) for c in chapter_ids or []), chapter_ids=sorted(int(c) for c in chapter_ids or []),
force_fields=sorted(force_fields or []), force_fields=sorted(force_fields or []),
options={"force": True} if force and mode_enum == DownloadMode.REBUILD else {},
) )
return job_queue.add(job).to_dict() return job_queue.add(job).to_dict()
@@ -674,6 +681,60 @@ def _build(story: dict[str, Any], overrides: dict[str, Any], out_path: Path) ->
) )
# Built books. The key of a story is a hash of all that goes into its epub, so the book changed when the key
# changed. The name of the built file holds the key. The story settings "built_key" and "sent_key" are the keys of
# the last build and of the book that went to CWA.
_build_lock = threading.Lock()
def _book_metadata(site: dict[str, Any], overrides: dict[str, Any]) -> dict[str, Any]:
"""The metadata of the book: the site values, with the kept values in their place."""
meta = dict(site)
for field, value in fff.effective_overrides(overrides).items():
if field == "series":
meta["series"], meta["series_index"] = value or ("", None)
else:
meta[field] = value
return meta
def _build_key(story: dict[str, Any], overrides: dict[str, Any], site: dict[str, Any]) -> str:
story_id = story["id"]
digest = hashlib.sha256()
state = [
story["url"],
_book_metadata(site, overrides),
store.rules(None),
store.rules(story_id),
_paragraphs(story_id),
store.image_sizes(story_id),
]
digest.update(json.dumps(state, sort_keys=True, default=str).encode())
for c in store.chapters_for_build(story_id):
digest.update(json.dumps([c["url"], c["title"], c["html"]]).encode())
return digest.hexdigest()
def _built_epub(story: dict[str, Any], overrides: dict[str, Any], key: str, force: bool = False) -> Path:
"""The epub of the story for this key. Bindery builds it only if it has no file for the key."""
path = config.TMP_DIR / f"story-{story['id']}-{key[:16]}.epub"
with _build_lock:
if force or not path.exists():
# A build under another name first, so a download never reads half a book.
partial = config.TMP_DIR / f"building-{uuid.uuid4().hex}.epub"
try:
_build(story, overrides, partial)
partial.replace(path)
finally:
partial.unlink(missing_ok=True)
for old in config.TMP_DIR.glob(f"story-{story['id']}-*.epub"):
if old != path:
old.unlink(missing_ok=True)
store.set_setting(story["id"], "built_key", key)
return path
def _find_book(story_url: str, titles: list[str | None]) -> tuple[Book, Path] | None: def _find_book(story_url: str, titles: list[str | None]) -> tuple[Book, Path] | None:
"""Search CWA for the book of a story. Returns the book and its downloaded epub.""" """Search CWA for the book of a story. Returns the book and its downloaded epub."""
linked = {s["calibre_id"] for s in store.list_stories() if s["calibre_id"]} linked = {s["calibre_id"] for s in store.list_stories() if s["calibre_id"]}
@@ -744,6 +805,8 @@ def _uploads_paused() -> bool:
def _link_changed(story_id: int) -> None: def _link_changed(story_id: int) -> None:
store.set_setting(story_id, "unapproved_link", True) store.set_setting(story_id, "unapproved_link", True)
# Another book, or no book, does not have the epub that Bindery sent.
store.set_setting(story_id, "sent_key", None)
def _needs_approval(story_id: int) -> bool: def _needs_approval(story_id: int) -> bool:
@@ -773,10 +836,6 @@ def _hold_reason(story: dict[str, Any], book: dict[str, Any] | None) -> str | No
def _deliver(story: dict[str, Any], epub: Path, book: dict[str, Any] | None, title: str, author: str) -> str: def _deliver(story: dict[str, Any], epub: Path, book: dict[str, Any] | None, title: str, author: str) -> str:
held = _hold_reason(story, book)
if held:
store.set_setting(story["id"], "upload_held", True)
return held
# With linking on, do not create a duplicate when the library book cannot be found. # With linking on, do not create a duplicate when the library book cannot be found.
if _link_books() and not book and story["ingested_at"]: if _link_books() and not book and story["ingested_at"]:
raise StoryError(NOT_FOUND) raise StoryError(NOT_FOUND)
@@ -794,6 +853,44 @@ def _deliver(story: dict[str, Any], epub: Path, book: dict[str, Any] | None, tit
return message return message
def _send(
story: dict[str, Any], overrides: dict[str, Any], key: str, title: str, author: str, force: bool = False
) -> tuple[str, bool]:
"""Build the book if it changed, and deliver it. Returns the message, and whether the book changed.
A held book that did not change is not built again. With `force`, Bindery builds the book in each case.
"""
story_id = story["id"]
book = _linked_book(story)
held = _hold_reason(story, book)
changed = key != store.get_setting(story_id, "built_key")
if held:
if changed or force:
_built_epub(story, overrides, key, force)
store.set_setting(story_id, "upload_held", True)
return held, changed
message = _deliver(story, _built_epub(story, overrides, key, force), book, title, author)
store.set_setting(story_id, "sent_key", key)
return message, changed
def adopt_build_keys() -> None:
"""Give the stories of an older Bindery their keys, so the first check does not send each book again."""
built = store.story_settings("built_key")
held = store.story_settings("upload_held")
for listed in store.list_stories():
story_id = listed["id"]
if story_id in built or listed["status"] == "error" or job_queue.has_active(story_id):
continue
story = _require_story(story_id)
if not story["metahtml"]:
continue
key = _build_key(story, _epub_overrides(story, story["pinned"]), story["site_values"])
store.set_setting(story_id, "built_key", key)
if (story["calibre_id"] or story["ingested_at"]) and not held.get(story_id):
store.set_setting(story_id, "sent_key", key)
# Jobs # Jobs
@@ -920,12 +1017,15 @@ def _run_import_file(job: Job) -> str:
def _run_rebuild(job: Job) -> str: def _run_rebuild(job: Job) -> str:
story = _require_story(job.story_id) story = _require_story(job.story_id)
job_queue.update(job, title=story["title"]) job_queue.update(job, title=story["title"])
book = _linked_book(story)
pinned = _resolve_metadata(story, story["site_values"], []) pinned = _resolve_metadata(story, story["site_values"], [])
epub = config.TMP_DIR / f"story-{story['id']}.epub" overrides = _epub_overrides(story, pinned)
_build(story, _epub_overrides(story, pinned), epub) key = _build_key(story, overrides, story["site_values"])
message = _deliver(story, epub, book, story["title"], story["author"]) force = bool(job.options.get("force"))
store.update_story(story["id"], pinned=pinned, status="ok", message=message, last_changed=now()) if not force and key == store.get_setting(story["id"], "sent_key"):
return "No changes"
message, changed = _send(story, overrides, key, story["title"], story["author"], force)
changes: dict[str, Any] = {"last_changed": now()} if changed or force else {}
store.update_story(story["id"], pinned=pinned, status="ok", message=message, **changes)
return message return message
@@ -1149,46 +1249,27 @@ def _run_site_job(job: Job) -> str:
job, result={"new": new_count, "changed": changed, "failed": failed, "skipped": skipped, "removed": removed} job, result={"new": new_count, "changed": changed, "failed": failed, "skipped": skipped, "removed": removed}
) )
metadata_changed = snap.calibre != story["site_values"]
delivered = bool(story["calibre_id"] or story["ingested_at"])
# A resumed job builds the book, because the run before the restart can have stored chapters.
needs_build = (
new_count
or changed
or renamed_count
or job.force_fields
or metadata_changed
or not delivered
or job.options.get("build")
or job.restarts
or mode in (DownloadMode.FULL, DownloadMode.METADATA)
or (mode == DownloadMode.NEW and story["status"] == "error" and not failed)
)
if mode in (DownloadMode.RECHECK_ALL, DownloadMode.FULL): if mode in (DownloadMode.RECHECK_ALL, DownloadMode.FULL):
store.set_setting(story_id, "last_full_recheck", now()) store.set_setting(story_id, "last_full_recheck", now())
if not store.chapters_for_build(story_id): if not store.chapters_for_build(story_id):
raise StoryError("; ".join(summary) + ". No chapters downloaded.") raise StoryError("; ".join(summary) + ". No chapters downloaded.")
if not needs_build:
store.update_story(story_id, last_checked=now(), status="ok", message="No changes")
return "No changes. " + ", ".join(summary)
book = _linked_book(story) # The book is built and sent only when its chapters, rules or metadata changed. A job that resumes, or that
# follows a failed job, finds the changes of the earlier run through the key.
pinned = _resolve_metadata(story, snap.calibre, job.force_fields) pinned = _resolve_metadata(story, snap.calibre, job.force_fields)
title = _display_title(story, pinned, snap.title) title = _display_title(story, pinned, snap.title)
overrides = _epub_overrides(story, pinned)
key = _build_key(story, overrides, snap.calibre)
checked: dict[str, Any] = {"pinned": pinned, "site_values": snap.calibre, "title": title, "last_checked": now()}
if key == store.get_setting(story_id, "sent_key"):
store.update_story(story_id, **checked, status="ok", message="No changes")
return "No changes. " + ", ".join(summary)
_progress(job, "Building epub", 1.0) _progress(job, "Building epub", 1.0)
epub = config.TMP_DIR / f"story-{story_id}.epub" message, book_changed = _send(story, overrides, key, title, snap.author)
_build(story, _epub_overrides(story, pinned), epub) if book_changed:
message = _deliver(story, epub, book, title, snap.author) checked["last_changed"] = now()
store.update_story( store.update_story(story_id, **checked, message=message, status="error" if failed else "ok")
story_id,
pinned=pinned,
site_values=snap.calibre,
title=title,
last_checked=now(),
last_changed=now(),
message=message,
status="error" if failed else "ok",
)
return ", ".join(summary) + ". " + message return ", ".join(summary) + ". " + message
@@ -1267,5 +1348,9 @@ def scheduler_loop() -> None:
def start_threads() -> None: def start_threads() -> None:
try:
adopt_build_keys()
except Exception as e:
logger.exception(f"Cannot check the built books: {e}")
for target in (download_loop, scheduler_loop): for target in (download_loop, scheduler_loop):
threading.Thread(target=target, daemon=True, name=target.__name__).start() threading.Thread(target=target, daemon=True, name=target.__name__).start()
+8
View File
@@ -388,6 +388,14 @@ class Store:
rows = conn.execute("SELECT url FROM images WHERE story_id = ? AND is_cover = 0", (story_id,)) rows = conn.execute("SELECT url FROM images WHERE story_id = ? AND is_cover = 0", (story_id,))
return {r["url"] for r in rows} return {r["url"] for r in rows}
def image_sizes(self, story_id: int) -> list[list[Any]]:
"""The URL, the cover mark and the size of each image, without its data."""
with self._connect() as conn:
rows = conn.execute(
"SELECT url, is_cover, LENGTH(data) FROM images WHERE story_id = ? ORDER BY url, is_cover", (story_id,)
)
return [list(r) for r in rows]
def images(self, story_id: int) -> list[dict[str, Any]]: def images(self, story_id: int) -> list[dict[str, Any]]:
with self._connect() as conn: with self._connect() as conn:
rows = conn.execute( rows = conn.execute(
+2 -1
View File
@@ -291,8 +291,9 @@ def delete_story(story_id: int) -> Any:
@app.post("/api/stories/<int:story_id>/download") @app.post("/api/stories/<int:story_id>/download")
def download(story_id: int) -> Any: def download(story_id: int) -> Any:
data = _json() data = _json()
# A rebuild that the user asks for also builds and sends a book that did not change.
return orchestrator.queue_story( return orchestrator.queue_story(
story_id, data.get("mode", DownloadMode.NEW.value), data.get("chapter_ids"), data.get("force_fields") story_id, data.get("mode", DownloadMode.NEW.value), data.get("chapter_ids"), data.get("force_fields"), True
) )
+205
View File
@@ -0,0 +1,205 @@
"""A book is built and sent only when its chapters, rules or metadata changed."""
import dataclasses
from unittest.mock import Mock
import pytest
from test_api import H
from test_api import client as client
from test_deliver import STORY_URL, add_story, ingest_files, linked_story, site, update
from test_deliver import fake as fake
from test_deliver import jobs as jobs
from test_deliver import story_epub as story_epub
from bindery import config, orchestrator
from bindery.jobs import DownloadMode, Job
ONE = [(1, "One", "a")]
@pytest.fixture(autouse=True)
def builds(fake, jobs, monkeypatch, tmp_path):
"""Count the builds, in a folder of built books that starts empty."""
monkeypatch.setattr(config, "TMP_DIR", tmp_path / "tmp")
config.TMP_DIR.mkdir()
spy = Mock(side_effect=orchestrator._build)
monkeypatch.setattr(orchestrator, "_build", spy)
return spy
def story(story_id):
return orchestrator.store.get_story(story_id)
def test_an_unchanged_story_is_not_built_or_sent_again(builds, monkeypatch):
story_id = linked_story(ONE)
site(monkeypatch, ONE)
assert "library book 1" in update(story_id)
changed = story(story_id)["last_changed"]
orchestrator.store.update_story(story_id, last_changed="2020-01-01T00:00:00")
for mode in (DownloadMode.NEW, DownloadMode.RECHECK_ALL, DownloadMode.METADATA, DownloadMode.FULL):
assert update(story_id, mode).startswith("No changes")
assert changed and story(story_id)["last_changed"] == "2020-01-01T00:00:00"
assert builds.call_count == 1
assert len(ingest_files()) == 2 # one epub and its sidecar
@pytest.mark.parametrize("change", ["chapter", "rule", "global rule", "paragraphs", "metadata", "included"])
def test_each_change_builds_and_sends_the_book(builds, monkeypatch, change):
story_id = linked_story([(1, "One", "a"), (2, "Two", "b")])
site(monkeypatch, [(1, "One", "a"), (2, "Two", "b")])
update(story_id)
if change == "chapter":
site(monkeypatch, [(1, "One", "a"), (2, "Two", "b"), (3, "Three", "c")])
elif change == "rule":
orchestrator.save_rules(story_id, [{"pattern": "a", "replacement": "b"}])
elif change == "global rule":
orchestrator.save_rules(None, [{"pattern": "a", "replacement": "b"}])
elif change == "paragraphs":
orchestrator.save_rules(story_id, [], paragraphs=True)
elif change == "metadata":
orchestrator.store.update_story(story_id, pinned={"title": "Edited"})
else:
chapter = orchestrator.store.list_chapters(story_id)[0]
orchestrator.store.update_chapter(chapter["id"], included=0)
assert "library book 1" in update(story_id)
assert builds.call_count == 2
assert len(ingest_files()) == 4
assert update(story_id).startswith("No changes")
def test_a_held_book_that_did_not_change_is_not_built_again(builds, monkeypatch):
monkeypatch.setattr(config, "INGEST_DIR", None)
site(monkeypatch, ONE)
job = Job(mode=DownloadMode.NEW, url=STORY_URL)
assert "Not sent. INGEST_DIR is not set." in orchestrator._run_site_job(job)
changed = story(job.story_id)["last_changed"]
orchestrator.store.update_story(job.story_id, last_changed="2020-01-01T00:00:00")
assert "Not sent. INGEST_DIR is not set." in update(job.story_id)
assert changed and story(job.story_id)["last_changed"] == "2020-01-01T00:00:00"
assert builds.call_count == 1
# The book changes. Bindery builds it, also while it cannot send it.
site(monkeypatch, [*ONE, (2, "Two", "b")])
update(job.story_id)
assert builds.call_count == 2
assert story(job.story_id)["last_changed"] != "2020-01-01T00:00:00"
def test_a_released_book_goes_out_without_another_build(builds, jobs):
story_id = add_story(ingested_at="2026-01-01T00:00:00", calibre_id=1, calibre_uuid="uuid-1")
orchestrator.save_settings(pause_uploads=True)
orchestrator._run_rebuild(Job(mode=DownloadMode.REBUILD, story_id=story_id))
assert builds.call_count == 1 and ingest_files() == []
orchestrator.save_settings(pause_uploads=False)
assert "library book 1" in orchestrator._run_rebuild(jobs.get_next())
assert builds.call_count == 1
assert len(ingest_files()) == 2
def test_a_lost_file_is_built_again_only_to_send_it(builds):
story_id = add_story(ingested_at="2026-01-01T00:00:00", calibre_id=1, calibre_uuid="uuid-1")
orchestrator.save_settings(pause_uploads=True)
job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
orchestrator._run_rebuild(job)
for path in config.TMP_DIR.iterdir():
path.unlink()
assert "pause" in orchestrator._run_rebuild(job)
assert builds.call_count == 1
orchestrator.store.set_setting(None, "pause_uploads", None)
assert "library book 1" in orchestrator._run_rebuild(job)
assert builds.call_count == 2
def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(builds, client, jobs):
story_id = add_story(ingested_at="2026-01-01T00:00:00", calibre_id=1, calibre_uuid="uuid-1")
job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
assert "library book 1" in orchestrator._run_rebuild(job)
orchestrator.store.update_story(story_id, last_changed=None, message="before")
assert orchestrator._run_rebuild(job) == "No changes"
assert (story(story_id)["last_changed"], story(story_id)["message"]) == (None, "before")
assert builds.call_count == 1
# The Rebuild button of the story page.
queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "rebuild"}, headers=H).json
assert queued["options"] == {"force": True}
assert "library book 1" in orchestrator._run_rebuild(jobs.get_next())
assert builds.call_count == 2
assert len(ingest_files()) == 4
assert story(story_id)["last_changed"]
# Other modes take no such option.
queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "new"}, headers=H).json
assert queued["options"] == {}
def test_a_changed_link_gets_the_book_without_a_build(builds, fake):
story_id = add_story(ingested_at="2026-01-01T00:00:00", calibre_id=1, calibre_uuid="uuid-1")
job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
orchestrator._run_rebuild(job)
fake.books.append(dataclasses.replace(fake.books[0], id=2, uuid="uuid-2", title="Other"))
orchestrator.link_story(story_id, "uuid-2")
assert "library book 2" in orchestrator._run_rebuild(job)
assert builds.call_count == 1
def test_a_failed_delivery_is_tried_again_without_a_build(builds, fake, monkeypatch):
story_id = linked_story(ONE)
site(monkeypatch, ONE)
send = orchestrator.IngestFolder.send_update
monkeypatch.setattr(orchestrator.IngestFolder, "send_update", Mock(side_effect=OSError("disk full")))
with pytest.raises(OSError, match="disk full"):
update(story_id)
monkeypatch.setattr(orchestrator.IngestFolder, "send_update", send)
assert "library book 1" in update(story_id)
assert builds.call_count == 1
def test_a_new_build_removes_the_older_file(builds, monkeypatch):
story_id = linked_story(ONE)
site(monkeypatch, ONE)
update(story_id)
(first,) = config.TMP_DIR.iterdir()
site(monkeypatch, [*ONE, (2, "Two", "b")])
update(story_id)
(second,) = config.TMP_DIR.iterdir()
assert first != second and second.name.startswith(f"story-{story_id}-")
@pytest.mark.parametrize(
("changes", "held", "built", "sent"),
[
({"calibre_id": 1, "calibre_uuid": "uuid-1"}, False, True, True),
({"ingested_at": "2026-01-01"}, False, True, True),
({"ingested_at": "2026-01-01"}, True, True, False),
({}, False, True, False),
({"ingested_at": "2026-01-01", "status": "error"}, False, False, False),
({"ingested_at": "2026-01-01", "no_metadata": True}, False, False, False),
],
)
def test_stories_of_an_older_bindery_get_their_keys(builds, changes, held, built, sent):
no_metadata = changes.pop("no_metadata", False)
story_id = add_story(**changes)
if no_metadata:
orchestrator.store.update_story(story_id, metahtml=None)
orchestrator.store.set_setting(story_id, "upload_held", held or None)
orchestrator.adopt_build_keys()
settings = orchestrator.store
assert bool(settings.get_setting(story_id, "built_key")) is built
assert bool(settings.get_setting(story_id, "sent_key")) is sent
if sent:
assert orchestrator._run_rebuild(Job(mode=DownloadMode.REBUILD, story_id=story_id)) == "No changes"
assert builds.call_count == 0
def test_a_story_with_a_waiting_job_gets_no_key(builds, jobs):
story_id = add_story(ingested_at="2026-01-01")
jobs.add(Job(mode=DownloadMode.NEW, story_id=story_id))
orchestrator.adopt_build_keys()
assert orchestrator.store.get_setting(story_id, "built_key") is None
def test_the_first_check_after_an_upgrade_sends_nothing(builds, monkeypatch):
story_id = linked_story(ONE)
orchestrator.store.update_story(story_id, pinned={"title": "My Title"}, site_values={})
orchestrator.adopt_build_keys()
site(monkeypatch, ONE)
assert update(story_id).startswith("No changes")
assert builds.call_count == 0 and ingest_files() == []
+2 -1
View File
@@ -60,7 +60,8 @@ def opf(name):
def rebuild(story_id): def rebuild(story_id):
return orchestrator._run_rebuild(Job(mode=DownloadMode.REBUILD, story_id=story_id)) """A rebuild that the user asks for. It builds and sends the book also when the book did not change."""
return orchestrator._run_rebuild(Job(mode=DownloadMode.REBUILD, story_id=story_id, options={"force": True}))
def test_first_version_goes_in_as_a_new_book(fake): def test_first_version_goes_in_as_a_new_book(fake):
+2 -2
View File
@@ -150,7 +150,7 @@ def test_build_without_ingest_succeeds_and_keeps_the_book_local(monkeypatch, tmp
story = orchestrator.store.get_story(story_id) story = orchestrator.store.get_story(story_id)
assert story["status"] == "ok" assert story["status"] == "ok"
assert story["ingested_at"] == ("2026-01-01" if existing else None) assert story["ingested_at"] == ("2026-01-01" if existing else None)
assert (config.TMP_DIR / f"story-{story_id}.epub").exists() assert list(config.TMP_DIR.glob(f"story-{story_id}-*.epub"))
assert not (tmp_path / "ingest").exists() assert not (tmp_path / "ingest").exists()
details = orchestrator.story_details(story_id) details = orchestrator.story_details(story_id)
assert details["uploads_paused"] is True assert details["uploads_paused"] is True
@@ -260,7 +260,7 @@ def test_new_story_download_skips_library_matching(monkeypatch, standalone):
assert ("Not sent." if standalone else "Sent to ingest folder") in message assert ("Not sent." if standalone else "Sent to ingest folder") in message
assert len(orchestrator.store.list_chapters(job.story_id)) == 1 assert len(orchestrator.store.list_chapters(job.story_id)) == 1
assert orchestrator.store.get_story(job.story_id)["status"] == "ok" assert orchestrator.store.get_story(job.story_id)["status"] == "ok"
assert (config.TMP_DIR / f"story-{job.story_id}.epub").exists() assert list(config.TMP_DIR.glob(f"story-{job.story_id}-*.epub"))
orchestrator.cwa.assert_not_called() orchestrator.cwa.assert_not_called()
orchestrator._find_book.assert_not_called() orchestrator._find_book.assert_not_called()
+4 -1
View File
@@ -25,6 +25,8 @@ def state(monkeypatch, tmp_path):
monkeypatch.setattr(o, "job_queue", queue) monkeypatch.setattr(o, "job_queue", queue)
monkeypatch.setattr(config, "CWA_URL", "") monkeypatch.setattr(config, "CWA_URL", "")
monkeypatch.setattr(config, "SITE_DELAY", 7) monkeypatch.setattr(config, "SITE_DELAY", 7)
monkeypatch.setattr(config, "TMP_DIR", tmp_path / "tmp")
config.TMP_DIR.mkdir()
return store, queue return store, queue
@@ -151,7 +153,8 @@ def site(state, monkeypatch):
calibre={"title": "Title"}, calibre={"title": "Title"},
) )
fetch = Mock(side_effect=lambda a, url, i: f"<p>fresh {url}</p>") fetch = Mock(side_effect=lambda a, url, i: f"<p>fresh {url}</p>")
build, deliver = Mock(), Mock(return_value="Sent") build = Mock(side_effect=lambda story, overrides, path: path.write_bytes(b"epub"))
deliver = Mock(return_value="Sent")
monkeypatch.setattr(fff, "get_adapter", lambda url: adapter) monkeypatch.setattr(fff, "get_adapter", lambda url: adapter)
monkeypatch.setattr(fff, "fetch_story", lambda a: snap) monkeypatch.setattr(fff, "fetch_story", lambda a: snap)
monkeypatch.setattr(fff, "fetch_chapter", fetch) monkeypatch.setattr(fff, "fetch_chapter", fetch)