Put the core flow first in the README

The README starts with the flow from a story URL to the OPDS catalog. One section near the end holds everything about CWA.

The example Compose file asks for no login and has the CWA values as comments.

AGENTS.md

@@ -28,6 +28,7 @@ The server has no reloader. If you change Python code, restart it. Flask reads `
 - `bindery/fff.py` is the only module that calls FanFicFare.
 - `bindery/cwa.py` talks to CWA through OPDS. Bindery never opens the CWA database.
 - `bindery/diffing.py` hashes and diffs text, not HTML.
+- `bindery/filename_template.py` expands Calibre's file name templates. It never runs template text as code.
 - `bindery/jobs.py` keeps the job queue in SQLite, so jobs survive a restart.
 - `tests/fake_cwa.py` is a fake CWA server.
 

README.md

@@ -1,10 +1,13 @@
 # Bindery
 
-Bindery downloads web fiction into a
+Bindery downloads web fiction and keeps each story up to date as an ePub. You give it the URL of a
-[Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library and
+story. Bindery downloads the chapters, checks for new chapters each day, and builds the book again
-keeps the books up to date. It uses [FanFicFare](https://github.com/JimmXinu/FanFicFare), so it
+after each change. Your reader app gets the books from the OPDS catalog of Bindery.
-supports the sites that FanFicFare supports. The tested site is RoyalRoad.
+
-CWA is optional. You can download ePubs from Bindery without it.
+Bindery uses [FanFicFare](https://github.com/JimmXinu/FanFicFare), so it supports the sites that
+FanFicFare supports. The tested site is RoyalRoad.
+
+Bindery can also send each book to a [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library. See "Calibre-Web-Automated".
 
 ## How it differs from FanFicFare
 
@@ -12,8 +15,8 @@ FanFicFare builds a book from the chapter list on the site, so a chapter that th
 also goes from your book. **Bindery keeps every chapter that it downloads** in its own database, and
 builds each ePub from that database.
 
-- **Chapters that the site removes stay in the book.** A new chapter goes after the chapter that
+- **Chapters that the publisher removes stay in the book.** A new chapter goes after the chapter that
-  comes before it on the site. For example, you have chapters 1 to 60, and the site now lists only 1
+  comes before it on the publishing site. For example, you have chapters 1 to 60, and the publishing site now lists only 1
   to 4 and 61. The book then has chapters 1 to 61.
 - **A changed chapter keeps its old text as a revision.** You can compare the revisions and choose
   the one that goes in the book.
@@ -22,9 +25,6 @@ builds each ePub from that database.
 
 ## Quick start
 
-For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update as a new book. See
-"Limitations".
-
 1. Get the code, and make the folder for Bindery's data.
 
    ```bash
@@ -33,8 +33,8 @@ For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update
    mkdir config
    ```
 
-2. Edit `docker-compose.yml` or environment variables. Set the CWA ingest folder, the CWA address and account, and the user
+2. Edit `docker-compose.yml`. Set the user and group of the container. **That user must be able to
-   and group of the container.
+   write to `./config`.**
 
 3. Build and start the container.
 
@@ -42,10 +42,54 @@ For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update
    docker compose up -d --build
    ```
 
-4. Open `http://localhost:8085`, and log in with a CWA user.
+4. Open `http://localhost:8085`. Under "Add stories", enter the URL of a story, and click "Add".
+   Bindery downloads the chapters and builds the book. To get the book at once, click "Download ePub"
+   on the story page.
 
-The container runs as the `user` in `docker-compose.yml`. **That user must be able to write to
+5. Open Settings. Under "Book Downloads", check "Enable OPDS", and save. In your reader app, add the
-`./config` and to the CWA ingest folder.** Usually this is the same user and group as CWA.
+   catalog `http://localhost:8085/opds`, with your server address in place of `localhost`.
+
+Bindery then checks each story for new chapters every 24 hours. When a story changes, Bindery builds
+its book again, and the catalog has the new book.
+
+The example Compose file asks for no login. Set a login before other people can reach Bindery. See
+"Logins".
+
+## OPDS catalog
+
+OPDS lets a reader app browse and download books from Bindery. An admin turns it on with "Enable OPDS"
+under Settings > Book Downloads. The catalog URL is `http://localhost:8085/opds`, with your server
+address in place of `localhost`. If Bindery runs under `URL_BASE`, put it before `/opds`.
+
+The catalog has three lists, with 50 books per page:
+
+- By Title, in title order without a leading "The", "A" or "An".
+- By Author, in surname order. Each author leads to their books. A book shows under each of its authors.
+- By Last Changed, with the newest change first.
+
+Last Changed records when Bindery stores a new chapter or a new revision of an existing chapter.
+Rebuilds, metadata edits and rule changes do not update this date.
+
+A book appears when it has a stored, included chapter. It shows its authors, series, description, tags,
+language and cover, with your kept values in place of the published values. A reader can search by title,
+author, series and tag. A download uses the stored chapters, the current ePub rules and the download
+template for the file name.
+
+`AUTH_METHOD` sets how a reader logs in:
+
+- `none`: The reader needs no login.
+- `local`: The reader sends your Bindery user name and password with HTTP Basic authentication.
+  A browser with a Bindery session works too.
+- `cwa`: The reader sends your CWA user name and password with HTTP Basic authentication.
+  A browser with a Bindery session works too.
+- `proxy`: Each user sets an OPDS password under Settings > Book Downloads. Settings shows the user
+  name for the reader, which is the name in `PROXY_AUTH_USER_HEADER`. The reader sends that name and
+  password with HTTP Basic authentication. The password has 8 to 256 characters, and Bindery keeps a
+  hash of it. Set the proxy to skip its login for the catalog. See "Paths without the proxy login".
+
+Bindery keeps a good login for five minutes. Wrong passwords count toward the lockouts in
+"Lockouts and sessions". With `TRUST_PROXY=true`, each reader has its own lockout.
+Use HTTPS outside a trusted local network, because HTTP Basic sends the password with each request.
 
 ## Configuration
 
@@ -53,17 +97,11 @@ Environment variables set the configuration. **Restart the container after a cha
 
 | Variable | Description | Default value |
 | --- | --- | --- |
-| `CWA_ENABLED` | `false` if you have no CWA. See "Without CWA". | `true` |
-| `CWA_URL` | The address of CWA, for example `http://calibre-web-automated:8083` | None |
-| `CWA_USERNAME`, `CWA_PASSWORD` | The CWA account of Bindery. Leave them unset if CWA asks for no login. | None |
-| `CWA_DB_PATH` | Calibre-Web's `app.db`, for logins. See "CWA logins". | None |
-| `CALIBRE_DB_PATH` | The calibre `metadata.db`, for book ids. See "How books reach the library". | None |
 | `UPDATE_INTERVAL_HOURS` | Hours between checks for new chapters. `0` turns them off. | `24` |
 | `DEFAULT_LOCKED_FIELDS` | Fields that new stories lock: `title`, `authors`, `comments`, `tags`, `series`, `pubdate`, `languages` | `title` |
 | `AUTH_METHOD` | `none`, `local`, `cwa` or `proxy`. See "Logins". | `none` |
 | `BINDERY_USERNAME` | The user name for `AUTH_METHOD=local`. Case-sensitive. | None |
 | `BINDERY_PASSWORD_HASH` | The password hash for `AUTH_METHOD=local`. See "Local login". | None |
-| `CWA_ALLOWED_USERS` | The CWA users that can log in. Empty allows every CWA user. | Empty |
 | `BINDERY_ADMIN_USERS` | The users that can change the global settings. See "Admins". | Empty |
 | `PROXY_AUTH_USER_HEADER` | The header in which a reverse proxy sends the user name | `X-Remote-User` |
 | `PROXY_AUTH_TRUSTED_IPS` | The addresses and networks that can send the user header. See "Logins". | Loopback and private addresses |
@@ -73,45 +111,12 @@ Environment variables set the configuration. **Restart the container after a cha
 | `SESSION_COOKIE_SECURE` | `true` if users reach Bindery only over HTTPS | `false` |
 | `TRUST_PROXY` | `true` if a reverse proxy sends the client address. See "Lockouts and sessions". | `false` |
 | `URL_BASE` | A path prefix, such as `/bindery`. See "Reverse proxy". | None |
-| `INGEST_DIR` | The CWA ingest folder. If unset, all uploads stay paused. Bindery does not make this folder. Without CWA, a folder for the built ePubs. | None |
+| `INGEST_DIR` | A folder that gets each built ePub. With CWA, the CWA ingest folder. Bindery does not make this folder. | None |
 | `CONFIG_DIR` | The story database and `personal.ini` | `/config` |
 | `TMP_DIR` | The folder for the built ePubs. Bindery keeps the last build of each story. | `/tmp/bindery` |
 | `LOG_LEVEL` | The log level | `INFO` |
 
-Separate the values in a list with commas.
+Separate the values in a list with commas. The variables for CWA are in "Calibre-Web-Automated".
-
-### Without CWA
-
-Set `CWA_ENABLED=false` to run Bindery without CWA. Bindery still downloads chapters and builds
-ePubs. Download the books from each story page.
-
-With `CWA_ENABLED=false`:
-
-- Bindery makes no request to CWA. It ignores `CWA_URL`, `CWA_USERNAME`, `CWA_PASSWORD`,
-  `CWA_DB_PATH` and `CALIBRE_DB_PATH`.
-- The web interface and the log show nothing about CWA. The pause, the library links and the
-  approvals do not apply.
-- If `INGEST_DIR` is set, Bindery puts each changed book in that folder as a plain ePub. Use this to
-  import the books into other software. Each file gets a new name, so Bindery never replaces a file
-  in the folder.
-- `AUTH_METHOD=cwa` does not work. Use `local` or `proxy`. On a private network, you can use `none`.
-
-If you use the example Compose file, remove its CWA connection values. Keep `INGEST_DIR` and its
-volume mount only if you want the ePubs in a folder.
-
-Without `CWA_ENABLED=false`, Bindery expects CWA. At each start, the log then has a warning for each
-function that a missing `INGEST_DIR`, `CWA_URL` or `CALIBRE_DB_PATH` turns off. Bindery still
-downloads chapters and builds ePubs.
-
-Without `INGEST_DIR`, Settings shows "Pause all uploads" checked and a highlighted explanation.
-You cannot clear that checkbox until you set `INGEST_DIR` and restart Bindery.
-If neither `CWA_URL` nor `CALIBRE_DB_PATH` is set, library linking is off and its checkbox is disabled.
-Set either value and restart Bindery to enable the checkbox. Bindery keeps your saved linking preference.
-Saved CWA book IDs still work for uploads if you configure an ingest folder.
-
-To export ePubs for other software, set `INGEST_DIR` and turn off "Link books to CWA library" in Settings.
-Each build without a saved CWA book ID then puts a plain ePub in that folder, including updates.
-"Pause all uploads" still stops these exports. If CWA imports the files, updates will create duplicate books.
 
 ### Logins
 
@@ -135,6 +140,17 @@ Each build without a saved CWA book ID then puts a plain ePub in that folder, in
   or publish the port of Bindery only on the host, as `127.0.0.1:8085:8085` in the Compose file.
   `TRUST_PROXY` does not change this check. It is only for the lockouts.
 
+  #### Paths without the proxy login
+
+  An OPDS reader cannot use the login page of the proxy, such as OIDC. Set the proxy to skip its login
+  for `/opds` and each path below it. If you set `URL_BASE`, put it before the path. As a regular
+  expression, this is `^/opds([/?].*)?$`. In Authelia, add an `access_control` rule with
+  `policy: bypass` and this expression in `resources`. In Authentik, add it to "Unauthenticated
+  Paths" of the proxy provider.
+
+  On these paths, Bindery checks the OPDS login itself. See "OPDS catalog". Keep the proxy login on
+  `/api` and on all other paths. The health check at `/api/health` needs no login.
+
 **If the login settings are incomplete or wrong, Bindery refuses every request**, also with the API
 key, and shows the reason.
 
@@ -179,32 +195,6 @@ key, and shows the reason.
 To change the account, change the user name or make a new hash, and restart Bindery. This ends the
 open sessions.
 
-#### CWA logins
-
-`AUTH_METHOD=cwa` needs `CWA_URL`, `CWA_DB_PATH` or both. Without `CWA_DB_PATH`, Bindery sends the
-user name and password to `CWA_URL/opds` as an HTTP Basic login. This is the OPDS login.
-
-**The OPDS login works only if CWA refuses a wrong password.** A CWA with anonymous browsing can
-accept every OPDS login. Bindery tests this with a login that cannot exist. If CWA accepts it,
-Bindery refuses each OPDS login, and the log names the reason. Set `CWA_DB_PATH` then, or use
-another `AUTH_METHOD`.
-
-With `CWA_DB_PATH`, Bindery checks the password against the account in Calibre-Web's `app.db`. A
-wrong password or an unknown account fails at once. If Bindery cannot read the file or its password
-hash, it uses the OPDS login.
-
-`CWA_DB_PATH` is the path of `app.db` inside the container. To use it, mount the CWA config folder
-read-only into the container. Mount the whole folder, so that SQLite can read the journal files
-beside the database. **Bindery does not write this database.**
-
-**Database logins check only the password.** CWA's external login providers and multi-factor
-authentication do not apply. If they must apply, use the `proxy` mode.
-
-- The CWA guest user cannot log in.
-- If `CWA_ALLOWED_USERS` is set, only those users can log in. Bindery checks this list on each
-  request.
-- `CWA_USERNAME` and `CWA_PASSWORD` are only for the library lookups of Bindery.
-
 #### Lockouts and sessions
 
 The `local` and `cwa` modes lock out password guesses:
@@ -234,7 +224,7 @@ In the `cwa` and `proxy` modes, `BINDERY_ADMIN_USERS` lists the users that are a
 
 - Open the Settings page, and change the settings on it.
 - Change the global replacement rules.
-- Read and change `personal.ini`, which can hold passwords for story sites.
+- Read and change `personal.ini`, which can hold passwords for publishing sites.
 
 Each other user can still add, update and delete stories, and change the settings and the rules of a
 story. Bindery ignores the case of the names.
@@ -308,13 +298,13 @@ explains the format, and
 setting. Bindery reads the file for each job, so a change applies to the next job.
 
 You can edit the file on the Settings page. Bindery refuses a file that FanFicFare cannot read, and
-shows FanFicFare's warnings for other problems. **The file can hold passwords for story sites, and
+shows FanFicFare's warnings for other problems. **The file can hold passwords for publishing sites, and
 each admin can read them.** If `BINDERY_ADMIN_USERS` is not set, each user is an admin. See "Admins".
 
 Bindery always sets `always_reload_first_chapter`, `mark_new_chapters`, `continue_on_chapter_error`,
 `make_directories`, `zip_output` and `calibre_series_meta`, because it needs fixed values for them.
 
-**Each book gets the cover from the site.** FanFicFare downloads the cover only if `include_images`
+**Each book gets the cover from the publisher.** FanFicFare downloads the cover only if `include_images`
 is on. If it is off, Bindery sets it to `coveronly`, which keeps the cover but leaves out the images
 in the story text.
 
@@ -327,7 +317,285 @@ To serve Bindery under a path, such as `/bindery/`, do one of these:
 
 The health check at `/api/health` works with and without the prefix.
 
-## How books reach the library
+## Import an ePub file
+
+If the publishing site removed chapters before Bindery tracked the story, import an ePub that you made earlier
+with FanFicFare. Bindery stores its chapters as if it downloaded them. Chapters that the site
+publishes later go after them.
+
+- To start to track a story from its ePub, use "Import an ePub file" on the stories page. For a
+  story that Bindery tracks already, it adds the chapters that it does not have.
+- To add chapters to one story, use "Import chapters from an ePub" on the story page. The ePub must
+  be of that story.
+
+**Bindery accepts only ePubs that FanFicFare made**, because it reads the story URL and chapter data
+that FanFicFare writes into each ePub. A chapter that Bindery stored already keeps its stored text.
+The other chapters go in the order of the ePub.
+
+After the import, Bindery checks the publishing site for new chapters and builds the book. If you clear
+"Check the site for new chapters after import", Bindery only rebuilds the book of a story that it
+downloaded before.
+
+## File names of downloaded books
+
+"Download ePub" on a story page and each download from the OPDS catalog send the book with a file
+name. A template gives the name. The default template is `{title_sort} - {authors}`. For "The Book"
+by Ann Writer, it gives `Book, The - Ann Writer.epub`. That is the name that Calibre's default template gives.
+
+An admin can change the template under "Book Downloads" in Settings. To go back to the default,
+clear the field and save. The template follows
+the rules of [Calibre's templates](https://manual.calibre-ebook.com/template_lang.html) for file
+names. Bindery adds `.epub`. For example, `{series:|| - }{series_index:0>2s|| - }{title} - {authors}`
+gives `The Saga - 02 - The Book - Ann Writer.epub`.
+
+In Calibre, `{title}` and `{series}` give the title sort in a file name, for example `Book, The`.
+In Bindery, they keep the article. This is the same as Calibre with the tweak
+`save_template_title_series_sorting` set to `strictly_alphabetic`. Use `{title_sort}` and
+`{series_sort}` for the title and the series with the article at the end. `{series_sort}` is a
+Bindery field. Calibre does not have it, and gives empty text for it.
+
+| Field | Value |
+| --- | --- |
+| `{title}` | The title: `The Book`. |
+| `{title_sort}` | The title with a leading "The", "A" or "An" at the end: `Book, The`. |
+| `{authors}`, `{author}` | The authors, with ` & ` between them. |
+| `{author_sort}` | The authors with the last name first: `Writer, Ann`. |
+| `{series}` | The series. |
+| `{series_sort}` | The series with a leading article at the end: `Saga, The`. |
+| `{series_index}` | The series number: `2`, or `2.5`. |
+| `{tags}` | The tags in sorted order, with `, ` between them. |
+| `{languages}` | The language, as Bindery stores it. |
+| `{publisher}` | The publishing site. |
+| `{pubdate}` | The published date. |
+| `{timestamp}` | The date when Bindery added the story. |
+| `{last_modified}` | The date when the chapters last changed. |
+| `{id}` | The Bindery story ID. With CWA on, the CWA book ID of a story that has a library book. |
+
+- A field with no value becomes empty text.
+- `{field:format|text before|text after}` adds the two texts only when the field has a value. Each
+  of the three parts can be empty. `{series:|| - }` adds ` - ` after the series.
+- The format is a Python format. `{series_index:0>3s}` gives `002`, and `{title:.20s}` gives the
+  first 20 characters. `{author_sort[0]}` gives the first character.
+- `{{` and `}}` give braces. Bindery makes each run of spaces one space.
+- A download has one file name. If the template has folders, Bindery uses the part after the last
+  `/`. That is the name that Calibre gives the file.
+- Bindery does not support template functions, such as `{title:uppercase()}`, or the program modes
+  of Calibre. It refuses such a template when you save it.
+- If the template fails for a book, or gives an empty name, Bindery names the file after the title.
+  The log has a warning.
+- Bindery replaces each character that a file name cannot have with `_`. A name has 100 characters
+  and 200 bytes at most.
+
+"Date format" sets the form of `{pubdate}`, `{timestamp}` and `{last_modified}`. It takes the
+[codes of Python's strftime](https://docs.python.org/3/library/datetime.html#format-codes). `%d`
+gives the day, `%b` the month name, `%m` the month number and `%Y` the year. The default is
+`%b, %Y`, as in Calibre. It gives `May, 2020`. To go back to the default, clear the field and save.
+A `/` in a date becomes `_`, as in Calibre.
+
+Bindery follows Calibre 9.2.1 for English, with the tweak above. The order of tags with letters
+outside ASCII can differ from Calibre. The template applies only to downloads in the browser. It
+does not change the names of the files in `INGEST_DIR`. The browser can ignore the name.
+
+## Download modes
+
+| Mode | What it does |
+| --- | --- |
+| Download new chapters | Downloads the chapters that are not stored yet. The default, and what scheduled updates use. |
+| Recheck selected chapters | Downloads the selected chapters again. Changed text becomes a new revision. |
+| Recheck all chapters | Downloads every chapter on the publishing site again, and new chapters too. |
+| Download and replace entire book | Downloads every chapter again. Chapters that the publishing site no longer lists leave the book. They stay stored, and you can put them back. |
+| Update metadata from site | Updates only the metadata in the ePub. |
+| Rebuild book | Builds the book again from the stored chapters. |
+
+**Bindery builds a book only if the book changed.** A change is a new, changed, moved or renamed
+chapter, a changed rule, or changed metadata. A job that finds no change answers "No changes". Only
+"Rebuild book" builds a book that did not change. With CWA on, Bindery sends each book that it
+builds to CWA.
+
+### Find edited chapters
+
+**Scheduled updates download only new chapters, so they miss most edits.** To find edits, run a
+recheck, or use one of these settings:
+
+- On the Settings page, "also recheck the last N chapters" rechecks the last N chapters with each
+  download of new chapters. Authors most often fix their latest chapters. The default is 0, which
+  turns it off.
+- In the "Checking for new chapters" section of a story page, "Recheck all chapters every N days" rechecks the whole
+  story. This recheck takes the place of the next scheduled update. The story then ignores the "last
+  N chapters" setting.
+
+Each rechecked chapter is one request to the publishing site, so keep the numbers small for long stories.
+
+If the publishing site changes the title of a chapter, "Download new chapters" and "Recheck selected chapters"
+download that chapter again. The chapter gets the new title only if the download works. On sites
+that number chapter URLs, such as FanFiction.net, this also finds chapters that moved when the
+author inserted a chapter.
+
+### Revisions
+
+A recheck compares the text of a chapter with its newest revision, which is the text that the publishing site
+sent last. Bindery stores the HTML as the publishing site sends it, before the rules, so a rule never makes a
+chapter look changed.
+
+- If the text changed, the new revision goes in the book. If you chose an older revision, it
+  stays in the book until the publishing site changes the chapter again.
+- On the page of a chapter, you can compare revisions, choose the one that goes in the book, and
+  delete the others. The revision in the book and the newest revision cannot be deleted.
+- A chapter imported from an ePub holds its text after the FanFicFare settings and rules of that
+  build. If the current rules turn the published text into the imported text, Bindery counts the chapter
+  as unchanged, and stores the published text in its place. Other differences, for example from settings
+  in `personal.ini`, show as a change.
+
+### Restarts
+
+Bindery stores each chapter as soon as it downloads it, and keeps the job queue in its database.
+After a restart, the waiting jobs run. A job that was running starts again at the front of the
+queue, and skips the chapters that it stored. If Bindery stops during the same job three times, the
+job fails.
+
+If a job thread stops, `/api/health` answers 503, so the container shows as unhealthy. Restart
+Bindery to start the thread again.
+
+### Backups
+
+Once a day, Bindery copies its database and `personal.ini` to `/config/backups`, and keeps the
+copies of the last 7 days. **The database is the only copy of the chapters that publishing sites removed.** The
+backups are on the same disk, so also copy `/config` to another disk or machine.
+
+To restore a backup, do these steps:
+
+1. Stop Bindery.
+2. Copy the backup, for example `/config/backups/bindery-2026-09-26.db`, to `/config/bindery.db`.
+3. If `/config/bindery.db-wal` or `/config/bindery.db-shm` exist, delete them. They belong to the
+   old database.
+4. Start Bindery.
+
+## Metadata
+
+Bindery keeps the metadata of each story in its database, and writes it into each ePub. The fields
+are title, authors, description, tags, series, published date and language. Each field is locked or
+unlocked for each story:
+
+- An unlocked field takes the published value on every update.
+- A locked field keeps its stored value. The first download sets that value. For a story that you
+  import from the library or from an ePub file, the value comes from that ePub.
+- To update a locked field once, select it under "Also take these locked or edited fields from the
+  site" when you start a download. The field stays locked.
+
+You can edit each field under "Metadata" on the story page. An edit of a locked field replaces its
+stored value. An edit of an unlocked field stays until the site changes that field. To go back to
+the site value at once, click "Use the site value" or unlock the field. If Bindery sent the book to
+CWA before, it rebuilds the book after an edit.
+
+- Separate authors with `&`, and tags with commas.
+- Put the number of a series in brackets, for example `Saga [2]`.
+- The description is plain text, with a blank line between paragraphs.
+
+`DEFAULT_LOCKED_FIELDS` sets the locks for new stories. The default is `title`. The publisher is
+always the site name.
+
+## Page count
+
+The story list shows an estimated page count beside the chapter count. The OPDS catalog shows it in the
+description of a book. Bindery counts the pages as the last step when it builds an ePub, so the count
+includes your replacement rules and the line-break setting. A story shows no page count until Bindery
+builds its book.
+
+The method comes from "Paragraphs (APNX Accurate)" in the
+[Count Pages plugin](https://github.com/kiwidude68/calibre_plugins/wiki/Count-Pages) for calibre:
+
+- A paragraph start is one line. Each 70 characters of text is one more line.
+- A page is 31 lines.
+- A page is at most 2400 characters. This limit sets the count of a book with almost no paragraphs.
+
+The plugin counts either the `<p>` elements or the `<div>` elements of a book as its paragraphs. Bindery
+counts each paragraph that a reader sees:
+
+- Text starts a paragraph after a `<br>`, and after the start or the end of a block element such as `<p>`,
+  `<div>`, `<li>` or a heading.
+- A run of `<br>` elements is one paragraph break.
+- An element that only holds other blocks is not a paragraph.
+- The cells of a table row share one line.
+- A paragraph with no text is not counted. This includes a paragraph that holds only a no-break space,
+  which some stories put between paragraphs as a blank line.
+
+So the same text gives the same count as `<p>` elements, as `<div>` elements, and as lines with `<br>`
+between them. For a book that uses `<p>` elements and has no empty paragraphs, the count is within about 1%
+of the count of the plugin. The plugin counts each empty paragraph as a line, so its count is higher for a
+book that has many of them.
+The count is an estimate and differs from the page count of a printed book.
+
+## Replacement rules
+
+A rule is a regular expression, a replacement and optional flags (`i`, `s`, `m`, `x`). Bindery uses
+the Python [`regex`](https://pypi.org/project/regex/) module, which accepts the [syntax of Python's
+`re` module](https://docs.python.org/3/library/re.html#regular-expression-syntax). For an
+introduction, read the [Regular Expression HOWTO](https://docs.python.org/3/howto/regex.html). The
+replacement can use groups, such as `\1` or `\g<name>`. When Bindery builds a book, it runs the
+global rules from the Settings page, then the rules of the story from its page. The rules run on the
+HTML of each chapter. Each rule editor has a preview that shows one chapter before and after the
+rules.
+
+**If a rule takes longer than 10 seconds on one chapter, the build stops** with an error that names
+the rule.
+
+Some sites separate paragraphs with `<br>` tags. For those, turn on "Turn line breaks into
+paragraphs before the rules run" in a rule editor. Bindery then puts each piece of text between
+`<br>` tags in its own `<p>`, and drops empty pieces. A `<br>` inside other markup, such as italics,
+stays. This step runs before the global rules. The global setting is off by default, and applies to
+each story that has no setting of its own. To see the new paragraphs in the preview, select "Compare
+HTML source".
+
+## Calibre-Web-Automated
+
+Bindery can send each book that it builds to a [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library, and update that book when the
+story changes. CWA is off by default. To turn it on:
+
+1. Set `CWA_ENABLED=true`.
+2. Set `INGEST_DIR` to the CWA ingest folder. Set `CWA_URL`, `CALIBRE_DB_PATH` or both.
+3. Restart Bindery.
+
+Use CWA v4.0.1 or later. Older versions can import an update as a new book. See "Limitations".
+
+An earlier Bindery used CWA unless `CWA_ENABLED` was `false`. After an upgrade, set `CWA_ENABLED=true`
+to keep the uploads to CWA.
+
+| Variable | Description | Default value |
+| --- | --- | --- |
+| `CWA_ENABLED` | `true` to use CWA | `false` |
+| `CWA_URL` | The address of CWA, for example `http://calibre-web-automated:8083` | None |
+| `CWA_USERNAME`, `CWA_PASSWORD` | The CWA account of Bindery. Leave them unset if CWA asks for no login. | None |
+| `CALIBRE_DB_PATH` | The calibre `metadata.db`, for book ids. See "How books reach the library". | None |
+| `CWA_DB_PATH` | Calibre-Web's `app.db`, for logins. See "CWA logins". | None |
+| `CWA_ALLOWED_USERS` | The CWA users that can log in. Empty allows every CWA user. | Empty |
+
+The example Compose file has these values as comments. The user of the container must be able to write
+to the CWA ingest folder. Usually this is the same user and group as CWA.
+
+Settings shows an alert for each missing value:
+
+- Neither `CWA_URL` nor `CALIBRE_DB_PATH` is set. The alert is under "Uploads to CWA". Library linking
+  is then off, and its checkbox is disabled. Set either value and restart Bindery. Bindery keeps your
+  saved linking preference.
+- `INGEST_DIR` is not set. The alert is under "Book Downloads". "Pause all uploads" is then checked
+  until you set `INGEST_DIR` and restart Bindery.
+
+At each start, the log has a warning for each missing value.
+
+### Without CWA
+
+Without `CWA_ENABLED=true`:
+
+- Bindery makes no request to CWA. It ignores `CWA_URL`, `CWA_USERNAME`, `CWA_PASSWORD`,
+  `CWA_DB_PATH` and `CALIBRE_DB_PATH`. If one of them is set, the log has a warning at each start.
+- The web interface shows nothing about CWA. The pause, the library links and the approvals do not
+  apply.
+- If `INGEST_DIR` is set, Bindery puts each changed book in that folder as a plain ePub. Use this to
+  import the books into other software. Each file gets a new name, so the earlier files stay in the
+  folder. If `INGEST_DIR` is not set, Settings shows an alert under "Book Downloads".
+- `AUTH_METHOD=cwa` does not work. Use `local` or `proxy`. On a private network, you can use `none`.
+
+### How books reach the library
 
 Bindery gives every book to CWA through the ingest folder. CWA makes all changes to the library.
 
@@ -362,6 +630,10 @@ because library imports and CWA lookups use them.
 A book that the FanFicFare calibre plugin made holds its story URL. If you add its URL, or import it
 from the library, Bindery links that book and imports its chapters from the ePub.
 
+**CWA reads the metadata and the cover of an ePub only when it imports a new book.** An update
+replaces only the ePub file. So edits that you make in CWA stay, and later changes on the site show
+in the ePub but not in the CWA catalog.
+
 ### If Bindery cannot find the book
 
 If library matching is on and configured, an update fails when Bindery cannot find the book.
@@ -381,7 +653,7 @@ To fix it, go to "Library link" on the story page, and do one of these:
 
 ### Upload settings
 
-The "Uploads to CWA" section controls library matching and uploads. Bindery still downloads chapters
+The "Uploads to CWA" section of Settings controls library matching and uploads. Bindery still downloads chapters
 and builds books when uploads stop. The story list marks a built book with "held" only when it waits
 for approval, library linking is on, and uploads are not paused. Bindery keeps track of all unsent books.
 
@@ -417,7 +689,7 @@ Make a CWA user for Bindery with these settings:
 - The download permission. Bindery reads story URLs and chapters from the ePubs.
 - "Show Publisher Section", in the sidebar settings of the user. CWA shows its list of publishers in
   OPDS only to users with this setting. "Import from the library" uses the list to find the books of
-  story sites, because FanFicFare sets the publisher of a book to the site name.
+  publishing sites, because FanFicFare sets the publisher of a book to the site name.
 - Access to all books. Bindery finds only the books that this account can see.
 
 Bindery sends the password of this account with each OPDS request. **Use a password that no other
@@ -426,146 +698,31 @@ service uses.**
 If CWA allows anonymous browsing, Bindery needs no account. Leave `CWA_USERNAME` and `CWA_PASSWORD`
 unset, and give the guest user of CWA the settings above.
 
-## Import an ePub file
+### CWA logins
 
-If the site removed chapters before Bindery tracked the story, import an ePub that you made earlier
+`AUTH_METHOD=cwa` needs `CWA_ENABLED=true`, and `CWA_URL`, `CWA_DB_PATH` or both. Without `CWA_DB_PATH`, Bindery sends the
-with FanFicFare. Bindery stores its chapters as if it downloaded them. Chapters that the site
+user name and password to `CWA_URL/opds` as an HTTP Basic login. This is the OPDS login.
-publishes later go after them.
 
-- To start to track a story from its ePub, use "Import an ePub file" on the stories page. For a
+**The OPDS login works only if CWA refuses a wrong password.** A CWA with anonymous browsing can
-  story that Bindery tracks already, it adds the chapters that it does not have.
+accept every OPDS login. Bindery tests this with a login that cannot exist. If CWA accepts it,
-- To add chapters to one story, use "Import chapters from an ePub" on the story page. The ePub must
+Bindery refuses each OPDS login, and the log names the reason. Set `CWA_DB_PATH` then, or use
-  be of that story.
+another `AUTH_METHOD`.
 
-**Bindery accepts only ePubs that FanFicFare made**, because it reads the story URL and chapter data
+With `CWA_DB_PATH`, Bindery checks the password against the account in Calibre-Web's `app.db`. A
-that FanFicFare writes into each ePub. A chapter that Bindery stored already keeps its stored text.
+wrong password or an unknown account fails at once. If Bindery cannot read the file or its password
-The other chapters go in the order of the ePub.
+hash, it uses the OPDS login.
 
-After the import, Bindery checks the site for new chapters and sends the book to CWA. If you clear
+`CWA_DB_PATH` is the path of `app.db` inside the container. To use it, mount the CWA config folder
-"Check the site for new chapters after import", Bindery only rebuilds the book of a story that it
+read-only into the container. Mount the whole folder, so that SQLite can read the journal files
-downloaded before.
+beside the database. **Bindery does not write this database.**
 
-## Download modes
+**Database logins check only the password.** CWA's external login providers and multi-factor
+authentication do not apply. If they must apply, use the `proxy` mode.
 
-| Mode | What it does |
+- The CWA guest user cannot log in.
-| --- | --- |
+- If `CWA_ALLOWED_USERS` is set, only those users can log in. Bindery checks this list on each
-| Download new chapters | Downloads the chapters that are not stored yet. The default, and what scheduled updates use. |
+  request.
-| Recheck selected chapters | Downloads the selected chapters again. Changed text becomes a new revision. |
+- `CWA_USERNAME` and `CWA_PASSWORD` are only for the library lookups of Bindery.
-| Recheck all chapters | Downloads every chapter on the site again, and new chapters too. |
-| Download and replace entire book | Downloads every chapter again. Chapters that the site no longer lists leave the book. They stay stored, and you can put them back. |
-| Update metadata from site | Updates only the metadata in the ePub. |
-| Rebuild book | Builds the book again from the stored chapters, and sends it. |
-
-**Bindery builds and sends a book only if the book changed.** A change is a new, changed, moved or
-renamed chapter, a changed rule, or changed metadata. A job that finds no change answers "No changes"
-and sends nothing to CWA. Only "Rebuild book" builds and sends a book that did not change.
-
-### Find edited chapters
-
-**Scheduled updates download only new chapters, so they miss most edits.** To find edits, run a
-recheck, or use one of these settings:
-
-- On the Settings page, "also recheck the last N chapters" rechecks the last N chapters with each
-  download of new chapters. Authors most often fix their latest chapters. The default is 0, which
-  turns it off.
-- In the "Checking for new chapters" section of a story page, "Recheck all chapters every N days" rechecks the whole
-  story. This recheck takes the place of the next scheduled update. The story then ignores the "last
-  N chapters" setting.
-
-Each rechecked chapter is one request to the site, so keep the numbers small for long stories.
-
-If the site changes the title of a chapter, "Download new chapters" and "Recheck selected chapters"
-download that chapter again. The chapter gets the new title only if the download works. On sites
-that number chapter URLs, such as FanFiction.net, this also finds chapters that moved when the
-author inserted a chapter.
-
-### Revisions
-
-A recheck compares the text of a chapter with its newest revision, which is the text that the site
-sent last. Bindery stores the HTML as the site sends it, before the rules, so a rule never makes a
-chapter look changed.
-
-- If the text changed, the new revision goes in the book. If you chose an older revision, it
-  stays in the book until the site changes the chapter again.
-- On the page of a chapter, you can compare revisions, choose the one that goes in the book, and
-  delete the others. The revision in the book and the newest revision cannot be deleted.
-- A chapter imported from an ePub holds its text after the FanFicFare settings and rules of that
-  build. If the current rules turn the site text into the imported text, Bindery counts the chapter
-  as unchanged, and stores the site text in its place. Other differences, for example from settings
-  in `personal.ini`, show as a change.
-
-### Restarts
-
-Bindery stores each chapter as soon as it downloads it, and keeps the job queue in its database.
-After a restart, the waiting jobs run. A job that was running starts again at the front of the
-queue, and skips the chapters that it stored. If Bindery stops during the same job three times, the
-job fails.
-
-If a job thread stops, `/api/health` answers 503, so the container shows as unhealthy. Restart
-Bindery to start the thread again.
-
-### Backups
-
-Once a day, Bindery copies its database and `personal.ini` to `/config/backups`, and keeps the
-copies of the last 7 days. **The database is the only copy of the chapters that sites removed.** The
-backups are on the same disk, so also copy `/config` to another disk or machine.
-
-To restore a backup, do these steps:
-
-1. Stop Bindery.
-2. Copy the backup, for example `/config/backups/bindery-2026-09-26.db`, to `/config/bindery.db`.
-3. If `/config/bindery.db-wal` or `/config/bindery.db-shm` exist, delete them. They belong to the
-   old database.
-4. Start Bindery.
-
-## Metadata
-
-Bindery keeps the metadata of each story in its database, and writes it into each ePub. The fields
-are title, authors, description, tags, series, published date and language. Each field is locked or
-unlocked for each story:
-
-- An unlocked field takes the site value on every update.
-- A locked field keeps its stored value. The first download sets that value. For a story that you
-  import from the library or from an ePub file, the value comes from that ePub.
-- To update a locked field once, select it under "Also take these locked or edited fields from the
-  site" when you start a download. The field stays locked.
-
-You can edit each field under "Metadata" on the story page. An edit of a locked field replaces its
-stored value. An edit of an unlocked field stays until the site changes that field. To go back to
-the site value at once, click "Use the site value" or unlock the field. If Bindery sent the book to
-CWA before, it rebuilds the book after an edit.
-
-- Separate authors with `&`, and tags with commas.
-- Put the number of a series in brackets, for example `Saga [2]`.
-- The description is plain text, with a blank line between paragraphs.
-
-`DEFAULT_LOCKED_FIELDS` sets the locks for new stories. The default is `title`. The publisher is
-always the site name.
-
-**CWA reads the metadata and the cover of an ePub only when it imports a new book.** An update
-replaces only the ePub file. So edits that you make in CWA stay, and later changes on the site show
-in the ePub but not in the CWA catalog.
-
-## Replacement rules
-
-A rule is a regular expression, a replacement and optional flags (`i`, `s`, `m`, `x`). Bindery uses
-the Python [`regex`](https://pypi.org/project/regex/) module, which accepts the [syntax of Python's
-`re` module](https://docs.python.org/3/library/re.html#regular-expression-syntax). For an
-introduction, read the [Regular Expression HOWTO](https://docs.python.org/3/howto/regex.html). The
-replacement can use groups, such as `\1` or `\g<name>`. When Bindery builds a book, it runs the
-global rules from the Settings page, then the rules of the story from its page. The rules run on the
-HTML of each chapter. Each rule editor has a preview that shows one chapter before and after the
-rules.
-
-**If a rule takes longer than 10 seconds on one chapter, the build stops** with an error that names
-the rule.
-
-Some sites separate paragraphs with `<br>` tags. For those, turn on "Turn line breaks into
-paragraphs before the rules run" in a rule editor. Bindery then puts each piece of text between
-`<br>` tags in its own `<p>`, and drops empty pieces. A `<br>` inside other markup, such as italics,
-stays. This step runs before the global rules. The global setting is off by default, and applies to
-each story that has no setting of its own. To see the new paragraphs in the preview, select "Compare
-HTML source".
 
 ## Limitations
 
@@ -575,8 +732,8 @@ HTML source".
   the same, run "Recheck all chapters". When an author removes a chapter there, the book loses that
   chapter, and the last chapter appears twice. The removed chapter stays as an older revision of the
   chapter that took its URL.
-- "Stop tracking" deletes only the data of Bindery. The book stays in CWA.
+- "Stop tracking" deletes only the data of Bindery. A book in CWA stays there.
-- **Updates need CWA v4.0.1 or later.** CWA v4.0.0 added the `.cwa.json` sidecar files. But v4.0.0
+- **Updates of a CWA library need CWA v4.0.1 or later.** CWA v4.0.0 added the `.cwa.json` sidecar files. But v4.0.0
   can delete a sidecar file before it imports the ePub, and then imports the update as a new book.
   Versions before v4.0.0 cannot read sidecar files, so each update becomes a new book.
 
@@ -599,14 +756,14 @@ The Python tests use a fake CWA server and temporary folders, so they need no CW
 sites. The frontend tests use the built-in test runner of Node, and need no npm packages. Also check
 the pages in a browser.
 
-### Run locally without CWA
+### Run locally
 
-Without CWA, Bindery puts each new build of a story into its ingest folder as an ePub. This is
+CWA is off by default. Bindery puts each new build of a story into its ingest folder as an
-enough to work on downloads, chapters, rules and the web interface.
+ePub. This is enough to work on downloads, chapters, rules and the web interface.
 
 ```bash
 mkdir -p dev/ingest
-CWA_ENABLED=false CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp uv run python -m bindery
+CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp uv run python -m bindery
 ```
 
 Open `http://127.0.0.1:8085`. The folders go in `dev/`, which git ignores. To look at an ePub, use
@@ -637,7 +794,7 @@ not use your real library.**
    test logins too.
 
    ```bash
-   CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp \
+   CWA_ENABLED=true CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp \
    CWA_URL=http://localhost:8083 CWA_USERNAME=admin CWA_PASSWORD=admin123 AUTH_METHOD=cwa \
    uv run python -m bindery
    ```

bindery/auth.py

@@ -1,23 +1,26 @@
-"""Logins: none, one local account, CWA accounts, or a proxy. API keys work in every mode.
+"""Logins: none, one local account, CWA accounts, or a proxy. API keys work in every mode. The OPDS catalog takes
+the account login, and in proxy mode a password of its own.
 
 CWA accounts use an optional read-only app.db, with OPDS as the fallback when
 the database cannot answer. A rejected database login never falls back. The
 OPDS login works only with a CWA that refuses a wrong password.
 """
 
+import base64
 import hashlib
 import hmac
 import ipaddress
 import logging
+import secrets
 import threading
 import time
-from collections.abc import Mapping
+from collections.abc import Callable, Mapping
 from dataclasses import dataclass
 from functools import lru_cache
 from typing import Any
 from urllib.parse import urlsplit
 
-from werkzeug.security import check_password_hash
+from werkzeug.security import check_password_hash, generate_password_hash
 
 from bindery import calibre_db, config
 from bindery.cwa import CwaClient, CwaError
@@ -120,7 +123,7 @@ def config_error() -> str | None:
         if not _valid_local_hash(config.LOCAL_PASSWORD_HASH):
             return "BINDERY_PASSWORD_HASH must be a Werkzeug scrypt or PBKDF2 hash of a nonempty password"
     if method == AUTH_CWA and not config.CWA_ENABLED:
-        return "AUTH_METHOD=cwa needs CWA, but CWA_ENABLED is false"
+        return "AUTH_METHOD=cwa needs CWA_ENABLED=true"
     if method == AUTH_CWA and not (config.CWA_URL or config.CWA_DB_PATH):
         return "AUTH_METHOD=cwa needs CWA_URL or CWA_DB_PATH"
     if method == AUTH_PROXY:
@@ -163,6 +166,20 @@ def _api_key_identity(headers: Mapping[str, str]) -> Identity | None:
     return None
 
 
+def _session_identity(session: Mapping[str, Any]) -> Identity | None:
+    """The user of a web session in the local or CWA mode."""
+    mode = config.AUTH_METHOD
+    name = session.get("user")
+    if not name or session.get("mode") != mode:
+        return None
+    if mode == AUTH_LOCAL:
+        if name == config.LOCAL_USERNAME and session.get("local_password_version") == local_password_version():
+            return Identity(name, "session")
+    elif mode == AUTH_CWA and _allowed(name):
+        return Identity(name, "session")
+    return None
+
+
 def current_identity(
     headers: Mapping[str, str], session: Mapping[str, Any], remote_addr: str | None = None
 ) -> Identity | None:
@@ -181,15 +198,7 @@ def current_identity(
             logger.warning(f"Ignored the user header from {remote_addr}, which is not in PROXY_AUTH_TRUSTED_IPS")
             return None
         return Identity(name, "proxy") if name else None
-    name = session.get("user")
+    return _session_identity(session)
-    if not name or session.get("mode") != mode:
-        return None
-    if mode == AUTH_LOCAL:
-        if name == config.LOCAL_USERNAME and session.get("local_password_version") == local_password_version():
-            return Identity(name, "session")
-    elif mode == AUTH_CWA and _allowed(name):
-        return Identity(name, "session")
-    return None
 
 
 def client_address(headers: Mapping[str, str], remote_addr: str | None) -> str:
@@ -233,6 +242,11 @@ class LoginLimiter:
             kept = [x for x in self._failures.get(address, []) if x[1] != name.casefold()]
             self._failures[address] = kept
 
+    def forget(self, name: str) -> None:
+        """Clear the wrong passwords of a user name from every address."""
+        with self._lock:
+            self._failures = {a: [x for x in f if x[1] != name.casefold()] for a, f in self._failures.items()}
+
 
 limiter = LoginLimiter()
 
@@ -291,3 +305,99 @@ def login_cwa(username: str, password: str, client: CwaClient, address: str) ->
         raise AuthError("Wrong user name or password")
     limiter.clear(username, address)
     return Identity(username, "session")
+
+
+KEPT_SECONDS = 5 * 60
+KEPT_LIMIT = 100
+MAX_PASSWORD = 256
+_kept_key = secrets.token_bytes(32)
+_kept_logins: dict[bytes, float] = {}  # keyed hash of a good login: end time
+_kept_lock = threading.Lock()
+
+
+def _basic_login(headers: Mapping[str, str]) -> tuple[str, str] | None:
+    """The user name and password of an HTTP Basic login, or None."""
+    scheme, _, value = headers.get("Authorization", "").partition(" ")
+    if scheme.lower() != "basic":
+        return None
+    try:
+        username, colon, password = base64.b64decode(value.strip(), validate=True).decode().partition(":")
+    except ValueError:
+        return None
+    return (username, password) if colon and password and len(password) <= MAX_PASSWORD else None
+
+
+def _kept(parts: tuple[Any, ...], check: Callable[[], bool]) -> bool:
+    """Whether `check` accepts the login. Bindery keeps a good login for five minutes, so a reader that loads 50
+    covers checks a password hash once. `parts` holds everything that the check depends on."""
+    key = hmac.digest(_kept_key, repr(parts).encode(), "sha256")
+    with _kept_lock:
+        if _kept_logins.get(key, 0) > time.monotonic():
+            return True
+    if not check():
+        return False
+    with _kept_lock:
+        if len(_kept_logins) >= KEPT_LIMIT:
+            _kept_logins.clear()
+        _kept_logins[key] = time.monotonic() + KEPT_SECONDS
+    return True
+
+
+def _account_login(username: str, password: str, address: str, client: Callable[[], CwaClient]) -> bool:
+    """Whether this is a good login of the local or the CWA account. A wrong password counts toward the lockout."""
+    try:
+        if config.AUTH_METHOD == AUTH_LOCAL:
+            login_local(username, password, address)
+        else:
+            login_cwa(username, password, client(), address)
+    except AuthError:
+        return False
+    return True
+
+
+@lru_cache(maxsize=1)
+def _dummy_hash() -> str:
+    return generate_password_hash("not a password")
+
+
+def _own_password_login(username: str, password: str, address: str, password_hash: str | None) -> bool:
+    """Whether this is the OPDS password of this user. A wrong password counts toward the lockout."""
+    if limiter.locked(username, address):
+        return False
+    # A user without a password meets the same work as one with a password, so the time shows no difference.
+    if not (check_password_hash(password_hash or _dummy_hash(), password) and password_hash):
+        limiter.failed(username, address)
+        return False
+    limiter.clear(username, address)
+    return True
+
+
+def catalog_login(
+    headers: Mapping[str, str],
+    session: Mapping[str, Any],
+    address: str,
+    password_hash_of: Callable[[str], str | None],
+    client: Callable[[], CwaClient],
+) -> bool:
+    """Whether the request may open the OPDS catalog.
+
+    The local and CWA modes take the web session or the account login in HTTP Basic. The proxy mode takes the user
+    name that the proxy gives for the web login, with the OPDS password of that user, because the proxy skips its
+    login on these paths and a client can send any user header there. The API key opens the catalog in no mode.
+    """
+    mode = config.AUTH_METHOD
+    if mode == AUTH_NONE:
+        return True
+    if mode in (AUTH_LOCAL, AUTH_CWA) and _session_identity(session):
+        return True
+    login = _basic_login(headers)
+    if not login:
+        return False
+    username, password = login
+    if mode == AUTH_PROXY:
+        username = username.strip()
+        password_hash = password_hash_of(username)
+        parts = (mode, username.casefold(), password_hash, password)
+        return _kept(parts, lambda: _own_password_login(username, password, address, password_hash))
+    parts = (mode, username, password, local_password_version(), tuple(config.CWA_ALLOWED_USERS))
+    return _kept(parts, lambda: _account_login(username, password, address, client))

bindery/config.py

@@ -74,9 +74,9 @@ CWA_PASSWORD = os.getenv("CWA_PASSWORD", "")
 # Independent, optional read-only databases. HTTP remains the fallback.
 CWA_DB_PATH = _optional_path("CWA_DB_PATH")  # Calibre-Web app.db, for logins
 CALIBRE_DB_PATH = _optional_path("CALIBRE_DB_PATH")  # calibre metadata.db, for book IDs
-# false if the user has no CWA. Bindery then ignores the CWA settings above, and shows nothing about CWA. INGEST_DIR
+# true if the user has CWA. Without it, Bindery ignores the CWA settings above, and shows nothing about CWA. INGEST_DIR
-# still gets each built book, as a plain epub. Only a clear "false" turns CWA off.
+# still gets each built book, as a plain epub. Only a clear "true" turns CWA on.
-CWA_ENABLED = _bool("CWA_ENABLED", True)
+CWA_ENABLED = _bool("CWA_ENABLED")
 CWA_SETTINGS = ("CWA_URL", "CWA_USERNAME", "CWA_PASSWORD", "CWA_DB_PATH", "CALIBRE_DB_PATH")
 IGNORED_SETTINGS = [] if CWA_ENABLED else [name for name in CWA_SETTINGS if os.getenv(name, "").strip()]
 if not CWA_ENABLED:
@@ -141,7 +141,10 @@ def prepare() -> None:
         logger.info(f"Created the default FanFicFare configuration at {PERSONAL_INI}")
     if not CWA_ENABLED:
         if IGNORED_SETTINGS:
-            logger.info(f"CWA_ENABLED is false. Bindery ignores {', '.join(IGNORED_SETTINGS)}.")
+            logger.warning(
+                f"CWA_ENABLED is not true. Bindery ignores {', '.join(IGNORED_SETTINGS)}. To use CWA, set "
+                "CWA_ENABLED=true."
+            )
         return
     if CWA_PASSWORD and not CWA_USERNAME:
         logger.warning("CWA_PASSWORD is set, but CWA_USERNAME is not. Bindery sends no login to CWA.")
@@ -149,7 +152,7 @@ def prepare() -> None:
     for warning in missing:
         logger.warning(warning)
     if missing:
-        logger.warning("If you do not use CWA, set CWA_ENABLED=false. Bindery then shows no messages about CWA.")
+        logger.warning("If you do not use CWA, remove CWA_ENABLED. Bindery then shows no messages about CWA.")
 
 
 def _missing_cwa() -> list[str]:

bindery/epub.py

@@ -1,9 +1,11 @@
 """Read the OPF metadata of an epub that came from CWA."""
 
+import posixpath
 import re
 import zipfile
 from pathlib import Path
 from typing import Any
+from urllib.parse import unquote
 from xml.etree.ElementTree import Element
 
 from defusedxml import ElementTree
@@ -17,14 +19,19 @@ class EpubError(Exception):
     pass
 
 
-def _metadata(path: Path) -> Element:
+def _opf(path: Path, z: zipfile.ZipFile) -> tuple[str, Element]:
-    try:
+    """The name of the OPF file in the epub, and its content."""
-        with zipfile.ZipFile(path) as z:
     container = ElementTree.fromstring(z.read("META-INF/container.xml"))
     rootfile = container.find(f".//{CONTAINER}rootfile")
     if rootfile is None or not rootfile.get("full-path"):
         raise EpubError(f"{path.name} has no OPF file")
-            opf = ElementTree.fromstring(z.read(rootfile.get("full-path")))
+    return rootfile.get("full-path"), ElementTree.fromstring(z.read(rootfile.get("full-path")))
+
+
+def _metadata(path: Path) -> Element:
+    try:
+        with zipfile.ZipFile(path) as z:
+            opf = _opf(path, z)[1]
     except (zipfile.BadZipFile, KeyError, ElementTree.ParseError) as e:
         raise EpubError(f"{path.name} is not a readable epub: {e}") from e
     metadata = opf.find(f"{OPF}metadata")
@@ -33,6 +40,21 @@ def _metadata(path: Path) -> Element:
     return metadata
 
 
+def spine_documents(path: Path) -> list[str]:
+    """The text of each document in the spine of an epub, in reading order."""
+    try:
+        with zipfile.ZipFile(path) as z:
+            name, opf = _opf(path, z)
+            hrefs = {item.get("id"): item.get("href") for item in opf.iter(f"{OPF}item")}
+            files = [
+                posixpath.join(posixpath.dirname(name), unquote(hrefs[ref.get("idref")]))
+                for ref in opf.iter(f"{OPF}itemref")
+            ]
+            return [z.read(file).decode("utf-8", "replace") for file in files]
+    except (zipfile.BadZipFile, KeyError, ElementTree.ParseError) as e:
+        raise EpubError(f"{path.name} is not a readable epub: {e}") from e
+
+
 def epub_urls(path: Path) -> list[str]:
     """Every URL in the identifiers and the source of an epub's OPF.
 

bindery/filename_template.py

@@ -0,0 +1,321 @@
+"""Calibre's templates for file names, for the name of a downloaded book.
+
+Bindery follows Calibre 9.2.1 with its default tweaks and the English articles. prepare() takes the tweak for
+the order of titles and series as an argument. Dates take a strftime format, with Calibre's "%b, %Y" as the
+default. Bindery takes {field}, {field[0]}, {field:format|prefix|suffix} and {{ }}. It refuses template
+functions and the program modes. It never runs template text as code.
+
+{series_sort} is a Bindery field. It is to {series} what {title_sort} is to {title}. Calibre has no such field.
+
+The tags sort close to Calibre's ICU rules, but their order can differ for characters outside ASCII.
+"""
+
+import re
+import unicodedata
+from datetime import datetime
+from string import Formatter
+from typing import Any
+
+from bindery.ingest import sanitize_filename
+
+# The name that Calibre's default template, {title} - {authors}, gives with its default tweaks.
+DEFAULT_TEMPLATE = "{title_sort} - {authors}"
+DEFAULT_DATE_FORMAT = "%b, %Y"
+MAX_TEMPLATE_LENGTH = 1024
+MAX_DATE_FORMAT_LENGTH = 100
+# The most characters that a format can ask for, as a width or a precision.
+MAX_WIDTH = 255
+MAX_NAME_BYTES = 200
+
+# The fields that Bindery has, with the help text for each. Any other field is empty, as it is in Calibre for
+# a book without that value.
+FIELDS = {
+    "title": "The title",
+    "title_sort": 'The title with a leading "The", "A" or "An" at the end, such as "Book, The"',
+    "authors": "The authors",
+    "author": "The authors",
+    "author_sort": "The author sort string. To use only the first letter of the name, use {author_sort[0]}",
+    "series": "The series",
+    "series_sort": 'The series with a leading "The", "A" or "An" at the end, such as "Saga, The". '
+    "Calibre does not have this field",
+    "series_index": "The series number. To get leading zeros, use {series_index:0>3s}",
+    "tags": "The tags",
+    "languages": "The language",
+    "publisher": "The story site",
+    "pubdate": "The published date",
+    "timestamp": "The date when Bindery added the story",
+    "last_modified": "The date when the chapters last changed",
+    "id": "The Bindery story ID",
+}
+
+
+class TemplateError(ValueError):
+    """A template that Bindery cannot use, or that fails for a book."""
+
+
+# Metadata, as Calibre prepares it for a file name
+
+_ARTICLE = re.compile(r"^(A\s+|The\s+|An\s+)", re.IGNORECASE)
+# Calibre's quote_pairs: each opening quote, with the quotes that close it.
+_QUOTE_PAIRS = {
+    '"': '"',
+    "'": "'",
+    "\u201c": "\u201d\u201c",
+    "\u201d": "\u201d",
+    "\u201e": "\u201d\u201c",
+    "\u201a": "\u2019\u2018",
+    "\u2019": "\u2019\u2018",
+    "\u2018": "\u2019\u2018",
+    "\u2039": "\u203a",
+    "\u203a": "\u2039",
+    "\u300a": "\u300b",
+    "\u3008": "\u3009",
+    "\u00bb": "\u00ab\u00bb",
+    "\u00ab": "\u00ab\u00bb",
+    "\u300c": "\u300d",
+    "\u300e": "\u300f",
+}
+# The ASCII punctuation in the order of Calibre's sort. Digits come next, then letters.
+_PUNCTUATION = " _-,;:!?.'\"()[]{}@*/\\&#%`^+<=>|~$"
+# Calibre's default tweaks for author names.
+_NAME_PREFIXES = {"mr", "mrs", "ms", "dr", "prof"}
+_NAME_SUFFIXES = {"jr", "sr", "inc", "ph.d", "phd", "md", "m.d", "i", "ii", "iii", "iv", "junior", "senior"}
+_COPY_WORDS = {
+    "agency",
+    "corporation",
+    "company",
+    "co.",
+    "council",
+    "committee",
+    "inc.",
+    "institute",
+    "national",
+    "society",
+    "club",
+    "team",
+    "software",
+    "games",
+    "entertainment",
+    "media",
+    "studios",
+}
+_BRACKETS = {"(": ")", "[": "]", "{": "}"}
+
+
+def _unquote(title: str) -> str:
+    closers = _QUOTE_PAIRS.get(title[:1])
+    if closers:
+        title = title[1:]
+        if title and title[-1] in closers:
+            title = title[:-1]
+    return title
+
+
+def title_sort(title: str) -> str:
+    """Calibre's title_sort(): "The Book" gives "Book, The"."""
+    title = _unquote(title.strip())
+    match = _ARTICLE.search(title)
+    if match:
+        title = _unquote(f"{title[match.end() :]}, {match.group(1)}")
+    return title.strip()
+
+
+def _without_brackets(text: str) -> str:
+    depth = dict.fromkeys(_BRACKETS, 0)
+    closers = {close: opener for opener, close in _BRACKETS.items()}
+    kept = []
+    for char in text:
+        if char in depth:
+            depth[char] += 1
+        elif char in closers:
+            depth[closers[char]] = max(0, depth[closers[char]] - 1)
+        elif not any(depth.values()):
+            kept.append(char)
+    return "".join(kept)
+
+
+def author_sort(author: str) -> str:
+    """Calibre's author_to_author_sort() with its default tweaks: "Ann Writer" gives "Writer, Ann"."""
+    plain = _without_brackets(author).strip()
+    tokens = plain.split()
+    lower = [t.lower().removesuffix(".") for t in tokens]
+    if "," in plain or len(tokens) < 2 or _COPY_WORDS.intersection(t.lower() for t in tokens):
+        return author
+    first = next((n for n, t in enumerate(lower) if t not in _NAME_PREFIXES), None)
+    if first is None:
+        return author
+    last = next((n for n in range(len(tokens) - 1, first - 1, -1) if lower[n] not in _NAME_SUFFIXES), None)
+    if last is None:
+        return author
+    names = [tokens[last], *tokens[first:last]]
+    if len(names) > 1:
+        names[0] += ","
+    return " ".join(names + tokens[last + 1 :])
+
+
+def series_index(index: Any) -> str:
+    """Calibre's fmt_sidx(): 2.0 gives "2", and 2.345 gives "2.35"."""
+    try:
+        index = float(index)
+    except (TypeError, ValueError):
+        index = 1.0
+    if index.is_integer():
+        return str(int(index))
+    return f"{index:.2f}".rstrip("0")
+
+
+def check_date_format(date_format: str) -> None:
+    """Raise TemplateError if Bindery cannot use the date format."""
+    if len(date_format) > MAX_DATE_FORMAT_LENGTH:
+        raise TemplateError(f"A date format can have {MAX_DATE_FORMAT_LENGTH} characters at most")
+    try:
+        datetime(2020, 5, 6).strftime(date_format)
+    except ValueError as e:
+        raise TemplateError(f"The date format is not valid: {e}") from e
+
+
+def _date(value: Any, date_format: str) -> str:
+    try:
+        return datetime.fromisoformat(value).strftime(date_format)
+    except (TypeError, ValueError):
+        return ""
+
+
+def _tag_key(tag: str) -> tuple[list[tuple[int, str]], str]:
+    """Close to the key of Calibre's sort: no case, punctuation before digits before letters, accents last."""
+    text = unicodedata.normalize("NFKD", tag.casefold())
+    ranks = len(_PUNCTUATION)
+    letters = [
+        (_PUNCTUATION.index(c), "") if c in _PUNCTUATION else (ranks + c.isalpha(), c)
+        for c in text
+        if not unicodedata.combining(c)
+    ]
+    return letters, text
+
+
+def prepare(
+    meta: dict[str, Any], book_id: int, order: str = "library_order", date_format: str = DEFAULT_DATE_FORMAT
+) -> dict[str, Any]:
+    """Calibre's get_component_metadata(): the values of a book in the form that a file name takes.
+
+    `order` is Calibre's tweak save_template_title_series_sorting. With "strictly_alphabetic", the title and the
+    series keep a leading article. `date_format` is Calibre's timefmt.
+    """
+    library = order == "library_order"
+    values: dict[str, Any] = {"id": str(book_id), "publisher": meta.get("publisher") or ""}
+    title, authors, series = meta.get("title"), [a for a in meta.get("authors") or [] if a], meta.get("series")
+    if title:
+        values["title_sort"] = title_sort(title)
+        values["title"] = values["title_sort"] if library else title
+    if authors:
+        values["authors"] = values["author"] = " & ".join(a.replace("&", "&&") for a in authors)
+        values["author_sort"] = " & ".join(author_sort(a) for a in authors)
+    if series:
+        values["series_sort"] = title_sort(series)
+        values["series"] = values["series_sort"] if library else series.strip()
+        if meta.get("series_index") is not None:
+            values["series_index"] = series_index(meta["series_index"])
+    values["tags"] = ", ".join(sorted(meta.get("tags") or [], key=_tag_key)).removeprefix("/")
+    values["languages"] = meta.get("languages") or ""
+    for name in ("pubdate", "timestamp", "last_modified"):
+        values[name] = _date(meta.get(name), date_format)
+    return values
+
+
+# Templates
+
+_FIELD = re.compile(r"([^.\[\]]*)((?:\[\d+\])*)")
+_AFFIXES = re.compile(r"^(.*)\|([^|]*)\|(.*)$", re.DOTALL)
+_INT_CODES, _FLOAT_CODES = "bcdoxXn", "eEfFgG%"
+
+
+def _format(value: str, fmt: str) -> str:
+    """Calibre's _do_format(): a number code takes the text as a number."""
+    if not fmt or not value:
+        return value
+    code = fmt[-1]
+    try:
+        number = int(value) if code in _INT_CODES else float(value) if code in _FLOAT_CODES else value
+        return format(number, fmt)
+    except (ValueError, OverflowError) as e:
+        raise TemplateError(f"The format {fmt!r} cannot take the value {value!r}: {e}") from e
+
+
+def _parse(template: str) -> list[tuple[str, str | None, list[int], str, str, str]]:
+    """The parts of a template. Each part is the text before a field, then the name of the field, its indexes,
+    its format, its prefix and its suffix. The name is None for text with no field after it."""
+    if len(template) > MAX_TEMPLATE_LENGTH:
+        raise TemplateError(f"A template can have {MAX_TEMPLATE_LENGTH} characters at most")
+    if template.startswith(("program:", "python:")):
+        raise TemplateError("Bindery does not support the program modes of Calibre")
+    try:
+        fields = list(Formatter().parse(template))
+    except ValueError as e:
+        raise TemplateError(f"The braces of the template do not match: {e}") from e
+    parts = []
+    for text, name, spec, conversion in fields:
+        if name is None:
+            parts.append((text, None, [], "", "", ""))
+            continue
+        field = _FIELD.fullmatch(name)
+        if not field or conversion or "{" in spec:
+            raise TemplateError(f"Bindery does not support the field {{{name}}} in this form")
+        if not field.group(1) or field.group(1).isdigit():
+            raise TemplateError("A field needs a name, such as {title}")
+        affixes = _AFFIXES.match(spec)
+        fmt, prefix, suffix = affixes.groups() if affixes else (spec, "", "")
+        if fmt.startswith("'") or (":'" in fmt and fmt.endswith("'")) or ("(" in fmt and fmt.endswith(")")):
+            raise TemplateError("Bindery does not support template functions, such as {title:uppercase()}")
+        if any(int(n) > MAX_WIDTH for n in re.findall(r"\d+", fmt)):
+            raise TemplateError(f"A format can ask for {MAX_WIDTH} characters at most")
+        _format("1", fmt)
+        indexes = [int(n) for n in re.findall(r"\d+", field.group(2))]
+        parts.append((text, field.group(1).lower(), indexes, fmt, prefix, suffix))
+    return parts
+
+
+def check(template: str) -> None:
+    """Raise TemplateError if Bindery cannot use the template."""
+    _parse(template)
+
+
+def expand(template: str, values: dict[str, Any]) -> str:
+    """The text of a template for the prepared values of a book, as Calibre gives it before it makes a path."""
+    out = []
+    for text, name, indexes, fmt, prefix, suffix in _parse(template):
+        out.append(text)
+        if name is None:
+            continue
+        value = values.get(name, "")
+        if isinstance(value, list):
+            value = ",".join(value)
+        value = value.replace("/", "_").replace("\\", "_")
+        try:
+            for index in indexes:
+                value = value[index]
+        except IndexError as e:
+            raise TemplateError(f"The value of {{{name}}} is too short for [{index}]") from e
+        if value:
+            value = _format(value, fmt)
+        if value:
+            out += prefix, value, suffix
+    return re.sub(r"\s+", " ", "".join(out)).strip(" ")
+
+
+_RESERVED = re.compile(r"(?:CON|PRN|AUX|NUL|COM\d|LPT\d)(?:\.|$)", re.IGNORECASE)
+
+
+def file_name(template: str, values: dict[str, Any]) -> str:
+    """The name of one file from a template, without the extension. Empty if the template gives no name.
+
+    A download has one name, so Bindery uses the last part of a template with folders. That is the name that
+    Calibre gives the file. Windows cannot use some names, such as CON. Those get a leading underscore.
+    """
+    parts = [part.strip() for part in expand(template, values).split("/")]
+    name = next((part for part in reversed(parts) if part), "")
+    if not name:
+        return ""
+    name = sanitize_filename(name)
+    if _RESERVED.match(name):
+        name = f"_{name}"
+    return name.encode()[:MAX_NAME_BYTES].decode(errors="ignore").rstrip(" .")

bindery/opds.py

@@ -0,0 +1,301 @@
+"""An OPDS 1.2 catalog of the books stored in Bindery."""
+
+import hashlib
+import io
+import os
+import re
+from collections.abc import Callable
+from datetime import UTC, datetime
+from typing import Any
+from urllib.parse import quote
+from xml.etree import ElementTree as ET
+
+from flask import Blueprint, Response, abort, request, send_file, url_for
+
+from bindery import diffing, filename_template, orchestrator
+from bindery.store import sort_title
+
+bp = Blueprint("opds", __name__)
+ATOM = "http://www.w3.org/2005/Atom"
+DC = "http://purl.org/dc/terms/"
+OPENSEARCH = "http://a9.com/-/spec/opensearch/1.1/"
+NAVIGATION = "application/atom+xml;profile=opds-catalog;kind=navigation"
+ACQUISITION = "application/atom+xml;profile=opds-catalog;kind=acquisition"
+EPUB = "application/epub+zip"
+PAGE_SIZE = 50
+UNKNOWN_AUTHOR = "Unknown"  # as in Calibre
+# Each list: its title, its kind, and the text of its entry in the start feed.
+LISTS = {
+    "title": ("By Title", ACQUISITION, "All books in title order"),
+    "author": ("By Author", NAVIGATION, "The authors, with the books of each"),
+    "changed": ("By Last Changed", ACQUISITION, "All books, with the most recently changed books first"),
+}
+# XML 1.0 excludes control characters, even in escaped text.
+INVALID_XML = re.compile("[^\x09\x0a\x0d\x20-퟿-�\U00010000-\U0010ffff]")
+
+
+def _element(parent: ET.Element, tag: str, text: str = "", **attributes: str) -> ET.Element:
+    element = ET.SubElement(parent, tag, attributes)
+    element.text = INVALID_XML.sub("", text)
+    return element
+
+
+def _updated(book: dict[str, Any]) -> str:
+    # Store timestamps without an offset use the server's local time.
+    date = datetime.fromisoformat(book["last_changed"] or book["created_at"])
+    return date.astimezone(UTC).isoformat(timespec="seconds").replace("+00:00", "Z")
+
+
+def _link(parent: ET.Element, rel: str, href: str, media_type: str) -> None:
+    _element(parent, "link", rel=rel, href=href, type=media_type)
+
+
+def _feed(title: str, identifier: str, updated: str, href: str, media_type: str) -> ET.Element:
+    feed = ET.Element("feed", {"xmlns": ATOM, "xmlns:dc": DC})
+    _element(feed, "id", identifier)
+    _element(feed, "title", title)
+    _element(feed, "updated", updated)
+    _element(_element(feed, "author"), "name", "Bindery")
+    _link(feed, "self", href, media_type)
+    _link(feed, "start", url_for("opds.catalog"), NAVIGATION)
+    _link(feed, "search", url_for("opds.search_description"), "application/opensearchdescription+xml")
+    # Many readers take the search template from the feed and never read the description.
+    _link(feed, "search", f"{url_for('opds.search')}?q={{searchTerms}}", ACQUISITION)
+    return feed
+
+
+def _response(root: ET.Element, media_type: str) -> Response:
+    response = Response(ET.tostring(root, encoding="utf-8", xml_declaration=True), content_type=media_type)
+    response.headers["Cache-Control"] = "private, no-store"
+    return response
+
+
+def _languages(value: Any) -> list[str]:
+    return [value] if isinstance(value, str) else list(value or [])
+
+
+def _books() -> list[dict[str, Any]]:
+    """The books of the catalog, in title order."""
+    books = []
+    page_counts = orchestrator.store.story_settings("page_count")
+    for story in orchestrator.store.catalog():
+        meta = orchestrator.book_metadata(story)
+        title = (meta["title"] or "").strip() or story["url"]
+        authors = [a.strip() for a in meta["authors"] if a and a.strip()] or [UNKNOWN_AUTHOR]
+        cover = story["cover_key"] if story["cover_mime"] in orchestrator.COVER_TYPES else None
+        books.append(
+            {
+                **story,
+                "meta": meta,
+                "title": title,
+                "authors": authors,
+                "updated": _updated(story),
+                "page_count": page_counts.get(story["id"]),
+                "cover": hashlib.sha256(cover.encode()).hexdigest()[:16] if cover else None,
+            }
+        )
+    # The ID breaks ties, so a book keeps its page.
+    return sorted(books, key=lambda book: (sort_title(book["title"]), book["id"]))
+
+
+def _latest(books: list[dict[str, Any]]) -> str:
+    return max((book["updated"] for book in books), default="1970-01-01T00:00:00Z")
+
+
+def _content(book: dict[str, Any]) -> str:
+    meta = book["meta"]
+    lines = []
+    if meta.get("series"):
+        index = filename_template.series_index(meta.get("series_index"))
+        lines.append(f"Series: {meta['series']}" + (f", book {index}" if index else ""))
+    lines.append(f"Chapters: {book['chapter_count']}")
+    if book["page_count"] is not None:
+        lines.append(f"Pages: {book['page_count']}")
+    description = "\n\n".join(diffing.html_to_blocks(meta.get("comments") or ""))
+    return "\n".join(lines) + (f"\n\n{description}" if description else "")
+
+
+def _book_entry(feed: ET.Element, book: dict[str, Any]) -> None:
+    meta = book["meta"]
+    entry = _element(feed, "entry")
+    _element(entry, "id", book["url"])
+    _element(entry, "title", book["title"])
+    _element(entry, "updated", book["updated"])
+    for name in book["authors"]:
+        author = _element(entry, "author")
+        _element(author, "name", name)
+        _element(author, "uri", url_for("opds.books", order="author", name=name))
+    _element(entry, "dc:publisher", book["site"])
+    if meta.get("pubdate"):
+        _element(entry, "dc:issued", meta["pubdate"][:10])
+    for language in _languages(meta.get("languages")):
+        _element(entry, "dc:language", language)
+    for tag in meta.get("tags") or []:
+        _element(entry, "category", term=tag, label=tag)
+    _element(entry, "content", _content(book), type="text")
+    _link(entry, "http://opds-spec.org/acquisition", url_for("opds.download", story_id=book["id"]), EPUB)
+    if book["cover"]:
+        href = url_for("opds.cover", story_id=book["id"], v=book["cover"])
+        for rel in ("http://opds-spec.org/image", "http://opds-spec.org/image/thumbnail"):
+            _link(entry, rel, href, book["cover_mime"])
+
+
+def _author_entry(feed: ET.Element, author: dict[str, Any]) -> None:
+    entry = _element(feed, "entry")
+    _element(entry, "id", f"urn:bindery:author:{quote(author['name'])}")
+    _element(entry, "title", author["name"])
+    _element(entry, "updated", _latest(author["books"]))
+    count = len(author["books"])
+    _element(entry, "content", f"{count} book" + ("" if count == 1 else "s"), type="text")
+    _link(entry, "subsection", url_for("opds.books", order="author", name=author["name"]), ACQUISITION)
+
+
+def _authors(books: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    """The authors of the books, in Calibre's author sort order. A name in other capitals is the same author."""
+    authors: dict[str, dict[str, Any]] = {}
+    for book in books:
+        for name in book["authors"]:
+            authors.setdefault(name.casefold(), {"name": name, "books": []})["books"].append(book)
+    return sorted(authors.values(), key=lambda a: (filename_template.author_sort(a["name"]).casefold(), a["name"]))
+
+
+def _page() -> int:
+    text = request.args.get("page", "1")
+    if not re.fullmatch(r"[0-9]{1,9}", text) or int(text) < 1:
+        raise ValueError("The page must be a positive whole number")
+    return int(text)
+
+
+def _paged(
+    title: str,
+    identifier: str,
+    items: list[dict[str, Any]],
+    kind: str,
+    entry: Callable[[ET.Element, dict[str, Any]], None],
+    up: str,
+    endpoint: str,
+    **args: str,
+) -> Response:
+    """A feed that shows one page of the items. `entry` adds the entry of one item."""
+    page = _page()
+    pages = max(1, (len(items) + PAGE_SIZE - 1) // PAGE_SIZE)
+    if page > pages:
+        abort(404)
+    books = [book for item in items for book in item.get("books", [item])]
+    feed = _feed(title, identifier, _latest(books), url_for(endpoint, **args, page=page), kind)
+    _link(feed, "up", up, NAVIGATION)
+    if page > 1:
+        _link(feed, "previous", url_for(endpoint, **args, page=page - 1), kind)
+    if page < pages:
+        _link(feed, "next", url_for(endpoint, **args, page=page + 1), kind)
+    for item in items[(page - 1) * PAGE_SIZE : page * PAGE_SIZE]:
+        entry(feed, item)
+    return _response(feed, kind)
+
+
+@bp.get("/opds/")
+@bp.get("/opds")
+def catalog() -> Response:
+    updated = _latest(_books())
+    feed = _feed("Bindery", "urn:bindery:catalog", updated, url_for("opds.catalog"), NAVIGATION)
+    for order, (title, kind, text) in LISTS.items():
+        entry = _element(feed, "entry")
+        _element(entry, "id", f"urn:bindery:catalog:{order}")
+        _element(entry, "title", title)
+        _element(entry, "updated", updated)
+        _element(entry, "content", text, type="text")
+        _link(entry, "subsection", url_for("opds.books", order=order), kind)
+    return _response(feed, NAVIGATION)
+
+
+@bp.get("/opds/<order>")
+def books(order: str) -> Response:
+    if order not in LISTS:
+        abort(404)
+    books = _books()
+    up = url_for("opds.catalog")
+    if order == "author":
+        authors = _authors(books)
+        name = request.args.get("name")
+        if name is None:
+            title, kind, _ = LISTS[order]
+            return _paged(
+                title, "urn:bindery:catalog:author", authors, kind, _author_entry, up, "opds.books", order=order
+            )
+        author = next((a for a in authors if a["name"].casefold() == name.casefold()), None)
+        if author is None:
+            abort(404)
+        identifier = f"urn:bindery:author:{quote(author['name'])}"
+        up = url_for("opds.books", order="author")
+        return _paged(
+            author["name"],
+            identifier,
+            author["books"],
+            ACQUISITION,
+            _book_entry,
+            up,
+            "opds.books",
+            order=order,
+            name=author["name"],
+        )
+    if order == "changed":
+        books.sort(key=lambda book: book["updated"], reverse=True)
+    identifier = f"urn:bindery:catalog:{order}"
+    return _paged(LISTS[order][0], identifier, books, ACQUISITION, _book_entry, up, "opds.books", order=order)
+
+
+@bp.get("/opds/search")
+def search() -> Response:
+    """The books whose title, authors, series or tags hold each word of the search, in title order."""
+    query = request.args.get("q", "")
+    words = query.casefold().split()
+    found = []
+    for book in _books() if words else []:
+        meta = book["meta"]
+        text = " ".join([book["title"], *book["authors"], meta.get("series") or "", *(meta.get("tags") or [])])
+        if all(word in text.casefold() for word in words):
+            found.append(book)
+    identifier = f"urn:bindery:search:{quote(query)}"
+    up = url_for("opds.catalog")
+    return _paged(f"Search: {query}", identifier, found, ACQUISITION, _book_entry, up, "opds.search", q=query)
+
+
+@bp.get("/opds/search.xml")
+def search_description() -> Response:
+    root = ET.Element("OpenSearchDescription", xmlns=OPENSEARCH)
+    _element(root, "ShortName", "Bindery")
+    _element(root, "Description", "Search the books in Bindery by title, author, series or tag")
+    _element(root, "InputEncoding", "UTF-8")
+    _element(root, "OutputEncoding", "UTF-8")
+    _element(root, "Url", type=ACQUISITION, template=f"{url_for('opds.search')}?q={{searchTerms}}")
+    return _response(root, "application/opensearchdescription+xml")
+
+
+def send_epub(story_id: int) -> Response:
+    """The ePub of a story as a download, built from the stored chapters."""
+    file, name = orchestrator.build_for_download(story_id)
+    response = send_file(file, download_name=name, as_attachment=True, mimetype=EPUB)
+    response.content_length = os.fstat(file.fileno()).st_size
+    return response
+
+
+def send_cover(story_id: int) -> Response:
+    cover = orchestrator.story_cover(story_id)
+    if not cover:
+        raise KeyError("This story has no cover")
+    # The URL holds the cover version, so a client can keep the image for a year.
+    response = send_file(io.BytesIO(cover["data"]), mimetype=cover["mime"], max_age=365 * 24 * 60 * 60)
+    # A cover needs a login, so a shared cache must not keep it.
+    response.cache_control.public = False
+    response.cache_control.private = True
+    return response
+
+
+@bp.get("/opds/books/<int:story_id>.epub")
+def download(story_id: int) -> Response:
+    return send_epub(story_id)
+
+
+@bp.get("/opds/books/<int:story_id>/cover")
+def cover(story_id: int) -> Response:
+    return send_cover(story_id)

bindery/orchestrator.py

@@ -11,13 +11,15 @@ from datetime import datetime, timedelta
 from pathlib import Path
 from typing import Any, BinaryIO
 
-from bindery import config, fff
+from werkzeug.security import generate_password_hash
+
+from bindery import config, fff, filename_template, pages
 from bindery.backup import daily_backup
 from bindery.cwa import Book, BookFinder, CwaClient, CwaError
 from bindery.diffing import diff_html, text_hash
 from bindery.epub import EpubError, epub_metadata, epub_urls
 from bindery.fff import StoryError
-from bindery.ingest import IngestError, IngestFolder
+from bindery.ingest import IngestError, IngestFolder, sanitize_filename
 from bindery.jobs import METADATA_FIELDS, DownloadMode, Job, QueueStatus, job_queue
 from bindery.merge import merge_chapter_lists
 from bindery.rules import RuleError, Step, apply_rules, compile_rules, normalize_rules
@@ -224,9 +226,11 @@ def list_stories() -> list[dict[str, Any]]:
     stories = store.list_stories()
     held = store.story_settings("upload_held")
     unapproved = store.story_settings("unapproved_link")
+    page_counts = store.story_settings("page_count")
     show_holds = _link_books() and store.get_setting(None, "hold_new_links") and _can_upload()
     for s in stories:
         s["busy"] = job_queue.has_active(s["id"])
+        s["page_count"] = page_counts.get(s["id"])
         s["upload_held"] = bool(held.get(s["id"]))
         s["approval_held"] = bool(
             show_holds
@@ -379,13 +383,25 @@ def get_settings() -> dict[str, Any]:
         **_upload_state(),
         "linking_available": _linking_available(),
         "hold_new_links": bool(store.get_setting(None, "hold_new_links")),
+        "opds_enabled": bool(store.get_setting(None, "opds_enabled")),
         "personal_ini": ini.read_text(encoding="utf-8") if ini.exists() else "",
+        **{name: _download_setting(name) for name in DOWNLOAD_DEFAULTS},
         **job_list_settings(),
     }
 
 
 # The settings that save_settings takes.
-SETTINGS = ("recent_chapters", "personal_ini", "link_books", "hold_new_links", "pause_uploads", *JOB_LIST_DEFAULTS)
+SETTINGS = (
+    "recent_chapters",
+    "personal_ini",
+    "link_books",
+    "hold_new_links",
+    "pause_uploads",
+    "opds_enabled",
+    "download_template",
+    "download_date_format",
+    *JOB_LIST_DEFAULTS,
+)
 
 
 def save_settings(**changes: Any) -> dict[str, Any]:
@@ -401,6 +417,9 @@ def save_settings(**changes: Any) -> dict[str, Any]:
     link_books, personal_ini = changes.get("link_books"), changes.get("personal_ini")
     if link_books is not None and not isinstance(link_books, bool):
         raise ValueError("Link books to CWA library must be true or false")
+    opds_enabled = changes.get("opds_enabled")
+    if opds_enabled is not None and not isinstance(opds_enabled, bool):
+        raise ValueError("Enable OPDS must be true or false")
     numbers: dict[str, int] = {}
     if "recent_chapters" in changes:
         numbers["recent_chapters"] = whole_number(changes["recent_chapters"], "The number of recent chapters")
@@ -408,8 +427,20 @@ def save_settings(**changes: Any) -> dict[str, Any]:
         if name in changes:
             most = config.JOB_LIST_MAX_HOURS if name.endswith("hours") else config.JOB_LIST_MAX_COUNT
             numbers[name] = whole_number(changes[name], name, most)
+    texts: dict[str, str] = {}
+    for name, label, check in (
+        ("download_template", "The download template", filename_template.check),
+        ("download_date_format", "The date format", filename_template.check_date_format),
+    ):
+        if name in changes:
+            if not isinstance(changes[name], str):
+                raise ValueError(f"{label} must be text")
+            texts[name] = changes[name].strip()
+            check(texts[name])
     warnings = fff.check_personal_ini(personal_ini) if personal_ini is not None else []
 
+    if opds_enabled is not None:
+        store.set_setting(None, "opds_enabled", opds_enabled)
     released = False
     if link_books is not None and _linking_available():
         released = _link_books() != link_books
@@ -422,6 +453,9 @@ def save_settings(**changes: Any) -> dict[str, Any]:
         store.set_setting(None, name, True if value else None)
     for name, number in numbers.items():
         store.set_setting(None, name, number)
+    # An empty value restores the default.
+    for name, text in texts.items():
+        store.set_setting(None, name, text or None)
     if personal_ini is not None:
         # Write a new file, then replace the old one, so a job never reads half a file.
         partial = config.PERSONAL_INI.with_name("personal.ini.part")
@@ -623,15 +657,81 @@ def library_candidates() -> list[dict[str, Any]]:
     return [b.to_dict() for b in cwa().site_books(fff.is_supported_site) if b.id not in tracked_ids]
 
 
+def download_fields() -> dict[str, str]:
+    """The fields of the download template, with the help text for each."""
+    fields = dict(filename_template.FIELDS)
+    if config.CWA_ENABLED:
+        fields["id"] = "The CWA book ID. For a story with no library book, the Bindery story ID"
+    return fields
+
+
+DOWNLOAD_DEFAULTS = {
+    "download_template": filename_template.DEFAULT_TEMPLATE,
+    "download_date_format": filename_template.DEFAULT_DATE_FORMAT,
+}
+
+
+def _download_setting(name: str) -> str:
+    return store.get_setting(None, name) or DOWNLOAD_DEFAULTS[name]
+
+
+OPDS_PASSWORD_LENGTH = (8, 256)
+
+
+def _opds_password_name(username: str) -> str:
+    return f"opds_password_hash:{username.strip().casefold()}"
+
+
+def opds_password_hash(username: str) -> str | None:
+    """The hash of the OPDS password of a user in the proxy mode, or None."""
+    return store.get_setting(None, _opds_password_name(username))
+
+
+def set_opds_password(username: str, password: Any) -> None:
+    """Set the OPDS password of a user. Bindery keeps only the hash."""
+    least, most = OPDS_PASSWORD_LENGTH
+    if not isinstance(password, str) or not least <= len(password) <= most:
+        raise ValueError(f"The OPDS password must be text of {least} to {most} characters")
+    store.set_setting(None, _opds_password_name(username), generate_password_hash(password))
+
+
+def book_metadata(story: dict[str, Any]) -> dict[str, Any]:
+    """The metadata of the book of a story, with the stored title and author if the site gave none."""
+    meta = _book_metadata(story["site_values"], _epub_overrides(story["pinned"]))
+    meta["title"] = meta.get("title") or story["title"]
+    meta["authors"] = meta.get("authors") or [story["author"]]
+    return meta
+
+
+def _download_name(story: dict[str, Any]) -> str:
+    """The file name of a downloaded book, from the download template. If the template fails or gives no name for
+    this book, the name is the title."""
+    meta = book_metadata(story)
+    meta.update(publisher=story["site"], timestamp=story["created_at"], last_modified=story["last_changed"])
+    # Without CWA, the saved CWA book IDs do not apply.
+    book_id = story["calibre_id"] if config.CWA_ENABLED and story["calibre_id"] is not None else story["id"]
+    name = ""
+    try:
+        # {title} and {series} keep a leading article, as in Calibre with this value of its tweak
+        # save_template_title_series_sorting. {title_sort} moves it to the end.
+        date_format = _download_setting("download_date_format")
+        values = filename_template.prepare(meta, book_id, order="strictly_alphabetic", date_format=date_format)
+        name = filename_template.file_name(_download_setting("download_template"), values)
+    except filename_template.TemplateError as e:
+        logger.warning(f"The download template fails for {story['url']}: {e}")
+    return f"{name or sanitize_filename(story['title'])}.epub"
+
+
 def build_for_download(story_id: int) -> tuple[BinaryIO, str]:
-    """The epub of a story for a browser download, as an open file. Bindery builds it only if the book changed
+    """The epub of a story for a browser download, as an open file with its file name. Bindery builds it only if
-    after the last build."""
+    the book changed after the last build."""
     story = _require_story(story_id)
     overrides = _epub_overrides(story["pinned"])
     key = _build_key(story, overrides, story["site_values"])
+    name = _download_name(story)
     # The file is opened under the lock, so a new build cannot delete it first.
     with _build_lock:
-        return open(_built_epub(story, overrides, key, record=False), "rb"), story["title"]
+        return open(_built_epub(story, overrides, key, record=False), "rb"), name
 
 
 # Metadata locks
@@ -735,6 +835,14 @@ def _build_key(story: dict[str, Any], overrides: dict[str, Any], site: dict[str,
     return digest.hexdigest()
 
 
+def _count_pages(story_id: int, path: Path) -> None:
+    """Store the estimated pages of a built epub, for the story list and the OPDS catalog."""
+    try:
+        store.set_setting(story_id, "page_count", pages.epub_pages(path))
+    except EpubError as e:
+        logger.warning(f"Cannot count the pages of story {story_id}: {e}")
+
+
 def _built_epub(
     story: dict[str, Any], overrides: dict[str, Any], key: str, force: bool = False, record: bool = True
 ) -> Path:
@@ -744,7 +852,8 @@ def _built_epub(
     """
     path = config.TMP_DIR / f"story-{story['id']}-{key[:16]}.epub"
     with _build_lock:
-        if force or not path.exists():
+        built = force or not path.exists()
+        if built:
             # A build under another name first, so a download never reads half a book.
             partial = config.TMP_DIR / f"building-{uuid.uuid4().hex}.epub"
             try:
@@ -755,6 +864,9 @@ def _built_epub(
             for old in config.TMP_DIR.glob(f"story-{story['id']}-*.epub"):
                 if old != path:
                     old.unlink(missing_ok=True)
+        # A book from before the page count has no count yet.
+        if built or store.get_setting(story["id"], "page_count") is None:
+            _count_pages(story["id"], path)
         if record:
             store.set_setting(story["id"], "built_key", key)
     return path
@@ -1084,9 +1196,8 @@ def _run_rebuild(job: Job) -> str:
     force = bool(job.options.get("force"))
     if not force and key == _done_key(story["id"]):
         return "No changes"
-    message, changed = _send(story, overrides, key, story["title"], story["author"], force)
+    message, _ = _send(story, overrides, key, story["title"], story["author"], force)
-    changes: dict[str, Any] = {"last_changed": now()} if changed or force else {}
+    store.update_story(story["id"], pinned=pinned, status="ok", message=message)
-    store.update_story(story["id"], pinned=pinned, status="ok", message=message, **changes)
     return message
 
 
@@ -1338,9 +1449,7 @@ def _run_site_job(job: Job) -> str:
         return f"No changes. {summary}"
 
     _progress(job, "Building epub", 1.0)
-    message, book_changed = _send(story, overrides, key, title, snap.author)
+    message, _ = _send(story, overrides, key, title, snap.author)
-    if book_changed:
-        changes["last_changed"] = now()
     store.update_story(story_id, **changes, message=message, status="error" if failed else "ok")
     return f"{summary}. {message}"
 

bindery/pages.py

@@ -0,0 +1,78 @@
+"""An estimate of the pages of a built epub, after the "Paragraphs (APNX Accurate)" method of the Count Pages
+plugin for calibre.
+
+The method counts lines. A paragraph start is one line, and so is each 70 characters of text. A page is 31
+lines. A page is also at most 2400 characters, which sets the count of a book with almost no paragraphs.
+
+The plugin counts either the <p> or the <div> elements of a book as its paragraphs. Bindery counts what a
+reader sees as a paragraph: text that follows a <br> or the start or end of a block element. So the same text
+gives the same count as <p> elements, as <div> elements, and as lines with <br> between them. A paragraph with
+no text, or with only a no-break space, is not counted.
+"""
+
+import html as htmllib
+import re
+from pathlib import Path
+from typing import NamedTuple
+
+from bindery.epub import spine_documents
+
+LINE_CHARS = 70
+LINES_PER_PAGE = 31
+CHARS_PER_PAGE = 2400
+# Elements that stay in the line of the text around them. Each other element, and <br>, ends a paragraph.
+# The cells of a table row share a line.
+INLINE = frozenset(
+    "a abbr acronym b bdi bdo big cite code data del dfn em font i img ins kbd mark q rp rt ruby s samp small "
+    "span strike strong sub sup td th time tt u var wbr".split()
+)
+# Elements whose text is not text of the book.
+HIDDEN = frozenset(("head", "script", "style"))
+_TOKEN = re.compile(r"<(/?)([a-zA-Z][a-zA-Z0-9]*)[^>]*>|<[^>]*>|([^<]+)")
+
+
+class Measure(NamedTuple):
+    paragraphs: int = 0
+    chars: int = 0
+
+
+def measure(html: str) -> Measure:
+    """The paragraphs and the characters of text of one document."""
+    paragraphs = chars = 0
+    hidden = False
+    new_paragraph = True
+    for match in _TOKEN.finditer(html):
+        closing, name, text = match.groups()
+        if name:
+            name = name.lower()
+            if name in HIDDEN:
+                hidden = not closing and not match.group().endswith("/>")
+            elif name not in INLINE:
+                new_paragraph = True
+        elif text and not hidden:
+            if "&" in text:
+                text = htmllib.unescape(text)
+            # A no-break space is white space here, so a paragraph that only spaces the text is no paragraph.
+            if text.strip():
+                if new_paragraph:
+                    paragraphs += 1
+                    new_paragraph = False
+                    text = text.lstrip()
+                chars += len(text)
+            elif not new_paragraph:
+                chars += 1  # the space between two inline elements
+    return Measure(paragraphs, chars)
+
+
+def page_count(total: Measure) -> int:
+    """The pages of a book, from the sums of its documents. A book with no text has no pages."""
+    if not total.chars:
+        return 0
+    lines = total.paragraphs + total.chars // LINE_CHARS
+    return max(lines // LINES_PER_PAGE, total.chars // CHARS_PER_PAGE + 1)
+
+
+def epub_pages(path: Path) -> int:
+    """The estimated pages of an epub, from each document in its spine."""
+    measures = [measure(text) for text in spine_documents(path)]
+    return page_count(Measure(sum(m.paragraphs for m in measures), sum(m.chars for m in measures)))

bindery/static/js/rules.js

@@ -227,9 +227,9 @@ export function globalRulesCard() {
     .catch(showError);
 
   return h(
-    'section',
+    'details',
-    { class: 'card' },
+    { class: 'card', open: true },
-    h('h2', {}, 'Global replacement rules'),
+    h('summary', {}, h('h2', {}, 'Global replacement rules')),
     h(
       'p',
       { class: 'hint' },

bindery/static/js/settings.js

@@ -1,10 +1,26 @@
-// Settings: the recheck of recent chapters, the uploads to CWA, the global rules, and FanFicFare's personal.ini.
+// Settings: the recheck of recent chapters, the names of downloaded books, the uploads to CWA, the global rules,
+// and FanFicFare's personal.ini.
 
 import { api, button, checkbox, externalLink, fill, h, showError, toast } from './lib.js';
 import { globalRulesCard } from './rules.js';
 
 const INI_DOCS = 'https://github.com/JimmXinu/FanFicFare/wiki/INI-File';
 const DEFAULTS_INI = 'https://github.com/JimmXinu/FanFicFare/blob/main/fanficfare/defaults.ini';
+const TEMPLATE_DOCS = 'https://manual.calibre-ebook.com/template_lang.html';
+const DATE_CODES = 'https://docs.python.org/3/library/datetime.html#format-codes';
+const code = (text) => h('code', {}, text);
+// How a reader logs in to the catalog, for each login mode.
+const OPDS_LOGIN = {
+  none: 'Bindery asks for no login, so the catalog needs none.',
+  local: 'The reader sends your Bindery user name and password with HTTP Basic authentication.',
+  cwa: 'The reader sends your CWA user name and password with HTTP Basic authentication.',
+  proxy:
+    'The proxy skips its login for the catalog. Each user sets an OPDS password below. The reader sends that ' +
+    'password and the user name from the proxy with HTTP Basic authentication.',
+};
+// A card of settings that folds shut. It starts open.
+const card = (title, ...children) =>
+  h('details', { class: 'card', open: true }, h('summary', {}, h('h2', {}, title)), ...children);
 
 export function settingsPage(main, _params, config) {
   document.title = 'Settings · Bindery';
@@ -23,7 +39,7 @@ export function settingsPage(main, _params, config) {
   const linkBooks = checkbox(true, showLinkState);
   const pauseUploads = checkbox(false, () => {});
   const ingestNote = h('p', { class: 'notice', id: 'ingest-note', hidden: true },
-    'INGEST_DIR is not set. Uploads stay paused until you set it and restart Bindery.');
+    'INGEST_DIR is not set. Bindery sends no book to the ingest folder until you set it and restart Bindery.');
   const showUploadSettings = (settings) => {
     linkBooks.disabled = settings.linking_available === false;
     linkBooks.checked = !linkBooks.disabled && (settings.link_books ?? true);
@@ -35,6 +51,63 @@ export function settingsPage(main, _params, config) {
     pauseUploads.setAttribute('aria-describedby', pauseUploads.disabled ? 'ingest-note' : '');
     ingestNote.hidden = !pauseUploads.disabled;
   };
+  const opdsEnabled = checkbox(false, (checked) => showPassword(passwordSet, checked), {
+    id: 'opds-enabled',
+    'aria-describedby': 'opds-note',
+  });
+  let passwordSet = false;
+  const opdsUser = h('input', {
+    id: 'opds-user',
+    type: 'text',
+    readonly: true,
+    spellcheck: 'false',
+    autocomplete: 'off',
+    'aria-describedby': 'opds-password-status',
+  });
+  const opdsPassword = h('input', {
+    id: 'opds-password',
+    type: 'password',
+    minlength: 8,
+    maxlength: 256,
+    autocomplete: 'new-password',
+  });
+  const opdsPasswordStatus = h('p', { class: 'hint', id: 'opds-password-status' });
+  const setPassword = button('Set password', async () => {
+    await api('settings/opds-password', 'PUT', { password: opdsPassword.value });
+    opdsPassword.value = '';
+    showPassword(true, opdsEnabled.checked);
+    toast('Saved');
+  });
+  // The password applies only when the catalog is on, so the field is grey while the box is not checked.
+  const showPassword = (isSet, enabled) => {
+    passwordSet = isSet;
+    opdsPassword.disabled = !enabled;
+    setPassword.disabled = !enabled;
+    setPassword.textContent = isSet ? 'Reset password' : 'Set password';
+    fill(
+      opdsPasswordStatus,
+      isSet
+        ? 'You have an OPDS password. Bindery keeps only a hash of it. ' +
+          'Enter the user name and the password in your reader.'
+        : 'You have no OPDS password yet. Set one, then enter the user name and the password in your reader.',
+    );
+  };
+  const template = h('input', {
+    id: 'download-template',
+    class: 'template',
+    type: 'text',
+    maxlength: 1024,
+    spellcheck: 'false',
+    autocomplete: 'off',
+  });
+  const dateFormat = h('input', {
+    id: 'download-date-format',
+    class: 'template',
+    type: 'text',
+    maxlength: 100,
+    spellcheck: 'false',
+    autocomplete: 'off',
+  });
   const hoursField = (id) => h('input', { id, type: 'number', min: 0, max: config.job_list_max_hours, step: 1 });
   const countField = (id) => h('input', { id, type: 'number', min: 0, max: config.job_list_max_count, step: 1 });
   const storyJobsHours = hoursField('story-jobs-hours');
@@ -73,6 +146,11 @@ export function settingsPage(main, _params, config) {
       recent.value = settings.recent_chapters;
       showUploadSettings(settings);
       ini.value = settings.personal_ini;
+      opdsEnabled.checked = settings.opds_enabled ?? false;
+      opdsUser.value = settings.opds_user ?? '';
+      showPassword(settings.opds_password_set ?? false, opdsEnabled.checked);
+      template.value = settings.download_template;
+      dateFormat.value = settings.download_date_format;
       storyJobsHours.value = settings.story_jobs_hours;
       storyJobsCount.value = settings.story_jobs_count;
       mainJobsHours.value = settings.main_jobs_hours;
@@ -82,11 +160,9 @@ export function settingsPage(main, _params, config) {
 
   fill(
     main,
-    h(
-      'section',
-      { class: 'card' },
     h('h1', {}, 'Settings'),
-      h('h2', {}, 'Recheck recent chapters'),
+    card(
+      'Recheck recent chapters',
       h(
         'div',
         { class: 'controls' },
@@ -102,10 +178,142 @@ export function settingsPage(main, _params, config) {
       ),
       saveButton(() => ({ recent_chapters: recent.value === '' ? 0 : Number(recent.value) })),
     ),
-    config.cwa_enabled && h(
+    card(
-      'section',
+      'Book Downloads',
-      { class: 'card' },
+      h('label', {}, opdsEnabled, ' Enable OPDS'),
-      h('h2', {}, 'Uploads to CWA'),
+      h(
+        'p',
+        { class: 'hint', id: 'opds-note' },
+        'OPDS lets a reader browse and download books. It is off by default. After you enable it and save, use ',
+        h('a', { href: 'opds' }, 'the OPDS catalog'),
+        '. ',
+        OPDS_LOGIN[config.auth_mode] ?? OPDS_LOGIN.local,
+        ' Use HTTPS outside a trusted local network.',
+      ),
+      config.auth_mode === 'proxy' && [
+        h(
+          'div',
+          { class: 'credentials' },
+          h('label', { for: 'opds-user' }, 'OPDS user name'),
+          opdsUser,
+          h('label', { for: 'opds-password' }, 'OPDS password'),
+          h('div', { class: 'controls' }, opdsPassword, setPassword),
+        ),
+        opdsPasswordStatus,
+      ],
+      h('div', { class: 'controls' }, h('label', { for: 'download-template' }, 'File name template'), template),
+      h(
+        'p',
+        { class: 'hint' },
+        'This template gives the file name of a book that you download from a story page or OPDS. ' +
+          'To go back to the default, ',
+        code('{title_sort} - {authors}'),
+        ', clear the field and save.',
+      ),
+      h(
+        'details',
+        { class: 'help' },
+        h('summary', {}, 'File name template help'),
+        h(
+          'p',
+          { class: 'hint' },
+          'The template follows the ',
+          externalLink(TEMPLATE_DOCS, 'template rules of Calibre'),
+          ' for file names. Bindery replaces each field in braces with the value of the book, and adds .epub. ' +
+            'A field with no value becomes empty text. For example, ',
+          code('{series:|| - }{series_index:0>2s|| - }{title} - {authors}'),
+          ' gives "The Saga - 02 - The Book - Ann Writer.epub".',
+        ),
+        h('p', { class: 'hint' }, 'You can use these fields:'),
+        h(
+          'dl',
+          { class: 'fields' },
+          Object.entries(config.download_fields).map(([name, text]) => [h('dt', {}, code(`{${name}}`)), h('dd', {}, text)]),
+        ),
+        h(
+          'ul',
+          { class: 'hint' },
+          h('li', {}, code('{title:.20s}'), ' gives the first 20 characters of the title.'),
+          h('li', {}, code('{author_sort[0]}'), ' gives the first character of the author sort string.'),
+          h(
+            'li',
+            {},
+            code('{series:|| - }'),
+            ' adds " - " after the series, but only for a book with a series. The form is ',
+            code('{field:format|text before|text after}'),
+            '.',
+          ),
+          h('li', {}, code('{{'), ' and ', code('}}'), ' give braces.'),
+          h(
+            'li',
+            {},
+            '{title} and {series} keep a leading "The", "A" or "An". {title_sort} and {series_sort} move it to the ' +
+              'end, so ' +
+              '"The Book" becomes "Book, The". Calibre puts the title sort in {title} as well, unless you set its ' +
+              'tweak save_template_title_series_sorting to strictly_alphabetic.',
+          ),
+          h(
+            'li',
+            {},
+            'A download has one file name. If the template has folders, Bindery uses the part after the last ',
+            code('/'),
+            '.',
+          ),
+          h(
+            'li',
+            {},
+            'Bindery does not support template functions, such as ',
+            code('{title:uppercase()}'),
+            ', or the program modes of Calibre.',
+          ),
+          h('li', {}, 'If the template fails or gives no name for a book, Bindery names the file after the title.'),
+        ),
+      ),
+      h('div', { class: 'controls' }, h('label', { for: 'download-date-format' }, 'Date format'), dateFormat),
+      h(
+        'p',
+        { class: 'hint' },
+        'The form of template dates in {pubdate}, {timestamp} and {last_modified}. To go back to the default, ',
+        code('%b, %Y'),
+        ', clear the field and save.',
+      ),
+      h(
+        'details',
+        { class: 'help' },
+        h('summary', {}, 'Date format help'),
+        h(
+          'p',
+          { class: 'hint' },
+          code('%d'),
+          ' gives the day, ',
+          code('%b'),
+          ' the month name, ',
+          code('%m'),
+          ' the month number and ',
+          code('%Y'),
+          ' the year. See the ',
+          externalLink(DATE_CODES, 'list of all codes'),
+          '. The default gives "May, 2020". A / in a date becomes _, as in Calibre.',
+        ),
+      ),
+      ingestNote,
+      saveButton(
+        () => ({
+          opds_enabled: opdsEnabled.checked,
+          download_template: template.value,
+          download_date_format: dateFormat.value,
+        }),
+        (settings) => {
+          opdsEnabled.checked = settings.opds_enabled ?? false;
+          showPassword(passwordSet, opdsEnabled.checked);
+          template.value = settings.download_template;
+          dateFormat.value = settings.download_date_format;
+          toast('Saved');
+        },
+      ),
+    ),
+    config.cwa_enabled && card(
+      'Uploads to CWA',
       h('label', {}, linkBooks, ' Link books to CWA library'),
       h(
         'p',
@@ -124,7 +332,6 @@ export function settingsPage(main, _params, config) {
           'Later updates of an approved book go to CWA as usual.',
       ),
       h('label', {}, pauseUploads, ' Pause all uploads'),
-      ingestNote,
       h(
         'p',
         { class: 'hint' },
@@ -139,10 +346,8 @@ export function settingsPage(main, _params, config) {
         },
       ),
     ),
-    h(
+    card(
-      'section',
+      'Jobs list',
-      { class: 'card' },
-      h('h2', {}, 'Jobs list'),
       h(
         'div',
         { class: 'controls' },
@@ -175,10 +380,8 @@ export function settingsPage(main, _params, config) {
       })),
     ),
     globalRulesCard(),
-    h(
+    card(
-      'section',
+      'FanFicFare settings (personal.ini)',
-      { class: 'card' },
-      h('h2', {}, 'FanFicFare settings (personal.ini)'),
       h(
         'p',
         { class: 'hint' },

bindery/static/js/stories.js

@@ -9,6 +9,7 @@ const STORY_COLUMNS = [
   { label: 'Author', key: 'author', type: 'text' },
   { label: 'Site', key: 'site', type: 'text' },
   { label: 'Chapters', key: 'chapter_count', type: 'number' },
+  { label: 'Pages', key: 'page_count', type: 'number' },
   { label: 'Status', key: 'status', type: 'text' },
   { label: 'Last change', key: 'last_changed', type: 'date' },
   { label: 'Last check', key: 'last_checked', type: 'date' },
@@ -21,11 +22,14 @@ function libraryImport(config) {
   let candidates = [];
   let selected = new Set();
   let check = true;
+  let empty = '';
+  // The load of the list, kept for each search, because CWA takes a while to send the list.
+  let list;
   const results = h('div');
 
   const render = () => {
     if (!candidates.length) {
-      fill(results, h('p', { class: 'empty' }, 'Every library book from a story site is tracked already.'));
+      fill(results, h('p', { class: 'empty' }, empty));
       return;
     }
     const toggle = (id, on) => {
@@ -81,9 +85,14 @@ function libraryImport(config) {
           const books = candidates
             .filter((c) => selected.has(c.id))
             .map(({ id, uuid, title }) => ({ id, uuid, title }));
-          if (!books.length) return;
+          if (!books.length) {
+            toast('Select books first', true);
+            return;
+          }
           const jobs = await api('library/import', 'POST', { books, check });
           toast(`Queued ${jobs.length} imports`);
+          const queued = new Set(books.map((b) => b.id));
+          list = list?.then((all) => all.filter((c) => !queued.has(c.id)));
           fill(results);
           refreshJobs();
         },
@@ -92,22 +101,68 @@ function libraryImport(config) {
     );
   };
 
+  // The full list starts with each book selected. A search starts with no book selected.
+  const show = (books, all) => {
+    candidates = books;
+    selected = new Set(all ? books.map((c) => c.id) : []);
+    empty = all
+      ? 'Every library book from a story site is tracked already.'
+      : 'No library books from a story site match. A book that Bindery tracks already does not show.';
+    render();
+  };
+  const query = h('input', {
+    type: 'search',
+    placeholder: 'Search the library by title, author or site',
+    'aria-label': 'Search the library',
+  });
+  const load = () => {
+    list ??= api('library/candidates').catch((e) => {
+      list = undefined;
+      throw e;
+    });
+    return list;
+  };
+  // A book matches if its title, authors and site hold each word of the search.
+  const search = async () => {
+    const words = query.value.toLowerCase().split(/\s+/).filter(Boolean);
+    if (!words.length) return;
+    const books = await load();
+    show(books.filter((c) => words.every((w) => [c.title, ...c.authors, c.publisher].join(' ').toLowerCase().includes(w))), false);
+  };
+  const submit = h('button', { type: 'submit' }, 'Search');
+
   return h(
     'div',
     {},
     h(
       'p',
       { class: 'hint' },
-      'Finds library books whose publisher is a story site, for example books from the FanFicFare calibre plugin. ' +
+      'Search the library, or show all the stories in it. Both find only books whose publisher is a story site, ' +
+        'for example books from the FanFicFare calibre plugin, and that Bindery does not track yet. ' +
         'Importing downloads each ePub from CWA and keeps its chapters, including chapters the site no longer has.',
     ),
+    h(
+      'form',
+      {
+        class: 'library-search',
+        onsubmit: (e) => {
+          e.preventDefault();
+          if (submit.disabled) return;
+          submit.disabled = true;
+          search()
+            .catch(showError)
+            .finally(() => (submit.disabled = false));
+        },
+      },
+      query,
+      submit,
+    ),
     h(
       'div',
       { class: 'controls' },
-      button('Find stories in the library', async () => {
+      button('Show all stories in the library', async () => {
-        candidates = await api('library/candidates');
+        list = undefined;
-        selected = new Set(candidates.map((c) => c.id));
+        show(await load(), true);
-        render();
       }),
       h('label', {}, checkbox(true, (on) => (check = on)), ' Check the site for new chapters after import'),
     ),
@@ -199,6 +254,8 @@ function storyRow(s) {
       s.offsite_count > 0 && h('small', { title: 'Chapters the site no longer lists' }, `${s.offsite_count} removed`),
       s.revised_count > 0 && h('small', { title: 'Chapters the site changed since Bindery added them' }, `${s.revised_count} revised`),
     ),
+    // A story has no page count until Bindery builds its book.
+    h('td', { 'data-label': 'Pages', title: 'Estimated from the built book' }, s.page_count != null && String(s.page_count)),
     h(
       'td',
       { 'data-label': 'Status' },
@@ -244,6 +301,8 @@ export function storiesPage(main, _params, config) {
         const order = (left - right) * direction;
         if (order) return order;
       } else if (sort !== 'title') {
+        // A story with no number goes last in each direction.
+        if (sortType === 'number' && (a[sort] == null) !== (b[sort] == null)) return a[sort] == null ? 1 : -1;
         const order = sortType === 'number'
           ? a[sort] - b[sort]
           : (a[sort] || '').localeCompare(b[sort] || '', undefined, { sensitivity: 'base' });

bindery/static/js/story.js

@@ -68,7 +68,7 @@ const table = (headings, rows, extra = '') =>
 export function storyPage(main, { storyId }, config) {
   const id = Number(storyId);
   const fields = config.metadata_fields;
-  // With CWA_ENABLED=false, the page shows nothing about CWA.
+  // Without CWA_ENABLED=true, the page shows nothing about CWA.
   const cwa = config.cwa_enabled;
   const libraryStays = cwa ? ' The library book stays.' : '';
   let details = null;

bindery/static/style.css

@@ -451,6 +451,25 @@ label:has(> input[type='checkbox']:disabled) {
   margin: 0;
 }
 
+ul.hint {
+  padding-left: 20px;
+}
+
+/* The fields of the download template: each name with its meaning beside it. */
+.fields {
+  display: grid;
+  grid-template-columns: max-content 1fr;
+  gap: 2px 12px;
+  max-width: 80ch;
+  margin: 4px 0 8px;
+  color: var(--muted);
+  font-size: var(--text-sm);
+
+  dd {
+    margin: 0;
+  }
+}
+
 .notice {
   margin: 12px 0;
   padding: 10px 14px;
@@ -556,6 +575,11 @@ button.danger:hover:not(:disabled) {
   background: var(--err-bg);
 }
 
+input:not([type='checkbox'], [type='radio']):disabled {
+  opacity: 0.55;
+  cursor: not-allowed;
+}
+
 button:disabled {
   opacity: 0.55;
   cursor: default;
@@ -699,6 +723,35 @@ textarea {
   line-height: 1.45;
 }
 
+/* A user name over a password, with the labels in one column. */
+.credentials {
+  display: grid;
+  grid-template-columns: max-content minmax(0, 1fr);
+  align-items: center;
+  gap: 8px 12px;
+  max-width: 48em;
+  margin-top: 8px;
+
+  input {
+    width: 18em;
+  }
+
+  input[readonly] {
+    background: var(--card-bg);
+    font-family: var(--mono);
+    cursor: default;
+  }
+
+  .controls {
+    margin-top: 0;
+  }
+}
+
+.template {
+  flex: 1 1 24em;
+  max-width: 48em;
+}
+
 .metadata input:not([type='checkbox']) {
   width: 100%;
   max-width: 32em;
@@ -822,18 +875,45 @@ textarea {
     display: block;
   }
 
+  /* The column headings become a row of sort chips. The chip of the sorted column is tinted and
+     shows the direction. The other chips show no arrow. */
   .stories-table thead tr {
     display: flex;
     flex-wrap: wrap;
-    gap: 8px 16px;
+    gap: 6px;
-    margin-bottom: 8px;
+    margin-bottom: 10px;
+  }
+
+  .stories-table thead tr::before {
+    content: 'Sort by';
+    flex-basis: 100%;
+    color: var(--muted);
+    font-size: var(--text-sm);
   }
 
   .stories-table th {
-    padding: 4px 0;
+    padding: 0;
     border: 0;
   }
 
+  .stories-table th button {
+    inline-size: auto;
+    padding: 3px 10px;
+    border: 1px solid var(--border);
+    border-radius: 999px;
+    background: var(--surface);
+    font-size: var(--text-sm);
+  }
+
+  .stories-table th[aria-sort='none'] button::after {
+    display: none;
+  }
+
+  .stories-table th:not([aria-sort='none']) button {
+    background: var(--info-bg);
+    color: var(--info);
+  }
+
   .stories-table tbody tr,
   .chapters-table tbody tr {
     border: 1px solid var(--border);
@@ -860,6 +940,11 @@ textarea {
     color: var(--muted);
   }
 
+  /* A story with no page count yet shows no line for it. */
+  .stories-table td:empty {
+    display: none;
+  }
+
   /* The chapter list has far more rows than the story list, so each row packs its cells onto
      three lines instead of one line per cell: checkbox and number, title, then the rest. */
   .chapters-table tbody tr {
@@ -1039,6 +1124,16 @@ textarea {
   margin-top: 12px;
 }
 
+/* Help that folds shut under a setting, such as the help for the download template. */
+.help {
+  margin: 0 0 12px;
+
+  > summary {
+    color: var(--muted);
+    font-size: var(--text-sm);
+  }
+}
+
 /* Toasts */
 
 #toasts {

bindery/store.py

@@ -204,6 +204,18 @@ class Store:
             stories = [dict(_story(r), sort_title=sort_title(r["title"] or r["url"])) for r in rows]
             return sorted(stories, key=lambda s: s["sort_title"])
 
+    def catalog(self) -> list[dict[str, Any]]:
+        """The stories with included chapters, with the values of the OPDS catalog. A story has one cover at most,
+        and `cover_key` changes with it."""
+        with self._connect() as conn:
+            rows = conn.execute("""
+                SELECT s.id, s.url, s.site, s.title, s.author, s.pinned, s.site_values, s.last_changed, s.created_at,
+                       (SELECT COUNT(*) FROM chapters c WHERE c.story_id = s.id AND c.included = 1) AS chapter_count,
+                       i.mime AS cover_mime, i.url || ' ' || length(i.data) AS cover_key
+                FROM stories s LEFT JOIN images i ON i.story_id = s.id AND i.is_cover = 1
+                WHERE EXISTS (SELECT 1 FROM chapters c WHERE c.story_id = s.id AND c.included = 1)""").fetchall()
+            return [_story(r) for r in rows]
+
     def update_story(self, story_id: int, **fields: Any) -> None:
         if not fields:
             return
@@ -261,15 +273,17 @@ class Store:
         conn: sqlite3.Connection, story_id: int, chapter: dict[str, str], source: str, position: int
     ) -> int:
         """Store a chapter with its first revision. The chapter needs url, title, html and text_hash."""
+        changed_at = now()
         cur = conn.execute(
             "INSERT INTO chapters (story_id, position, url, title, added_at) VALUES (?, ?, ?, ?, ?)",
-            (story_id, position, chapter["url"], chapter["title"], now()),
+            (story_id, position, chapter["url"], chapter["title"], changed_at),
         )
         chapter_id = int(cur.lastrowid)
         conn.execute(
             "INSERT INTO revisions (chapter_id, rev, html, text_hash, source, fetched_at) VALUES (?, 1, ?, ?, ?, ?)",
-            (chapter_id, chapter["html"], chapter["text_hash"], source, now()),
+            (chapter_id, chapter["html"], chapter["text_hash"], source, changed_at),
         )
+        conn.execute("UPDATE stories SET last_changed = ? WHERE id = ?", (changed_at, story_id))
         return chapter_id
 
     def add_chapter(
@@ -314,6 +328,7 @@ class Store:
 
     def add_revision(self, chapter_id: int, html: str, text_hash: str, source: str) -> int:
         """Store a new revision and make it the current one."""
+        changed_at = now()
         with self._connect() as conn:
             rev = conn.execute(
                 "SELECT COALESCE(MAX(rev), 0) + 1 FROM revisions WHERE chapter_id = ?", (chapter_id,)
@@ -321,9 +336,15 @@ class Store:
             conn.execute(
                 "INSERT INTO revisions (chapter_id, rev, html, text_hash, source, fetched_at) "
                 "VALUES (?, ?, ?, ?, ?, ?)",
-                (chapter_id, rev, html, text_hash, source, now()),
+                (chapter_id, rev, html, text_hash, source, changed_at),
+            )
+            conn.execute(
+                "UPDATE chapters SET current_rev = ?, changed_at = ? WHERE id = ?", (rev, changed_at, chapter_id)
+            )
+            conn.execute(
+                "UPDATE stories SET last_changed = ? WHERE id = (SELECT story_id FROM chapters WHERE id = ?)",
+                (changed_at, chapter_id),
             )
-            conn.execute("UPDATE chapters SET current_rev = ?, changed_at = ? WHERE id = ?", (rev, now(), chapter_id))
             return int(rev)
 
     def update_chapter(self, chapter_id: int, **fields: Any) -> None:

bindery/web.py

@@ -2,7 +2,6 @@
 
 import base64
 import hashlib
-import io
 import logging
 import os
 import re
@@ -13,14 +12,13 @@ from datetime import timedelta
 from pathlib import Path
 from typing import Any
 
-from flask import Flask, Response, g, jsonify, request, send_file, session
+from flask import Flask, Response, g, jsonify, request, session
 from werkzeug.exceptions import Forbidden, HTTPException
 
-from bindery import auth, config, orchestrator
+from bindery import auth, config, opds, orchestrator
 from bindery.cwa import CwaError
 from bindery.epub import EpubError
 from bindery.fff import StoryError
-from bindery.ingest import sanitize_filename
 from bindery.jobs import METADATA_FIELDS, MODE_LABELS, DownloadMode
 from bindery.rules import RuleError
 
@@ -70,6 +68,7 @@ def _secret_key() -> bytes:
 config.prepare()
 
 app = Flask(__name__)
+app.register_blueprint(opds.bp)
 app.config.update(
     SECRET_KEY=_secret_key(),
     SEND_FILE_MAX_AGE_DEFAULT=0,
@@ -149,13 +148,27 @@ def check_host() -> Any:
     return jsonify({"error": f"Bindery does not answer to the host name {request.host}. Add it to ALLOWED_HOSTS."}), 400
 
 
+def _catalog_login() -> Any:
+    """The login of the OPDS catalog. It takes no API key. See `auth.catalog_login`."""
+    if not orchestrator.store.get_setting(None, "opds_enabled"):
+        return jsonify({"error": "Not found"}), 404
+    address = auth.client_address(request.headers, request.remote_addr)
+    if auth.catalog_login(request.headers, session, address, orchestrator.opds_password_hash, orchestrator.cwa):
+        return None
+    # The header makes a reader ask for the user name and password.
+    return jsonify({"error": "Unauthorized"}), 401, {"WWW-Authenticate": 'Basic realm="Bindery", charset="UTF-8"'}
+
+
 @app.before_request
 def check_login() -> Any:
-    if not request.path.startswith("/api/") or request.path == "/api/health":
+    catalog = request.path.startswith("/opds/") or request.path == "/opds"
+    if not (catalog or request.path.startswith("/api/")) or request.path == "/api/health":
         return None
     error = auth.config_error()
     if error:
         return jsonify({"error": f"{error}. Fix the setting and restart Bindery."}), 503
+    if catalog:
+        return _catalog_login()
     identity = _identity()
     if (
         request.method in MUTATING
@@ -269,6 +282,8 @@ def app_config() -> Any:
         "update_interval": config.UPDATE_INTERVAL_HOURS,
         "job_list_max_hours": config.JOB_LIST_MAX_HOURS,
         "job_list_max_count": config.JOB_LIST_MAX_COUNT,
+        "download_fields": orchestrator.download_fields(),
+        "auth_mode": config.AUTH_METHOD,
         **orchestrator.job_list_settings(),
     }
 
@@ -278,13 +293,31 @@ def settings() -> Any:
     if request.method == "PUT":
         _admin_only()
         data = _json()
-        return orchestrator.save_settings(**{name: data.get(name) for name in orchestrator.SETTINGS})
+        return _with_opds_user(orchestrator.save_settings(**{name: data.get(name) for name in orchestrator.SETTINGS}))
-    settings = orchestrator.get_settings()
+    settings = _with_opds_user(orchestrator.get_settings())
     if not auth.is_admin(g.identity):
         del settings["personal_ini"]  # it can hold passwords for story sites
     return settings
 
 
+def _with_opds_user(settings: dict[str, Any]) -> dict[str, Any]:
+    """Adds the user name that the catalog expects from this user, and whether this user has an OPDS password."""
+    user = g.identity.name if config.AUTH_METHOD == auth.AUTH_PROXY and g.identity.via == "proxy" else None
+    return {**settings, "opds_user": user, "opds_password_set": bool(user and orchestrator.opds_password_hash(user))}
+
+
+@app.put("/api/settings/opds-password")
+def opds_password() -> Any:
+    """Each user in the proxy mode sets their own OPDS password. The user name is the one from the proxy."""
+    identity = g.identity
+    if config.AUTH_METHOD != auth.AUTH_PROXY or identity.via != "proxy" or not identity.name:
+        raise ValueError("The OPDS password is for users who log in through the proxy")
+    orchestrator.set_opds_password(identity.name, _json().get("password"))
+    # Tries before the password existed must not lock the user out of the new password.
+    auth.limiter.forget(identity.name)
+    return {"opds_user": identity.name, "opds_password_set": True}
+
+
 # Jobs
 
 
@@ -399,24 +432,12 @@ def delete_chapters(story_id: int) -> Any:
 
 @app.get("/api/stories/<int:story_id>/epub")
 def story_epub(story_id: int) -> Any:
-    file, title = orchestrator.build_for_download(story_id)
+    return opds.send_epub(story_id)
-    name = f"{sanitize_filename(title)}.epub"
-    response = send_file(file, download_name=name, as_attachment=True, mimetype="application/epub+zip")
-    response.content_length = os.fstat(file.fileno()).st_size
-    return response
 
 
 @app.get("/api/stories/<int:story_id>/cover")
 def story_cover(story_id: int) -> Any:
-    cover = orchestrator.story_cover(story_id)
+    return opds.send_cover(story_id)
-    if not cover:
-        raise KeyError("This story has no cover")
-    # The page puts the cover version in the URL, so the browser can keep the image for a year.
-    response = send_file(io.BytesIO(cover["data"]), mimetype=cover["mime"], max_age=365 * 24 * 60 * 60)
-    # A cover needs a login, so a shared cache must not keep it.
-    response.cache_control.public = False
-    response.cache_control.private = True
-    return response
 
 
 # Replacement rules. Story rules run after the global rules.

docker-compose.yml

@@ -2,26 +2,14 @@ services:
   bindery:
     build: .
     image: bindery:latest
-    # The user and group that own ./config and can write CWA's ingest folder. Usually the same as CWA's.
+    # The user and group that own ./config.
     user: "1000:1000"
     environment:
-      # Without CWA: set this, and remove the CWA values below. INGEST_DIR then gets the built ePubs.
-      # CWA_ENABLED: "false"
-      # Omit this and its volume mount to keep uploads paused.
-      INGEST_DIR: /cwa-book-ingest
-      # The address of CWA, as the Bindery container reaches it, and the CWA account that Bindery uses.
-      # Leave the account out if CWA asks for no login.
-      CWA_URL: http://calibre-web-automated:8083
-      CWA_USERNAME: bindery
-      CWA_PASSWORD: change-me
       # none, local (one Bindery account), cwa (CWA accounts), or proxy (a trusted user header)
-      AUTH_METHOD: cwa
+      AUTH_METHOD: none
       # For local logins, put these values in .env. See "Local login" in the README.
       # BINDERY_USERNAME: ${BINDERY_USERNAME:-}
       # BINDERY_PASSWORD_HASH: ${BINDERY_PASSWORD_HASH:-}
-      # Optional and independent. app.db is for logins, metadata.db is for book IDs.
-      # CWA_DB_PATH: /cwa-config/app.db
-      # CALIBRE_DB_PATH: /calibre-library/metadata.db
       # URL_BASE: /bindery
       # The host names that Bindery answers to. AUTH_METHOD=none needs this for a public name such as this one.
       # ALLOWED_HOSTS: bindery.example.com
@@ -32,6 +20,19 @@ services:
       # For the cwa and proxy modes: the users that can change the global settings. Without a list, each user can.
       # BINDERY_ADMIN_USERS: alice, bob
       # BINDERY_API_KEY: a-long-random-string
+      # Optional: a folder that gets each built ePub. With CWA, this is CWA's ingest folder.
+      # INGEST_DIR: /cwa-book-ingest
+      # Optional: Calibre-Web-Automated. The user above must then be able to write CWA's ingest folder.
+      # Usually it is the same as CWA's.
+      # CWA_ENABLED: "true"
+      # The address of CWA, as the Bindery container reaches it, and the CWA account that Bindery uses.
+      # Leave the account out if CWA asks for no login.
+      # CWA_URL: http://calibre-web-automated:8083
+      # CWA_USERNAME: bindery
+      # CWA_PASSWORD: change-me
+      # Optional and independent. app.db is for logins, metadata.db is for book IDs.
+      # CWA_DB_PATH: /cwa-config/app.db
+      # CALIBRE_DB_PATH: /calibre-library/metadata.db
     ports:
       # If a reverse proxy on this host is the only way in, publish the port only on the host: 127.0.0.1:8085:8085
       - 8085:8085
@@ -44,11 +45,11 @@ services:
       start_period: 30s
       retries: 3
     volumes:
-      # CWA's ingest folder. New stories and updates go here. Bindery does not make the folder.
-      - /path/to/cwa/ingest:/cwa-book-ingest
       # The story database and personal.ini. Create the folder before the first start.
       - ./config:/config
-      # Optional read-only folders. Include SQLite journal files beside each database.
+      # Optional: the folder of INGEST_DIR. With CWA, CWA's ingest folder. Bindery does not make the folder.
+      # - /path/to/cwa/ingest:/cwa-book-ingest
+      # Optional read-only folders of CWA. Include SQLite journal files beside each database.
       # Mount the library that receives the ingest files, not a copy or another library.
       # - /path/to/cwa/config:/cwa-config:ro
       # - /path/to/calibre/library:/calibre-library:ro

tests/conftest.py

@@ -10,6 +10,7 @@ for name, sub in [("CONFIG_DIR", "config"), ("INGEST_DIR", "ingest"), ("TMP_DIR"
 (_tmp / "ingest").mkdir()  # Bindery does not make the ingest folder
 os.environ["BINDERY_START_THREADS"] = "false"
 os.environ["UPDATE_INTERVAL_HOURS"] = "0"
+os.environ["CWA_ENABLED"] = "true"  # most tests are about a Bindery with CWA
 for name in (
     "CWA_URL",
     "CWA_USERNAME",

tests/js/admin.test.mjs

@@ -9,7 +9,7 @@ test('a user who is not an admin gets no settings link and no settings page', as
   t.after(stopPolling);
   const backend = server(t);
   backend.on('GET', 'auth/check', { authenticated: true, mode: 'proxy', user: 'bob', admin: false });
-  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false });
+  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false, download_fields: {} });
   backend.on('GET', 'jobs', []);
   backend.on('GET', 'stories', []);
   await import('../../bindery/static/js/main.js');

tests/js/auth-flows.test.mjs

@@ -11,7 +11,7 @@ test('login retry, navigation, session expiry and logout preserve the authentica
   const backend = server(t);
   let auth = { authenticated: false, mode: 'cwa', user: null };
   backend.on('GET', 'auth/check', () => auth);
-  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false });
+  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false, download_fields: {} });
   backend.on('GET', 'jobs', []);
   backend.on('GET', 'stories', []);
   backend.on('GET', 'rules', { rules: [], paragraphs: false });

tests/js/flows.test.mjs

@@ -16,6 +16,7 @@ const config = {
   metadata_fields: { title: 'Title', authors: 'Authors', comments: 'Description', series: 'Series', pubdate: 'Published', tags: 'Tags', languages: 'Language' },
   mode_labels: { new: 'New chapters' }, library_available: true, cwa_enabled: true, update_interval: 2,
   job_list_max_hours: 720, job_list_max_count: 500,
+  download_fields: { title: 'The title', id: 'The story ID' },
 };
 const rule = { pattern: 'before', replacement: 'after', flags: 'i', note: '', enabled: true };
 const emptyRules = { rules: [], paragraphs: null, global_paragraphs: true };
@@ -54,7 +55,7 @@ for (const value of ['7', '']) {
     await settle();
     api.on('PUT', 'settings', {});
     field(main, 'recent-chapters').value = value;
-    await click(section(main, 'Settings'), 'Save');
+    await click(section(main, 'Recheck recent chapters'), 'Save');
     assert.deepEqual(api.sent('PUT', 'settings')[0].body, { recent_chapters: value === '' ? 0 : 7 });
     assert.match(ui.toasts.textContent, /Saved/);
   });
@@ -108,6 +109,52 @@ test('the jobs list settings load and save all four fields', async () => {
   assert.match(ui.toasts.textContent, /Saved/);
 });
 
+test('each group of settings is an open card, and the download help starts closed', async () => {
+  settingsPage(main, {}, config);
+  await settle();
+  const cards = main.children.filter((el) => el.tagName !== 'h1');
+  assert.deepEqual(cards.map((el) => [el.tagName, el.className, el.open, el.children[0].textContent]), [
+    ['details', 'card', true, 'Recheck recent chapters'],
+    ['details', 'card', true, 'Book Downloads'],
+    ['details', 'card', true, 'Uploads to CWA'],
+    ['details', 'card', true, 'Jobs list'],
+    ['details', 'card', true, 'Global replacement rules'],
+    ['details', 'card', true, 'FanFicFare settings (personal.ini)'],
+  ]);
+  const help = nodes(section(main, 'Book Downloads')).filter((el) => el.className === 'help');
+  assert.deepEqual(help.map((el) => [el.tagName, el.open, el.children[0].textContent]), [
+    ['details', undefined, 'File name template help'],
+    ['details', undefined, 'Date format help'],
+  ]);
+});
+
+test('the download template loads, lists its fields, and saves the stored form', async () => {
+  api.on('GET', 'settings', {
+    recent_chapters: 0, personal_ini: '', download_template: '{title}', download_date_format: '%b, %Y',
+  });
+  settingsPage(main, {}, config);
+  await settle();
+  const card = section(main, 'Book Downloads');
+  assert.equal(field(main, 'download-template').value, '{title}');
+  assert.equal(field(main, 'download-date-format').value, '%b, %Y');
+  assert.ok(find(card, 'code', '{id}'));
+  assert.match(card.textContent, /\{title\}The title\{id\}The story ID/);
+  field(main, 'download-template').value = ' {title} - {authors} ';
+  field(main, 'download-date-format').value = '';
+  api.on('PUT', 'settings', { download_template: '{title} - {authors}', download_date_format: '%b, %Y' });
+  await click(card, 'Save');
+  assert.deepEqual(api.sent('PUT', 'settings')[0].body, {
+    opds_enabled: false, download_template: ' {title} - {authors} ', download_date_format: '',
+  });
+  assert.equal(field(main, 'download-template').value, '{title} - {authors}');
+  assert.equal(field(main, 'download-date-format').value, '%b, %Y');
+  api.on('PUT', 'settings', { error: 'Bindery does not support template functions' }, 400);
+  field(main, 'download-template').value = '{title:uppercase()}';
+  await click(card, 'Save');
+  assert.equal(field(main, 'download-template').value, '{title:uppercase()}');
+  assert.match(ui.toasts.textContent, /does not support template functions/);
+});
+
 test('adding stories keeps input on rejection and clears it after a mixed result', async () => {
   await mountStories({ ...config, library_available: false });
   const urls = nodes(main).find((el) => el.tagName === 'textarea');
@@ -186,11 +233,11 @@ test('dropping a file on a closed epub import card opens it and fills the file f
 test('library import sends only selected books and handles an empty candidate list', async () => {
   await mountStories();
   api.on('GET', 'library/candidates', []);
-  await click(main, 'Find stories in the library');
+  await click(main, 'Show all stories in the library');
   assert.match(main.textContent, /tracked already/);
   const books = [{ id: 8, uuid: 'a', title: 'A', authors: ['W'] }, { id: 9, uuid: 'b', title: 'B', authors: [] }];
   api.on('GET', 'library/candidates', books);
-  await click(main, 'Find stories in the library');
+  await click(main, 'Show all stories in the library');
   change(field(main, 'Select all'), false);
   await click(main, 'Import selected');
   assert.equal(writes().length, 0);
@@ -205,6 +252,91 @@ test('library import sends only selected books and handles an empty candidate li
   assert.deepEqual(api.sent('POST', 'library/import')[0].body, { books: [{ id: 8, uuid: 'a', title: 'A' }], check: false });
 });
 
+test('library import search filters the loaded list, selects no book at first and imports the ticked books', async () => {
+  await mountStories();
+  const card = section(main, 'Import from the library');
+  const form = nodes(card).find((el) => el.tagName === 'form');
+  const submit = find(form, 'button', 'Search');
+  assert.equal(form.className, 'library-search');
+  assert.equal(submit.attributes.type, 'submit');
+  const search = async (text) => {
+    field(card, 'Search the library').value = text;
+    emit(form, 'submit');
+    await settle();
+  };
+  const shown = () => ['Ash & Bone', 'Bright Sky', 'Ash Road'].filter((title) => field(card, title));
+  await search('   ');
+  assert.equal(api.requests.some((r) => r.path.startsWith('library/candidates')), false);
+  const books = [
+    { id: 8, uuid: 'a', title: 'Ash & Bone', authors: ['Wren'], publisher: 'www.royalroad.com' },
+    { id: 9, uuid: 'b', title: 'Bright Sky', authors: ['Ashley Poe', 'Wren'], publisher: 'archiveofourown.org' },
+    { id: 10, uuid: 'c', title: 'Ash Road', authors: [], publisher: 'www.royalroad.com' },
+  ];
+  api.on('GET', 'library/candidates', books);
+  await search('nothing');
+  assert.match(card.textContent, /No library books from a story site match/);
+  assert.equal(find(card, 'button', 'Import selected'), undefined);
+  // Each word must be in the title, the authors or the site. Capitals do not matter.
+  await search(' ASH ');
+  assert.deepEqual(shown(), ['Ash & Bone', 'Bright Sky', 'Ash Road']);
+  await search('ash  royalroad');
+  assert.deepEqual(shown(), ['Ash & Bone', 'Ash Road']);
+  await search('wren sky');
+  assert.deepEqual(shown(), ['Bright Sky']);
+  assert.equal(api.sent('GET', 'library/candidates').length, 1, 'One load serves each search');
+  await search('ash');
+  assert.deepEqual(['Select all', ...shown()].map((label) => field(card, label).checked), [false, false, false, false]);
+  await click(card, 'Import selected');
+  assert.equal(writes().length, 0);
+  change(field(card, 'Bright Sky'), true);
+  api.on('POST', 'library/import', [{ id: 'job' }]);
+  await click(card, 'Import selected');
+  assert.deepEqual(api.sent('POST', 'library/import')[0].body, { books: [{ id: 9, uuid: 'b', title: 'Bright Sky' }], check: true });
+  // A queued book leaves the list, without another load.
+  await search('ash');
+  assert.deepEqual(shown(), ['Ash & Bone', 'Ash Road']);
+  assert.equal(api.sent('GET', 'library/candidates').length, 1);
+  // The full list loads again, and selects each book.
+  await click(card, 'Show all stories in the library');
+  assert.equal(api.sent('GET', 'library/candidates').length, 2);
+  assert.deepEqual(['Select all', ...shown()].map((label) => field(card, label).checked), [true, true, true, true]);
+});
+
+test('the library import buttons cannot be used again until the answer comes', async () => {
+  await mountStories();
+  const card = section(main, 'Import from the library');
+  const form = nodes(card).find((el) => el.tagName === 'form');
+  const submit = find(form, 'button', 'Search');
+  const all = find(card, 'button', 'Show all stories in the library');
+  let answer;
+  api.on('GET', 'library/candidates', () => new Promise((resolve) => { answer = resolve; }));
+  field(card, 'Search the library').value = 'a';
+  emit(form, 'submit');
+  await settle();
+  assert.equal(submit.disabled, true);
+  emit(form, 'submit');
+  await settle();
+  assert.equal(api.sent('GET', 'library/candidates').length, 1);
+  answer([{ id: 8, uuid: 'a', title: 'A', authors: [] }]);
+  await settle();
+  assert.equal(submit.disabled, false);
+  assert.ok(field(card, 'A'));
+  emit(all, 'click');
+  await settle();
+  assert.equal(all.disabled, true);
+  answer([]);
+  await settle();
+  assert.equal(all.disabled, false);
+  // A failed load frees the button, and the next search loads again.
+  api.on('GET', 'library/candidates', { error: 'CWA is offline' }, 502);
+  await click(card, 'Show all stories in the library');
+  emit(form, 'submit');
+  await settle();
+  assert.equal(submit.disabled, false);
+  assert.equal(all.disabled, false);
+  assert.equal(api.sent('GET', 'library/candidates').length, 4);
+});
+
 test('rules edits, moves, removal and paragraph choices are saved in order', async () => {
   api.on('GET', 'stories/1/rules', { ...emptyRules, rules: [rule, { ...rule, pattern: 'second' }] });
   const editor = rulesEditor('stories/1/rules');
@@ -289,7 +421,7 @@ test('story list stacks on a narrow screen instead of scrolling sideways', async
   const wrap = nodes(main).find((el) => el.className === 'table-wrap stories-table');
   assert.ok(wrap);
   const dataLabels = nodes(wrap).filter((el) => el.tagName === 'td').map((el) => el.attributes['data-label']);
-  assert.deepEqual(dataLabels, [undefined, 'Author', 'Site', 'Chapters', 'Status', 'Last change', 'Last check']);
+  assert.deepEqual(dataLabels, [undefined, 'Author', 'Site', 'Chapters', 'Pages', 'Status', 'Last change', 'Last check']);
 });
 
 test('the story list shows removed and revised chapter counts, only when there are any', async () => {
@@ -657,9 +789,17 @@ test('with CWA off, the pages show nothing about CWA', async () => {
   assert.ok(section(main, 'Add stories'));
   assert.equal(section(main, 'Import from the library'), undefined);
   assert.doesNotMatch(main.textContent, /CWA|INGEST_DIR|library/i);
+  for (const ingest of [true, false]) {
+    api.on('GET', 'settings', { recent_chapters: 3, personal_ini: '[defaults]', ingest_available: ingest });
     settingsPage(main, {}, off);
     await settle();
     assert.ok(section(main, 'Jobs list'));
     assert.equal(section(main, 'Uploads to CWA'), undefined);
-  assert.doesNotMatch(main.textContent, /CWA|INGEST_DIR|library/i);
+    assert.doesNotMatch(main.textContent, /CWA|library/i);
+    // The alert about INGEST_DIR is under Book Downloads, also with CWA off.
+    const note = nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note');
+    assert.equal(note.hidden, ingest);
+    assert.equal(note.className, 'notice');
+    assert.match(note.textContent, /INGEST_DIR is not set\. Bindery sends no book to the ingest folder/);
+  }
 });

tests/js/frontend.test.mjs

@@ -48,6 +48,7 @@ const settle = () => new Promise((resolve) => setImmediate(resolve));
 const config = {
   metadata_fields: { title: 'Title' }, mode_labels: {}, library_available: true, cwa_enabled: true, update_interval: 0,
   job_list_max_hours: 720, job_list_max_count: 500,
+  download_fields: { title: 'The title', id: 'The story ID' },
 };
 const story = {
   id: 1, title: 'Story', author: 'Author', url: 'https://example.com/story', site: 'example.com',

tests/js/layout.test.mjs

@@ -114,6 +114,36 @@ test('sortable headings show direction and remain usable above story cards on na
   assert.doesNotMatch(css, /\.stories-table thead\s*\{[^}]*clip-path/);
 });
 
+test('on narrow screens a story cell with no value shows no line', () => {
+  const start = css.indexOf('@media (max-width: 600px)', css.indexOf('.stories-table th button'));
+  assert.equal(declarations('.stories-table td:empty', css.slice(start)).display, 'none');
+  assert.doesNotMatch(css.slice(0, start), /\.stories-table td:empty/);
+});
+
+test('on narrow screens the headings are sort chips, and only the sorted chip is tinted with an arrow', () => {
+  const start = css.indexOf('@media (max-width: 600px)', css.indexOf('.stories-table th button'));
+  const mobile = css.slice(start);
+  assert.equal(declarations('.stories-table thead tr', mobile)['flex-wrap'], 'wrap');
+  const caption = declarations('.stories-table thead tr::before', mobile);
+  assert.equal(caption.content, "'Sort by'");
+  assert.equal(caption['flex-basis'], '100%');
+  const chip = declarations('.stories-table th button', mobile);
+  assert.equal(chip['border-radius'], '999px');
+  assert.equal(chip['inline-size'], 'auto');
+  assert.equal(chip.padding, '3px 10px');
+  assert.equal(chip['font-size'], 'var(--text-sm)');
+  assert.equal(chip.background, 'var(--surface)');
+  assert.equal(declarations(".stories-table th[aria-sort='none'] button::after", mobile).display, 'none');
+  const sorted = declarations(".stories-table th:not([aria-sort='none']) button", mobile);
+  assert.equal(sorted.background, 'var(--info-bg)');
+  assert.equal(sorted.color, 'var(--info)');
+  assert.equal(sorted['border-color'], undefined, 'The sorted chip keeps the soft border');
+  // The wide layout keeps plain headings.
+  const wide = css.slice(0, start);
+  assert.doesNotMatch(wide, /\.stories-table thead tr::before/);
+  assert.equal(declarations('.stories-table th button', wide).border, '0');
+});
+
 test('sortable headings fill the column with one clickable highlight without shifting layout', () => {
   assert.equal(declarations('.stories-table th').padding, '0');
   const button = declarations('.stories-table th button');

tests/js/local-auth.test.mjs

@@ -11,7 +11,7 @@ test('local login, retry, session expiry and logout use the existing form', asyn
   const backend = server(t);
   let auth = { authenticated: false, mode: 'local', user: null };
   backend.on('GET', 'auth/check', () => auth);
-  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false });
+  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false, download_fields: {} });
   backend.on('GET', 'stories', []);
   backend.on('GET', 'jobs', []);
   backend.on('GET', 'expired', { error: 'Session ended' }, 401);

tests/js/opds-settings.test.mjs

@@ -0,0 +1,127 @@
+import assert from 'node:assert/strict';
+import { test } from 'node:test';
+import { Element, change, click, field, find, install, nodes, section, server, settle } from './flow-helpers.mjs';
+
+install();
+const { settingsPage } = await import('../../bindery/static/js/settings.js');
+
+async function open(t, settings, authMode = 'proxy') {
+  install();
+  t.mock.timers.enable({ apis: ['setTimeout'] });
+  const backend = server(t);
+  const stored = { download_template: '{title}', download_date_format: '%Y', personal_ini: '', ...settings };
+  backend.on('GET', 'settings', stored);
+  backend.on('GET', 'rules', { rules: [] });
+  backend.on('GET', 'stories', []);
+  backend.on('PUT', 'settings', ({ body }) => ({ ...stored, ...body }));
+  backend.on('PUT', 'settings/opds-password', { opds_user: 'alice', opds_password_set: true });
+  const main = new Element('main');
+  settingsPage(main, {}, { cwa_enabled: false, download_fields: {}, auth_mode: authMode });
+  await settle();
+  return { backend, main, card: section(main, 'Book Downloads') };
+}
+
+for (const enabled of [false, true]) {
+  test(`OPDS loads ${enabled}, appears first in Book Downloads, and saves both states`, async (t) => {
+    const { backend, main, card } = await open(t, { opds_enabled: enabled });
+    const toggle = field(card, 'opds-enabled');
+    assert.equal(toggle.checked, enabled);
+    assert.equal(card.children[1].tagName, 'label');
+    assert.equal(card.children[1].children[0], toggle);
+    assert.equal(card.children[1].textContent, ' Enable OPDS');
+    assert.equal(toggle.getAttribute('aria-describedby'), 'opds-note');
+    assert.match(field(card, 'opds-note').textContent, /off by default/);
+    assert.equal(find(field(card, 'opds-note'), 'a', 'the OPDS catalog').getAttribute('href'), 'opds');
+    assert.equal(nodes(main).filter((node) => node.getAttribute('id') === 'opds-enabled').length, 1);
+    for (const value of [!enabled, enabled]) {
+      change(toggle, value);
+      await click(card, 'Save');
+      assert.deepEqual(backend.sent('PUT', 'settings').at(-1).body, {
+        opds_enabled: value, download_template: '{title}', download_date_format: '%Y',
+      });
+      assert.equal(toggle.checked, value);
+    }
+    backend.on('PUT', 'settings', { opds_enabled: false, download_template: '{title}', download_date_format: '%Y' });
+    change(toggle, true);
+    await click(card, 'Save');
+    assert.equal(toggle.checked, false, 'The saved response sets the checkbox state');
+  });
+}
+
+const LOGIN = {
+  none: /Bindery asks for no login, so the catalog needs none/,
+  local: /sends your Bindery user name and password/,
+  cwa: /sends your CWA user name and password/,
+  proxy: /The proxy skips its login for the catalog.*Each user sets an OPDS password.*user name from the proxy/,
+};
+for (const [mode, text] of Object.entries(LOGIN)) {
+  test(`the note names the login of the ${mode} mode, and only the proxy mode shows the password`, async (t) => {
+    const { card } = await open(t, { opds_enabled: true, opds_password_set: false, opds_user: 'alice' }, mode);
+    assert.match(field(card, 'opds-note').textContent, text);
+    assert.doesNotMatch(field(card, 'opds-note').textContent, /BINDERY_API_KEY|Bearer|X-Api-Key/);
+    for (const id of ['opds-user', 'opds-password', 'opds-password-status']) {
+      assert.equal(field(card, id) !== undefined, mode === 'proxy', id);
+    }
+    assert.equal(find(card, 'button', 'Set password') !== undefined, mode === 'proxy');
+  });
+}
+
+test('the password field is grey until the box is checked, and follows the box at once', async (t) => {
+  const { card } = await open(t, { opds_enabled: false, opds_password_set: true, opds_user: 'alice' });
+  const password = field(card, 'opds-password');
+  const button = find(card, 'button', 'Reset password');
+  assert.equal(password.disabled, true);
+  assert.equal(button.disabled, true);
+  change(field(card, 'opds-enabled'), true);
+  assert.equal(password.disabled, false);
+  assert.equal(button.disabled, false);
+  change(field(card, 'opds-enabled'), false);
+  assert.equal(password.disabled, true);
+  assert.equal(button.disabled, true);
+});
+
+for (const isSet of [false, true]) {
+  test(`the OPDS password ${isSet ? 'is set' : 'is not set'}, and the button sets or resets it`, async (t) => {
+    const { backend, card } = await open(t, { opds_enabled: true, opds_password_set: isSet, opds_user: 'alice' });
+    const password = field(card, 'opds-password');
+    const status = field(card, 'opds-password-status');
+    const user = field(card, 'opds-user');
+    assert.equal(user.value, 'alice');
+    assert.equal(user.getAttribute('readonly'), '');
+    assert.equal(user.disabled, false, 'The user name stays selectable');
+    assert.equal(user.getAttribute('aria-describedby'), 'opds-password-status');
+    assert.equal(find(card, 'label', 'OPDS user name').getAttribute('for'), 'opds-user');
+    assert.equal(find(card, 'label', 'OPDS password').getAttribute('for'), 'opds-password');
+    assert.equal(password.disabled, false);
+    assert.equal(password.getAttribute('type'), 'password');
+    assert.equal(password.getAttribute('autocomplete'), 'new-password');
+    assert.equal(password.getAttribute('minlength'), '8');
+    assert.equal(password.value, '');
+    assert.match(status.textContent, /enter the user name and the password in your reader/i);
+    assert.match(status.textContent, isSet ? /You have an OPDS password/ : /You have no OPDS password yet/);
+    assert.ok(find(card, 'button', isSet ? 'Reset password' : 'Set password'));
+    password.value = 'long enough';
+    await click(card, isSet ? 'Reset password' : 'Set password');
+    assert.deepEqual(backend.sent('PUT', 'settings/opds-password')[0].body, { password: 'long enough' });
+    assert.equal(password.value, '', 'The field is empty after the save');
+    assert.match(status.textContent, /You have an OPDS password/);
+    assert.equal(password.disabled, false);
+    assert.ok(find(card, 'button', 'Reset password'));
+    // The Save button of the card never sends the password.
+    password.value = 'not sent';
+    await click(card, 'Save');
+    assert.equal(backend.sent('PUT', 'settings/opds-password').length, 1);
+    assert.equal('password' in backend.sent('PUT', 'settings')[0].body, false);
+    assert.equal(password.disabled, false, 'A save keeps the field in step with the box');
+  });
+}
+
+test('a rejected OPDS password keeps the field and the status', async (t) => {
+  const { backend, card } = await open(t, { opds_enabled: true, opds_password_set: false });
+  backend.on('PUT', 'settings/opds-password', { error: 'The OPDS password must be text of 8 to 256 characters' }, 400);
+  field(card, 'opds-password').value = 'short';
+  await click(card, 'Set password');
+  assert.equal(field(card, 'opds-password').value, 'short');
+  assert.match(field(card, 'opds-password-status').textContent, /You have no OPDS password yet/);
+  assert.ok(find(card, 'button', 'Set password'));
+});

tests/js/proxy-reload.test.mjs

@@ -15,7 +15,7 @@ test('a 401 while logged in through the proxy reloads the page', async (t) => {
   });
   const backend = server(t);
   backend.on('GET', 'auth/check', { authenticated: true, mode: 'proxy', user: 'alice' });
-  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false });
+  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false, download_fields: {} });
   backend.on('GET', 'jobs', []);
   backend.on('GET', 'stories', []);
   backend.on('GET', 'expired', { error: 'Session ended' }, 401);

tests/js/story-list.test.mjs

@@ -21,7 +21,7 @@ const story = (id, title, changes = {}) => ({
   id, title, url: `https://example.com/${id}`,
   sort_title: (title || `https://example.com/${id}`).trim().replace(/^(?:the|an?)\s+/i, '').toLowerCase(),
   author: 'Reader', site: 'example.com',
-  chapter_count: 42, offsite_count: 0, revised_count: 0, status: 'ok', message: 'No changes',
+  chapter_count: 42, page_count: 120, offsite_count: 0, revised_count: 0, status: 'ok', message: 'No changes',
   last_changed: null, last_checked: null, ...changes,
 });
 const rows = () => {
@@ -106,6 +106,39 @@ test('chapter sorting compares counts numerically and keeps equal counts in titl
   assert.equal(heading('Chapters').attributes['aria-sort'], 'ascending');
 });
 
+test('pages show beside the chapters and sort numerically', async (t) => {
+  await mount(t, [
+    story(1, 'Zeta', { page_count: 9 }),
+    story(2, 'Alpha', { page_count: 1200 }),
+    story(3, 'Beta', { page_count: 0 }),
+    story(4, 'Gamma', { page_count: 9 }),
+  ]);
+  const cells = nodes(rows()[0]).filter((el) => el.tagName === 'td').map((el) => el.attributes['data-label']);
+  assert.deepEqual(cells.slice(3, 5), ['Chapters', 'Pages']);
+  assert.equal(find(app, 'button', 'Pages').attributes['aria-label'], 'Sort by pages, most first');
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Alpha', 'Gamma', 'Zeta', 'Beta']);
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Beta', 'Gamma', 'Zeta', 'Alpha']);
+  assert.deepEqual(JSON.parse(window.localStorage.getItem(SORT_KEY)), { column: 'page_count', direction: 1 });
+});
+
+test('a story with no built book shows no page count and sorts last in each direction', async (t) => {
+  await mount(t, [
+    story(1, 'Alpha', { page_count: null }),
+    story(2, 'Beta', { page_count: 300 }),
+    story(3, 'Gamma', { page_count: 0 }),
+    story(4, 'Delta', { page_count: null }),
+  ]);
+  const pages = () => rows().map((row) => cell(row, 'Pages'));
+  assert.deepEqual(pages().map((el) => el.textContent), ['', '300', '', '0']);
+  assert.deepEqual(pages().map((el) => el.children.length), [0, 1, 0, 1], 'An empty cell hides on a narrow screen');
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Beta', 'Gamma', 'Alpha', 'Delta']);
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Gamma', 'Beta', 'Alpha', 'Delta']);
+});
+
 for (const [label, key] of [['Author', 'author'], ['Site', 'site'], ['Status', 'status']]) {
   test(`${label} sorts alphabetically in either direction and uses titles to break ties`, async (t) => {
     await mount(t, [
@@ -248,7 +281,7 @@ test('date columns keep each timestamp on one line, with last change before last
   const checked = '2026-01-03T05:06:00Z';
   await mount(t, [story(1, 'Alpha', { last_changed: changed, last_checked: checked })]);
   const headings = nodes(app).filter((el) => el.tagName === 'th').map((el) => el.textContent);
-  assert.deepEqual(headings, ['Title', 'Author', 'Site', 'Chapters', 'Status', 'Last change', 'Last check']);
+  assert.deepEqual(headings, ['Title', 'Author', 'Site', 'Chapters', 'Pages', 'Status', 'Last change', 'Last check']);
   for (const [label, value] of [['Last change', changed], ['Last check', checked]]) {
     const dateCell = cell(rows()[0], label);
     const time = nodes(dateCell).find((el) => el.tagName === 'time');

tests/js/story-tab.test.mjs

@@ -31,7 +31,7 @@ test('an open story has a tab with its title, also on its chapter pages', async
   t.after(stopPolling);
   const backend = server(t);
   backend.on('GET', 'auth/check', { authenticated: true, mode: 'none', user: '' });
-  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false });
+  backend.on('GET', 'config', { metadata_fields: {}, mode_labels: {}, library_available: false, download_fields: {} });
   backend.on('GET', 'jobs', []);
   backend.on('GET', 'stories', []);
   backend.on('GET', 'settings', { recent_chapters: 0, personal_ini: '' });

tests/js/upload-hold.test.mjs

@@ -7,7 +7,7 @@ const { settingsPage } = await import('../../bindery/static/js/settings.js');
 const { storyPage } = await import('../../bindery/static/js/story.js');
 const { storiesPage } = await import('../../bindery/static/js/stories.js');
 
-const config = { cwa_enabled: true, job_list_max_hours: 720, job_list_max_count: 500 };
+const config = { cwa_enabled: true, job_list_max_hours: 720, job_list_max_count: 500, download_fields: {} };
 const book = { id: 7, uuid: 'book-7', title: 'Library Title', authors: ['Writer'] };
 const details = (story, extra = {}) => ({
   story: { id: 1, title: 'Story', author: 'Writer', url: 'https://example.com/story', site: 'example.com',
@@ -54,7 +54,7 @@ test('the upload switches load and save on the settings page', async (t) => {
   assert.equal(warning.hidden, true);
   assert.equal(link.attributes['aria-describedby'], '');
   assert.match(warning.textContent, /Updates without a saved CWA book ID create duplicate books if CWA imports them/);
-  assert.equal(nodes(card).find((el) => el.attributes.id === 'ingest-note').hidden, true);
+  assert.equal(nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note').hidden, true);
   assert.equal(pause.disabled, false);
   assert.equal(hold.disabled, false);
   assert.equal(link.disabled, false);
@@ -129,7 +129,7 @@ for (const linkBooks of [true, false]) {
     assert.equal(nodes(card).find((el) => el.attributes.id === 'link-note').hidden, linkBooks);
     assert.equal(pause.checked, true);
     assert.equal(pause.disabled, true);
-    const note = nodes(card).find((el) => el.attributes.id === 'ingest-note');
+    const note = nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note');
     assert.equal(note.hidden, false);
     assert.equal(note.className, 'notice');
     assert.equal(pause.attributes['aria-describedby'], note.attributes.id);

tests/test_api.py

@@ -472,6 +472,11 @@ def test_settings(client, monkeypatch, tmp_path):
         "pause_uploads": False,
         "ingest_available": True,
         "personal_ini": "[defaults]\nis_adult:true\n",
+        "opds_enabled": False,
+        "opds_password_set": False,
+        "opds_user": None,
+        "download_template": "{title_sort} - {authors}",
+        "download_date_format": "%b, %Y",
         "story_jobs_hours": 36,
         "story_jobs_count": 10,
         "main_jobs_hours": 6,

tests/test_api_flows.py

@@ -142,11 +142,85 @@ def test_download_builds_from_stored_chapters_without_changing_them(client, stor
     response = client.get(f"/api/stories/{sid}/epub")
     assert response.status_code == 200
     assert response.data.startswith(b"PK")
-    assert "filename=A_B.epub" in response.headers["Content-Disposition"]
+    assert 'filename="A_B - Writer.epub"' in response.headers["Content-Disposition"]
     assert o.store.revision_html(cid, 1) == original
     response.close()
 
 
+def download_name(client, sid):
+    response = client.get(f"/api/stories/{sid}/epub")
+    response.close()
+    return response.headers["Content-Disposition"]
+
+
+def test_download_template_names_the_file(client, story):
+    sid, _ = story
+    site = {"title": "The Book", "authors": ["Ann Writer"], "series": "The Saga", "series_index": 2.0}
+    o.store.update_story(sid, metahtml=metahtml(), title="The Book", site_values=site)
+    assert client.get("/api/config").json["download_fields"]["id"].startswith("The CWA book ID")
+    saved = client.put("/api/settings", json={"download_template": " {series:|| - }{title} - {id} "}, headers=H)
+    assert saved.json["download_template"] == "{series:|| - }{title} - {id}"
+    assert f'filename="The Saga - The Book - {sid}.epub"' in download_name(client, sid)
+    # The default moves the article to the end, as Calibre's default does.
+    client.put("/api/settings", json={"download_template": ""}, headers=H)
+    assert 'filename="Book, The - Ann Writer.epub"' in download_name(client, sid)
+    client.put("/api/settings", json={"download_template": "{series_sort} - {title} - {id}"}, headers=H)
+    assert f'filename="Saga, The - The Book - {sid}.epub"' in download_name(client, sid)
+    # The ID is that of the CWA book, when the story has one.
+    o.store.update_story(sid, calibre_id=77, pinned={"title": "Caf\u00e9", "series": ["", None]})
+    client.put("/api/settings", json={"download_template": "{series:|| - }{title} - {id}"}, headers=H)
+    name = download_name(client, sid)
+    assert 'filename="Cafe - 77.epub"' in name and "filename*=UTF-8''Caf%C3%A9%20-%2077.epub" in name
+    # A template with folders gives the name of the file alone.
+    client.put("/api/settings", json={"download_template": "{author_sort}/{title}"}, headers=H)
+    assert "filename*=UTF-8''Caf%C3%A9.epub" in download_name(client, sid)
+    # An empty template restores the default.
+    saved = client.put("/api/settings", json={"download_template": " "}, headers=H).json
+    assert saved["download_template"] == "{title_sort} - {authors}"
+    assert "filename*=UTF-8''Caf%C3%A9%20-%20Ann%20Writer.epub" in download_name(client, sid)
+
+
+def test_date_format_of_download_names(client, story):
+    sid, _ = story
+    o.store.update_story(sid, metahtml=metahtml(), site_values={"pubdate": "2020-05-06T00:00:00"})
+    client.put("/api/settings", json={"download_template": "{title} {pubdate}"}, headers=H)
+    assert 'filename="Story May, 2020.epub"' in download_name(client, sid)
+    saved = client.put("/api/settings", json={"download_date_format": " %Y-%m-%d "}, headers=H).json
+    assert saved["download_date_format"] == "%Y-%m-%d"
+    assert "filename=Story 2020-05-06.epub" in download_name(client, sid).replace('"', "")
+    response = client.put("/api/settings", json={"download_date_format": 5}, headers=H)
+    assert (response.status_code, response.json) == (400, {"error": "The date format must be text"})
+    response = client.put("/api/settings", json={"download_date_format": "%Y" * 51}, headers=H)
+    assert (response.status_code, response.json) == (400, {"error": "A date format can have 100 characters at most"})
+    # An empty format restores the default.
+    assert client.put("/api/settings", json={"download_date_format": ""}, headers=H).json["download_date_format"] == (
+        "%b, %Y"
+    )
+
+
+def test_download_template_that_fails_gives_the_title(client, story, caplog):
+    sid, _ = story
+    o.store.update_story(sid, metahtml=metahtml())
+    for template in ("{title:d}", "{series}/{series_index}"):
+        client.put("/api/settings", json={"download_template": template}, headers=H)
+        assert "filename=Story.epub" in download_name(client, sid)
+    assert "The download template fails for https://www.royalroad.com/fiction/1234" in caplog.text
+    assert caplog.text.count("The download template fails") == 1
+
+
+def test_bad_download_template_is_not_saved(client):
+    client.put("/api/settings", json={"download_template": "{title}"}, headers=H)
+    for template, error in [
+        ("{title:uppercase()}", "Bindery does not support template functions, such as {title:uppercase()}"),
+        ("{title", "The braces of the template do not match: expected '}' before end of string"),
+        (7, "The download template must be text"),
+    ]:
+        response = client.put("/api/settings", json={"download_template": template, "recent_chapters": 9}, headers=H)
+        assert (response.status_code, response.json) == (400, {"error": error})
+    assert client.get("/api/settings").json["download_template"] == "{title}"
+    assert client.get("/api/settings").json["recent_chapters"] == 0
+
+
 def test_unexpected_backend_error_returns_json(client, monkeypatch, caplog):
     monkeypatch.setattr(o, "list_stories", Mock(side_effect=RuntimeError("broken")))
     response = client.get("/api/stories")

tests/test_build_keys.py

@@ -11,7 +11,8 @@ from test_deliver import fake as fake
 from test_deliver import jobs as jobs
 from test_deliver import story_epub as story_epub
 
-from bindery import config, orchestrator
+from bindery import config, orchestrator, pages
+from bindery import store as storage
 from bindery.jobs import DownloadMode, Job
 
 ONE = [(1, "One", "a")]
@@ -44,13 +45,21 @@ def test_an_unchanged_story_is_not_built_or_sent_again(builds, monkeypatch):
     assert len(ingest_files()) == 2  # one epub and its sidecar
 
 
-@pytest.mark.parametrize("change", ["chapter", "rule", "global rule", "paragraphs", "metadata", "included"])
+@pytest.mark.parametrize(
+    "change", ["chapter", "revision", "renamed", "rule", "global rule", "paragraphs", "metadata", "included"]
+)
 def test_each_change_builds_and_sends_the_book(builds, monkeypatch, change):
     story_id = linked_story([(1, "One", "a"), (2, "Two", "b")])
     site(monkeypatch, [(1, "One", "a"), (2, "Two", "b")])
     update(story_id)
+    previous = "2020-01-01T00:00:00"
+    orchestrator.store.update_story(story_id, last_changed=previous)
     if change == "chapter":
         site(monkeypatch, [(1, "One", "a"), (2, "Two", "b"), (3, "Three", "c")])
+    elif change == "revision":
+        site(monkeypatch, [(1, "One", "a"), (2, "Two", "new text")])
+    elif change == "renamed":
+        site(monkeypatch, [(1, "One", "a"), (2, "Renamed", "b")])
     elif change == "rule":
         orchestrator.save_rules(story_id, [{"pattern": "a", "replacement": "b"}])
     elif change == "global rule":
@@ -62,10 +71,14 @@ def test_each_change_builds_and_sends_the_book(builds, monkeypatch, change):
     else:
         chapter = orchestrator.store.list_chapters(story_id)[0]
         orchestrator.store.update_chapter(chapter["id"], included=0)
-    assert "library book 1" in update(story_id)
+    mode = DownloadMode.RECHECK_ALL if change == "revision" else DownloadMode.NEW
+    assert "library book 1" in update(story_id, mode)
     assert builds.call_count == 2
     assert len(ingest_files()) == 4
+    changed_at = story(story_id)["last_changed"]
+    assert (changed_at != previous) == (change in ("chapter", "revision"))
     assert update(story_id).startswith("No changes")
+    assert story(story_id)["last_changed"] == changed_at
 
 
 def test_a_held_book_that_did_not_change_is_not_built_again(builds, monkeypatch):
@@ -110,13 +123,14 @@ def test_a_lost_file_is_built_again_only_to_send_it(builds):
     assert builds.call_count == 2
 
 
-def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(builds, client, jobs):
+@pytest.mark.parametrize("last_changed", [None, "2020-01-01T00:00:00"])
+def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(builds, client, jobs, last_changed):
     story_id = add_story(ingested_at="2026-01-01T00:00:00", calibre_id=1, calibre_uuid="uuid-1")
     job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
     assert "library book 1" in orchestrator._run_rebuild(job)
-    orchestrator.store.update_story(story_id, last_changed=None, message="before")
+    orchestrator.store.update_story(story_id, last_changed=last_changed, message="before")
     assert orchestrator._run_rebuild(job) == "No changes"
-    assert (story(story_id)["last_changed"], story(story_id)["message"]) == (None, "before")
+    assert (story(story_id)["last_changed"], story(story_id)["message"]) == (last_changed, "before")
     assert builds.call_count == 1
     # The Rebuild button of the story page.
     queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "rebuild"}, headers=H).json
@@ -124,7 +138,7 @@ def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(bui
     assert "library book 1" in orchestrator._run_rebuild(jobs.get_next())
     assert builds.call_count == 2
     assert len(ingest_files()) == 4
-    assert story(story_id)["last_changed"]
+    assert story(story_id)["last_changed"] == last_changed
     # Other modes take no such option.
     queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "new"}, headers=H).json
     assert queued["options"] == {}
@@ -152,6 +166,31 @@ def test_a_failed_delivery_is_tried_again_without_a_build(builds, fake, monkeypa
     assert builds.call_count == 1
 
 
+@pytest.mark.parametrize("change", ["chapter", "revision"])
+def test_chapter_timestamp_survives_a_failed_build_and_retry(builds, monkeypatch, change):
+    story_id = linked_story(ONE)
+    site(monkeypatch, ONE)
+    update(story_id)
+    previous = "2020-01-01T00:00:00"
+    orchestrator.store.update_story(story_id, last_changed=previous)
+    clock = ["2024-01-01T00:00:00"]
+    monkeypatch.setattr(storage, "now", lambda: clock[0])
+    site(monkeypatch, [*ONE, (2, "Two", "b")] if change == "chapter" else [(1, "One", "revised")])
+    job = Job(mode=DownloadMode.RECHECK_ALL, story_id=story_id)
+    build = orchestrator._build
+    monkeypatch.setattr(orchestrator, "_build", Mock(side_effect=OSError("disk full")))
+    with pytest.raises(OSError, match="disk full"):
+        orchestrator._run_site_job(job)
+    assert story(story_id)["last_changed"] == clock[0]
+    assert job.result["new"] or job.result["changed"]
+    clock[0] = "2024-02-01T00:00:00"
+    monkeypatch.setattr(orchestrator, "_build", build)
+    assert "library book 1" in orchestrator._run_site_job(job)
+    assert story(story_id)["last_changed"] == "2024-01-01T00:00:00"
+    assert update(story_id).startswith("No changes")
+    assert story(story_id)["last_changed"] == "2024-01-01T00:00:00"
+
+
 def test_a_new_build_removes_the_older_file(builds, monkeypatch):
     story_id = linked_story(ONE)
     site(monkeypatch, ONE)
@@ -227,6 +266,7 @@ def test_a_download_uses_the_built_book(builds, client, monkeypatch):
 
 def test_a_download_builds_a_book_that_has_no_file_or_that_changed(builds, client, monkeypatch):
     story_id = add_story()
+    changed_at = story(story_id)["last_changed"]
     first = download(client, story_id)
     download(client, story_id)
     assert builds.call_count == 1
@@ -234,12 +274,65 @@ def test_a_download_builds_a_book_that_has_no_file_or_that_changed(builds, clien
     assert download(client, story_id) != first
     assert builds.call_count == 2
     assert len(list(config.TMP_DIR.iterdir())) == 1
-    # The build of a download is not the build of a job. The job sends the book, and marks the change.
+    # The build of a download is not the build of a job. The job sends the book.
     assert orchestrator.store.get_setting(story_id, "built_key") is None
     job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
     assert "library book 1" in orchestrator._run_rebuild(job)
     assert builds.call_count == 2
-    assert story(story_id)["last_changed"]
+    assert story(story_id)["last_changed"] == changed_at
+
+
+# Page counts
+
+LONG = "A line of the story. " * 20
+
+
+def listed_pages(story_id):
+    return {s["id"]: s["page_count"] for s in orchestrator.list_stories()}[story_id]
+
+
+def test_a_story_has_no_page_count_until_its_book_is_built(builds, monkeypatch):
+    story_id = linked_story(ONE)
+    assert listed_pages(story_id) is None
+    site(monkeypatch, ONE)
+    update(story_id)
+    (built,) = config.TMP_DIR.iterdir()
+    assert listed_pages(story_id) == pages.epub_pages(built) == 1
+
+
+def test_each_build_counts_the_pages_of_the_new_book(builds, client, monkeypatch):
+    story_id = linked_story(ONE)
+    site(monkeypatch, [*ONE, *[(n, f"Chapter {n}", "<br/>".join([LONG] * 31)) for n in range(2, 6)]])
+    update(story_id)
+    long_book = listed_pages(story_id)
+    # Each of the 124 lines is a paragraph and 6 lines of text.
+    assert long_book >= 4 * 31 * 7 // 31
+    # The count is of the built book, so a rule that removes text changes it without a download.
+    orchestrator.save_rules(story_id, [{"pattern": "(A line of the story. ){19}", "replacement": ""}])
+    download(client, story_id)
+    assert builds.call_count == 2
+    assert 0 < listed_pages(story_id) < long_book / 3
+
+
+def test_a_book_from_before_the_page_count_is_counted_without_a_build(builds, client, monkeypatch):
+    story_id = linked_story(ONE)
+    site(monkeypatch, ONE)
+    update(story_id)
+    orchestrator.store.set_setting(story_id, "page_count", None)
+    download(client, story_id)
+    assert builds.call_count == 1
+    assert listed_pages(story_id) == 1
+
+
+def test_a_book_that_cannot_be_counted_is_still_built(builds, monkeypatch):
+    def fail(path):
+        raise orchestrator.EpubError("unreadable")
+
+    monkeypatch.setattr(pages, "epub_pages", fail)
+    story_id = linked_story(ONE)
+    site(monkeypatch, ONE)
+    assert "library book 1" in update(story_id)
+    assert listed_pages(story_id) is None
 
 
 def test_stop_tracking_deletes_the_built_book(builds, client):

tests/test_config.py

@@ -143,7 +143,7 @@ def test_defaults(monkeypatch, reload_config):
     ]
     assert config.ALLOWED_HOSTS == []
     assert config.ADMIN_USERS == []
-    assert config.CWA_ENABLED is True
+    assert config.CWA_ENABLED is False
     assert config.IGNORED_SETTINGS == []
     assert config.SESSION_COOKIE_SECURE is False
     assert config.TRUST_PROXY is False
@@ -161,8 +161,11 @@ def test_empty_ingest_directory_disables_uploads(monkeypatch, reload_config, val
     assert config.INGEST_DIR is None
 
 
-@pytest.mark.parametrize("value", ["false", " No ", "0", "off"])
+@pytest.mark.parametrize("value", [None, "", "false", " No ", "0", "maybe"])
 def test_cwa_off_ignores_the_cwa_settings_but_not_the_ingest_folder(monkeypatch, reload_config, value):
+    if value is None:
+        monkeypatch.delenv("CWA_ENABLED")
+    else:
         monkeypatch.setenv("CWA_ENABLED", value)
     monkeypatch.setenv("CWA_URL", "https://cwa.invalid")
     monkeypatch.setenv("CWA_USERNAME", "bindery")
@@ -178,8 +181,8 @@ def test_cwa_off_ignores_the_cwa_settings_but_not_the_ingest_folder(monkeypatch,
     assert str(config.INGEST_DIR) == "/books"
 
 
-@pytest.mark.parametrize("value", ["", "true", "maybe"])
+@pytest.mark.parametrize("value", ["true", " Yes ", "1", "on"])
-def test_only_a_clear_false_turns_cwa_off(monkeypatch, reload_config, value):
+def test_only_a_clear_true_turns_cwa_on(monkeypatch, reload_config, value):
     monkeypatch.setenv("CWA_ENABLED", value)
     monkeypatch.setenv("CWA_URL", "https://cwa.invalid")
     reload_config()
@@ -197,17 +200,19 @@ def test_prepare_with_cwa_off_logs_nothing_about_missing_cwa(folders, monkeypatc
     with caplog.at_level("INFO"):
         config.prepare()
     lines = [r.getMessage() for r in caplog.records]
-    expected = ["CWA_ENABLED is false. Bindery ignores CWA_URL, CALIBRE_DB_PATH."] if ignored else []
+    ignores = "CWA_ENABLED is not true. Bindery ignores CWA_URL, CALIBRE_DB_PATH. To use CWA, set CWA_ENABLED=true."
+    expected = [ignores] if ignored else []
+    assert [r.levelname for r in caplog.records if "CWA" in r.getMessage()] == ["WARNING"] * len(expected)
     assert [line for line in lines if "CWA" in line or "INGEST" in line] == expected
 
 
 @pytest.mark.parametrize(
     ("ingest", "url", "database", "expected"),
     [
-        (False, "", None, ["INGEST_DIR is not set", "cannot find the library book of a story.", "CWA_ENABLED=false"]),
+        (False, "", None, ["INGEST_DIR is not set", "cannot find the library book of a story.", "remove CWA_ENABLED"]),
-        (True, "", None, ["so each update adds a new book to CWA.", "CWA_ENABLED=false"]),
+        (True, "", None, ["so each update adds a new book to CWA.", "remove CWA_ENABLED"]),
-        (True, "", Path("/library/metadata.db"), ["CWA_URL is not set. Bindery cannot search", "CWA_ENABLED=false"]),
+        (True, "", Path("/library/metadata.db"), ["CWA_URL is not set. Bindery cannot search", "remove CWA_ENABLED"]),
-        (False, "https://cwa.invalid", None, ["INGEST_DIR is not set", "CWA_ENABLED=false"]),
+        (False, "https://cwa.invalid", None, ["INGEST_DIR is not set", "remove CWA_ENABLED"]),
         (True, "https://cwa.invalid", None, []),
     ],
 )

tests/test_filename_template.py

@@ -0,0 +1,228 @@
+"""The file name templates, against the results of Calibre 9.2.1.
+
+Each expected text is what Calibre gives for the same book and template.
+"""
+
+import pytest
+
+from bindery import filename_template as ft
+
+BOOK = {
+    "title": "The Book",
+    "authors": ["Ann Writer", "Ben Reader"],
+    "series": "The Saga",
+    "series_index": 2.345,
+    "languages": ["eng", "fra"],
+    "tags": ["Space", "Fiction"],
+    "pubdate": "2020-05-06T12:00:00+00:00",
+    "publisher": "site.example",
+}
+NO_SERIES = {**BOOK, "series": "", "series_index": None}
+
+
+def expand(template, order="library_order", **changes):
+    return ft.expand(template, ft.prepare({**BOOK, **changes}, 42, order))
+
+
+# name, template, text from Calibre, series index
+REFERENCE = [
+    ("title", "{title}", "Book, The", 2.345),
+    ("series", "{series}", "Saga, The", 2.345),
+    ("fraction", "{series_index}", "2.35", 2.345),
+    ("languages", "{languages}", "eng,fra", 2.345),
+    ("tags", "{tags}", "Fiction, Space", 2.345),
+    ("send_date", "{pubdate}", "May, 2020", 2.345),
+    ("space_padding", "x{series_index:>5s}x", "x 2x", 2),
+    ("zero_padding", "{series_index:0>3s}", "002", 2),
+    ("numeric_format", "{series_index:03d}", "002", 2),
+    ("first_character", "{author_sort[0]}", "W", 2),
+    ("conditional", "{series:|| - }{title}", "Saga, The - Book, The", 2),
+    ("case_insensitive", "{TITLE}", "Book, The", 2),
+    ("unknown", "x{not_a_field}x", "xx", 2),
+    ("literal_braces", "{{title}}", "{title}", 2),
+]
+
+
+@pytest.mark.parametrize("name,template,expected,index", REFERENCE, ids=[case[0] for case in REFERENCE])
+def test_reference_cases(name, template, expected, index):
+    assert expand(template, series_index=index) == expected
+
+
+def test_reference_cases_for_a_missing_series_and_for_alphabetic_order():
+    values = ft.prepare(NO_SERIES, 42)
+    assert ft.expand("{series}{series_index:| - | - }{title}", values) == "Book, The"
+    assert expand("{title}", "strictly_alphabetic") == "The Book"
+    assert expand("{series}", "strictly_alphabetic") == "The Saga"
+
+
+def test_series_sort_is_a_bindery_field():
+    assert expand("{series_sort}") == expand("{series_sort}", "strictly_alphabetic") == "Saga, The"
+    assert ft.expand("x{series_sort}x", ft.prepare(NO_SERIES, 42)) == "xx"
+
+
+@pytest.mark.parametrize(
+    "template,expected",
+    [
+        ("{authors}", "Ann Writer & Ben Reader"),
+        ("{author}", "Ann Writer & Ben Reader"),
+        ("{author_sort}", "Writer, Ann & Reader, Ben"),
+        ("{title_sort}", "Book, The"),
+        ("{publisher} {id}", "site.example 42"),
+        ("{title:.3s|<|>}", "<Boo>"),
+        ("{title:*^15}", "***Book, The***"),
+        ("{id:05d} {id:x} {id:.2f}", "00042 2a 42.00"),
+        ("{title[0]}{title[1]}", "Bo"),
+        ("{title} {{{id}}}", "Book, The {42}"),
+        ("  {title}   -\t{authors}  ", "Book, The - Ann Writer & Ben Reader"),
+        ("{ title }{other}", ""),
+        ("a|b|c{title:|-|}", "a|b|c-Book, The"),
+        ("{series}/{series_index:0>2s| | - }{title}", "Saga, The/ 2.35 - Book, The"),
+    ],
+)
+def test_more_templates_give_the_text_of_calibre(template, expected):
+    assert expand(template) == expected
+
+
+def test_values_follow_the_rules_of_calibre():
+    assert expand("{title} - {authors}", title="A/B: C\\D", authors=["X/Y"]) == "A_B: C_D - X_Y"
+    assert expand("{authors}", authors=["Rock & Roll", "", "Solo"]) == "Rock && Roll & Solo"
+    assert expand("{tags}", tags=["/root", "alpha", "Beta", "1st", "_x"]) == "_x, _root, 1st, alpha, Beta"
+    assert expand("{tags}", tags=["F/M", "Hurt-Comfort", "hurt comfort", "Hurt", "x2", "x10", "é", "e", "f"]) == (
+        "e, é, f, F_M, Hurt, hurt comfort, Hurt-Comfort, x10, x2"
+    )
+    assert expand("{tags}|{languages}|{pubdate}", tags=[], languages="", pubdate="") == "||"
+    assert expand("{languages}", languages="en") == "en"
+    assert expand("{pubdate} {timestamp}", pubdate="1999-12-31T23:59:59", timestamp="2026-10-03") == (
+        "Dec, 1999 Oct, 2026"
+    )
+    assert expand("{pubdate}", pubdate="not a date") == ""
+
+
+def test_date_format():
+    # Calibre's helper_date reference case.
+    assert ft.expand("{pubdate}", ft.prepare(BOOK, 42, date_format="%b %Y")) == "May 2020"
+    assert ft.expand("{pubdate}", ft.prepare(BOOK, 42, date_format="%d/%m/%Y")) == "06_05_2020"
+    assert ft.expand("x{pubdate}x", ft.prepare(BOOK, 42, date_format="")) == "xx"
+    ft.check_date_format("%Y-%m-%d " * 10)
+    with pytest.raises(ft.TemplateError, match="A date format can have 100 characters at most"):
+        ft.check_date_format("x" * 101)
+    # A series without a number has no series_index, and a book without a series has neither.
+    assert expand("{series}{series_index:| [|]}", series_index=None) == "Saga, The"
+    assert ft.expand("{series_index:03d}|{series}", ft.prepare(NO_SERIES, 42)) == "|"
+
+
+@pytest.mark.parametrize(
+    "title,expected",
+    [
+        ("A Tale", "Tale, A"),
+        ("An Apple", "Apple, An"),
+        ("the lower", "lower, the"),
+        ("Theory", "Theory"),
+        ("The", "The"),
+        ("  The   Spaced  ", "Spaced, The"),
+        ("'The Quoted'", "Quoted, The"),
+        ("“The Curly”", "Curly, The"),
+        ("„The Low“", "Low, The"),
+        ("The 'Inner", "Inner, The"),
+        ('""', ""),
+        ("Le Monde", "Le Monde"),
+    ],
+)
+def test_title_sort(title, expected):
+    assert ft.title_sort(title) == expected
+
+
+@pytest.mark.parametrize(
+    "author,expected",
+    [
+        ("Ann Writer", "Writer, Ann"),
+        ("Dr. John Smith Jr.", "Smith, John Jr."),
+        ("John Smith MD PhD", "Smith, John MD PhD"),
+        ("Mr Smith", "Smith"),
+        ("Dr Mr", "Dr Mr"),
+        ("Jr Sr", "Jr Sr"),
+        ("xX_Dark_Xx", "xX_Dark_Xx"),
+        ("Smith, John", "Smith, John"),
+        ("John (Jack) Doe", "Doe, John"),
+        ("A (b c", "A (b c"),
+        ("Acme Media", "Acme Media"),
+        ("Ludwig van Beethoven", "Beethoven, Ludwig van"),
+        ("J. R. R. Tolkien", "Tolkien, J. R. R."),
+    ],
+)
+def test_author_sort(author, expected):
+    assert ft.author_sort(author) == expected
+
+
+@pytest.mark.parametrize(
+    "index,expected",
+    [(1, "1"), (2.0, "2"), (0, "0"), (2.5, "2.5"), (1.10, "1.1"), (3.14159, "3.14"), (2.001, "2."), (10.999, "11.")],
+)
+def test_series_index(index, expected):
+    assert ft.series_index(index) == expected
+
+
+@pytest.mark.parametrize(
+    "template,error",
+    [
+        ("{title:uppercase()}", "does not support template functions"),
+        ("{series:ifempty(none)}", "does not support template functions"),
+        ("{title:'uppercase($)'}", "does not support template functions"),
+        ("{title:.3s:'uppercase($)'}", "does not support template functions"),
+        ("program: field('title')", "does not support the program modes"),
+        ("python: def evaluate(book, context): return ''", "does not support the program modes"),
+        ("{title!r}", "does not support the field"),
+        ("{title.upper}", "does not support the field"),
+        ("{title[x]}", "does not support the field"),
+        ("{title:{id}}", "does not support the field"),
+        ("{}", "A field needs a name"),
+        ("{0}", "A field needs a name"),
+        ("{title", "The braces of the template do not match"),
+        ("title}", "The braces of the template do not match"),
+        ("{title:5q}", "The format '5q' cannot take"),
+        ("{series:|a|b|c}", "The format '|a' cannot take"),
+        ("{id:.3d}", "The format '.3d' cannot take"),
+        ("{title:>256s}", "A format can ask for 255 characters at most"),
+        ("{id:.999f}", "A format can ask for 255 characters at most"),
+        ("x" * 1025, "A template can have 1024 characters at most"),
+    ],
+)
+def test_check_refuses_what_bindery_does_not_support(template, error):
+    with pytest.raises(ft.TemplateError, match=error.replace("|", "\\|")):
+        ft.check(template)
+
+
+def test_check_accepts_the_templates_of_calibre():
+    for template in ("", "{author_sort}/{title}/{title} - {authors}", "{title:>255s}", "x" * 1024):
+        ft.check(template)
+
+
+@pytest.mark.parametrize("template", ["{series_index:03d}", "{title:d}", "{title[50]}", "{id:c}{title:c}"])
+def test_a_template_can_fail_for_one_book(template):
+    ft.check(template)
+    with pytest.raises(ft.TemplateError):
+        expand(template)
+
+
+def file_name(template, **changes):
+    return ft.file_name(template, ft.prepare({**BOOK, **changes}, 42))
+
+
+def test_file_name_is_the_last_part_of_the_path():
+    assert file_name("{author_sort}/{title}/{title} - {authors}") == "Book, The - Ann Writer & Ben Reader"
+    assert file_name("{series}//{title} /") == "Book, The"
+    assert file_name("{author_sort[0]}/{series}", series="") == "W"
+    assert file_name("{series}/{other}", series="") == ""
+    assert file_name(" / ") == ""
+
+
+def test_file_name_is_safe():
+    assert file_name("{title}?", title="A/B: C\\D") == "A_B_ C_D_"
+    assert file_name("<{id:c}>") == "___"
+    assert file_name("{title}", title="con") == "_con"
+    assert file_name("{title}.{id}", title="NUL") == "_NUL.42"
+    assert file_name("{title}", title="Console") == "Console"
+    assert file_name("{title}...") == "Book, The"
+    long = file_name("{title}", title="é" * 150)
+    assert long == "é" * 100
+    assert len(file_name("{title}", title="\U0001f600" * 150).encode()) == 200

tests/test_last_changed.py

@@ -0,0 +1,66 @@
+"""Last Changed records stored chapters and revisions, not changes to the built book."""
+
+import sqlite3
+
+import pytest
+
+from bindery import store as storage
+
+
+@pytest.fixture
+def state(tmp_path, monkeypatch):
+    clock = ["2024-01-01T00:00:00"]
+    monkeypatch.setattr(storage, "now", lambda: clock[0])
+    store = storage.Store(tmp_path / "stories.db")
+    sid = store.create_story("story", "site", "Title", "Author", [])
+    return store, sid, clock
+
+
+@pytest.mark.parametrize("source", ["new", "import"])
+def test_new_chapters_and_revisions_set_the_timestamp_in_the_store(state, source):
+    store, sid, clock = state
+    other = store.create_story("other", "site", "Other", "Author", [])
+    assert store.get_story(sid)["last_changed"] is None
+    first = store.add_chapter(sid, "one", "One", "first", "h1", source, 0)
+    assert store.get_story(sid)["last_changed"] == clock[0]
+    assert store.get_chapter(first)["added_at"] == clock[0]
+    clock[0] = "2024-02-01T00:00:00"
+    store.add_chapters(sid, [{"url": "two", "title": "Two", "html": "second", "text_hash": "h2"}], source, [])
+    assert store.get_story(sid)["last_changed"] == clock[0]
+    clock[0] = "2024-03-01T00:00:00"
+    store.add_revision(first, "revised", "h3", "recheck")
+    assert store.get_story(sid)["last_changed"] == clock[0]
+    assert store.get_chapter(first)["changed_at"] == clock[0]
+    assert store.list_revisions(first)[-1]["fetched_at"] == clock[0]
+    assert storage.Store(store.path).get_story(sid)["last_changed"] == clock[0]
+    assert store.get_story(other)["last_changed"] is None
+
+
+def test_changes_without_new_chapters_or_revisions_keep_the_timestamp(state):
+    store, sid, clock = state
+    first = store.add_chapter(sid, "one", "One", "first", "h1", "new", 0)
+    second = store.add_chapter(sid, "two", "Two", "second", "h2", "new", 1)
+    store.add_revision(first, "revised", "h3", "recheck")
+    previous = clock[0]
+    clock[0] = "2024-02-01T00:00:00"
+    store.update_story(sid, title="Edited", pinned={"title": "Edited"})
+    store.set_rules(sid, [{"pattern": "first", "replacement": "1st"}])
+    store.update_chapter(first, title="Renamed", included=False, on_site=False, current_rev=1)
+    store.replace_revision(first, 1, "original site text", "h4")
+    store.reorder_chapters(sid, lambda order: list(reversed(order)))
+    store.add_chapters(sid, [], "import", ["one", "two"])
+    store.delete_revision(first, 2)
+    store.delete_chapters(sid, [second])
+    assert store.get_story(sid)["last_changed"] == previous
+
+
+def test_a_failed_chapter_transaction_does_not_change_the_timestamp(state):
+    store, sid, clock = state
+    store.add_chapter(sid, "one", "One", "first", "h1", "new", 0)
+    previous = clock[0]
+    clock[0] = "2024-02-01T00:00:00"
+    chapters = [{"url": url, "title": url, "html": "text", "text_hash": "h"} for url in ("two", "one")]
+    with pytest.raises(sqlite3.IntegrityError):
+        store.add_chapters(sid, chapters, "import", [])
+    assert store.get_story(sid)["last_changed"] == previous
+    assert [chapter["url"] for chapter in store.list_chapters(sid)] == ["one"]

tests/test_opds.py

@@ -0,0 +1,278 @@
+"""OPDS navigation, book lists, downloads and the shared login gate."""
+
+from datetime import UTC, datetime
+from xml.etree import ElementTree as ET
+
+import pytest
+from test_api import client as client
+from test_api import cwa as cwa
+from test_build import PNG
+
+from bindery import diffing, fff, opds, orchestrator
+
+NS = {"a": opds.ATOM}
+ACQUIRE = "http://opds-spec.org/acquisition"
+PATHS = ["/opds", "/opds/", "/opds/title", "/opds/author", "/opds/changed"]
+
+
+@pytest.fixture(autouse=True)
+def enabled_catalog(client):
+    orchestrator.store.set_setting(None, "opds_enabled", True)
+
+
+def feed(response):
+    assert response.status_code == 200
+    assert response.mimetype == "application/atom+xml"
+    assert response.headers["Cache-Control"] == "private, no-store"
+    root = ET.fromstring(response.data)
+    assert root.tag == f"{{{opds.ATOM}}}feed"
+    assert root.findtext("a:id", namespaces=NS)
+    assert root.findtext("a:title", namespaces=NS)
+    assert datetime.fromisoformat(root.findtext("a:updated", namespaces=NS)).tzinfo is not None
+    return root
+
+
+def links(element):
+    return {link.attrib["rel"]: link.attrib["href"] for link in element.findall("a:link", NS)}
+
+
+def entries(root):
+    return root.findall("a:entry", NS)
+
+
+def titles(root):
+    return [entry.findtext("a:title", namespaces=NS) for entry in entries(root)]
+
+
+def add_book(title="A book", author="Writer", changed=None, url=None, included=True, meta=None):
+    store = orchestrator.store
+    url = url or f"https://example.com/books/{len(store.list_stories()) + 1}"
+    sid = store.create_story(url, "example.com", title, author, [])
+    if included:
+        html = "<p>Stored text.</p>"
+        store.add_chapter(sid, url + "/1", "One", html, diffing.text_hash(html), "site", 0)
+    store.update_story(sid, last_changed=changed, site_values=meta or {})
+    return sid
+
+
+@pytest.fixture(autouse=True)
+def no_site_requests(monkeypatch):
+    def fail(*args, **kwargs):
+        pytest.fail("OPDS must not contact a story site")
+
+    for name in ("fetch_story", "fetch_chapter", "collected_images"):
+        monkeypatch.setattr(fff, name, fail)
+
+
+def test_navigation_and_empty_lists(client):
+    root = feed(client.get("/opds"))
+    assert titles(root) == [title for title, _, _ in opds.LISTS.values()]
+    assert len({entry.findtext("a:id", namespaces=NS) for entry in entries(root)}) == 3
+    assert client.get("/opds/").data == client.get("/opds").data
+    for entry, (_, kind, text) in zip(entries(root), opds.LISTS.values(), strict=True):
+        assert entry.findtext("a:content", namespaces=NS) == text
+        link = entry.find("a:link", NS)
+        assert link.attrib["type"] == kind
+        books = feed(client.get(link.attrib["href"]))
+        assert entries(books) == []
+        assert links(books)["start"] == links(books)["up"] == "/opds"
+        assert "next" not in links(books)
+
+
+def test_sort_orders_dates_and_stable_book_ids(client):
+    add_book("The Zebra", "alice", "2024-01-01T12:00:00+00:00")
+    add_book("An Apple", "Bob", "2024-03-01T12:00:00+00:00")
+    add_book("Banana", "ALICE", "2024-02-01T12:00:00+00:00")
+    expected = {
+        "title": ["An Apple", "Banana", "The Zebra"],
+        "author?name=Alice": ["Banana", "The Zebra"],
+        "changed": ["An Apple", "Banana", "The Zebra"],
+    }
+    ids = []
+    for order, names in expected.items():
+        root = feed(client.get(f"/opds/{order}"))
+        assert titles(root) == names
+        if order != "author?name=Alice":
+            assert root.findtext("a:updated", namespaces=NS) == "2024-03-01T12:00:00Z"
+        ids.append({e.findtext("a:id", namespaces=NS) for e in entries(root)})
+        for entry in entries(root):
+            assert entry.findtext("a:author/a:name", namespaces=NS)
+            assert entry.findtext("a:content", namespaces=NS) == "Chapters: 1"
+            assert entry.find("a:link", NS).attrib["type"] == "application/epub+zip"
+    assert ids[0] == ids[2] > ids[1]
+
+
+def test_dates_use_creation_fallback_and_compare_offsets(client):
+    sid = add_book("Created")
+    created = orchestrator.store.get_story(sid)["created_at"]
+    expected = datetime.fromisoformat(created).astimezone(UTC).isoformat().replace("+00:00", "Z")
+    root = feed(client.get("/opds/changed"))
+    assert entries(root)[0].findtext("a:updated", namespaces=NS) == expected
+    orchestrator.store.update_story(sid, last_changed="2024-01-01T12:00:00+02:00")
+    add_book("Later", changed="2024-01-01T11:00:00+00:00")
+    assert titles(feed(client.get("/opds/changed"))) == ["Later", "Created"]
+
+
+def test_xml_escaping_unicode_and_missing_metadata(client):
+    add_book('Books <&> "日本語"\x00', "A & B <writers>\x0b")
+    empty = add_book("", "")
+    root = feed(client.get("/opds/title"))
+    assert titles(root)[0] == 'Books <&> "日本語"'
+    assert entries(root)[0].findtext("a:author/a:name", namespaces=NS) == "A & B <writers>"
+    assert titles(root)[1] == orchestrator.store.get_story(empty)["url"]
+    assert entries(root)[1].findtext("a:author/a:name", namespaces=NS) == "Unknown"
+
+
+def test_only_books_with_included_chapters_appear(client):
+    add_book("Pending", included=False)
+    sid = add_book("Excluded")
+    chapter = orchestrator.store.list_chapters(sid)[0]
+    orchestrator.store.update_chapter(chapter["id"], included=False)
+    add_book("Ready")
+    assert titles(feed(client.get("/opds/title"))) == ["Ready"]
+
+
+@pytest.mark.parametrize("order", ["title", "changed", "author?name=Same", "search?q=same"])
+def test_pagination_with_ties(client, monkeypatch, order):
+    monkeypatch.setattr(opds, "PAGE_SIZE", 2)
+    for _ in range(5):
+        add_book("Same", "Same", "2024-01-01T00:00:00+00:00")
+    first = feed(client.get(f"/opds/{order}"))
+    second = feed(client.get(links(first)["next"]))
+    last = feed(client.get(links(second)["next"]))
+    assert [len(entries(root)) for root in (first, second, last)] == [2, 2, 1]
+    assert "previous" not in links(first)
+    assert "next" not in links(last)
+    assert links(second)["previous"] == links(first)["self"]
+    assert links(last)["previous"] == links(second)["self"]
+    ids = [e.findtext("a:id", namespaces=NS) for root in (first, second, last) for e in entries(root)]
+    assert ids == [f"https://example.com/books/{i}" for i in range(1, 6)]
+    assert client.get(links(first)["self"].replace("page=1", "page=4")).status_code == 404
+
+
+@pytest.mark.parametrize("page", ["0", "-1", "1.5", "abc", "", "999999999999999999999"])
+def test_bad_page(client, page):
+    assert client.get("/opds/title", query_string={"page": page}).status_code == 400
+
+
+def test_unknown_list(client):
+    assert client.get("/opds/missing").status_code == 404
+
+
+DC = {"a": opds.ATOM, "dc": opds.DC}
+
+
+def test_entry_shows_the_book_metadata_and_cover(client):
+    sid = add_book(
+        "Stored title",
+        meta={
+            "title": "The Book",
+            "authors": ["Ann Writer", "Ben Reader"],
+            "comments": "<p>First <b>part</b>.</p><p>Second part.</p>",
+            "tags": ["Space", "Fiction"],
+            "series": "The Saga",
+            "series_index": 2.0,
+            "pubdate": "2020-05-06T12:00:00+00:00",
+            "languages": "en",
+        },
+    )
+    orchestrator.store.update_story(sid, pinned={"title": "The Kept Title"})
+    entry = entries(feed(client.get("/opds/title")))[0]
+    assert entry.findtext("a:title", namespaces=NS) == "The Kept Title"
+    assert [a.findtext("a:name", namespaces=NS) for a in entry.findall("a:author", NS)] == ["Ann Writer", "Ben Reader"]
+    assert entry.find("a:author/a:uri", NS).text == "/opds/author?name=Ann+Writer"
+    assert entry.find("a:content", NS).attrib["type"] == "text"
+    assert entry.findtext("a:content", namespaces=NS) == (
+        "Series: The Saga, book 2\nChapters: 1\n\nFirst part.\n\nSecond part."
+    )
+    assert [c.attrib["term"] for c in entry.findall("a:category", NS)] == ["Space", "Fiction"]
+    assert entry.findtext("dc:language", namespaces=DC) == "en"
+    assert entry.findtext("dc:issued", namespaces=DC) == "2020-05-06"
+    assert entry.findtext("dc:publisher", namespaces=DC) == "example.com"
+    assert "http://opds-spec.org/image" not in links(entry)
+
+    orchestrator.store.put_image(sid, "https://example.com/cover.png", "png", "image/png", PNG, is_cover=True)
+    entry = entries(feed(client.get("/opds/title")))[0]
+    image = links(entry)["http://opds-spec.org/image"]
+    assert image == links(entry)["http://opds-spec.org/image/thumbnail"]
+    assert image.startswith(f"/opds/books/{sid}/cover?v=")
+    response = client.get(image)
+    assert response.data == PNG
+    assert response.mimetype == "image/png"
+    assert "private" in response.headers["Cache-Control"]
+    orchestrator.store.put_image(sid, "https://example.com/new.png", "png", "image/png", PNG, is_cover=True)
+    assert links(entries(feed(client.get("/opds/title")))[0])["http://opds-spec.org/image"] != image
+    assert client.get(f"/opds/books/{add_book()}/cover").status_code == 404
+
+
+def test_entry_shows_the_page_count_of_a_built_book(client):
+    sid = add_book()
+    orchestrator.store.set_setting(sid, "page_count", 321)
+    add_book(title="Not built")
+    contents = {
+        e.findtext("a:title", namespaces=NS): e.findtext("a:content", namespaces=NS)
+        for e in entries(feed(client.get("/opds/title")))
+    }
+    assert contents == {"A book": "Chapters: 1\nPages: 321", "Not built": "Chapters: 1"}
+
+
+def test_cover_of_an_unsafe_type_is_left_out(client):
+    sid = add_book()
+    orchestrator.store.put_image(sid, "https://example.com/c.svg", "svg", "image/svg+xml", b"<svg/>", is_cover=True)
+    assert "http://opds-spec.org/image" not in links(entries(feed(client.get("/opds/title")))[0])
+    assert client.get(f"/opds/books/{sid}/cover").status_code == 404
+
+
+def test_authors_in_surname_order_with_their_books(client):
+    add_book("One", "Zed Adams")
+    add_book("Two", "ann writer")
+    add_book("Three", "", meta={"authors": ["Ann Writer", "Bob Brown"]})
+    add_book("Four", "")
+    root = feed(client.get("/opds/author"))
+    assert titles(root) == ["Zed Adams", "Bob Brown", "Unknown", "Ann Writer"]
+    assert [e.findtext("a:content", namespaces=NS) for e in entries(root)] == ["1 book", "1 book", "1 book", "2 books"]
+    assert links(root)["up"] == "/opds"
+    writer = feed(client.get(links(entries(root)[3])["subsection"]))
+    assert titles(writer) == ["Three", "Two"]
+    assert links(writer)["up"] == "/opds/author"
+    assert client.get("/opds/author", query_string={"name": "Nobody"}).status_code == 404
+
+
+def test_author_list_pages(client, monkeypatch):
+    monkeypatch.setattr(opds, "PAGE_SIZE", 2)
+    for name in ("A One", "B Two", "C Three"):
+        add_book(author=name)
+    first = feed(client.get("/opds/author"))
+    assert titles(first) == ["A One", "C Three"]
+    assert titles(feed(client.get(links(first)["next"]))) == ["B Two"]
+
+
+def test_search(client):
+    add_book("The Long Road", "Ann Writer", meta={"tags": ["Adventure"]})
+    add_book("Short Road", "Ben Reader", meta={"series": "Roads"})
+    add_book("Other", "Cat Smith")
+
+    def search(q):
+        return titles(feed(client.get("/opds/search", query_string={"q": q})))
+
+    assert search("road") == ["The Long Road", "Short Road"]
+    assert search("ROAD writer") == ["The Long Road"]
+    assert search("adventure") == ["The Long Road"]
+    assert search("roads") == ["Short Road"]
+    assert search("missing") == search("") == search("  ") == []
+    assert feed(client.get("/opds/search")).findtext("a:title", namespaces=NS) == "Search: "
+
+
+def test_every_feed_links_the_search(client):
+    add_book()
+    for path in [*PATHS, "/opds/author?name=Writer", "/opds/search?q=book"]:
+        searches = [link for link in feed(client.get(path)).findall("a:link", NS) if link.attrib["rel"] == "search"]
+        found = {link.attrib["type"]: link.attrib["href"] for link in searches}
+        assert found == {
+            "application/opensearchdescription+xml": "/opds/search.xml",
+            opds.ACQUISITION: "/opds/search?q={searchTerms}",
+        }
+    response = client.get("/opds/search.xml")
+    assert response.mimetype == "application/opensearchdescription+xml"
+    url = ET.fromstring(response.data).find(f"{{{opds.OPENSEARCH}}}Url")
+    assert url.attrib == {"type": opds.ACQUISITION, "template": "/opds/search?q={searchTerms}"}

tests/test_opds_auth.py

@@ -0,0 +1,329 @@
+"""The login of the OPDS catalog: the account login in the local and CWA modes, a password of its own in proxy mode."""
+
+import base64
+import io
+import zipfile
+
+import pytest
+from test_api import H, login
+from test_api import client as client
+from test_api import cwa as cwa
+from test_build import URL, metahtml
+from test_opds import PATHS, add_book, enabled_catalog, entries, feed, links, no_site_requests
+from werkzeug.security import generate_password_hash
+
+from bindery import auth, config, opds, orchestrator, web
+
+__all__ = ["enabled_catalog", "no_site_requests"]  # fixtures that every test here uses
+
+PASSWORD = "reader-password"
+CHALLENGE = 'Basic realm="Bindery", charset="UTF-8"'
+ACQUIRE = "http://opds-spec.org/acquisition"
+GATED = [*PATHS, "/opds/search?q=a", "/opds/search.xml", "/opds/books/1.epub", "/opds/books/1/cover"]
+OTHER_LOGINS = [
+    {"Authorization": "Bearer api-secret"},
+    {"X-Api-Key": "api-secret"},
+    {"X-Remote-User": "admin"},
+]
+
+
+def basic(password, user="reader"):
+    return {"Authorization": "Basic " + base64.b64encode(f"{user}:{password}".encode()).decode()}
+
+
+def set_password(password=PASSWORD, user="reader"):
+    hashed = generate_password_hash(password, method="pbkdf2:sha256:1000")
+    orchestrator.store.set_setting(None, orchestrator._opds_password_name(user), hashed)
+
+
+def fresh_lockout(monkeypatch):
+    monkeypatch.setattr(auth, "limiter", auth.LoginLimiter())
+
+
+@pytest.fixture
+def proxy(monkeypatch):
+    monkeypatch.setattr(config, "AUTH_METHOD", "proxy")
+    monkeypatch.setattr(config, "PROXY_AUTH_TRUSTED_IPS", ["127.0.0.1"])
+    monkeypatch.setattr(config, "API_KEY", "api-secret")
+    set_password()
+
+
+@pytest.fixture
+def local(monkeypatch):
+    monkeypatch.setattr(config, "AUTH_METHOD", "local")
+    monkeypatch.setattr(config, "LOCAL_USERNAME", "reader")
+    monkeypatch.setattr(config, "LOCAL_PASSWORD_HASH", generate_password_hash("web-pw", method="pbkdf2:sha256:1000"))
+    monkeypatch.setattr(config, "API_KEY", "api-secret")
+    set_password()  # an OPDS password does not apply here
+
+
+def test_proxy_mode_takes_the_proxy_user_name_with_the_password_of_that_user(client, proxy, monkeypatch):
+    add_book()
+    set_password("alice-password", "alice")
+    for path in GATED:
+        response = client.get(path)
+        assert response.status_code == 401
+        assert response.headers["WWW-Authenticate"] == CHALLENGE
+    for path in PATHS:
+        response = client.get(path, headers=basic(PASSWORD, "reader"))
+        feed(response)
+        assert "Set-Cookie" not in response.headers
+    assert client.get("/opds/search.xml", headers=basic("alice-password", "alice")).status_code == 200
+    # The user name matches without regard to case and spaces. Each user has their own password.
+    assert client.get("/opds", headers=basic(PASSWORD, " READER ")).status_code == 200
+    for headers in [
+        basic(PASSWORD, "alice"),
+        basic("alice-password", "reader"),
+        basic(PASSWORD, "nobody"),
+        basic(PASSWORD, ""),
+        *OTHER_LOGINS,
+        basic("api-secret"),
+        {**OTHER_LOGINS[2], **basic("wrong")},
+        {"X-Remote-User": "alice", **basic(PASSWORD, "nobody")},
+    ]:
+        fresh_lockout(monkeypatch)
+        for path in PATHS[:3]:
+            assert client.get(path, headers=headers).status_code == 401
+
+
+def test_the_catalog_password_does_not_open_the_api(client, proxy):
+    for headers in (basic(PASSWORD), {"Authorization": f"Bearer {PASSWORD}"}, {"X-Api-Key": PASSWORD}):
+        response = client.get("/api/stories", headers=headers)
+        assert response.status_code == 401
+        assert "WWW-Authenticate" not in response.headers
+    assert client.get("/api/stories", headers={"X-Api-Key": "api-secret"}).status_code == 200
+    assert client.post("/api/stories", json={"urls": []}, headers={**H, **basic(PASSWORD)}).status_code == 401
+
+
+def test_a_user_without_a_password_meets_the_same_work_as_one_with_a_password(client, proxy, monkeypatch):
+    calls = []
+    real = auth.check_password_hash
+    monkeypatch.setattr(auth, "check_password_hash", lambda *args: calls.append(args[0]) or real(*args))
+    for user in ("nobody", "reader"):
+        calls.clear()
+        response = client.get("/opds", headers=basic("wrong-password", user))
+        assert response.status_code == 401
+        assert response.json == {"error": "Unauthorized"}
+        assert len(calls) == 1
+
+
+def test_local_mode_takes_the_account_login(client, local, monkeypatch):
+    for path in GATED:
+        response = client.get(path)
+        assert response.status_code == 401
+        assert response.headers["WWW-Authenticate"] == CHALLENGE
+        assert response.json["error"] == "Unauthorized"
+    for path in PATHS:
+        response = client.get(path, headers=basic("web-pw"))
+        feed(response)
+        assert "Set-Cookie" not in response.headers
+    for headers in [basic(PASSWORD), basic("web-pw", "other"), basic("wrong"), *OTHER_LOGINS]:
+        fresh_lockout(monkeypatch)
+        assert client.get("/opds", headers=headers).status_code == 401
+    assert client.get("/opds", headers=basic("web-pw", " reader ")).status_code == 200
+
+
+def test_local_mode_takes_the_web_session_but_not_the_api_key_with_it(client, local):
+    assert client.get("/opds").status_code == 401
+    assert login(client, password="web-pw").status_code == 200
+    for path in PATHS:
+        feed(client.get(path))
+    assert client.get("/opds", headers={"X-Api-Key": "api-secret"}).status_code == 200  # the session counts
+    client.post("/api/auth/logout", headers=H)
+    assert client.get("/opds").status_code == 401
+    assert client.get("/opds", headers={"X-Api-Key": "api-secret"}).status_code == 401
+
+
+def test_cwa_mode_takes_the_cwa_login(client, cwa, monkeypatch):
+    monkeypatch.setattr(config, "API_KEY", "api-secret")
+    set_password()
+    assert client.get("/opds").status_code == 401
+    assert client.get("/opds", headers=basic("pw")).status_code == 200
+    for headers in [basic("wrong"), basic("pw", "other"), basic(PASSWORD), *OTHER_LOGINS]:
+        fresh_lockout(monkeypatch)
+        assert client.get("/opds", headers=headers).status_code == 401
+    assert login(client).status_code == 200
+    feed(client.get("/opds/title"))
+    client.post("/api/auth/logout", headers=H)
+    monkeypatch.setattr(config, "CWA_ALLOWED_USERS", ["someone"])
+    assert client.get("/opds", headers=basic("pw")).status_code == 401
+
+
+def test_none_mode_needs_no_login(client):
+    assert orchestrator.store.get_setting(None, "opds_password_hash") is None
+    for path in PATHS:
+        feed(client.get(path))
+    set_password()
+    feed(client.get("/opds"))
+
+
+def test_a_good_account_login_is_kept_for_five_minutes(client, local, monkeypatch):
+    clock = [1000.0]
+    monkeypatch.setattr(auth.time, "monotonic", lambda: clock[0])
+    calls = []
+    real = auth.login_local
+    monkeypatch.setattr(auth, "login_local", lambda *args: calls.append(args) or real(*args))
+    for seconds in (0, auth.KEPT_SECONDS - 1, 1):
+        clock[0] += seconds
+        assert client.get("/opds", headers=basic("web-pw")).status_code == 200
+    assert len(calls) == 2
+    # A wrong password is never kept. A new web password ends a kept login.
+    assert client.get("/opds", headers=basic("wrong")).status_code == 401
+    assert client.get("/opds", headers=basic("wrong")).status_code == 401
+    assert len(calls) == 4
+    monkeypatch.setattr(config, "LOCAL_PASSWORD_HASH", generate_password_hash("new-pw", method="pbkdf2:sha256:1000"))
+    assert client.get("/opds", headers=basic("web-pw")).status_code == 401
+    assert client.get("/opds", headers=basic("new-pw")).status_code == 200
+
+
+def test_a_good_catalog_password_is_kept_and_a_new_password_ends_it(client, proxy, monkeypatch):
+    calls = []
+    real = auth.check_password_hash
+    monkeypatch.setattr(auth, "check_password_hash", lambda *args: calls.append(args) or real(*args))
+    for _ in range(3):
+        assert client.get("/opds", headers=basic(PASSWORD)).status_code == 200
+    assert len(calls) == 1
+    set_password("a new password")
+    assert client.get("/opds", headers=basic(PASSWORD)).status_code == 401
+    assert client.get("/opds", headers=basic("a new password")).status_code == 200
+
+
+def test_a_disabled_catalog_answers_404_before_any_login(client, proxy):
+    orchestrator.store.set_setting(None, "opds_enabled", False)
+    for headers in ({}, basic(PASSWORD), basic("wrong")):
+        assert client.get("/opds", headers=headers).status_code == 404
+
+
+def test_wrong_catalog_passwords_lock_out_a_user_at_an_address(client, proxy):
+    set_password("alice-password", "alice")
+    for _ in range(auth.MAX_FAILURES):
+        assert client.get("/opds", headers=basic("wrong")).status_code == 401
+    assert client.get("/opds", headers=basic(PASSWORD)).status_code == 401
+    # Another user at that address, and the same user at another address, can log in.
+    assert client.get("/opds", headers=basic("alice-password", "alice")).status_code == 200
+    other = {"environ_base": {"REMOTE_ADDR": "198.51.100.7"}}
+    assert client.get("/opds", headers=basic(PASSWORD), **other).status_code == 200
+
+
+def test_wrong_account_passwords_lock_out_an_address(client, local):
+    for _ in range(auth.MAX_FAILURES):
+        assert client.get("/opds", headers=basic("wrong")).status_code == 401
+    assert client.get("/opds", headers=basic("web-pw")).status_code == 401
+    assert client.get("/opds", headers=basic("web-pw"), environ_base={"REMOTE_ADDR": "198.51.100.7"}).status_code == 200
+
+
+@pytest.mark.parametrize(
+    "value",
+    ["Basic", "Basic !!!", "Digest cmVhZGVyOnB3", basic("")["Authorization"], basic("x" * 257)["Authorization"]]
+    + ["Basic " + base64.b64encode(text).decode() for text in (b"reader", b"\xff:pw")],
+)
+def test_bad_basic_headers(client, proxy, monkeypatch, value):
+    def fail(*args):
+        pytest.fail("A bad header must not reach the hash")
+
+    monkeypatch.setattr(auth, "check_password_hash", fail)
+    assert client.get("/opds", headers={"Authorization": value}).status_code == 401
+
+
+def test_bad_config_fails_closed(client, monkeypatch):
+    set_password()
+    monkeypatch.setattr(config, "AUTH_METHOD", "invalid")
+    for path in PATHS:
+        assert client.get(path, headers=basic(PASSWORD)).status_code == 503
+
+
+def test_host_protection(client):
+    assert client.get("/opds", base_url="http://attacker.example").status_code == 400
+
+
+def test_url_base_and_download_auth(client, proxy, monkeypatch, tmp_path):
+    monkeypatch.setattr(web.app, "wsgi_app", web.PrefixMiddleware(web.app.wsgi_app, "/bindery"))
+    monkeypatch.setattr(opds, "PAGE_SIZE", 1)
+    sid = add_book()
+    add_book("Second")
+    headers = basic(PASSWORD)
+    root = feed(client.get("/bindery/opds", headers=headers))
+    for entry in entries(root):
+        books = feed(client.get(links(entry)["subsection"], headers=headers))
+        assert all(href.startswith("/bindery/") for href in links(books).values())
+        if entry.find("a:link", {"a": opds.ATOM}).attrib["type"] == opds.ACQUISITION:
+            assert client.get(links(books)["next"], headers=headers).status_code == 200
+    books = feed(client.get("/bindery/opds/title", headers=headers))
+    download = links(entries(books)[0])[ACQUIRE]
+    assert download == f"/bindery/opds/books/{sid}.epub"
+    assert client.get(download).status_code == 401
+    assert client.get(download, headers=basic("wrong")).status_code == 401
+    file = tmp_path / "book.epub"
+    file.write_bytes(b"epub bytes")
+    monkeypatch.setattr(orchestrator, "build_for_download", lambda story_id: (file.open("rb"), "book.epub"))
+    with client.get(download, headers=headers) as response:
+        assert response.data == b"epub bytes"
+        assert response.mimetype == "application/epub+zip"
+    assert client.get("/opds", headers=headers).status_code == 404
+
+
+def test_download_builds_from_store_without_changing_chapters(client, local, monkeypatch, tmp_path):
+    monkeypatch.setattr(config, "TMP_DIR", tmp_path)
+    sid = add_book(url=URL)
+    orchestrator.store.update_story(sid, metahtml=metahtml())
+    before = orchestrator.store.chapters_for_build(sid)
+    root = feed(client.get("/opds/title", headers=basic("web-pw")))
+    with client.get(links(entries(root)[0])[ACQUIRE], headers=basic("web-pw")) as response:
+        assert response.status_code == 200
+        assert response.mimetype == "application/epub+zip"
+        with zipfile.ZipFile(io.BytesIO(response.data)) as epub:
+            assert epub.read("mimetype") == b"application/epub+zip"
+    assert orchestrator.store.chapters_for_build(sid) == before
+
+
+def test_the_catalog_takes_no_changes(client, proxy):
+    assert client.post("/opds", headers=basic(PASSWORD)).status_code == 405
+
+
+def test_an_opds_download_is_named_by_the_download_template(client, local, monkeypatch, tmp_path):
+    monkeypatch.setattr(config, "TMP_DIR", tmp_path)
+    sid = add_book(url=URL, meta={"title": "The Book", "authors": ["Ann Writer"]})
+    orchestrator.store.update_story(sid, metahtml=metahtml())
+    root = feed(client.get("/opds/title", headers=basic("web-pw")))
+    href = links(entries(root)[0])[ACQUIRE]
+
+    def disposition():
+        with client.get(href, headers=basic("web-pw")) as response:
+            assert response.status_code == 200
+            return response.headers["Content-Disposition"]
+
+    assert 'filename="Book, The - Ann Writer.epub"' in disposition()
+    orchestrator.store.set_setting(None, "download_template", "{title} - {authors}")
+    assert 'filename="The Book - Ann Writer.epub"' in disposition()
+
+
+def test_forget_clears_the_wrong_passwords_of_one_user_from_every_address():
+    limiter = auth.LoginLimiter()
+    for address in ("10.0.0.1", "10.0.0.2"):
+        for _ in range(auth.MAX_FAILURES):
+            limiter.failed("Alice", address)
+            limiter.failed("bob", address)
+    limiter.forget("ALICE")
+    assert not limiter.locked("alice", "10.0.0.1")
+    assert not limiter.locked("alice", "10.0.0.2")
+    assert limiter.locked("bob", "10.0.0.1")
+
+
+def test_setting_a_password_ends_the_lockout_of_that_user(client, proxy):
+    set_password("bob-password", "bob")
+    far = {"environ_base": {"REMOTE_ADDR": "198.51.100.7"}}
+    # A user without a password tries from two addresses, and meets the lockout.
+    for kwargs in ({}, far):
+        for _ in range(auth.MAX_FAILURES):
+            assert client.get("/opds", headers=basic("guess", "alice"), **kwargs).status_code == 401
+    for _ in range(auth.MAX_FAILURES):
+        assert client.get("/opds", headers=basic("guess", "bob")).status_code == 401
+    assert client.get("/opds", headers=basic("alice-password", "alice")).status_code == 401
+    saved = client.put(
+        "/api/settings/opds-password", json={"password": "alice-password"}, headers={**H, "X-Remote-User": "Alice"}
+    )
+    assert saved.status_code == 200
+    for kwargs in ({}, far):
+        assert client.get("/opds", headers=basic("alice-password", "alice"), **kwargs).status_code == 200
+    # The lockout of another user stays.
+    assert client.get("/opds", headers=basic("bob-password", "bob")).status_code == 401

tests/test_opds_settings.py

@@ -0,0 +1,186 @@
+"""The OPDS catalog stays off until an admin enables it in Settings. An admin also sets its password there."""
+
+import base64
+
+import pytest
+from test_api import H
+from test_api import client as client
+from werkzeug.security import check_password_hash, generate_password_hash
+
+from bindery import config, orchestrator, web
+from bindery.store import Store
+
+PATHS = ["/opds", "/opds/", "/opds/title", "/opds/author", "/opds/changed"]
+
+
+def test_catalog_defaults_to_off_and_can_be_toggled_without_restart(client, monkeypatch):
+    assert client.get("/api/settings").json["opds_enabled"] is False
+    assert orchestrator.store.get_setting(None, "opds_enabled") is None
+    for enabled in (False, True, False):
+        if enabled or orchestrator.store.get_setting(None, "opds_enabled") is not None:
+            saved = client.put("/api/settings", json={"opds_enabled": enabled}, headers=H)
+            assert saved.status_code == 200
+            assert saved.json["opds_enabled"] is enabled
+        for path in PATHS:
+            for method in (client.get, client.head):
+                assert method(path).status_code == (200 if enabled else 404)
+        if enabled:
+            # A fresh store instance reads the saved setting, as after a restart.
+            monkeypatch.setattr(orchestrator, "store", Store(orchestrator.store.path))
+            assert client.get("/api/settings").json["opds_enabled"] is True
+            unchanged = client.put("/api/settings", json={"recent_chapters": 2}, headers=H)
+            assert unchanged.json["opds_enabled"] is True
+            unchanged = client.put("/api/settings", json={"opds_enabled": None}, headers=H)
+            assert unchanged.json["opds_enabled"] is True
+
+
+@pytest.mark.parametrize("value", ["false", "true", 0, 1, [], {}])
+def test_opds_setting_requires_a_boolean_and_rejects_all_changes(client, value):
+    before = client.get("/api/settings").json
+    response = client.put("/api/settings", json={"opds_enabled": value, "recent_chapters": 9}, headers=H)
+    assert response.status_code == 400
+    assert "true or false" in response.json["error"]
+    assert client.get("/api/settings").json == before
+
+
+def test_invalid_download_settings_do_not_enable_opds(client):
+    response = client.put(
+        "/api/settings", json={"opds_enabled": True, "download_template": "{title:uppercase()}"}, headers=H
+    )
+    assert response.status_code == 400
+    assert client.get("/api/settings").json["opds_enabled"] is False
+    assert client.get("/opds").status_code == 404
+
+
+def test_only_an_admin_can_enable_opds_and_csrf_header_is_required(client, monkeypatch):
+    monkeypatch.setattr(config, "AUTH_METHOD", "proxy")
+    monkeypatch.setattr(config, "ADMIN_USERS", ["admin"])
+    monkeypatch.setattr(config, "PROXY_AUTH_TRUSTED_IPS", ["127.0.0.1"])
+    assert client.put("/api/settings", json={"opds_enabled": True}, headers=H).status_code == 401
+    assert (
+        client.put("/api/settings", json={"opds_enabled": True}, headers={**H, "X-Remote-User": "reader"}).status_code
+        == 403
+    )
+    assert (
+        client.put("/api/settings", json={"opds_enabled": True}, headers={"X-Remote-User": "admin"}).status_code == 403
+    )
+    assert orchestrator.store.get_setting(None, "opds_enabled") is None
+    response = client.put("/api/settings", json={"opds_enabled": True}, headers={**H, "X-Remote-User": "admin"})
+    assert response.status_code == 200
+    assert response.json["opds_enabled"] is True
+    assert client.get("/opds").status_code == 401
+    # The proxy user header does not open the catalog. Only the catalog password does.
+    assert client.get("/opds", headers={"X-Remote-User": "reader"}).status_code == 401
+
+
+PASSWORD_URL = "/api/settings/opds-password"
+
+
+def basic(password, user="reader"):
+    return {"Authorization": "Basic " + base64.b64encode(f"{user}:{password}".encode()).decode()}
+
+
+def as_user(name):
+    return {**H, "X-Remote-User": name}
+
+
+@pytest.fixture
+def proxy(monkeypatch):
+    monkeypatch.setattr(config, "AUTH_METHOD", "proxy")
+    monkeypatch.setattr(config, "PROXY_AUTH_TRUSTED_IPS", ["127.0.0.1"])
+    monkeypatch.setattr(config, "ADMIN_USERS", ["admin"])
+    monkeypatch.setattr(config, "API_KEY", "key")
+
+
+def test_disabled_catalog_answers_404_under_url_base_with_any_credentials(client, proxy, monkeypatch):
+    monkeypatch.setattr(web.app, "wsgi_app", web.PrefixMiddleware(web.app.wsgi_app, "/bindery"))
+    keyed = {"X-Api-Key": "key"}
+    for path in PATHS:
+        for headers in (keyed, basic("reader-password")):
+            assert client.get("/bindery" + path, headers=headers).status_code == 404
+    assert client.put("/bindery/api/settings", json={"opds_enabled": True}, headers=keyed).status_code == 200
+    saved = client.put("/bindery" + PASSWORD_URL, json={"password": "reader-password"}, headers=as_user("reader"))
+    assert saved.status_code == 200
+    for path in PATHS:
+        assert client.get("/bindery" + path, headers=keyed).status_code == 401
+        assert client.get("/bindery" + path, headers=basic("reader-password")).status_code == 200
+
+
+def test_each_user_sets_their_own_password_and_it_is_stored_as_a_hash(client, proxy):
+    client.put("/api/settings", json={"opds_enabled": True}, headers=as_user("admin"))
+    for user in ("reader", "admin"):
+        info = client.get("/api/settings", headers=as_user(user)).json
+        assert (info["opds_user"], info["opds_password_set"]) == (user, False)
+    saved = client.put(PASSWORD_URL, json={"password": "first-password"}, headers=as_user("Reader"))
+    assert saved.json == {"opds_user": "Reader", "opds_password_set": True}
+    stored = orchestrator.store.get_setting(None, "opds_password_hash:reader")
+    assert "first-password" not in stored
+    assert check_password_hash(stored, "first-password")
+    # The answers show the user name and the state, never the password or its hash.
+    for response in (saved, client.get("/api/settings", headers=as_user("reader"))):
+        assert "first-password" not in response.text
+        assert stored not in response.text
+    assert client.get("/api/settings", headers=as_user("reader")).json["opds_password_set"] is True
+    assert client.get("/api/settings", headers=as_user("admin")).json["opds_password_set"] is False
+    assert client.get("/opds", headers=basic("first-password", "reader")).status_code == 200
+    assert client.get("/opds", headers=basic("first-password", "admin")).status_code == 401
+    # A new password replaces the old one. Other changes leave it as it is.
+    client.put(PASSWORD_URL, json={"password": "second-password"}, headers=as_user("reader"))
+    client.put("/api/settings", json={"recent_chapters": 2}, headers=as_user("admin"))
+    assert client.get("/opds", headers=basic("first-password")).status_code == 401
+    assert client.get("/opds", headers=basic("second-password")).status_code == 200
+    client.put(PASSWORD_URL, json={"password": "admin-password"}, headers=as_user("admin"))
+    assert client.get("/opds", headers=basic("admin-password", "admin")).status_code == 200
+    assert client.get("/opds", headers=basic("second-password")).status_code == 200
+
+
+@pytest.mark.parametrize("value", [None, "", "short", "x" * 7, "x" * 257, 12345678, ["password"], True])
+def test_bad_password_changes_nothing(client, proxy, value):
+    response = client.put(PASSWORD_URL, json={"password": value}, headers=as_user("reader"))
+    assert response.status_code == 400
+    assert "OPDS password" in response.json["error"]
+    assert orchestrator.store.get_setting(None, "opds_password_hash:reader") is None
+
+
+def test_a_password_of_the_limits_is_accepted(client, proxy):
+    for password in ("x" * 8, "é" * 256):
+        assert client.put(PASSWORD_URL, json={"password": password}, headers=as_user("reader")).status_code == 200
+
+
+def test_the_setting_route_ignores_a_password(client, proxy):
+    client.put("/api/settings", json={"opds_password": "reader-password"}, headers=as_user("admin"))
+    assert orchestrator.store.get_setting(None, "opds_password_hash:admin") is None
+
+
+@pytest.mark.parametrize("mode", ["none", "local", "cwa"])
+def test_only_the_proxy_mode_has_an_opds_password(client, monkeypatch, mode):
+    monkeypatch.setattr(config, "AUTH_METHOD", mode)
+    monkeypatch.setattr(config, "LOCAL_USERNAME", "reader")
+    monkeypatch.setattr(config, "LOCAL_PASSWORD_HASH", generate_password_hash("pw", method="pbkdf2:sha256:1000"))
+    monkeypatch.setattr(config, "CWA_URL", "http://localhost:9999")
+    monkeypatch.setattr(config, "API_KEY", "key")
+    keyed = {"X-Api-Key": "key"}
+    response = client.put(PASSWORD_URL, json={"password": "reader-password"}, headers=keyed)
+    assert response.status_code == 400
+    assert "through the proxy" in response.json["error"]
+    info = client.get("/api/settings", headers=keyed).json
+    assert (info["opds_user"], info["opds_password_set"]) == (None, False)
+
+
+def test_a_user_needs_the_login_of_the_proxy_to_set_a_password(client, proxy):
+    body = {"password": "reader-password"}
+    assert client.put(PASSWORD_URL, json=body, headers=H).status_code == 401
+    assert client.put(PASSWORD_URL, json=body, headers={"X-Api-Key": "key"}).status_code == 400
+    assert client.put(PASSWORD_URL, json=body, headers={"X-Remote-User": "reader"}).status_code == 403  # no CSRF header
+    assert client.put(PASSWORD_URL, json=body, headers=as_user("reader")).status_code == 200
+    untrusted = client.put(
+        PASSWORD_URL, json=body, headers=as_user("reader"), environ_base={"REMOTE_ADDR": "198.51.100.7"}
+    )
+    assert untrusted.status_code == 401
+
+
+def test_config_tells_the_page_the_login_mode(client, monkeypatch):
+    assert client.get("/api/config").json["auth_mode"] == "none"
+    monkeypatch.setattr(config, "AUTH_METHOD", "proxy")
+    monkeypatch.setattr(config, "PROXY_AUTH_TRUSTED_IPS", ["127.0.0.1"])
+    assert client.get("/api/config", headers={"X-Remote-User": "reader"}).json["auth_mode"] == "proxy"

tests/test_optional_cwa.py

@@ -40,7 +40,7 @@ def library(isolated, monkeypatch):
     monkeypatch.setattr(orchestrator, "_find_book", Mock(return_value=None))
 
 
-@pytest.mark.parametrize("enabled", [False, True])
+@pytest.mark.parametrize("enabled", [None, False, True])
 def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
     env = {
         **os.environ,
@@ -51,7 +51,14 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
         "URL_BASE": "",
         "CWA_ENABLED": str(enabled),
     }
-    for name in ("INGEST_DIR", "CWA_URL", "CWA_DB_PATH", "CALIBRE_DB_PATH"):
+    # CWA is off unless CWA_ENABLED turns it on.
+    for name in (
+        "INGEST_DIR",
+        "CWA_URL",
+        "CWA_DB_PATH",
+        "CALIBRE_DB_PATH",
+        *(["CWA_ENABLED"] if enabled is None else []),
+    ):
         env.pop(name, None)
     result = subprocess.run(
         [
@@ -60,6 +67,7 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
             "from bindery.web import app; "
             "client = app.test_client(); "
             "assert client.get('/api/health').status_code == 200; "
+            f"assert client.get('/api/config').json['cwa_enabled'] is {bool(enabled)}; "
             "settings = client.get('/api/settings').json; "
             "assert settings['pause_uploads'] is True; "
             "assert settings['ingest_available'] is False; "
@@ -73,9 +81,9 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
     )
     assert result.returncode == 0, result.stderr
     assert "ERROR" not in result.stderr
-    # Bindery names what is missing, until the user says that there is no CWA.
+    # Bindery names what is missing only for a user who turned CWA on.
-    for text in ("WARNING", "INGEST_DIR is not set", "CWA_URL and CALIBRE_DB_PATH are not set", "CWA_ENABLED=false"):
+    for text in ("WARNING", "INGEST_DIR is not set", "CWA_URL and CALIBRE_DB_PATH are not set", "remove CWA_ENABLED"):
-        assert (text in result.stderr) is enabled
+        assert (text in result.stderr) is bool(enabled)
     if not enabled:
         assert "CWA" not in result.stderr and "INGEST" not in result.stderr
 
@@ -134,7 +142,16 @@ def test_cwa_off_shows_in_the_config_and_refuses_cwa_logins(client, monkeypatch,
     monkeypatch.setattr(config, "CWA_DB_PATH", "/cwa/app.db")
     response = client.get("/api/stories")
     assert response.status_code == 503
-    assert response.json["error"].startswith("AUTH_METHOD=cwa needs CWA, but CWA_ENABLED is false.")
+    assert response.json["error"].startswith("AUTH_METHOD=cwa needs CWA_ENABLED=true.")
+
+
+def test_without_cwa_the_download_template_uses_the_story_id(client, cwa_off):
+    assert client.get("/api/config").json["download_fields"]["id"] == "The Bindery story ID"
+    story_id = add_story(calibre_id=77)
+    client.put("/api/settings", json={"download_template": "{title} {id}"}, headers=H)
+    response = client.get(f"/api/stories/{story_id}/epub")
+    response.close()
+    assert f'filename="My Title {story_id}.epub"' in response.headers["Content-Disposition"]
 
 
 def test_link_setting_defaults_on_and_persists(client, monkeypatch, tmp_path, library):

tests/test_pages.py

@@ -0,0 +1,122 @@
+"""The page estimate of a document and of a built epub."""
+
+import zipfile
+
+import pytest
+
+from bindery import pages
+from bindery.epub import EpubError, spine_documents
+from bindery.pages import Measure
+
+TEXT = ["One line of the story.", "Another, longer line of the story that goes on for a while.", "The end."]
+CHARS = sum(map(len, TEXT))
+
+
+def test_a_paragraph_is_text_after_a_block_or_a_line_break():
+    assert pages.measure("<p>One</p><p>Two</p>") == Measure(2, 6)
+    assert pages.measure("<div>One<br/><br/>Two<br>Three</div>") == Measure(3, 11)
+    assert pages.measure("<div><div>One</div><div>Two</div></div>") == Measure(2, 6)
+    # Text before, between and after the blocks of an element counts too.
+    assert pages.measure("<div>Intro<p>One</p>Middle<p>Two</p>End</div>") == Measure(5, 20)
+    assert pages.measure("<h3>Title</h3><ul><li>One</li><li>Two</li></ul><blockquote>Three</blockquote>") == Measure(
+        4, 16
+    )
+
+
+@pytest.mark.parametrize(
+    "book",
+    [
+        "".join(f"<p>{line}</p>" for line in TEXT),
+        "".join(f'<p class="a">{line}</p>\n' for line in TEXT),
+        "<div>" + "".join(f"<div>{line}</div>" for line in TEXT) + "</div>",
+        "<div>" + "<br/><br/>".join(TEXT) + "</div>",
+        "<div>" + "<br>\n".join(TEXT) + "</div>",
+        f"<div>{TEXT[0]}<p>{TEXT[1]}</p>{TEXT[2]}</div>",
+    ],
+)
+def test_the_same_text_gives_the_same_measure_in_each_markup(book):
+    assert pages.measure(book) == Measure(3, CHARS)
+
+
+def test_inline_elements_and_table_cells_stay_in_the_line():
+    assert pages.measure('<p>ab<b>cd</b><a href="x">ef</a> <span><i>gh</i></span></p>') == Measure(1, 9)
+    assert pages.measure("<table><tr><td>Name</td><td>Value</td></tr><tr><th>a</th><td>b</td></tr></table>") == Measure(
+        2, 11
+    )
+    assert pages.measure('<p>Before<img src="a.png"/>after</p>') == Measure(1, 11)
+
+
+def test_only_the_text_of_the_book_counts():
+    document = (
+        '<?xml version="1.0"?><html><head><title>Hidden title</title><style>p { color: red }</style>'
+        '<link href="a.css"/></head><body><script>var hidden;</script><script src="x.js"/>'
+        "<!-- a note --><p>Shown &amp; counted</p>\n\n<p> </p><p></p></body></html>"
+    )
+    assert pages.measure(document) == Measure(1, len("Shown & counted"))
+    assert pages.measure("") == Measure(0, 0)
+    assert pages.measure("<p>\n</p>") == Measure(0, 0)
+
+
+def test_a_paragraph_that_only_spaces_the_text_is_not_counted():
+    spaced = "<p>One</p><p>&nbsp;</p><p>\xa0</p><p> &#160; </p><p><br/></p><p><span>&nbsp;</span></p><p>Two</p>"
+    assert pages.measure(spaced) == pages.measure("<p>One</p><p>Two</p>") == Measure(2, 6)
+    assert pages.measure("<div>One<br/>&nbsp;<br/>Two</div>") == Measure(2, 6)
+    # A no-break space inside a paragraph is a character of its text.
+    assert pages.measure("<p>One&nbsp;<b>two</b>\xa0three</p>") == Measure(1, 13)
+
+
+def test_a_page_is_31_lines_and_a_line_is_a_paragraph_or_70_characters():
+    assert pages.page_count(Measure(62, 620)) == (62 + 620 // 70) // 31 == 2
+    assert pages.page_count(Measure(31 * 9, 31 * 9)) == 9
+    assert pages.page_count(Measure(1000, 177_000)) == (1000 + 177_000 // 70) // 31
+
+
+def test_a_short_book_has_one_page_and_a_book_with_no_text_has_none():
+    assert pages.page_count(Measure(1, 10)) == 1
+    assert pages.page_count(Measure(30, 30)) == 1
+    assert pages.page_count(Measure()) == 0
+    # One block of text: 2400 characters start the second page before 31 lines do.
+    assert pages.page_count(Measure(1, 2399)) == 1
+    assert pages.page_count(Measure(1, 2400)) == 2
+
+
+def epub(path, documents, spine=None):
+    names = [f"file {n}.xhtml" for n in range(len(documents))]
+    items = "".join(f'<item id="f{n}" href="{name.replace(" ", "%20")}"/>' for n, name in enumerate(names))
+    items += '<item id="css" href="stylesheet.css"/>'
+    refs = "".join(f'<itemref idref="f{n}"/>' for n in (spine or range(len(documents))))
+    with zipfile.ZipFile(path, "w") as z:
+        z.writestr(
+            "META-INF/container.xml",
+            '<container xmlns="urn:oasis:names:tc:opendocument:xmlns:container"><rootfiles>'
+            '<rootfile full-path="OEBPS/content.opf"/></rootfiles></container>',
+        )
+        z.writestr(
+            "OEBPS/content.opf",
+            f'<package xmlns="http://www.idpf.org/2007/opf"><metadata/><manifest>{items}</manifest>'
+            f"<spine>{refs}</spine></package>",
+        )
+        z.writestr("OEBPS/stylesheet.css", "p { margin: 0 }")
+        for name, document in zip(names, documents, strict=True):
+            z.writestr(f"OEBPS/{name}", document)
+    return path
+
+
+def test_an_epub_is_counted_from_the_documents_of_its_spine(tmp_path):
+    chapter = "<html><head><title>T</title></head><body>" + "<p>x</p>" * 31 + "</body></html>"
+    book = epub(tmp_path / "book.epub", [chapter, "<html><body><p>Not in the spine</p></body></html>", chapter], [2, 0])
+    assert spine_documents(book) == [chapter, chapter]
+    # 62 paragraphs of one character.
+    assert pages.epub_pages(book) == 2
+    assert pages.epub_pages(epub(tmp_path / "empty.epub", ["<html><body><img src='a.png'/></body></html>"])) == 0
+
+
+def test_a_file_that_is_not_an_epub_cannot_be_counted(tmp_path):
+    path = tmp_path / "bad.epub"
+    path.write_bytes(b"not a zip")
+    with pytest.raises(EpubError):
+        pages.epub_pages(path)
+    with zipfile.ZipFile(path, "w") as z:
+        z.writestr("META-INF/container.xml", "<container/>")
+    with pytest.raises(EpubError):
+        pages.epub_pages(path)