Put the core flow first in the README

The README starts with the flow from a story URL to the OPDS catalog. One section near the end holds everything about CWA.

The example Compose file asks for no login and has the CWA values as comments.

README.md

@@ -1,10 +1,13 @@
 # Bindery
 
-Bindery downloads web fiction into a
+Bindery downloads web fiction and keeps each story up to date as an ePub. You give it the URL of a
-[Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library and
+story. Bindery downloads the chapters, checks for new chapters each day, and builds the book again
-keeps the books up to date. It uses [FanFicFare](https://github.com/JimmXinu/FanFicFare), so it
+after each change. Your reader app gets the books from the OPDS catalog of Bindery.
-supports the sites that FanFicFare supports. The tested site is RoyalRoad.
+
-CWA is optional. You can download ePubs from Bindery without it.
+Bindery uses [FanFicFare](https://github.com/JimmXinu/FanFicFare), so it supports the sites that
+FanFicFare supports. The tested site is RoyalRoad.
+
+Bindery can also send each book to a [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library. See "Calibre-Web-Automated".
 
 ## How it differs from FanFicFare
 
@@ -22,9 +25,6 @@ builds each ePub from that database.
 
 ## Quick start
 
-For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update as a new book. See
-"Limitations".
-
 1. Get the code, and make the folder for Bindery's data.
 
    ```bash
@@ -33,8 +33,8 @@ For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update
    mkdir config
    ```
 
-2. Edit `docker-compose.yml` or environment variables. Set the CWA ingest folder, the CWA address and account, and the user
+2. Edit `docker-compose.yml`. Set the user and group of the container. **That user must be able to
-   and group of the container.
+   write to `./config`.**
 
 3. Build and start the container.
 
@@ -42,10 +42,54 @@ For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update
    docker compose up -d --build
    ```
 
-4. Open `http://localhost:8085`, and log in with a CWA user.
+4. Open `http://localhost:8085`. Under "Add stories", enter the URL of a story, and click "Add".
+   Bindery downloads the chapters and builds the book. To get the book at once, click "Download ePub"
+   on the story page.
 
-The container runs as the `user` in `docker-compose.yml`. **That user must be able to write to
+5. Open Settings. Under "Book Downloads", check "Enable OPDS", and save. In your reader app, add the
-`./config` and to the CWA ingest folder.** Usually this is the same user and group as CWA.
+   catalog `http://localhost:8085/opds`, with your server address in place of `localhost`.
+
+Bindery then checks each story for new chapters every 24 hours. When a story changes, Bindery builds
+its book again, and the catalog has the new book.
+
+The example Compose file asks for no login. Set a login before other people can reach Bindery. See
+"Logins".
+
+## OPDS catalog
+
+OPDS lets a reader app browse and download books from Bindery. An admin turns it on with "Enable OPDS"
+under Settings > Book Downloads. The catalog URL is `http://localhost:8085/opds`, with your server
+address in place of `localhost`. If Bindery runs under `URL_BASE`, put it before `/opds`.
+
+The catalog has three lists, with 50 books per page:
+
+- By Title, in title order without a leading "The", "A" or "An".
+- By Author, in surname order. Each author leads to their books. A book shows under each of its authors.
+- By Last Changed, with the newest change first.
+
+Last Changed records when Bindery stores a new chapter or a new revision of an existing chapter.
+Rebuilds, metadata edits and rule changes do not update this date.
+
+A book appears when it has a stored, included chapter. It shows its authors, series, description, tags,
+language and cover, with your kept values in place of the published values. A reader can search by title,
+author, series and tag. A download uses the stored chapters, the current ePub rules and the download
+template for the file name.
+
+`AUTH_METHOD` sets how a reader logs in:
+
+- `none`: The reader needs no login.
+- `local`: The reader sends your Bindery user name and password with HTTP Basic authentication.
+  A browser with a Bindery session works too.
+- `cwa`: The reader sends your CWA user name and password with HTTP Basic authentication.
+  A browser with a Bindery session works too.
+- `proxy`: Each user sets an OPDS password under Settings > Book Downloads. Settings shows the user
+  name for the reader, which is the name in `PROXY_AUTH_USER_HEADER`. The reader sends that name and
+  password with HTTP Basic authentication. The password has 8 to 256 characters, and Bindery keeps a
+  hash of it. Set the proxy to skip its login for the catalog. See "Paths without the proxy login".
+
+Bindery keeps a good login for five minutes. Wrong passwords count toward the lockouts in
+"Lockouts and sessions". With `TRUST_PROXY=true`, each reader has its own lockout.
+Use HTTPS outside a trusted local network, because HTTP Basic sends the password with each request.
 
 ## Configuration
 
@@ -53,17 +97,11 @@ Environment variables set the configuration. **Restart the container after a cha
 
 | Variable | Description | Default value |
 | --- | --- | --- |
-| `CWA_ENABLED` | `false` if you have no CWA. See "Without CWA". | `true` |
-| `CWA_URL` | The address of CWA, for example `http://calibre-web-automated:8083` | None |
-| `CWA_USERNAME`, `CWA_PASSWORD` | The CWA account of Bindery. Leave them unset if CWA asks for no login. | None |
-| `CWA_DB_PATH` | Calibre-Web's `app.db`, for logins. See "CWA logins". | None |
-| `CALIBRE_DB_PATH` | The calibre `metadata.db`, for book ids. See "How books reach the library". | None |
 | `UPDATE_INTERVAL_HOURS` | Hours between checks for new chapters. `0` turns them off. | `24` |
 | `DEFAULT_LOCKED_FIELDS` | Fields that new stories lock: `title`, `authors`, `comments`, `tags`, `series`, `pubdate`, `languages` | `title` |
 | `AUTH_METHOD` | `none`, `local`, `cwa` or `proxy`. See "Logins". | `none` |
 | `BINDERY_USERNAME` | The user name for `AUTH_METHOD=local`. Case-sensitive. | None |
 | `BINDERY_PASSWORD_HASH` | The password hash for `AUTH_METHOD=local`. See "Local login". | None |
-| `CWA_ALLOWED_USERS` | The CWA users that can log in. Empty allows every CWA user. | Empty |
 | `BINDERY_ADMIN_USERS` | The users that can change the global settings. See "Admins". | Empty |
 | `PROXY_AUTH_USER_HEADER` | The header in which a reverse proxy sends the user name | `X-Remote-User` |
 | `PROXY_AUTH_TRUSTED_IPS` | The addresses and networks that can send the user header. See "Logins". | Loopback and private addresses |
@@ -73,45 +111,12 @@ Environment variables set the configuration. **Restart the container after a cha
 | `SESSION_COOKIE_SECURE` | `true` if users reach Bindery only over HTTPS | `false` |
 | `TRUST_PROXY` | `true` if a reverse proxy sends the client address. See "Lockouts and sessions". | `false` |
 | `URL_BASE` | A path prefix, such as `/bindery`. See "Reverse proxy". | None |
-| `INGEST_DIR` | The CWA ingest folder. If unset, all uploads stay paused. Bindery does not make this folder. Without CWA, a folder for the built ePubs. | None |
+| `INGEST_DIR` | A folder that gets each built ePub. With CWA, the CWA ingest folder. Bindery does not make this folder. | None |
 | `CONFIG_DIR` | The story database and `personal.ini` | `/config` |
 | `TMP_DIR` | The folder for the built ePubs. Bindery keeps the last build of each story. | `/tmp/bindery` |
 | `LOG_LEVEL` | The log level | `INFO` |
 
-Separate the values in a list with commas.
+Separate the values in a list with commas. The variables for CWA are in "Calibre-Web-Automated".
-
-### Without CWA
-
-Set `CWA_ENABLED=false` to run Bindery without CWA. Bindery still downloads chapters and builds
-ePubs. Download the books from each story page.
-
-With `CWA_ENABLED=false`:
-
-- Bindery makes no request to CWA. It ignores `CWA_URL`, `CWA_USERNAME`, `CWA_PASSWORD`,
-  `CWA_DB_PATH` and `CALIBRE_DB_PATH`.
-- The web interface and the log show nothing about CWA. The pause, the library links and the
-  approvals do not apply.
-- If `INGEST_DIR` is set, Bindery puts each changed book in that folder as a plain ePub. Use this to
-  import the books into other software. Each file gets a new name, so Bindery never replaces a file
-  in the folder.
-- `AUTH_METHOD=cwa` does not work. Use `local` or `proxy`. On a private network, you can use `none`.
-
-If you use the example Compose file, remove its CWA connection values. Keep `INGEST_DIR` and its
-volume mount only if you want the ePubs in a folder.
-
-Without `CWA_ENABLED=false`, Bindery expects CWA. At each start, the log then has a warning for each
-function that a missing `INGEST_DIR`, `CWA_URL` or `CALIBRE_DB_PATH` turns off. Bindery still
-downloads chapters and builds ePubs.
-
-Without `INGEST_DIR`, Settings shows "Pause all uploads" checked and a highlighted explanation.
-You cannot clear that checkbox until you set `INGEST_DIR` and restart Bindery.
-If neither `CWA_URL` nor `CALIBRE_DB_PATH` is set, library linking is off and its checkbox is disabled.
-Set either value and restart Bindery to enable the checkbox. Bindery keeps your saved linking preference.
-Saved CWA book IDs still work for uploads if you configure an ingest folder.
-
-To export ePubs for other software, set `INGEST_DIR` and turn off "Link books to CWA library" in Settings.
-Each build without a saved CWA book ID then puts a plain ePub in that folder, including updates.
-"Pause all uploads" still stops these exports. If CWA imports the files, updates will create duplicate books.
 
 ### Logins
 
@@ -190,32 +195,6 @@ key, and shows the reason.
 To change the account, change the user name or make a new hash, and restart Bindery. This ends the
 open sessions.
 
-#### CWA logins
-
-`AUTH_METHOD=cwa` needs `CWA_URL`, `CWA_DB_PATH` or both. Without `CWA_DB_PATH`, Bindery sends the
-user name and password to `CWA_URL/opds` as an HTTP Basic login. This is the OPDS login.
-
-**The OPDS login works only if CWA refuses a wrong password.** A CWA with anonymous browsing can
-accept every OPDS login. Bindery tests this with a login that cannot exist. If CWA accepts it,
-Bindery refuses each OPDS login, and the log names the reason. Set `CWA_DB_PATH` then, or use
-another `AUTH_METHOD`.
-
-With `CWA_DB_PATH`, Bindery checks the password against the account in Calibre-Web's `app.db`. A
-wrong password or an unknown account fails at once. If Bindery cannot read the file or its password
-hash, it uses the OPDS login.
-
-`CWA_DB_PATH` is the path of `app.db` inside the container. To use it, mount the CWA config folder
-read-only into the container. Mount the whole folder, so that SQLite can read the journal files
-beside the database. **Bindery does not write this database.**
-
-**Database logins check only the password.** CWA's external login providers and multi-factor
-authentication do not apply. If they must apply, use the `proxy` mode.
-
-- The CWA guest user cannot log in.
-- If `CWA_ALLOWED_USERS` is set, only those users can log in. Bindery checks this list on each
-  request.
-- `CWA_USERNAME` and `CWA_PASSWORD` are only for the library lookups of Bindery.
-
 #### Lockouts and sessions
 
 The `local` and `cwa` modes lock out password guesses:
@@ -309,39 +288,6 @@ The answer lists the queued jobs and the unsupported URLs: `{"jobs": [...], "err
 
 If Bindery runs under `URL_BASE`, put the prefix before `/api`.
 
-### OPDS catalog
-
-OPDS lets a reader app browse and download books from Bindery. An admin turns it on with "Enable OPDS"
-under Settings > Book Downloads. The catalog URL is `http://localhost:8085/opds`, with your server
-address in place of `localhost`. If Bindery runs under `URL_BASE`, put it before `/opds`.
-
-The catalog has three lists, with 50 books per page:
-
-- By Title, in title order without a leading "The", "A" or "An".
-- By Author, in surname order. Each author leads to their books. A book shows under each of its authors.
-- By Last Changed, with the newest change first.
-
-A book appears when it has a stored, included chapter. It shows its authors, series, description, tags,
-language and cover, with your kept values in place of the published values. A reader can search by title,
-author, series and tag. A download uses the stored chapters, the current ePub rules and the download
-template for the file name.
-
-`AUTH_METHOD` sets how a reader logs in:
-
-- `none`: The reader needs no login.
-- `local`: The reader sends your Bindery user name and password with HTTP Basic authentication.
-  A browser with a Bindery session works too.
-- `cwa`: The reader sends your CWA user name and password with HTTP Basic authentication.
-  A browser with a Bindery session works too.
-- `proxy`: Each user sets an OPDS password under Settings > Book Downloads. Settings shows the user
-  name for the reader, which is the name in `PROXY_AUTH_USER_HEADER`. The reader sends that name and
-  password with HTTP Basic authentication. The password has 8 to 256 characters, and Bindery keeps a
-  hash of it. Set the proxy to skip its login for the catalog. See "Paths without the proxy login".
-
-Bindery keeps a good login for five minutes. Wrong passwords count toward the lockouts in
-"Lockouts and sessions". With `TRUST_PROXY=true`, each reader has its own lockout.
-Use HTTPS outside a trusted local network, because HTTP Basic sends the password with each request.
-
 ### FanFicFare settings
 
 On first start, Bindery copies `data/personal.ini` to `/config/personal.ini`. It uses the format of
@@ -371,105 +317,6 @@ To serve Bindery under a path, such as `/bindery/`, do one of these:
 
 The health check at `/api/health` works with and without the prefix.
 
-## How books reach the library
-
-Bindery gives every book to CWA through the ingest folder. CWA makes all changes to the library.
-
-**Bindery does not make the ingest folder.** If the folder in `INGEST_DIR` does not exist, most often
-because its volume mount is missing, the log shows an error at the start and each upload fails. The
-books go to CWA with the next update after you fix the mount.
-
-1. Bindery puts the ePub of a new story into the ingest folder. CWA imports it as a new book.
-2. At the next update, Bindery finds the CWA book of the story. If you set `CALIBRE_DB_PATH`, it
-   first looks for the story URL in the library database. Without a single match there, it searches
-   the CWA OPDS feed (using `CWA_URL`, `CWA_USERNAME` and `CWA_PASSWORD`) for the story title. The
-   search ignores the parts of a title in parentheses or brackets, such as tags or "(Book 1 Complete)".
-   Bindery downloads up to 5 ePubs with a matching title, closest title first. The book whose ePub
-   holds the story URL is the match. Bindery saves the id and uuid of the book with the story.
-3. For the update, Bindery puts two files into the ingest folder: the new ePub, and a sidecar file
-   `<name>.epub.cwa.json` that holds `{"action": "add_format", "book_id": N}`. CWA then replaces the
-   ePub of book N.
-
-Before each later update, Bindery finds the book by its uuid, first in the library database if you
-set `CALIBRE_DB_PATH`, then through CWA. If you deleted the book in CWA, Bindery unlinks the story.
-
-`CALIBRE_DB_PATH` is optional. It is the path of the calibre `metadata.db` inside the container, and
-it lets Bindery find a book by its story URL instead of its title. To use it, mount the library
-folder read-only into the container. Mount the whole folder, so that SQLite can read the journal
-files beside the database. The Compose file has an example. **Bindery does not write to this database.**
-
-Be sure to use the library that gets the ingest files. A different library can make Bindery update the
-wrong book. Database lookups also find books that the CWA account of Bindery cannot see. If Bindery
-cannot read the database, it uses CWA for every lookup. Keep `CWA_URL` and the CWA account set,
-because library imports and CWA lookups use them.
-
-A book that the FanFicFare calibre plugin made holds its story URL. If you add its URL, or import it
-from the library, Bindery links that book and imports its chapters from the ePub.
-
-### If Bindery cannot find the book
-
-If library matching is on and configured, an update fails when Bindery cannot find the book.
-The story page shows the error "Cannot find this story in CWA. Link to library book, or send as new book." The
-chapters stay stored, and the next successful update includes them. This happens if:
-
-- CWA did not import the ePub.
-- The title of the book changed in CWA before Bindery linked it.
-- The CWA account of Bindery cannot see the book.
-- You deleted the book in CWA.
-
-To fix it, go to "Library link" on the story page, and do one of these:
-
-- Search for the book, and click "Link". Then click "Rebuild book" to send the update.
-- Click "Send to CWA as a new book". Use this if CWA did not import the first version, or if you
-  deleted the book.
-
-### Upload settings
-
-The "Uploads to CWA" section controls library matching and uploads. Bindery still downloads chapters
-and builds books when uploads stop. The story list marks a built book with "held" only when it waits
-for approval, library linking is on, and uploads are not paused. Bindery keeps track of all unsent books.
-
-- Link books to CWA library. This defaults to on when `CWA_URL` or `CALIBRE_DB_PATH` is set.
-  Without either value, it stays off and disabled. Turn it off to skip matching with a configured library connection.
-  Uploads use saved CWA book IDs when available. Without an ID, every build goes to `INGEST_DIR` as a plain ePub.
-  If CWA imports these files, updates will create duplicate books. Settings and the story page show a highlighted warning.
-  A change of this setting sends the unsent books, unless uploads are paused.
-- **Hold new links for approval.** This setting applies only when library linking is on.
-  Use this to catch a story that is linked to the wrong CWA book.
-  The first upload of a new story waits for your approval. The first upload after a change of the
-  library link also waits. This includes a link that you make, and a book that Bindery finds after
-  the linked book is gone. The story page names the book that gets the update, or says that the
-  story goes to CWA as a new book.
-- **Pause all uploads.** No book goes to CWA. This is always checked when `INGEST_DIR` is unset.
-  When you turn the pause off, Bindery sends the unsent books. The other upload settings still apply.
-  While paused, use "Approve" or "Approve as a new book instead" to save a decision without sending the book.
-
-To approve a link, open the story page and do one of these:
-
-- If the book is right, click "Approve and send".
-- If the book is wrong, search for the right book under "Library link", and click "Link". Then
-  approve the new link.
-- If CWA does not have the story, click "Send as a new book instead".
-
-The book that CWA imports from the first upload of a story needs no approval. After an approval,
-later updates go to CWA as usual.
-
-### The CWA account
-
-Make a CWA user for Bindery with these settings:
-
-- The download permission. Bindery reads story URLs and chapters from the ePubs.
-- "Show Publisher Section", in the sidebar settings of the user. CWA shows its list of publishers in
-  OPDS only to users with this setting. "Import from the library" uses the list to find the books of
-  publishing sites, because FanFicFare sets the publisher of a book to the site name.
-- Access to all books. Bindery finds only the books that this account can see.
-
-Bindery sends the password of this account with each OPDS request. **Use a password that no other
-service uses.**
-
-If CWA allows anonymous browsing, Bindery needs no account. Leave `CWA_USERNAME` and `CWA_PASSWORD`
-unset, and give the guest user of CWA the settings above.
-
 ## Import an ePub file
 
 If the publishing site removed chapters before Bindery tracked the story, import an ePub that you made earlier
@@ -485,7 +332,7 @@ publishes later go after them.
 that FanFicFare writes into each ePub. A chapter that Bindery stored already keeps its stored text.
 The other chapters go in the order of the ePub.
 
-After the import, Bindery checks the publishing site for new chapters and sends the book to CWA. If you clear
+After the import, Bindery checks the publishing site for new chapters and builds the book. If you clear
 "Check the site for new chapters after import", Bindery only rebuilds the book of a story that it
 downloaded before.
 
@@ -522,7 +369,7 @@ Bindery field. Calibre does not have it, and gives empty text for it.
 | `{pubdate}` | The published date. |
 | `{timestamp}` | The date when Bindery added the story. |
 | `{last_modified}` | The date when the chapters last changed. |
-| `{id}` | The CWA book ID. For a story with no library book, and without CWA, the Bindery story ID. |
+| `{id}` | The Bindery story ID. With CWA on, the CWA book ID of a story that has a library book. |
 
 - A field with no value becomes empty text.
 - `{field:format|text before|text after}` adds the two texts only when the field has a value. Each
@@ -558,11 +405,12 @@ does not change the names of the files in `INGEST_DIR`. The browser can ignore t
 | Recheck all chapters | Downloads every chapter on the publishing site again, and new chapters too. |
 | Download and replace entire book | Downloads every chapter again. Chapters that the publishing site no longer lists leave the book. They stay stored, and you can put them back. |
 | Update metadata from site | Updates only the metadata in the ePub. |
-| Rebuild book | Builds the book again from the stored chapters, and sends it. |
+| Rebuild book | Builds the book again from the stored chapters. |
 
-**Bindery builds and sends a book only if the book changed.** A change is a new, changed, moved or
+**Bindery builds a book only if the book changed.** A change is a new, changed, moved or renamed
-renamed chapter, a changed rule, or changed metadata. A job that finds no change answers "No changes"
+chapter, a changed rule, or changed metadata. A job that finds no change answers "No changes". Only
-and sends nothing to CWA. Only "Rebuild book" builds and sends a book that did not change.
+"Rebuild book" builds a book that did not change. With CWA on, Bindery sends each book that it
+builds to CWA.
 
 ### Find edited chapters
 
@@ -646,9 +494,36 @@ CWA before, it rebuilds the book after an edit.
 `DEFAULT_LOCKED_FIELDS` sets the locks for new stories. The default is `title`. The publisher is
 always the site name.
 
-**CWA reads the metadata and the cover of an ePub only when it imports a new book.** An update
+## Page count
-replaces only the ePub file. So edits that you make in CWA stay, and later changes on the site show
+
-in the ePub but not in the CWA catalog.
+The story list shows an estimated page count beside the chapter count. The OPDS catalog shows it in the
+description of a book. Bindery counts the pages as the last step when it builds an ePub, so the count
+includes your replacement rules and the line-break setting. A story shows no page count until Bindery
+builds its book.
+
+The method comes from "Paragraphs (APNX Accurate)" in the
+[Count Pages plugin](https://github.com/kiwidude68/calibre_plugins/wiki/Count-Pages) for calibre:
+
+- A paragraph start is one line. Each 70 characters of text is one more line.
+- A page is 31 lines.
+- A page is at most 2400 characters. This limit sets the count of a book with almost no paragraphs.
+
+The plugin counts either the `<p>` elements or the `<div>` elements of a book as its paragraphs. Bindery
+counts each paragraph that a reader sees:
+
+- Text starts a paragraph after a `<br>`, and after the start or the end of a block element such as `<p>`,
+  `<div>`, `<li>` or a heading.
+- A run of `<br>` elements is one paragraph break.
+- An element that only holds other blocks is not a paragraph.
+- The cells of a table row share one line.
+- A paragraph with no text is not counted. This includes a paragraph that holds only a no-break space,
+  which some stories put between paragraphs as a blank line.
+
+So the same text gives the same count as `<p>` elements, as `<div>` elements, and as lines with `<br>`
+between them. For a book that uses `<p>` elements and has no empty paragraphs, the count is within about 1%
+of the count of the plugin. The plugin counts each empty paragraph as a line, so its count is higher for a
+book that has many of them.
+The count is an estimate and differs from the page count of a printed book.
 
 ## Replacement rules
 
@@ -671,6 +546,184 @@ stays. This step runs before the global rules. The global setting is off by defa
 each story that has no setting of its own. To see the new paragraphs in the preview, select "Compare
 HTML source".
 
+## Calibre-Web-Automated
+
+Bindery can send each book that it builds to a [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library, and update that book when the
+story changes. CWA is off by default. To turn it on:
+
+1. Set `CWA_ENABLED=true`.
+2. Set `INGEST_DIR` to the CWA ingest folder. Set `CWA_URL`, `CALIBRE_DB_PATH` or both.
+3. Restart Bindery.
+
+Use CWA v4.0.1 or later. Older versions can import an update as a new book. See "Limitations".
+
+An earlier Bindery used CWA unless `CWA_ENABLED` was `false`. After an upgrade, set `CWA_ENABLED=true`
+to keep the uploads to CWA.
+
+| Variable | Description | Default value |
+| --- | --- | --- |
+| `CWA_ENABLED` | `true` to use CWA | `false` |
+| `CWA_URL` | The address of CWA, for example `http://calibre-web-automated:8083` | None |
+| `CWA_USERNAME`, `CWA_PASSWORD` | The CWA account of Bindery. Leave them unset if CWA asks for no login. | None |
+| `CALIBRE_DB_PATH` | The calibre `metadata.db`, for book ids. See "How books reach the library". | None |
+| `CWA_DB_PATH` | Calibre-Web's `app.db`, for logins. See "CWA logins". | None |
+| `CWA_ALLOWED_USERS` | The CWA users that can log in. Empty allows every CWA user. | Empty |
+
+The example Compose file has these values as comments. The user of the container must be able to write
+to the CWA ingest folder. Usually this is the same user and group as CWA.
+
+Settings shows an alert for each missing value:
+
+- Neither `CWA_URL` nor `CALIBRE_DB_PATH` is set. The alert is under "Uploads to CWA". Library linking
+  is then off, and its checkbox is disabled. Set either value and restart Bindery. Bindery keeps your
+  saved linking preference.
+- `INGEST_DIR` is not set. The alert is under "Book Downloads". "Pause all uploads" is then checked
+  until you set `INGEST_DIR` and restart Bindery.
+
+At each start, the log has a warning for each missing value.
+
+### Without CWA
+
+Without `CWA_ENABLED=true`:
+
+- Bindery makes no request to CWA. It ignores `CWA_URL`, `CWA_USERNAME`, `CWA_PASSWORD`,
+  `CWA_DB_PATH` and `CALIBRE_DB_PATH`. If one of them is set, the log has a warning at each start.
+- The web interface shows nothing about CWA. The pause, the library links and the approvals do not
+  apply.
+- If `INGEST_DIR` is set, Bindery puts each changed book in that folder as a plain ePub. Use this to
+  import the books into other software. Each file gets a new name, so the earlier files stay in the
+  folder. If `INGEST_DIR` is not set, Settings shows an alert under "Book Downloads".
+- `AUTH_METHOD=cwa` does not work. Use `local` or `proxy`. On a private network, you can use `none`.
+
+### How books reach the library
+
+Bindery gives every book to CWA through the ingest folder. CWA makes all changes to the library.
+
+**Bindery does not make the ingest folder.** If the folder in `INGEST_DIR` does not exist, most often
+because its volume mount is missing, the log shows an error at the start and each upload fails. The
+books go to CWA with the next update after you fix the mount.
+
+1. Bindery puts the ePub of a new story into the ingest folder. CWA imports it as a new book.
+2. At the next update, Bindery finds the CWA book of the story. If you set `CALIBRE_DB_PATH`, it
+   first looks for the story URL in the library database. Without a single match there, it searches
+   the CWA OPDS feed (using `CWA_URL`, `CWA_USERNAME` and `CWA_PASSWORD`) for the story title. The
+   search ignores the parts of a title in parentheses or brackets, such as tags or "(Book 1 Complete)".
+   Bindery downloads up to 5 ePubs with a matching title, closest title first. The book whose ePub
+   holds the story URL is the match. Bindery saves the id and uuid of the book with the story.
+3. For the update, Bindery puts two files into the ingest folder: the new ePub, and a sidecar file
+   `<name>.epub.cwa.json` that holds `{"action": "add_format", "book_id": N}`. CWA then replaces the
+   ePub of book N.
+
+Before each later update, Bindery finds the book by its uuid, first in the library database if you
+set `CALIBRE_DB_PATH`, then through CWA. If you deleted the book in CWA, Bindery unlinks the story.
+
+`CALIBRE_DB_PATH` is optional. It is the path of the calibre `metadata.db` inside the container, and
+it lets Bindery find a book by its story URL instead of its title. To use it, mount the library
+folder read-only into the container. Mount the whole folder, so that SQLite can read the journal
+files beside the database. The Compose file has an example. **Bindery does not write to this database.**
+
+Be sure to use the library that gets the ingest files. A different library can make Bindery update the
+wrong book. Database lookups also find books that the CWA account of Bindery cannot see. If Bindery
+cannot read the database, it uses CWA for every lookup. Keep `CWA_URL` and the CWA account set,
+because library imports and CWA lookups use them.
+
+A book that the FanFicFare calibre plugin made holds its story URL. If you add its URL, or import it
+from the library, Bindery links that book and imports its chapters from the ePub.
+
+**CWA reads the metadata and the cover of an ePub only when it imports a new book.** An update
+replaces only the ePub file. So edits that you make in CWA stay, and later changes on the site show
+in the ePub but not in the CWA catalog.
+
+### If Bindery cannot find the book
+
+If library matching is on and configured, an update fails when Bindery cannot find the book.
+The story page shows the error "Cannot find this story in CWA. Link to library book, or send as new book." The
+chapters stay stored, and the next successful update includes them. This happens if:
+
+- CWA did not import the ePub.
+- The title of the book changed in CWA before Bindery linked it.
+- The CWA account of Bindery cannot see the book.
+- You deleted the book in CWA.
+
+To fix it, go to "Library link" on the story page, and do one of these:
+
+- Search for the book, and click "Link". Then click "Rebuild book" to send the update.
+- Click "Send to CWA as a new book". Use this if CWA did not import the first version, or if you
+  deleted the book.
+
+### Upload settings
+
+The "Uploads to CWA" section of Settings controls library matching and uploads. Bindery still downloads chapters
+and builds books when uploads stop. The story list marks a built book with "held" only when it waits
+for approval, library linking is on, and uploads are not paused. Bindery keeps track of all unsent books.
+
+- Link books to CWA library. This defaults to on when `CWA_URL` or `CALIBRE_DB_PATH` is set.
+  Without either value, it stays off and disabled. Turn it off to skip matching with a configured library connection.
+  Uploads use saved CWA book IDs when available. Without an ID, every build goes to `INGEST_DIR` as a plain ePub.
+  If CWA imports these files, updates will create duplicate books. Settings and the story page show a highlighted warning.
+  A change of this setting sends the unsent books, unless uploads are paused.
+- **Hold new links for approval.** This setting applies only when library linking is on.
+  Use this to catch a story that is linked to the wrong CWA book.
+  The first upload of a new story waits for your approval. The first upload after a change of the
+  library link also waits. This includes a link that you make, and a book that Bindery finds after
+  the linked book is gone. The story page names the book that gets the update, or says that the
+  story goes to CWA as a new book.
+- **Pause all uploads.** No book goes to CWA. This is always checked when `INGEST_DIR` is unset.
+  When you turn the pause off, Bindery sends the unsent books. The other upload settings still apply.
+  While paused, use "Approve" or "Approve as a new book instead" to save a decision without sending the book.
+
+To approve a link, open the story page and do one of these:
+
+- If the book is right, click "Approve and send".
+- If the book is wrong, search for the right book under "Library link", and click "Link". Then
+  approve the new link.
+- If CWA does not have the story, click "Send as a new book instead".
+
+The book that CWA imports from the first upload of a story needs no approval. After an approval,
+later updates go to CWA as usual.
+
+### The CWA account
+
+Make a CWA user for Bindery with these settings:
+
+- The download permission. Bindery reads story URLs and chapters from the ePubs.
+- "Show Publisher Section", in the sidebar settings of the user. CWA shows its list of publishers in
+  OPDS only to users with this setting. "Import from the library" uses the list to find the books of
+  publishing sites, because FanFicFare sets the publisher of a book to the site name.
+- Access to all books. Bindery finds only the books that this account can see.
+
+Bindery sends the password of this account with each OPDS request. **Use a password that no other
+service uses.**
+
+If CWA allows anonymous browsing, Bindery needs no account. Leave `CWA_USERNAME` and `CWA_PASSWORD`
+unset, and give the guest user of CWA the settings above.
+
+### CWA logins
+
+`AUTH_METHOD=cwa` needs `CWA_ENABLED=true`, and `CWA_URL`, `CWA_DB_PATH` or both. Without `CWA_DB_PATH`, Bindery sends the
+user name and password to `CWA_URL/opds` as an HTTP Basic login. This is the OPDS login.
+
+**The OPDS login works only if CWA refuses a wrong password.** A CWA with anonymous browsing can
+accept every OPDS login. Bindery tests this with a login that cannot exist. If CWA accepts it,
+Bindery refuses each OPDS login, and the log names the reason. Set `CWA_DB_PATH` then, or use
+another `AUTH_METHOD`.
+
+With `CWA_DB_PATH`, Bindery checks the password against the account in Calibre-Web's `app.db`. A
+wrong password or an unknown account fails at once. If Bindery cannot read the file or its password
+hash, it uses the OPDS login.
+
+`CWA_DB_PATH` is the path of `app.db` inside the container. To use it, mount the CWA config folder
+read-only into the container. Mount the whole folder, so that SQLite can read the journal files
+beside the database. **Bindery does not write this database.**
+
+**Database logins check only the password.** CWA's external login providers and multi-factor
+authentication do not apply. If they must apply, use the `proxy` mode.
+
+- The CWA guest user cannot log in.
+- If `CWA_ALLOWED_USERS` is set, only those users can log in. Bindery checks this list on each
+  request.
+- `CWA_USERNAME` and `CWA_PASSWORD` are only for the library lookups of Bindery.
+
 ## Limitations
 
 - **Bindery identifies a chapter by its URL.** Most sites, such as RoyalRoad and AO3, keep the URL
@@ -679,8 +732,8 @@ HTML source".
   the same, run "Recheck all chapters". When an author removes a chapter there, the book loses that
   chapter, and the last chapter appears twice. The removed chapter stays as an older revision of the
   chapter that took its URL.
-- "Stop tracking" deletes only the data of Bindery. The book stays in CWA.
+- "Stop tracking" deletes only the data of Bindery. A book in CWA stays there.
-- **Updates need CWA v4.0.1 or later.** CWA v4.0.0 added the `.cwa.json` sidecar files. But v4.0.0
+- **Updates of a CWA library need CWA v4.0.1 or later.** CWA v4.0.0 added the `.cwa.json` sidecar files. But v4.0.0
   can delete a sidecar file before it imports the ePub, and then imports the update as a new book.
   Versions before v4.0.0 cannot read sidecar files, so each update becomes a new book.
 
@@ -703,14 +756,14 @@ The Python tests use a fake CWA server and temporary folders, so they need no CW
 sites. The frontend tests use the built-in test runner of Node, and need no npm packages. Also check
 the pages in a browser.
 
-### Run locally without CWA
+### Run locally
 
-Without CWA, Bindery puts each new build of a story into its ingest folder as an ePub. This is
+CWA is off by default. Bindery puts each new build of a story into its ingest folder as an
-enough to work on downloads, chapters, rules and the web interface.
+ePub. This is enough to work on downloads, chapters, rules and the web interface.
 
 ```bash
 mkdir -p dev/ingest
-CWA_ENABLED=false CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp uv run python -m bindery
+CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp uv run python -m bindery
 ```
 
 Open `http://127.0.0.1:8085`. The folders go in `dev/`, which git ignores. To look at an ePub, use
@@ -741,7 +794,7 @@ not use your real library.**
    test logins too.
 
    ```bash
-   CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp \
+   CWA_ENABLED=true CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp \
    CWA_URL=http://localhost:8083 CWA_USERNAME=admin CWA_PASSWORD=admin123 AUTH_METHOD=cwa \
    uv run python -m bindery
    ```

bindery/auth.py

@@ -123,7 +123,7 @@ def config_error() -> str | None:
         if not _valid_local_hash(config.LOCAL_PASSWORD_HASH):
             return "BINDERY_PASSWORD_HASH must be a Werkzeug scrypt or PBKDF2 hash of a nonempty password"
     if method == AUTH_CWA and not config.CWA_ENABLED:
-        return "AUTH_METHOD=cwa needs CWA, but CWA_ENABLED is false"
+        return "AUTH_METHOD=cwa needs CWA_ENABLED=true"
     if method == AUTH_CWA and not (config.CWA_URL or config.CWA_DB_PATH):
         return "AUTH_METHOD=cwa needs CWA_URL or CWA_DB_PATH"
     if method == AUTH_PROXY:

bindery/config.py

@@ -74,9 +74,9 @@ CWA_PASSWORD = os.getenv("CWA_PASSWORD", "")
 # Independent, optional read-only databases. HTTP remains the fallback.
 CWA_DB_PATH = _optional_path("CWA_DB_PATH")  # Calibre-Web app.db, for logins
 CALIBRE_DB_PATH = _optional_path("CALIBRE_DB_PATH")  # calibre metadata.db, for book IDs
-# false if the user has no CWA. Bindery then ignores the CWA settings above, and shows nothing about CWA. INGEST_DIR
+# true if the user has CWA. Without it, Bindery ignores the CWA settings above, and shows nothing about CWA. INGEST_DIR
-# still gets each built book, as a plain epub. Only a clear "false" turns CWA off.
+# still gets each built book, as a plain epub. Only a clear "true" turns CWA on.
-CWA_ENABLED = _bool("CWA_ENABLED", True)
+CWA_ENABLED = _bool("CWA_ENABLED")
 CWA_SETTINGS = ("CWA_URL", "CWA_USERNAME", "CWA_PASSWORD", "CWA_DB_PATH", "CALIBRE_DB_PATH")
 IGNORED_SETTINGS = [] if CWA_ENABLED else [name for name in CWA_SETTINGS if os.getenv(name, "").strip()]
 if not CWA_ENABLED:
@@ -141,7 +141,10 @@ def prepare() -> None:
         logger.info(f"Created the default FanFicFare configuration at {PERSONAL_INI}")
     if not CWA_ENABLED:
         if IGNORED_SETTINGS:
-            logger.info(f"CWA_ENABLED is false. Bindery ignores {', '.join(IGNORED_SETTINGS)}.")
+            logger.warning(
+                f"CWA_ENABLED is not true. Bindery ignores {', '.join(IGNORED_SETTINGS)}. To use CWA, set "
+                "CWA_ENABLED=true."
+            )
         return
     if CWA_PASSWORD and not CWA_USERNAME:
         logger.warning("CWA_PASSWORD is set, but CWA_USERNAME is not. Bindery sends no login to CWA.")
@@ -149,7 +152,7 @@ def prepare() -> None:
     for warning in missing:
         logger.warning(warning)
     if missing:
-        logger.warning("If you do not use CWA, set CWA_ENABLED=false. Bindery then shows no messages about CWA.")
+        logger.warning("If you do not use CWA, remove CWA_ENABLED. Bindery then shows no messages about CWA.")
 
 
 def _missing_cwa() -> list[str]:

bindery/epub.py

@@ -1,9 +1,11 @@
 """Read the OPF metadata of an epub that came from CWA."""
 
+import posixpath
 import re
 import zipfile
 from pathlib import Path
 from typing import Any
+from urllib.parse import unquote
 from xml.etree.ElementTree import Element
 
 from defusedxml import ElementTree
@@ -17,14 +19,19 @@ class EpubError(Exception):
     pass
 
 
-def _metadata(path: Path) -> Element:
+def _opf(path: Path, z: zipfile.ZipFile) -> tuple[str, Element]:
-    try:
+    """The name of the OPF file in the epub, and its content."""
-        with zipfile.ZipFile(path) as z:
     container = ElementTree.fromstring(z.read("META-INF/container.xml"))
     rootfile = container.find(f".//{CONTAINER}rootfile")
     if rootfile is None or not rootfile.get("full-path"):
         raise EpubError(f"{path.name} has no OPF file")
-            opf = ElementTree.fromstring(z.read(rootfile.get("full-path")))
+    return rootfile.get("full-path"), ElementTree.fromstring(z.read(rootfile.get("full-path")))
+
+
+def _metadata(path: Path) -> Element:
+    try:
+        with zipfile.ZipFile(path) as z:
+            opf = _opf(path, z)[1]
     except (zipfile.BadZipFile, KeyError, ElementTree.ParseError) as e:
         raise EpubError(f"{path.name} is not a readable epub: {e}") from e
     metadata = opf.find(f"{OPF}metadata")
@@ -33,6 +40,21 @@ def _metadata(path: Path) -> Element:
     return metadata
 
 
+def spine_documents(path: Path) -> list[str]:
+    """The text of each document in the spine of an epub, in reading order."""
+    try:
+        with zipfile.ZipFile(path) as z:
+            name, opf = _opf(path, z)
+            hrefs = {item.get("id"): item.get("href") for item in opf.iter(f"{OPF}item")}
+            files = [
+                posixpath.join(posixpath.dirname(name), unquote(hrefs[ref.get("idref")]))
+                for ref in opf.iter(f"{OPF}itemref")
+            ]
+            return [z.read(file).decode("utf-8", "replace") for file in files]
+    except (zipfile.BadZipFile, KeyError, ElementTree.ParseError) as e:
+        raise EpubError(f"{path.name} is not a readable epub: {e}") from e
+
+
 def epub_urls(path: Path) -> list[str]:
     """Every URL in the identifiers and the source of an epub's OPF.
 

bindery/opds.py

@@ -77,6 +77,7 @@ def _languages(value: Any) -> list[str]:
 def _books() -> list[dict[str, Any]]:
     """The books of the catalog, in title order."""
     books = []
+    page_counts = orchestrator.store.story_settings("page_count")
     for story in orchestrator.store.catalog():
         meta = orchestrator.book_metadata(story)
         title = (meta["title"] or "").strip() or story["url"]
@@ -89,6 +90,7 @@ def _books() -> list[dict[str, Any]]:
                 "title": title,
                 "authors": authors,
                 "updated": _updated(story),
+                "page_count": page_counts.get(story["id"]),
                 "cover": hashlib.sha256(cover.encode()).hexdigest()[:16] if cover else None,
             }
         )
@@ -107,6 +109,8 @@ def _content(book: dict[str, Any]) -> str:
         index = filename_template.series_index(meta.get("series_index"))
         lines.append(f"Series: {meta['series']}" + (f", book {index}" if index else ""))
     lines.append(f"Chapters: {book['chapter_count']}")
+    if book["page_count"] is not None:
+        lines.append(f"Pages: {book['page_count']}")
     description = "\n\n".join(diffing.html_to_blocks(meta.get("comments") or ""))
     return "\n".join(lines) + (f"\n\n{description}" if description else "")
 

bindery/orchestrator.py

@@ -13,7 +13,7 @@ from typing import Any, BinaryIO
 
 from werkzeug.security import generate_password_hash
 
-from bindery import config, fff, filename_template
+from bindery import config, fff, filename_template, pages
 from bindery.backup import daily_backup
 from bindery.cwa import Book, BookFinder, CwaClient, CwaError
 from bindery.diffing import diff_html, text_hash
@@ -226,9 +226,11 @@ def list_stories() -> list[dict[str, Any]]:
     stories = store.list_stories()
     held = store.story_settings("upload_held")
     unapproved = store.story_settings("unapproved_link")
+    page_counts = store.story_settings("page_count")
     show_holds = _link_books() and store.get_setting(None, "hold_new_links") and _can_upload()
     for s in stories:
         s["busy"] = job_queue.has_active(s["id"])
+        s["page_count"] = page_counts.get(s["id"])
         s["upload_held"] = bool(held.get(s["id"]))
         s["approval_held"] = bool(
             show_holds
@@ -833,6 +835,14 @@ def _build_key(story: dict[str, Any], overrides: dict[str, Any], site: dict[str,
     return digest.hexdigest()
 
 
+def _count_pages(story_id: int, path: Path) -> None:
+    """Store the estimated pages of a built epub, for the story list and the OPDS catalog."""
+    try:
+        store.set_setting(story_id, "page_count", pages.epub_pages(path))
+    except EpubError as e:
+        logger.warning(f"Cannot count the pages of story {story_id}: {e}")
+
+
 def _built_epub(
     story: dict[str, Any], overrides: dict[str, Any], key: str, force: bool = False, record: bool = True
 ) -> Path:
@@ -842,7 +852,8 @@ def _built_epub(
     """
     path = config.TMP_DIR / f"story-{story['id']}-{key[:16]}.epub"
     with _build_lock:
-        if force or not path.exists():
+        built = force or not path.exists()
+        if built:
             # A build under another name first, so a download never reads half a book.
             partial = config.TMP_DIR / f"building-{uuid.uuid4().hex}.epub"
             try:
@@ -853,6 +864,9 @@ def _built_epub(
             for old in config.TMP_DIR.glob(f"story-{story['id']}-*.epub"):
                 if old != path:
                     old.unlink(missing_ok=True)
+        # A book from before the page count has no count yet.
+        if built or store.get_setting(story["id"], "page_count") is None:
+            _count_pages(story["id"], path)
         if record:
             store.set_setting(story["id"], "built_key", key)
     return path
@@ -1182,9 +1196,8 @@ def _run_rebuild(job: Job) -> str:
     force = bool(job.options.get("force"))
     if not force and key == _done_key(story["id"]):
         return "No changes"
-    message, changed = _send(story, overrides, key, story["title"], story["author"], force)
+    message, _ = _send(story, overrides, key, story["title"], story["author"], force)
-    changes: dict[str, Any] = {"last_changed": now()} if changed or force else {}
+    store.update_story(story["id"], pinned=pinned, status="ok", message=message)
-    store.update_story(story["id"], pinned=pinned, status="ok", message=message, **changes)
     return message
 
 
@@ -1436,9 +1449,7 @@ def _run_site_job(job: Job) -> str:
         return f"No changes. {summary}"
 
     _progress(job, "Building epub", 1.0)
-    message, book_changed = _send(story, overrides, key, title, snap.author)
+    message, _ = _send(story, overrides, key, title, snap.author)
-    if book_changed:
-        changes["last_changed"] = now()
     store.update_story(story_id, **changes, message=message, status="error" if failed else "ok")
     return f"{summary}. {message}"
 

bindery/pages.py

@@ -0,0 +1,78 @@
+"""An estimate of the pages of a built epub, after the "Paragraphs (APNX Accurate)" method of the Count Pages
+plugin for calibre.
+
+The method counts lines. A paragraph start is one line, and so is each 70 characters of text. A page is 31
+lines. A page is also at most 2400 characters, which sets the count of a book with almost no paragraphs.
+
+The plugin counts either the <p> or the <div> elements of a book as its paragraphs. Bindery counts what a
+reader sees as a paragraph: text that follows a <br> or the start or end of a block element. So the same text
+gives the same count as <p> elements, as <div> elements, and as lines with <br> between them. A paragraph with
+no text, or with only a no-break space, is not counted.
+"""
+
+import html as htmllib
+import re
+from pathlib import Path
+from typing import NamedTuple
+
+from bindery.epub import spine_documents
+
+LINE_CHARS = 70
+LINES_PER_PAGE = 31
+CHARS_PER_PAGE = 2400
+# Elements that stay in the line of the text around them. Each other element, and <br>, ends a paragraph.
+# The cells of a table row share a line.
+INLINE = frozenset(
+    "a abbr acronym b bdi bdo big cite code data del dfn em font i img ins kbd mark q rp rt ruby s samp small "
+    "span strike strong sub sup td th time tt u var wbr".split()
+)
+# Elements whose text is not text of the book.
+HIDDEN = frozenset(("head", "script", "style"))
+_TOKEN = re.compile(r"<(/?)([a-zA-Z][a-zA-Z0-9]*)[^>]*>|<[^>]*>|([^<]+)")
+
+
+class Measure(NamedTuple):
+    paragraphs: int = 0
+    chars: int = 0
+
+
+def measure(html: str) -> Measure:
+    """The paragraphs and the characters of text of one document."""
+    paragraphs = chars = 0
+    hidden = False
+    new_paragraph = True
+    for match in _TOKEN.finditer(html):
+        closing, name, text = match.groups()
+        if name:
+            name = name.lower()
+            if name in HIDDEN:
+                hidden = not closing and not match.group().endswith("/>")
+            elif name not in INLINE:
+                new_paragraph = True
+        elif text and not hidden:
+            if "&" in text:
+                text = htmllib.unescape(text)
+            # A no-break space is white space here, so a paragraph that only spaces the text is no paragraph.
+            if text.strip():
+                if new_paragraph:
+                    paragraphs += 1
+                    new_paragraph = False
+                    text = text.lstrip()
+                chars += len(text)
+            elif not new_paragraph:
+                chars += 1  # the space between two inline elements
+    return Measure(paragraphs, chars)
+
+
+def page_count(total: Measure) -> int:
+    """The pages of a book, from the sums of its documents. A book with no text has no pages."""
+    if not total.chars:
+        return 0
+    lines = total.paragraphs + total.chars // LINE_CHARS
+    return max(lines // LINES_PER_PAGE, total.chars // CHARS_PER_PAGE + 1)
+
+
+def epub_pages(path: Path) -> int:
+    """The estimated pages of an epub, from each document in its spine."""
+    measures = [measure(text) for text in spine_documents(path)]
+    return page_count(Measure(sum(m.paragraphs for m in measures), sum(m.chars for m in measures)))

bindery/static/js/settings.js

@@ -1,4 +1,4 @@
-// Settings: the recheck of recent chapters, the uploads to CWA, the names of downloaded books, the global rules,
+// Settings: the recheck of recent chapters, the names of downloaded books, the uploads to CWA, the global rules,
 // and FanFicFare's personal.ini.
 
 import { api, button, checkbox, externalLink, fill, h, showError, toast } from './lib.js';
@@ -39,7 +39,7 @@ export function settingsPage(main, _params, config) {
   const linkBooks = checkbox(true, showLinkState);
   const pauseUploads = checkbox(false, () => {});
   const ingestNote = h('p', { class: 'notice', id: 'ingest-note', hidden: true },
-    'INGEST_DIR is not set. Uploads stay paused until you set it and restart Bindery.');
+    'INGEST_DIR is not set. Bindery sends no book to the ingest folder until you set it and restart Bindery.');
   const showUploadSettings = (settings) => {
     linkBooks.disabled = settings.linking_available === false;
     linkBooks.checked = !linkBooks.disabled && (settings.link_books ?? true);
@@ -178,41 +178,6 @@ export function settingsPage(main, _params, config) {
       ),
       saveButton(() => ({ recent_chapters: recent.value === '' ? 0 : Number(recent.value) })),
     ),
-    config.cwa_enabled && card(
-      'Uploads to CWA',
-      h('label', {}, linkBooks, ' Link books to CWA library'),
-      h(
-        'p',
-        { class: 'hint' },
-        'Find library books through CWA_URL or CALIBRE_DB_PATH. When this is off, uploads use saved CWA book IDs ' +
-          'or send a plain ePub to INGEST_DIR.',
-      ),
-      libraryNote,
-      linkNote,
-      h('label', {}, holdNewLinks, ' Hold new links for approval'),
-      h(
-        'p',
-        { class: 'hint' },
-        'The first upload of a new story waits for your approval. So does the first upload after a change of the ' +
-          'library link, also a link that you make. Check the linked book on the story page, and approve it there. ' +
-          'Later updates of an approved book go to CWA as usual.',
-      ),
-      h('label', {}, pauseUploads, ' Pause all uploads'),
-      ingestNote,
-      h(
-        'p',
-        { class: 'hint' },
-        'Bindery downloads chapters and builds books, but sends no book to CWA. ' +
-          'When you turn the pause off, Bindery sends the books that wait.',
-      ),
-      saveButton(
-        () => ({ link_books: linkBooks.checked, hold_new_links: holdNewLinks.checked, pause_uploads: pauseUploads.checked }),
-        (settings) => {
-          showUploadSettings(settings);
-          toast('Saved');
-        },
-      ),
-    ),
     card(
       'Book Downloads',
       h('label', {}, opdsEnabled, ' Enable OPDS'),
@@ -331,6 +296,7 @@ export function settingsPage(main, _params, config) {
           '. The default gives "May, 2020". A / in a date becomes _, as in Calibre.',
         ),
       ),
+      ingestNote,
       saveButton(
         () => ({
           opds_enabled: opdsEnabled.checked,
@@ -346,6 +312,40 @@ export function settingsPage(main, _params, config) {
         },
       ),
     ),
+    config.cwa_enabled && card(
+      'Uploads to CWA',
+      h('label', {}, linkBooks, ' Link books to CWA library'),
+      h(
+        'p',
+        { class: 'hint' },
+        'Find library books through CWA_URL or CALIBRE_DB_PATH. When this is off, uploads use saved CWA book IDs ' +
+          'or send a plain ePub to INGEST_DIR.',
+      ),
+      libraryNote,
+      linkNote,
+      h('label', {}, holdNewLinks, ' Hold new links for approval'),
+      h(
+        'p',
+        { class: 'hint' },
+        'The first upload of a new story waits for your approval. So does the first upload after a change of the ' +
+          'library link, also a link that you make. Check the linked book on the story page, and approve it there. ' +
+          'Later updates of an approved book go to CWA as usual.',
+      ),
+      h('label', {}, pauseUploads, ' Pause all uploads'),
+      h(
+        'p',
+        { class: 'hint' },
+        'Bindery downloads chapters and builds books, but sends no book to CWA. ' +
+          'When you turn the pause off, Bindery sends the books that wait.',
+      ),
+      saveButton(
+        () => ({ link_books: linkBooks.checked, hold_new_links: holdNewLinks.checked, pause_uploads: pauseUploads.checked }),
+        (settings) => {
+          showUploadSettings(settings);
+          toast('Saved');
+        },
+      ),
+    ),
     card(
       'Jobs list',
       h(

bindery/static/js/stories.js

@@ -9,6 +9,7 @@ const STORY_COLUMNS = [
   { label: 'Author', key: 'author', type: 'text' },
   { label: 'Site', key: 'site', type: 'text' },
   { label: 'Chapters', key: 'chapter_count', type: 'number' },
+  { label: 'Pages', key: 'page_count', type: 'number' },
   { label: 'Status', key: 'status', type: 'text' },
   { label: 'Last change', key: 'last_changed', type: 'date' },
   { label: 'Last check', key: 'last_checked', type: 'date' },
@@ -21,11 +22,14 @@ function libraryImport(config) {
   let candidates = [];
   let selected = new Set();
   let check = true;
+  let empty = '';
+  // The load of the list, kept for each search, because CWA takes a while to send the list.
+  let list;
   const results = h('div');
 
   const render = () => {
     if (!candidates.length) {
-      fill(results, h('p', { class: 'empty' }, 'Every library book from a story site is tracked already.'));
+      fill(results, h('p', { class: 'empty' }, empty));
       return;
     }
     const toggle = (id, on) => {
@@ -81,9 +85,14 @@ function libraryImport(config) {
           const books = candidates
             .filter((c) => selected.has(c.id))
             .map(({ id, uuid, title }) => ({ id, uuid, title }));
-          if (!books.length) return;
+          if (!books.length) {
+            toast('Select books first', true);
+            return;
+          }
           const jobs = await api('library/import', 'POST', { books, check });
           toast(`Queued ${jobs.length} imports`);
+          const queued = new Set(books.map((b) => b.id));
+          list = list?.then((all) => all.filter((c) => !queued.has(c.id)));
           fill(results);
           refreshJobs();
         },
@@ -92,22 +101,68 @@ function libraryImport(config) {
     );
   };
 
+  // The full list starts with each book selected. A search starts with no book selected.
+  const show = (books, all) => {
+    candidates = books;
+    selected = new Set(all ? books.map((c) => c.id) : []);
+    empty = all
+      ? 'Every library book from a story site is tracked already.'
+      : 'No library books from a story site match. A book that Bindery tracks already does not show.';
+    render();
+  };
+  const query = h('input', {
+    type: 'search',
+    placeholder: 'Search the library by title, author or site',
+    'aria-label': 'Search the library',
+  });
+  const load = () => {
+    list ??= api('library/candidates').catch((e) => {
+      list = undefined;
+      throw e;
+    });
+    return list;
+  };
+  // A book matches if its title, authors and site hold each word of the search.
+  const search = async () => {
+    const words = query.value.toLowerCase().split(/\s+/).filter(Boolean);
+    if (!words.length) return;
+    const books = await load();
+    show(books.filter((c) => words.every((w) => [c.title, ...c.authors, c.publisher].join(' ').toLowerCase().includes(w))), false);
+  };
+  const submit = h('button', { type: 'submit' }, 'Search');
+
   return h(
     'div',
     {},
     h(
       'p',
       { class: 'hint' },
-      'Finds library books whose publisher is a story site, for example books from the FanFicFare calibre plugin. ' +
+      'Search the library, or show all the stories in it. Both find only books whose publisher is a story site, ' +
+        'for example books from the FanFicFare calibre plugin, and that Bindery does not track yet. ' +
         'Importing downloads each ePub from CWA and keeps its chapters, including chapters the site no longer has.',
     ),
+    h(
+      'form',
+      {
+        class: 'library-search',
+        onsubmit: (e) => {
+          e.preventDefault();
+          if (submit.disabled) return;
+          submit.disabled = true;
+          search()
+            .catch(showError)
+            .finally(() => (submit.disabled = false));
+        },
+      },
+      query,
+      submit,
+    ),
     h(
       'div',
       { class: 'controls' },
-      button('Find stories in the library', async () => {
+      button('Show all stories in the library', async () => {
-        candidates = await api('library/candidates');
+        list = undefined;
-        selected = new Set(candidates.map((c) => c.id));
+        show(await load(), true);
-        render();
       }),
       h('label', {}, checkbox(true, (on) => (check = on)), ' Check the site for new chapters after import'),
     ),
@@ -199,6 +254,8 @@ function storyRow(s) {
       s.offsite_count > 0 && h('small', { title: 'Chapters the site no longer lists' }, `${s.offsite_count} removed`),
       s.revised_count > 0 && h('small', { title: 'Chapters the site changed since Bindery added them' }, `${s.revised_count} revised`),
     ),
+    // A story has no page count until Bindery builds its book.
+    h('td', { 'data-label': 'Pages', title: 'Estimated from the built book' }, s.page_count != null && String(s.page_count)),
     h(
       'td',
       { 'data-label': 'Status' },
@@ -244,6 +301,8 @@ export function storiesPage(main, _params, config) {
         const order = (left - right) * direction;
         if (order) return order;
       } else if (sort !== 'title') {
+        // A story with no number goes last in each direction.
+        if (sortType === 'number' && (a[sort] == null) !== (b[sort] == null)) return a[sort] == null ? 1 : -1;
         const order = sortType === 'number'
           ? a[sort] - b[sort]
           : (a[sort] || '').localeCompare(b[sort] || '', undefined, { sensitivity: 'base' });

bindery/static/js/story.js

@@ -68,7 +68,7 @@ const table = (headings, rows, extra = '') =>
 export function storyPage(main, { storyId }, config) {
   const id = Number(storyId);
   const fields = config.metadata_fields;
-  // With CWA_ENABLED=false, the page shows nothing about CWA.
+  // Without CWA_ENABLED=true, the page shows nothing about CWA.
   const cwa = config.cwa_enabled;
   const libraryStays = cwa ? ' The library book stays.' : '';
   let details = null;

bindery/static/style.css

@@ -940,6 +940,11 @@ textarea {
     color: var(--muted);
   }
 
+  /* A story with no page count yet shows no line for it. */
+  .stories-table td:empty {
+    display: none;
+  }
+
   /* The chapter list has far more rows than the story list, so each row packs its cells onto
      three lines instead of one line per cell: checkbox and number, title, then the rest. */
   .chapters-table tbody tr {

bindery/store.py

@@ -273,15 +273,17 @@ class Store:
         conn: sqlite3.Connection, story_id: int, chapter: dict[str, str], source: str, position: int
     ) -> int:
         """Store a chapter with its first revision. The chapter needs url, title, html and text_hash."""
+        changed_at = now()
         cur = conn.execute(
             "INSERT INTO chapters (story_id, position, url, title, added_at) VALUES (?, ?, ?, ?, ?)",
-            (story_id, position, chapter["url"], chapter["title"], now()),
+            (story_id, position, chapter["url"], chapter["title"], changed_at),
         )
         chapter_id = int(cur.lastrowid)
         conn.execute(
             "INSERT INTO revisions (chapter_id, rev, html, text_hash, source, fetched_at) VALUES (?, 1, ?, ?, ?, ?)",
-            (chapter_id, chapter["html"], chapter["text_hash"], source, now()),
+            (chapter_id, chapter["html"], chapter["text_hash"], source, changed_at),
         )
+        conn.execute("UPDATE stories SET last_changed = ? WHERE id = ?", (changed_at, story_id))
         return chapter_id
 
     def add_chapter(
@@ -326,6 +328,7 @@ class Store:
 
     def add_revision(self, chapter_id: int, html: str, text_hash: str, source: str) -> int:
         """Store a new revision and make it the current one."""
+        changed_at = now()
         with self._connect() as conn:
             rev = conn.execute(
                 "SELECT COALESCE(MAX(rev), 0) + 1 FROM revisions WHERE chapter_id = ?", (chapter_id,)
@@ -333,9 +336,15 @@ class Store:
             conn.execute(
                 "INSERT INTO revisions (chapter_id, rev, html, text_hash, source, fetched_at) "
                 "VALUES (?, ?, ?, ?, ?, ?)",
-                (chapter_id, rev, html, text_hash, source, now()),
+                (chapter_id, rev, html, text_hash, source, changed_at),
+            )
+            conn.execute(
+                "UPDATE chapters SET current_rev = ?, changed_at = ? WHERE id = ?", (rev, changed_at, chapter_id)
+            )
+            conn.execute(
+                "UPDATE stories SET last_changed = ? WHERE id = (SELECT story_id FROM chapters WHERE id = ?)",
+                (changed_at, chapter_id),
             )
-            conn.execute("UPDATE chapters SET current_rev = ?, changed_at = ? WHERE id = ?", (rev, now(), chapter_id))
             return int(rev)
 
     def update_chapter(self, chapter_id: int, **fields: Any) -> None:

docker-compose.yml

@@ -2,26 +2,14 @@ services:
   bindery:
     build: .
     image: bindery:latest
-    # The user and group that own ./config and can write CWA's ingest folder. Usually the same as CWA's.
+    # The user and group that own ./config.
     user: "1000:1000"
     environment:
-      # Without CWA: set this, and remove the CWA values below. INGEST_DIR then gets the built ePubs.
-      # CWA_ENABLED: "false"
-      # Omit this and its volume mount to keep uploads paused.
-      INGEST_DIR: /cwa-book-ingest
-      # The address of CWA, as the Bindery container reaches it, and the CWA account that Bindery uses.
-      # Leave the account out if CWA asks for no login.
-      CWA_URL: http://calibre-web-automated:8083
-      CWA_USERNAME: bindery
-      CWA_PASSWORD: change-me
       # none, local (one Bindery account), cwa (CWA accounts), or proxy (a trusted user header)
-      AUTH_METHOD: cwa
+      AUTH_METHOD: none
       # For local logins, put these values in .env. See "Local login" in the README.
       # BINDERY_USERNAME: ${BINDERY_USERNAME:-}
       # BINDERY_PASSWORD_HASH: ${BINDERY_PASSWORD_HASH:-}
-      # Optional and independent. app.db is for logins, metadata.db is for book IDs.
-      # CWA_DB_PATH: /cwa-config/app.db
-      # CALIBRE_DB_PATH: /calibre-library/metadata.db
       # URL_BASE: /bindery
       # The host names that Bindery answers to. AUTH_METHOD=none needs this for a public name such as this one.
       # ALLOWED_HOSTS: bindery.example.com
@@ -32,6 +20,19 @@ services:
       # For the cwa and proxy modes: the users that can change the global settings. Without a list, each user can.
       # BINDERY_ADMIN_USERS: alice, bob
       # BINDERY_API_KEY: a-long-random-string
+      # Optional: a folder that gets each built ePub. With CWA, this is CWA's ingest folder.
+      # INGEST_DIR: /cwa-book-ingest
+      # Optional: Calibre-Web-Automated. The user above must then be able to write CWA's ingest folder.
+      # Usually it is the same as CWA's.
+      # CWA_ENABLED: "true"
+      # The address of CWA, as the Bindery container reaches it, and the CWA account that Bindery uses.
+      # Leave the account out if CWA asks for no login.
+      # CWA_URL: http://calibre-web-automated:8083
+      # CWA_USERNAME: bindery
+      # CWA_PASSWORD: change-me
+      # Optional and independent. app.db is for logins, metadata.db is for book IDs.
+      # CWA_DB_PATH: /cwa-config/app.db
+      # CALIBRE_DB_PATH: /calibre-library/metadata.db
     ports:
       # If a reverse proxy on this host is the only way in, publish the port only on the host: 127.0.0.1:8085:8085
       - 8085:8085
@@ -44,11 +45,11 @@ services:
       start_period: 30s
       retries: 3
     volumes:
-      # CWA's ingest folder. New stories and updates go here. Bindery does not make the folder.
-      - /path/to/cwa/ingest:/cwa-book-ingest
       # The story database and personal.ini. Create the folder before the first start.
       - ./config:/config
-      # Optional read-only folders. Include SQLite journal files beside each database.
+      # Optional: the folder of INGEST_DIR. With CWA, CWA's ingest folder. Bindery does not make the folder.
+      # - /path/to/cwa/ingest:/cwa-book-ingest
+      # Optional read-only folders of CWA. Include SQLite journal files beside each database.
       # Mount the library that receives the ingest files, not a copy or another library.
       # - /path/to/cwa/config:/cwa-config:ro
       # - /path/to/calibre/library:/calibre-library:ro

tests/conftest.py

@@ -10,6 +10,7 @@ for name, sub in [("CONFIG_DIR", "config"), ("INGEST_DIR", "ingest"), ("TMP_DIR"
 (_tmp / "ingest").mkdir()  # Bindery does not make the ingest folder
 os.environ["BINDERY_START_THREADS"] = "false"
 os.environ["UPDATE_INTERVAL_HOURS"] = "0"
+os.environ["CWA_ENABLED"] = "true"  # most tests are about a Bindery with CWA
 for name in (
     "CWA_URL",
     "CWA_USERNAME",

tests/js/flows.test.mjs

@@ -115,8 +115,8 @@ test('each group of settings is an open card, and the download help starts close
   const cards = main.children.filter((el) => el.tagName !== 'h1');
   assert.deepEqual(cards.map((el) => [el.tagName, el.className, el.open, el.children[0].textContent]), [
     ['details', 'card', true, 'Recheck recent chapters'],
-    ['details', 'card', true, 'Uploads to CWA'],
     ['details', 'card', true, 'Book Downloads'],
+    ['details', 'card', true, 'Uploads to CWA'],
     ['details', 'card', true, 'Jobs list'],
     ['details', 'card', true, 'Global replacement rules'],
     ['details', 'card', true, 'FanFicFare settings (personal.ini)'],
@@ -233,11 +233,11 @@ test('dropping a file on a closed epub import card opens it and fills the file f
 test('library import sends only selected books and handles an empty candidate list', async () => {
   await mountStories();
   api.on('GET', 'library/candidates', []);
-  await click(main, 'Find stories in the library');
+  await click(main, 'Show all stories in the library');
   assert.match(main.textContent, /tracked already/);
   const books = [{ id: 8, uuid: 'a', title: 'A', authors: ['W'] }, { id: 9, uuid: 'b', title: 'B', authors: [] }];
   api.on('GET', 'library/candidates', books);
-  await click(main, 'Find stories in the library');
+  await click(main, 'Show all stories in the library');
   change(field(main, 'Select all'), false);
   await click(main, 'Import selected');
   assert.equal(writes().length, 0);
@@ -252,6 +252,91 @@ test('library import sends only selected books and handles an empty candidate li
   assert.deepEqual(api.sent('POST', 'library/import')[0].body, { books: [{ id: 8, uuid: 'a', title: 'A' }], check: false });
 });
 
+test('library import search filters the loaded list, selects no book at first and imports the ticked books', async () => {
+  await mountStories();
+  const card = section(main, 'Import from the library');
+  const form = nodes(card).find((el) => el.tagName === 'form');
+  const submit = find(form, 'button', 'Search');
+  assert.equal(form.className, 'library-search');
+  assert.equal(submit.attributes.type, 'submit');
+  const search = async (text) => {
+    field(card, 'Search the library').value = text;
+    emit(form, 'submit');
+    await settle();
+  };
+  const shown = () => ['Ash & Bone', 'Bright Sky', 'Ash Road'].filter((title) => field(card, title));
+  await search('   ');
+  assert.equal(api.requests.some((r) => r.path.startsWith('library/candidates')), false);
+  const books = [
+    { id: 8, uuid: 'a', title: 'Ash & Bone', authors: ['Wren'], publisher: 'www.royalroad.com' },
+    { id: 9, uuid: 'b', title: 'Bright Sky', authors: ['Ashley Poe', 'Wren'], publisher: 'archiveofourown.org' },
+    { id: 10, uuid: 'c', title: 'Ash Road', authors: [], publisher: 'www.royalroad.com' },
+  ];
+  api.on('GET', 'library/candidates', books);
+  await search('nothing');
+  assert.match(card.textContent, /No library books from a story site match/);
+  assert.equal(find(card, 'button', 'Import selected'), undefined);
+  // Each word must be in the title, the authors or the site. Capitals do not matter.
+  await search(' ASH ');
+  assert.deepEqual(shown(), ['Ash & Bone', 'Bright Sky', 'Ash Road']);
+  await search('ash  royalroad');
+  assert.deepEqual(shown(), ['Ash & Bone', 'Ash Road']);
+  await search('wren sky');
+  assert.deepEqual(shown(), ['Bright Sky']);
+  assert.equal(api.sent('GET', 'library/candidates').length, 1, 'One load serves each search');
+  await search('ash');
+  assert.deepEqual(['Select all', ...shown()].map((label) => field(card, label).checked), [false, false, false, false]);
+  await click(card, 'Import selected');
+  assert.equal(writes().length, 0);
+  change(field(card, 'Bright Sky'), true);
+  api.on('POST', 'library/import', [{ id: 'job' }]);
+  await click(card, 'Import selected');
+  assert.deepEqual(api.sent('POST', 'library/import')[0].body, { books: [{ id: 9, uuid: 'b', title: 'Bright Sky' }], check: true });
+  // A queued book leaves the list, without another load.
+  await search('ash');
+  assert.deepEqual(shown(), ['Ash & Bone', 'Ash Road']);
+  assert.equal(api.sent('GET', 'library/candidates').length, 1);
+  // The full list loads again, and selects each book.
+  await click(card, 'Show all stories in the library');
+  assert.equal(api.sent('GET', 'library/candidates').length, 2);
+  assert.deepEqual(['Select all', ...shown()].map((label) => field(card, label).checked), [true, true, true, true]);
+});
+
+test('the library import buttons cannot be used again until the answer comes', async () => {
+  await mountStories();
+  const card = section(main, 'Import from the library');
+  const form = nodes(card).find((el) => el.tagName === 'form');
+  const submit = find(form, 'button', 'Search');
+  const all = find(card, 'button', 'Show all stories in the library');
+  let answer;
+  api.on('GET', 'library/candidates', () => new Promise((resolve) => { answer = resolve; }));
+  field(card, 'Search the library').value = 'a';
+  emit(form, 'submit');
+  await settle();
+  assert.equal(submit.disabled, true);
+  emit(form, 'submit');
+  await settle();
+  assert.equal(api.sent('GET', 'library/candidates').length, 1);
+  answer([{ id: 8, uuid: 'a', title: 'A', authors: [] }]);
+  await settle();
+  assert.equal(submit.disabled, false);
+  assert.ok(field(card, 'A'));
+  emit(all, 'click');
+  await settle();
+  assert.equal(all.disabled, true);
+  answer([]);
+  await settle();
+  assert.equal(all.disabled, false);
+  // A failed load frees the button, and the next search loads again.
+  api.on('GET', 'library/candidates', { error: 'CWA is offline' }, 502);
+  await click(card, 'Show all stories in the library');
+  emit(form, 'submit');
+  await settle();
+  assert.equal(submit.disabled, false);
+  assert.equal(all.disabled, false);
+  assert.equal(api.sent('GET', 'library/candidates').length, 4);
+});
+
 test('rules edits, moves, removal and paragraph choices are saved in order', async () => {
   api.on('GET', 'stories/1/rules', { ...emptyRules, rules: [rule, { ...rule, pattern: 'second' }] });
   const editor = rulesEditor('stories/1/rules');
@@ -336,7 +421,7 @@ test('story list stacks on a narrow screen instead of scrolling sideways', async
   const wrap = nodes(main).find((el) => el.className === 'table-wrap stories-table');
   assert.ok(wrap);
   const dataLabels = nodes(wrap).filter((el) => el.tagName === 'td').map((el) => el.attributes['data-label']);
-  assert.deepEqual(dataLabels, [undefined, 'Author', 'Site', 'Chapters', 'Status', 'Last change', 'Last check']);
+  assert.deepEqual(dataLabels, [undefined, 'Author', 'Site', 'Chapters', 'Pages', 'Status', 'Last change', 'Last check']);
 });
 
 test('the story list shows removed and revised chapter counts, only when there are any', async () => {
@@ -704,9 +789,17 @@ test('with CWA off, the pages show nothing about CWA', async () => {
   assert.ok(section(main, 'Add stories'));
   assert.equal(section(main, 'Import from the library'), undefined);
   assert.doesNotMatch(main.textContent, /CWA|INGEST_DIR|library/i);
+  for (const ingest of [true, false]) {
+    api.on('GET', 'settings', { recent_chapters: 3, personal_ini: '[defaults]', ingest_available: ingest });
     settingsPage(main, {}, off);
     await settle();
     assert.ok(section(main, 'Jobs list'));
     assert.equal(section(main, 'Uploads to CWA'), undefined);
-  assert.doesNotMatch(main.textContent, /CWA|INGEST_DIR|library/i);
+    assert.doesNotMatch(main.textContent, /CWA|library/i);
+    // The alert about INGEST_DIR is under Book Downloads, also with CWA off.
+    const note = nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note');
+    assert.equal(note.hidden, ingest);
+    assert.equal(note.className, 'notice');
+    assert.match(note.textContent, /INGEST_DIR is not set\. Bindery sends no book to the ingest folder/);
+  }
 });

tests/js/layout.test.mjs

@@ -114,6 +114,12 @@ test('sortable headings show direction and remain usable above story cards on na
   assert.doesNotMatch(css, /\.stories-table thead\s*\{[^}]*clip-path/);
 });
 
+test('on narrow screens a story cell with no value shows no line', () => {
+  const start = css.indexOf('@media (max-width: 600px)', css.indexOf('.stories-table th button'));
+  assert.equal(declarations('.stories-table td:empty', css.slice(start)).display, 'none');
+  assert.doesNotMatch(css.slice(0, start), /\.stories-table td:empty/);
+});
+
 test('on narrow screens the headings are sort chips, and only the sorted chip is tinted with an arrow', () => {
   const start = css.indexOf('@media (max-width: 600px)', css.indexOf('.stories-table th button'));
   const mobile = css.slice(start);

tests/js/story-list.test.mjs

@@ -21,7 +21,7 @@ const story = (id, title, changes = {}) => ({
   id, title, url: `https://example.com/${id}`,
   sort_title: (title || `https://example.com/${id}`).trim().replace(/^(?:the|an?)\s+/i, '').toLowerCase(),
   author: 'Reader', site: 'example.com',
-  chapter_count: 42, offsite_count: 0, revised_count: 0, status: 'ok', message: 'No changes',
+  chapter_count: 42, page_count: 120, offsite_count: 0, revised_count: 0, status: 'ok', message: 'No changes',
   last_changed: null, last_checked: null, ...changes,
 });
 const rows = () => {
@@ -106,6 +106,39 @@ test('chapter sorting compares counts numerically and keeps equal counts in titl
   assert.equal(heading('Chapters').attributes['aria-sort'], 'ascending');
 });
 
+test('pages show beside the chapters and sort numerically', async (t) => {
+  await mount(t, [
+    story(1, 'Zeta', { page_count: 9 }),
+    story(2, 'Alpha', { page_count: 1200 }),
+    story(3, 'Beta', { page_count: 0 }),
+    story(4, 'Gamma', { page_count: 9 }),
+  ]);
+  const cells = nodes(rows()[0]).filter((el) => el.tagName === 'td').map((el) => el.attributes['data-label']);
+  assert.deepEqual(cells.slice(3, 5), ['Chapters', 'Pages']);
+  assert.equal(find(app, 'button', 'Pages').attributes['aria-label'], 'Sort by pages, most first');
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Alpha', 'Gamma', 'Zeta', 'Beta']);
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Beta', 'Gamma', 'Zeta', 'Alpha']);
+  assert.deepEqual(JSON.parse(window.localStorage.getItem(SORT_KEY)), { column: 'page_count', direction: 1 });
+});
+
+test('a story with no built book shows no page count and sorts last in each direction', async (t) => {
+  await mount(t, [
+    story(1, 'Alpha', { page_count: null }),
+    story(2, 'Beta', { page_count: 300 }),
+    story(3, 'Gamma', { page_count: 0 }),
+    story(4, 'Delta', { page_count: null }),
+  ]);
+  const pages = () => rows().map((row) => cell(row, 'Pages'));
+  assert.deepEqual(pages().map((el) => el.textContent), ['', '300', '', '0']);
+  assert.deepEqual(pages().map((el) => el.children.length), [0, 1, 0, 1], 'An empty cell hides on a narrow screen');
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Beta', 'Gamma', 'Alpha', 'Delta']);
+  await click(app, 'Pages');
+  assert.deepEqual(titles(), ['Gamma', 'Beta', 'Alpha', 'Delta']);
+});
+
 for (const [label, key] of [['Author', 'author'], ['Site', 'site'], ['Status', 'status']]) {
   test(`${label} sorts alphabetically in either direction and uses titles to break ties`, async (t) => {
     await mount(t, [
@@ -248,7 +281,7 @@ test('date columns keep each timestamp on one line, with last change before last
   const checked = '2026-01-03T05:06:00Z';
   await mount(t, [story(1, 'Alpha', { last_changed: changed, last_checked: checked })]);
   const headings = nodes(app).filter((el) => el.tagName === 'th').map((el) => el.textContent);
-  assert.deepEqual(headings, ['Title', 'Author', 'Site', 'Chapters', 'Status', 'Last change', 'Last check']);
+  assert.deepEqual(headings, ['Title', 'Author', 'Site', 'Chapters', 'Pages', 'Status', 'Last change', 'Last check']);
   for (const [label, value] of [['Last change', changed], ['Last check', checked]]) {
     const dateCell = cell(rows()[0], label);
     const time = nodes(dateCell).find((el) => el.tagName === 'time');

tests/js/upload-hold.test.mjs

@@ -54,7 +54,7 @@ test('the upload switches load and save on the settings page', async (t) => {
   assert.equal(warning.hidden, true);
   assert.equal(link.attributes['aria-describedby'], '');
   assert.match(warning.textContent, /Updates without a saved CWA book ID create duplicate books if CWA imports them/);
-  assert.equal(nodes(card).find((el) => el.attributes.id === 'ingest-note').hidden, true);
+  assert.equal(nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note').hidden, true);
   assert.equal(pause.disabled, false);
   assert.equal(hold.disabled, false);
   assert.equal(link.disabled, false);
@@ -129,7 +129,7 @@ for (const linkBooks of [true, false]) {
     assert.equal(nodes(card).find((el) => el.attributes.id === 'link-note').hidden, linkBooks);
     assert.equal(pause.checked, true);
     assert.equal(pause.disabled, true);
-    const note = nodes(card).find((el) => el.attributes.id === 'ingest-note');
+    const note = nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note');
     assert.equal(note.hidden, false);
     assert.equal(note.className, 'notice');
     assert.equal(pause.attributes['aria-describedby'], note.attributes.id);

tests/test_build_keys.py

@@ -11,7 +11,8 @@ from test_deliver import fake as fake
 from test_deliver import jobs as jobs
 from test_deliver import story_epub as story_epub
 
-from bindery import config, orchestrator
+from bindery import config, orchestrator, pages
+from bindery import store as storage
 from bindery.jobs import DownloadMode, Job
 
 ONE = [(1, "One", "a")]
@@ -44,13 +45,21 @@ def test_an_unchanged_story_is_not_built_or_sent_again(builds, monkeypatch):
     assert len(ingest_files()) == 2  # one epub and its sidecar
 
 
-@pytest.mark.parametrize("change", ["chapter", "rule", "global rule", "paragraphs", "metadata", "included"])
+@pytest.mark.parametrize(
+    "change", ["chapter", "revision", "renamed", "rule", "global rule", "paragraphs", "metadata", "included"]
+)
 def test_each_change_builds_and_sends_the_book(builds, monkeypatch, change):
     story_id = linked_story([(1, "One", "a"), (2, "Two", "b")])
     site(monkeypatch, [(1, "One", "a"), (2, "Two", "b")])
     update(story_id)
+    previous = "2020-01-01T00:00:00"
+    orchestrator.store.update_story(story_id, last_changed=previous)
     if change == "chapter":
         site(monkeypatch, [(1, "One", "a"), (2, "Two", "b"), (3, "Three", "c")])
+    elif change == "revision":
+        site(monkeypatch, [(1, "One", "a"), (2, "Two", "new text")])
+    elif change == "renamed":
+        site(monkeypatch, [(1, "One", "a"), (2, "Renamed", "b")])
     elif change == "rule":
         orchestrator.save_rules(story_id, [{"pattern": "a", "replacement": "b"}])
     elif change == "global rule":
@@ -62,10 +71,14 @@ def test_each_change_builds_and_sends_the_book(builds, monkeypatch, change):
     else:
         chapter = orchestrator.store.list_chapters(story_id)[0]
         orchestrator.store.update_chapter(chapter["id"], included=0)
-    assert "library book 1" in update(story_id)
+    mode = DownloadMode.RECHECK_ALL if change == "revision" else DownloadMode.NEW
+    assert "library book 1" in update(story_id, mode)
     assert builds.call_count == 2
     assert len(ingest_files()) == 4
+    changed_at = story(story_id)["last_changed"]
+    assert (changed_at != previous) == (change in ("chapter", "revision"))
     assert update(story_id).startswith("No changes")
+    assert story(story_id)["last_changed"] == changed_at
 
 
 def test_a_held_book_that_did_not_change_is_not_built_again(builds, monkeypatch):
@@ -110,13 +123,14 @@ def test_a_lost_file_is_built_again_only_to_send_it(builds):
     assert builds.call_count == 2
 
 
-def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(builds, client, jobs):
+@pytest.mark.parametrize("last_changed", [None, "2020-01-01T00:00:00"])
+def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(builds, client, jobs, last_changed):
     story_id = add_story(ingested_at="2026-01-01T00:00:00", calibre_id=1, calibre_uuid="uuid-1")
     job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
     assert "library book 1" in orchestrator._run_rebuild(job)
-    orchestrator.store.update_story(story_id, last_changed=None, message="before")
+    orchestrator.store.update_story(story_id, last_changed=last_changed, message="before")
     assert orchestrator._run_rebuild(job) == "No changes"
-    assert (story(story_id)["last_changed"], story(story_id)["message"]) == (None, "before")
+    assert (story(story_id)["last_changed"], story(story_id)["message"]) == (last_changed, "before")
     assert builds.call_count == 1
     # The Rebuild button of the story page.
     queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "rebuild"}, headers=H).json
@@ -124,7 +138,7 @@ def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(bui
     assert "library book 1" in orchestrator._run_rebuild(jobs.get_next())
     assert builds.call_count == 2
     assert len(ingest_files()) == 4
-    assert story(story_id)["last_changed"]
+    assert story(story_id)["last_changed"] == last_changed
     # Other modes take no such option.
     queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "new"}, headers=H).json
     assert queued["options"] == {}
@@ -152,6 +166,31 @@ def test_a_failed_delivery_is_tried_again_without_a_build(builds, fake, monkeypa
     assert builds.call_count == 1
 
 
+@pytest.mark.parametrize("change", ["chapter", "revision"])
+def test_chapter_timestamp_survives_a_failed_build_and_retry(builds, monkeypatch, change):
+    story_id = linked_story(ONE)
+    site(monkeypatch, ONE)
+    update(story_id)
+    previous = "2020-01-01T00:00:00"
+    orchestrator.store.update_story(story_id, last_changed=previous)
+    clock = ["2024-01-01T00:00:00"]
+    monkeypatch.setattr(storage, "now", lambda: clock[0])
+    site(monkeypatch, [*ONE, (2, "Two", "b")] if change == "chapter" else [(1, "One", "revised")])
+    job = Job(mode=DownloadMode.RECHECK_ALL, story_id=story_id)
+    build = orchestrator._build
+    monkeypatch.setattr(orchestrator, "_build", Mock(side_effect=OSError("disk full")))
+    with pytest.raises(OSError, match="disk full"):
+        orchestrator._run_site_job(job)
+    assert story(story_id)["last_changed"] == clock[0]
+    assert job.result["new"] or job.result["changed"]
+    clock[0] = "2024-02-01T00:00:00"
+    monkeypatch.setattr(orchestrator, "_build", build)
+    assert "library book 1" in orchestrator._run_site_job(job)
+    assert story(story_id)["last_changed"] == "2024-01-01T00:00:00"
+    assert update(story_id).startswith("No changes")
+    assert story(story_id)["last_changed"] == "2024-01-01T00:00:00"
+
+
 def test_a_new_build_removes_the_older_file(builds, monkeypatch):
     story_id = linked_story(ONE)
     site(monkeypatch, ONE)
@@ -227,6 +266,7 @@ def test_a_download_uses_the_built_book(builds, client, monkeypatch):
 
 def test_a_download_builds_a_book_that_has_no_file_or_that_changed(builds, client, monkeypatch):
     story_id = add_story()
+    changed_at = story(story_id)["last_changed"]
     first = download(client, story_id)
     download(client, story_id)
     assert builds.call_count == 1
@@ -234,12 +274,65 @@ def test_a_download_builds_a_book_that_has_no_file_or_that_changed(builds, clien
     assert download(client, story_id) != first
     assert builds.call_count == 2
     assert len(list(config.TMP_DIR.iterdir())) == 1
-    # The build of a download is not the build of a job. The job sends the book, and marks the change.
+    # The build of a download is not the build of a job. The job sends the book.
     assert orchestrator.store.get_setting(story_id, "built_key") is None
     job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
     assert "library book 1" in orchestrator._run_rebuild(job)
     assert builds.call_count == 2
-    assert story(story_id)["last_changed"]
+    assert story(story_id)["last_changed"] == changed_at
+
+
+# Page counts
+
+LONG = "A line of the story. " * 20
+
+
+def listed_pages(story_id):
+    return {s["id"]: s["page_count"] for s in orchestrator.list_stories()}[story_id]
+
+
+def test_a_story_has_no_page_count_until_its_book_is_built(builds, monkeypatch):
+    story_id = linked_story(ONE)
+    assert listed_pages(story_id) is None
+    site(monkeypatch, ONE)
+    update(story_id)
+    (built,) = config.TMP_DIR.iterdir()
+    assert listed_pages(story_id) == pages.epub_pages(built) == 1
+
+
+def test_each_build_counts_the_pages_of_the_new_book(builds, client, monkeypatch):
+    story_id = linked_story(ONE)
+    site(monkeypatch, [*ONE, *[(n, f"Chapter {n}", "<br/>".join([LONG] * 31)) for n in range(2, 6)]])
+    update(story_id)
+    long_book = listed_pages(story_id)
+    # Each of the 124 lines is a paragraph and 6 lines of text.
+    assert long_book >= 4 * 31 * 7 // 31
+    # The count is of the built book, so a rule that removes text changes it without a download.
+    orchestrator.save_rules(story_id, [{"pattern": "(A line of the story. ){19}", "replacement": ""}])
+    download(client, story_id)
+    assert builds.call_count == 2
+    assert 0 < listed_pages(story_id) < long_book / 3
+
+
+def test_a_book_from_before_the_page_count_is_counted_without_a_build(builds, client, monkeypatch):
+    story_id = linked_story(ONE)
+    site(monkeypatch, ONE)
+    update(story_id)
+    orchestrator.store.set_setting(story_id, "page_count", None)
+    download(client, story_id)
+    assert builds.call_count == 1
+    assert listed_pages(story_id) == 1
+
+
+def test_a_book_that_cannot_be_counted_is_still_built(builds, monkeypatch):
+    def fail(path):
+        raise orchestrator.EpubError("unreadable")
+
+    monkeypatch.setattr(pages, "epub_pages", fail)
+    story_id = linked_story(ONE)
+    site(monkeypatch, ONE)
+    assert "library book 1" in update(story_id)
+    assert listed_pages(story_id) is None
 
 
 def test_stop_tracking_deletes_the_built_book(builds, client):

tests/test_config.py

@@ -143,7 +143,7 @@ def test_defaults(monkeypatch, reload_config):
     ]
     assert config.ALLOWED_HOSTS == []
     assert config.ADMIN_USERS == []
-    assert config.CWA_ENABLED is True
+    assert config.CWA_ENABLED is False
     assert config.IGNORED_SETTINGS == []
     assert config.SESSION_COOKIE_SECURE is False
     assert config.TRUST_PROXY is False
@@ -161,8 +161,11 @@ def test_empty_ingest_directory_disables_uploads(monkeypatch, reload_config, val
     assert config.INGEST_DIR is None
 
 
-@pytest.mark.parametrize("value", ["false", " No ", "0", "off"])
+@pytest.mark.parametrize("value", [None, "", "false", " No ", "0", "maybe"])
 def test_cwa_off_ignores_the_cwa_settings_but_not_the_ingest_folder(monkeypatch, reload_config, value):
+    if value is None:
+        monkeypatch.delenv("CWA_ENABLED")
+    else:
         monkeypatch.setenv("CWA_ENABLED", value)
     monkeypatch.setenv("CWA_URL", "https://cwa.invalid")
     monkeypatch.setenv("CWA_USERNAME", "bindery")
@@ -178,8 +181,8 @@ def test_cwa_off_ignores_the_cwa_settings_but_not_the_ingest_folder(monkeypatch,
     assert str(config.INGEST_DIR) == "/books"
 
 
-@pytest.mark.parametrize("value", ["", "true", "maybe"])
+@pytest.mark.parametrize("value", ["true", " Yes ", "1", "on"])
-def test_only_a_clear_false_turns_cwa_off(monkeypatch, reload_config, value):
+def test_only_a_clear_true_turns_cwa_on(monkeypatch, reload_config, value):
     monkeypatch.setenv("CWA_ENABLED", value)
     monkeypatch.setenv("CWA_URL", "https://cwa.invalid")
     reload_config()
@@ -197,17 +200,19 @@ def test_prepare_with_cwa_off_logs_nothing_about_missing_cwa(folders, monkeypatc
     with caplog.at_level("INFO"):
         config.prepare()
     lines = [r.getMessage() for r in caplog.records]
-    expected = ["CWA_ENABLED is false. Bindery ignores CWA_URL, CALIBRE_DB_PATH."] if ignored else []
+    ignores = "CWA_ENABLED is not true. Bindery ignores CWA_URL, CALIBRE_DB_PATH. To use CWA, set CWA_ENABLED=true."
+    expected = [ignores] if ignored else []
+    assert [r.levelname for r in caplog.records if "CWA" in r.getMessage()] == ["WARNING"] * len(expected)
     assert [line for line in lines if "CWA" in line or "INGEST" in line] == expected
 
 
 @pytest.mark.parametrize(
     ("ingest", "url", "database", "expected"),
     [
-        (False, "", None, ["INGEST_DIR is not set", "cannot find the library book of a story.", "CWA_ENABLED=false"]),
+        (False, "", None, ["INGEST_DIR is not set", "cannot find the library book of a story.", "remove CWA_ENABLED"]),
-        (True, "", None, ["so each update adds a new book to CWA.", "CWA_ENABLED=false"]),
+        (True, "", None, ["so each update adds a new book to CWA.", "remove CWA_ENABLED"]),
-        (True, "", Path("/library/metadata.db"), ["CWA_URL is not set. Bindery cannot search", "CWA_ENABLED=false"]),
+        (True, "", Path("/library/metadata.db"), ["CWA_URL is not set. Bindery cannot search", "remove CWA_ENABLED"]),
-        (False, "https://cwa.invalid", None, ["INGEST_DIR is not set", "CWA_ENABLED=false"]),
+        (False, "https://cwa.invalid", None, ["INGEST_DIR is not set", "remove CWA_ENABLED"]),
         (True, "https://cwa.invalid", None, []),
     ],
 )

tests/test_last_changed.py

@@ -0,0 +1,66 @@
+"""Last Changed records stored chapters and revisions, not changes to the built book."""
+
+import sqlite3
+
+import pytest
+
+from bindery import store as storage
+
+
+@pytest.fixture
+def state(tmp_path, monkeypatch):
+    clock = ["2024-01-01T00:00:00"]
+    monkeypatch.setattr(storage, "now", lambda: clock[0])
+    store = storage.Store(tmp_path / "stories.db")
+    sid = store.create_story("story", "site", "Title", "Author", [])
+    return store, sid, clock
+
+
+@pytest.mark.parametrize("source", ["new", "import"])
+def test_new_chapters_and_revisions_set_the_timestamp_in_the_store(state, source):
+    store, sid, clock = state
+    other = store.create_story("other", "site", "Other", "Author", [])
+    assert store.get_story(sid)["last_changed"] is None
+    first = store.add_chapter(sid, "one", "One", "first", "h1", source, 0)
+    assert store.get_story(sid)["last_changed"] == clock[0]
+    assert store.get_chapter(first)["added_at"] == clock[0]
+    clock[0] = "2024-02-01T00:00:00"
+    store.add_chapters(sid, [{"url": "two", "title": "Two", "html": "second", "text_hash": "h2"}], source, [])
+    assert store.get_story(sid)["last_changed"] == clock[0]
+    clock[0] = "2024-03-01T00:00:00"
+    store.add_revision(first, "revised", "h3", "recheck")
+    assert store.get_story(sid)["last_changed"] == clock[0]
+    assert store.get_chapter(first)["changed_at"] == clock[0]
+    assert store.list_revisions(first)[-1]["fetched_at"] == clock[0]
+    assert storage.Store(store.path).get_story(sid)["last_changed"] == clock[0]
+    assert store.get_story(other)["last_changed"] is None
+
+
+def test_changes_without_new_chapters_or_revisions_keep_the_timestamp(state):
+    store, sid, clock = state
+    first = store.add_chapter(sid, "one", "One", "first", "h1", "new", 0)
+    second = store.add_chapter(sid, "two", "Two", "second", "h2", "new", 1)
+    store.add_revision(first, "revised", "h3", "recheck")
+    previous = clock[0]
+    clock[0] = "2024-02-01T00:00:00"
+    store.update_story(sid, title="Edited", pinned={"title": "Edited"})
+    store.set_rules(sid, [{"pattern": "first", "replacement": "1st"}])
+    store.update_chapter(first, title="Renamed", included=False, on_site=False, current_rev=1)
+    store.replace_revision(first, 1, "original site text", "h4")
+    store.reorder_chapters(sid, lambda order: list(reversed(order)))
+    store.add_chapters(sid, [], "import", ["one", "two"])
+    store.delete_revision(first, 2)
+    store.delete_chapters(sid, [second])
+    assert store.get_story(sid)["last_changed"] == previous
+
+
+def test_a_failed_chapter_transaction_does_not_change_the_timestamp(state):
+    store, sid, clock = state
+    store.add_chapter(sid, "one", "One", "first", "h1", "new", 0)
+    previous = clock[0]
+    clock[0] = "2024-02-01T00:00:00"
+    chapters = [{"url": url, "title": url, "html": "text", "text_hash": "h"} for url in ("two", "one")]
+    with pytest.raises(sqlite3.IntegrityError):
+        store.add_chapters(sid, chapters, "import", [])
+    assert store.get_story(sid)["last_changed"] == previous
+    assert [chapter["url"] for chapter in store.list_chapters(sid)] == ["one"]

tests/test_opds.py

@@ -48,10 +48,10 @@ def add_book(title="A book", author="Writer", changed=None, url=None, included=T
     store = orchestrator.store
     url = url or f"https://example.com/books/{len(store.list_stories()) + 1}"
     sid = store.create_story(url, "example.com", title, author, [])
-    store.update_story(sid, last_changed=changed, site_values=meta or {})
     if included:
         html = "<p>Stored text.</p>"
         store.add_chapter(sid, url + "/1", "One", html, diffing.text_hash(html), "site", 0)
+    store.update_story(sid, last_changed=changed, site_values=meta or {})
     return sid
 
 
@@ -205,6 +205,17 @@ def test_entry_shows_the_book_metadata_and_cover(client):
     assert client.get(f"/opds/books/{add_book()}/cover").status_code == 404
 
 
+def test_entry_shows_the_page_count_of_a_built_book(client):
+    sid = add_book()
+    orchestrator.store.set_setting(sid, "page_count", 321)
+    add_book(title="Not built")
+    contents = {
+        e.findtext("a:title", namespaces=NS): e.findtext("a:content", namespaces=NS)
+        for e in entries(feed(client.get("/opds/title")))
+    }
+    assert contents == {"A book": "Chapters: 1\nPages: 321", "Not built": "Chapters: 1"}
+
+
 def test_cover_of_an_unsafe_type_is_left_out(client):
     sid = add_book()
     orchestrator.store.put_image(sid, "https://example.com/c.svg", "svg", "image/svg+xml", b"<svg/>", is_cover=True)

tests/test_optional_cwa.py

@@ -40,7 +40,7 @@ def library(isolated, monkeypatch):
     monkeypatch.setattr(orchestrator, "_find_book", Mock(return_value=None))
 
 
-@pytest.mark.parametrize("enabled", [False, True])
+@pytest.mark.parametrize("enabled", [None, False, True])
 def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
     env = {
         **os.environ,
@@ -51,7 +51,14 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
         "URL_BASE": "",
         "CWA_ENABLED": str(enabled),
     }
-    for name in ("INGEST_DIR", "CWA_URL", "CWA_DB_PATH", "CALIBRE_DB_PATH"):
+    # CWA is off unless CWA_ENABLED turns it on.
+    for name in (
+        "INGEST_DIR",
+        "CWA_URL",
+        "CWA_DB_PATH",
+        "CALIBRE_DB_PATH",
+        *(["CWA_ENABLED"] if enabled is None else []),
+    ):
         env.pop(name, None)
     result = subprocess.run(
         [
@@ -60,6 +67,7 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
             "from bindery.web import app; "
             "client = app.test_client(); "
             "assert client.get('/api/health').status_code == 200; "
+            f"assert client.get('/api/config').json['cwa_enabled'] is {bool(enabled)}; "
             "settings = client.get('/api/settings').json; "
             "assert settings['pause_uploads'] is True; "
             "assert settings['ingest_available'] is False; "
@@ -73,9 +81,9 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
     )
     assert result.returncode == 0, result.stderr
     assert "ERROR" not in result.stderr
-    # Bindery names what is missing, until the user says that there is no CWA.
+    # Bindery names what is missing only for a user who turned CWA on.
-    for text in ("WARNING", "INGEST_DIR is not set", "CWA_URL and CALIBRE_DB_PATH are not set", "CWA_ENABLED=false"):
+    for text in ("WARNING", "INGEST_DIR is not set", "CWA_URL and CALIBRE_DB_PATH are not set", "remove CWA_ENABLED"):
-        assert (text in result.stderr) is enabled
+        assert (text in result.stderr) is bool(enabled)
     if not enabled:
         assert "CWA" not in result.stderr and "INGEST" not in result.stderr
 
@@ -134,7 +142,7 @@ def test_cwa_off_shows_in_the_config_and_refuses_cwa_logins(client, monkeypatch,
     monkeypatch.setattr(config, "CWA_DB_PATH", "/cwa/app.db")
     response = client.get("/api/stories")
     assert response.status_code == 503
-    assert response.json["error"].startswith("AUTH_METHOD=cwa needs CWA, but CWA_ENABLED is false.")
+    assert response.json["error"].startswith("AUTH_METHOD=cwa needs CWA_ENABLED=true.")
 
 
 def test_without_cwa_the_download_template_uses_the_story_id(client, cwa_off):

tests/test_pages.py

@@ -0,0 +1,122 @@
+"""The page estimate of a document and of a built epub."""
+
+import zipfile
+
+import pytest
+
+from bindery import pages
+from bindery.epub import EpubError, spine_documents
+from bindery.pages import Measure
+
+TEXT = ["One line of the story.", "Another, longer line of the story that goes on for a while.", "The end."]
+CHARS = sum(map(len, TEXT))
+
+
+def test_a_paragraph_is_text_after_a_block_or_a_line_break():
+    assert pages.measure("<p>One</p><p>Two</p>") == Measure(2, 6)
+    assert pages.measure("<div>One<br/><br/>Two<br>Three</div>") == Measure(3, 11)
+    assert pages.measure("<div><div>One</div><div>Two</div></div>") == Measure(2, 6)
+    # Text before, between and after the blocks of an element counts too.
+    assert pages.measure("<div>Intro<p>One</p>Middle<p>Two</p>End</div>") == Measure(5, 20)
+    assert pages.measure("<h3>Title</h3><ul><li>One</li><li>Two</li></ul><blockquote>Three</blockquote>") == Measure(
+        4, 16
+    )
+
+
+@pytest.mark.parametrize(
+    "book",
+    [
+        "".join(f"<p>{line}</p>" for line in TEXT),
+        "".join(f'<p class="a">{line}</p>\n' for line in TEXT),
+        "<div>" + "".join(f"<div>{line}</div>" for line in TEXT) + "</div>",
+        "<div>" + "<br/><br/>".join(TEXT) + "</div>",
+        "<div>" + "<br>\n".join(TEXT) + "</div>",
+        f"<div>{TEXT[0]}<p>{TEXT[1]}</p>{TEXT[2]}</div>",
+    ],
+)
+def test_the_same_text_gives_the_same_measure_in_each_markup(book):
+    assert pages.measure(book) == Measure(3, CHARS)
+
+
+def test_inline_elements_and_table_cells_stay_in_the_line():
+    assert pages.measure('<p>ab<b>cd</b><a href="x">ef</a> <span><i>gh</i></span></p>') == Measure(1, 9)
+    assert pages.measure("<table><tr><td>Name</td><td>Value</td></tr><tr><th>a</th><td>b</td></tr></table>") == Measure(
+        2, 11
+    )
+    assert pages.measure('<p>Before<img src="a.png"/>after</p>') == Measure(1, 11)
+
+
+def test_only_the_text_of_the_book_counts():
+    document = (
+        '<?xml version="1.0"?><html><head><title>Hidden title</title><style>p { color: red }</style>'
+        '<link href="a.css"/></head><body><script>var hidden;</script><script src="x.js"/>'
+        "<!-- a note --><p>Shown &amp; counted</p>\n\n<p> </p><p></p></body></html>"
+    )
+    assert pages.measure(document) == Measure(1, len("Shown & counted"))
+    assert pages.measure("") == Measure(0, 0)
+    assert pages.measure("<p>\n</p>") == Measure(0, 0)
+
+
+def test_a_paragraph_that_only_spaces_the_text_is_not_counted():
+    spaced = "<p>One</p><p>&nbsp;</p><p>\xa0</p><p> &#160; </p><p><br/></p><p><span>&nbsp;</span></p><p>Two</p>"
+    assert pages.measure(spaced) == pages.measure("<p>One</p><p>Two</p>") == Measure(2, 6)
+    assert pages.measure("<div>One<br/>&nbsp;<br/>Two</div>") == Measure(2, 6)
+    # A no-break space inside a paragraph is a character of its text.
+    assert pages.measure("<p>One&nbsp;<b>two</b>\xa0three</p>") == Measure(1, 13)
+
+
+def test_a_page_is_31_lines_and_a_line_is_a_paragraph_or_70_characters():
+    assert pages.page_count(Measure(62, 620)) == (62 + 620 // 70) // 31 == 2
+    assert pages.page_count(Measure(31 * 9, 31 * 9)) == 9
+    assert pages.page_count(Measure(1000, 177_000)) == (1000 + 177_000 // 70) // 31
+
+
+def test_a_short_book_has_one_page_and_a_book_with_no_text_has_none():
+    assert pages.page_count(Measure(1, 10)) == 1
+    assert pages.page_count(Measure(30, 30)) == 1
+    assert pages.page_count(Measure()) == 0
+    # One block of text: 2400 characters start the second page before 31 lines do.
+    assert pages.page_count(Measure(1, 2399)) == 1
+    assert pages.page_count(Measure(1, 2400)) == 2
+
+
+def epub(path, documents, spine=None):
+    names = [f"file {n}.xhtml" for n in range(len(documents))]
+    items = "".join(f'<item id="f{n}" href="{name.replace(" ", "%20")}"/>' for n, name in enumerate(names))
+    items += '<item id="css" href="stylesheet.css"/>'
+    refs = "".join(f'<itemref idref="f{n}"/>' for n in (spine or range(len(documents))))
+    with zipfile.ZipFile(path, "w") as z:
+        z.writestr(
+            "META-INF/container.xml",
+            '<container xmlns="urn:oasis:names:tc:opendocument:xmlns:container"><rootfiles>'
+            '<rootfile full-path="OEBPS/content.opf"/></rootfiles></container>',
+        )
+        z.writestr(
+            "OEBPS/content.opf",
+            f'<package xmlns="http://www.idpf.org/2007/opf"><metadata/><manifest>{items}</manifest>'
+            f"<spine>{refs}</spine></package>",
+        )
+        z.writestr("OEBPS/stylesheet.css", "p { margin: 0 }")
+        for name, document in zip(names, documents, strict=True):
+            z.writestr(f"OEBPS/{name}", document)
+    return path
+
+
+def test_an_epub_is_counted_from_the_documents_of_its_spine(tmp_path):
+    chapter = "<html><head><title>T</title></head><body>" + "<p>x</p>" * 31 + "</body></html>"
+    book = epub(tmp_path / "book.epub", [chapter, "<html><body><p>Not in the spine</p></body></html>", chapter], [2, 0])
+    assert spine_documents(book) == [chapter, chapter]
+    # 62 paragraphs of one character.
+    assert pages.epub_pages(book) == 2
+    assert pages.epub_pages(epub(tmp_path / "empty.epub", ["<html><body><img src='a.png'/></body></html>"])) == 0
+
+
+def test_a_file_that_is_not_an_epub_cannot_be_counted(tmp_path):
+    path = tmp_path / "bad.epub"
+    path.write_bytes(b"not a zip")
+    with pytest.raises(EpubError):
+        pages.epub_pages(path)
+    with zipfile.ZipFile(path, "w") as z:
+        z.writestr("META-INF/container.xml", "<container/>")
+    with pytest.raises(EpubError):
+        pages.epub_pages(path)