Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2f59b0e125 | ||
|
|
ca3d730129 | ||
|
|
9a48356155 | ||
|
|
eea6adcc91 | ||
|
|
46de77c228 | ||
|
|
b73b2808ca |
@@ -1,10 +1,13 @@
|
||||
# Bindery
|
||||
|
||||
Bindery downloads web fiction into a
|
||||
[Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library and
|
||||
keeps the books up to date. It uses [FanFicFare](https://github.com/JimmXinu/FanFicFare), so it
|
||||
supports the sites that FanFicFare supports. The tested site is RoyalRoad.
|
||||
CWA is optional. You can download ePubs from Bindery without it.
|
||||
Bindery downloads web fiction and keeps each story up to date as an ePub. You give it the URL of a
|
||||
story. Bindery downloads the chapters, checks for new chapters each day, and builds the book again
|
||||
after each change. Your reader app gets the books from the OPDS catalog of Bindery.
|
||||
|
||||
Bindery uses [FanFicFare](https://github.com/JimmXinu/FanFicFare), so it supports the sites that
|
||||
FanFicFare supports. The tested site is RoyalRoad.
|
||||
|
||||
Bindery can also send each book to a [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library. See "Calibre-Web-Automated".
|
||||
|
||||
## How it differs from FanFicFare
|
||||
|
||||
@@ -22,9 +25,6 @@ builds each ePub from that database.
|
||||
|
||||
## Quick start
|
||||
|
||||
For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update as a new book. See
|
||||
"Limitations".
|
||||
|
||||
1. Get the code, and make the folder for Bindery's data.
|
||||
|
||||
```bash
|
||||
@@ -33,8 +33,8 @@ For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update
|
||||
mkdir config
|
||||
```
|
||||
|
||||
2. Edit `docker-compose.yml` or environment variables. Set the CWA ingest folder, the CWA address and account, and the user
|
||||
and group of the container.
|
||||
2. Edit `docker-compose.yml`. Set the user and group of the container. **That user must be able to
|
||||
write to `./config`.**
|
||||
|
||||
3. Build and start the container.
|
||||
|
||||
@@ -42,10 +42,54 @@ For uploads to CWA, use CWA v4.0.1 or later. Older versions can import an update
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
4. Open `http://localhost:8085`, and log in with a CWA user.
|
||||
4. Open `http://localhost:8085`. Under "Add stories", enter the URL of a story, and click "Add".
|
||||
Bindery downloads the chapters and builds the book. To get the book at once, click "Download ePub"
|
||||
on the story page.
|
||||
|
||||
The container runs as the `user` in `docker-compose.yml`. **That user must be able to write to
|
||||
`./config` and to the CWA ingest folder.** Usually this is the same user and group as CWA.
|
||||
5. Open Settings. Under "Book Downloads", check "Enable OPDS", and save. In your reader app, add the
|
||||
catalog `http://localhost:8085/opds`, with your server address in place of `localhost`.
|
||||
|
||||
Bindery then checks each story for new chapters every 24 hours. When a story changes, Bindery builds
|
||||
its book again, and the catalog has the new book.
|
||||
|
||||
The example Compose file asks for no login. Set a login before other people can reach Bindery. See
|
||||
"Logins".
|
||||
|
||||
## OPDS catalog
|
||||
|
||||
OPDS lets a reader app browse and download books from Bindery. An admin turns it on with "Enable OPDS"
|
||||
under Settings > Book Downloads. The catalog URL is `http://localhost:8085/opds`, with your server
|
||||
address in place of `localhost`. If Bindery runs under `URL_BASE`, put it before `/opds`.
|
||||
|
||||
The catalog has three lists, with 50 books per page:
|
||||
|
||||
- By Title, in title order without a leading "The", "A" or "An".
|
||||
- By Author, in surname order. Each author leads to their books. A book shows under each of its authors.
|
||||
- By Last Changed, with the newest change first.
|
||||
|
||||
Last Changed records when Bindery stores a new chapter or a new revision of an existing chapter.
|
||||
Rebuilds, metadata edits and rule changes do not update this date.
|
||||
|
||||
A book appears when it has a stored, included chapter. It shows its authors, series, description, tags,
|
||||
language and cover, with your kept values in place of the published values. A reader can search by title,
|
||||
author, series and tag. A download uses the stored chapters, the current ePub rules and the download
|
||||
template for the file name.
|
||||
|
||||
`AUTH_METHOD` sets how a reader logs in:
|
||||
|
||||
- `none`: The reader needs no login.
|
||||
- `local`: The reader sends your Bindery user name and password with HTTP Basic authentication.
|
||||
A browser with a Bindery session works too.
|
||||
- `cwa`: The reader sends your CWA user name and password with HTTP Basic authentication.
|
||||
A browser with a Bindery session works too.
|
||||
- `proxy`: Each user sets an OPDS password under Settings > Book Downloads. Settings shows the user
|
||||
name for the reader, which is the name in `PROXY_AUTH_USER_HEADER`. The reader sends that name and
|
||||
password with HTTP Basic authentication. The password has 8 to 256 characters, and Bindery keeps a
|
||||
hash of it. Set the proxy to skip its login for the catalog. See "Paths without the proxy login".
|
||||
|
||||
Bindery keeps a good login for five minutes. Wrong passwords count toward the lockouts in
|
||||
"Lockouts and sessions". With `TRUST_PROXY=true`, each reader has its own lockout.
|
||||
Use HTTPS outside a trusted local network, because HTTP Basic sends the password with each request.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -53,17 +97,11 @@ Environment variables set the configuration. **Restart the container after a cha
|
||||
|
||||
| Variable | Description | Default value |
|
||||
| --- | --- | --- |
|
||||
| `CWA_ENABLED` | `false` if you have no CWA. See "Without CWA". | `true` |
|
||||
| `CWA_URL` | The address of CWA, for example `http://calibre-web-automated:8083` | None |
|
||||
| `CWA_USERNAME`, `CWA_PASSWORD` | The CWA account of Bindery. Leave them unset if CWA asks for no login. | None |
|
||||
| `CWA_DB_PATH` | Calibre-Web's `app.db`, for logins. See "CWA logins". | None |
|
||||
| `CALIBRE_DB_PATH` | The calibre `metadata.db`, for book ids. See "How books reach the library". | None |
|
||||
| `UPDATE_INTERVAL_HOURS` | Hours between checks for new chapters. `0` turns them off. | `24` |
|
||||
| `DEFAULT_LOCKED_FIELDS` | Fields that new stories lock: `title`, `authors`, `comments`, `tags`, `series`, `pubdate`, `languages` | `title` |
|
||||
| `AUTH_METHOD` | `none`, `local`, `cwa` or `proxy`. See "Logins". | `none` |
|
||||
| `BINDERY_USERNAME` | The user name for `AUTH_METHOD=local`. Case-sensitive. | None |
|
||||
| `BINDERY_PASSWORD_HASH` | The password hash for `AUTH_METHOD=local`. See "Local login". | None |
|
||||
| `CWA_ALLOWED_USERS` | The CWA users that can log in. Empty allows every CWA user. | Empty |
|
||||
| `BINDERY_ADMIN_USERS` | The users that can change the global settings. See "Admins". | Empty |
|
||||
| `PROXY_AUTH_USER_HEADER` | The header in which a reverse proxy sends the user name | `X-Remote-User` |
|
||||
| `PROXY_AUTH_TRUSTED_IPS` | The addresses and networks that can send the user header. See "Logins". | Loopback and private addresses |
|
||||
@@ -73,45 +111,12 @@ Environment variables set the configuration. **Restart the container after a cha
|
||||
| `SESSION_COOKIE_SECURE` | `true` if users reach Bindery only over HTTPS | `false` |
|
||||
| `TRUST_PROXY` | `true` if a reverse proxy sends the client address. See "Lockouts and sessions". | `false` |
|
||||
| `URL_BASE` | A path prefix, such as `/bindery`. See "Reverse proxy". | None |
|
||||
| `INGEST_DIR` | The CWA ingest folder. If unset, all uploads stay paused. Bindery does not make this folder. Without CWA, a folder for the built ePubs. | None |
|
||||
| `INGEST_DIR` | A folder that gets each built ePub. With CWA, the CWA ingest folder. Bindery does not make this folder. | None |
|
||||
| `CONFIG_DIR` | The story database and `personal.ini` | `/config` |
|
||||
| `TMP_DIR` | The folder for the built ePubs. Bindery keeps the last build of each story. | `/tmp/bindery` |
|
||||
| `LOG_LEVEL` | The log level | `INFO` |
|
||||
|
||||
Separate the values in a list with commas.
|
||||
|
||||
### Without CWA
|
||||
|
||||
Set `CWA_ENABLED=false` to run Bindery without CWA. Bindery still downloads chapters and builds
|
||||
ePubs. Download the books from each story page.
|
||||
|
||||
With `CWA_ENABLED=false`:
|
||||
|
||||
- Bindery makes no request to CWA. It ignores `CWA_URL`, `CWA_USERNAME`, `CWA_PASSWORD`,
|
||||
`CWA_DB_PATH` and `CALIBRE_DB_PATH`.
|
||||
- The web interface and the log show nothing about CWA. The pause, the library links and the
|
||||
approvals do not apply.
|
||||
- If `INGEST_DIR` is set, Bindery puts each changed book in that folder as a plain ePub. Use this to
|
||||
import the books into other software. Each file gets a new name, so Bindery never replaces a file
|
||||
in the folder.
|
||||
- `AUTH_METHOD=cwa` does not work. Use `local` or `proxy`. On a private network, you can use `none`.
|
||||
|
||||
If you use the example Compose file, remove its CWA connection values. Keep `INGEST_DIR` and its
|
||||
volume mount only if you want the ePubs in a folder.
|
||||
|
||||
Without `CWA_ENABLED=false`, Bindery expects CWA. At each start, the log then has a warning for each
|
||||
function that a missing `INGEST_DIR`, `CWA_URL` or `CALIBRE_DB_PATH` turns off. Bindery still
|
||||
downloads chapters and builds ePubs.
|
||||
|
||||
Without `INGEST_DIR`, Settings shows "Pause all uploads" checked and a highlighted explanation.
|
||||
You cannot clear that checkbox until you set `INGEST_DIR` and restart Bindery.
|
||||
If neither `CWA_URL` nor `CALIBRE_DB_PATH` is set, library linking is off and its checkbox is disabled.
|
||||
Set either value and restart Bindery to enable the checkbox. Bindery keeps your saved linking preference.
|
||||
Saved CWA book IDs still work for uploads if you configure an ingest folder.
|
||||
|
||||
To export ePubs for other software, set `INGEST_DIR` and turn off "Link books to CWA library" in Settings.
|
||||
Each build without a saved CWA book ID then puts a plain ePub in that folder, including updates.
|
||||
"Pause all uploads" still stops these exports. If CWA imports the files, updates will create duplicate books.
|
||||
Separate the values in a list with commas. The variables for CWA are in "Calibre-Web-Automated".
|
||||
|
||||
### Logins
|
||||
|
||||
@@ -190,32 +195,6 @@ key, and shows the reason.
|
||||
To change the account, change the user name or make a new hash, and restart Bindery. This ends the
|
||||
open sessions.
|
||||
|
||||
#### CWA logins
|
||||
|
||||
`AUTH_METHOD=cwa` needs `CWA_URL`, `CWA_DB_PATH` or both. Without `CWA_DB_PATH`, Bindery sends the
|
||||
user name and password to `CWA_URL/opds` as an HTTP Basic login. This is the OPDS login.
|
||||
|
||||
**The OPDS login works only if CWA refuses a wrong password.** A CWA with anonymous browsing can
|
||||
accept every OPDS login. Bindery tests this with a login that cannot exist. If CWA accepts it,
|
||||
Bindery refuses each OPDS login, and the log names the reason. Set `CWA_DB_PATH` then, or use
|
||||
another `AUTH_METHOD`.
|
||||
|
||||
With `CWA_DB_PATH`, Bindery checks the password against the account in Calibre-Web's `app.db`. A
|
||||
wrong password or an unknown account fails at once. If Bindery cannot read the file or its password
|
||||
hash, it uses the OPDS login.
|
||||
|
||||
`CWA_DB_PATH` is the path of `app.db` inside the container. To use it, mount the CWA config folder
|
||||
read-only into the container. Mount the whole folder, so that SQLite can read the journal files
|
||||
beside the database. **Bindery does not write this database.**
|
||||
|
||||
**Database logins check only the password.** CWA's external login providers and multi-factor
|
||||
authentication do not apply. If they must apply, use the `proxy` mode.
|
||||
|
||||
- The CWA guest user cannot log in.
|
||||
- If `CWA_ALLOWED_USERS` is set, only those users can log in. Bindery checks this list on each
|
||||
request.
|
||||
- `CWA_USERNAME` and `CWA_PASSWORD` are only for the library lookups of Bindery.
|
||||
|
||||
#### Lockouts and sessions
|
||||
|
||||
The `local` and `cwa` modes lock out password guesses:
|
||||
@@ -309,39 +288,6 @@ The answer lists the queued jobs and the unsupported URLs: `{"jobs": [...], "err
|
||||
|
||||
If Bindery runs under `URL_BASE`, put the prefix before `/api`.
|
||||
|
||||
### OPDS catalog
|
||||
|
||||
OPDS lets a reader app browse and download books from Bindery. An admin turns it on with "Enable OPDS"
|
||||
under Settings > Book Downloads. The catalog URL is `http://localhost:8085/opds`, with your server
|
||||
address in place of `localhost`. If Bindery runs under `URL_BASE`, put it before `/opds`.
|
||||
|
||||
The catalog has three lists, with 50 books per page:
|
||||
|
||||
- By Title, in title order without a leading "The", "A" or "An".
|
||||
- By Author, in surname order. Each author leads to their books. A book shows under each of its authors.
|
||||
- By Last Changed, with the newest change first.
|
||||
|
||||
A book appears when it has a stored, included chapter. It shows its authors, series, description, tags,
|
||||
language and cover, with your kept values in place of the published values. A reader can search by title,
|
||||
author, series and tag. A download uses the stored chapters, the current ePub rules and the download
|
||||
template for the file name.
|
||||
|
||||
`AUTH_METHOD` sets how a reader logs in:
|
||||
|
||||
- `none`: The reader needs no login.
|
||||
- `local`: The reader sends your Bindery user name and password with HTTP Basic authentication.
|
||||
A browser with a Bindery session works too.
|
||||
- `cwa`: The reader sends your CWA user name and password with HTTP Basic authentication.
|
||||
A browser with a Bindery session works too.
|
||||
- `proxy`: Each user sets an OPDS password under Settings > Book Downloads. Settings shows the user
|
||||
name for the reader, which is the name in `PROXY_AUTH_USER_HEADER`. The reader sends that name and
|
||||
password with HTTP Basic authentication. The password has 8 to 256 characters, and Bindery keeps a
|
||||
hash of it. Set the proxy to skip its login for the catalog. See "Paths without the proxy login".
|
||||
|
||||
Bindery keeps a good login for five minutes. Wrong passwords count toward the lockouts in
|
||||
"Lockouts and sessions". With `TRUST_PROXY=true`, each reader has its own lockout.
|
||||
Use HTTPS outside a trusted local network, because HTTP Basic sends the password with each request.
|
||||
|
||||
### FanFicFare settings
|
||||
|
||||
On first start, Bindery copies `data/personal.ini` to `/config/personal.ini`. It uses the format of
|
||||
@@ -371,105 +317,6 @@ To serve Bindery under a path, such as `/bindery/`, do one of these:
|
||||
|
||||
The health check at `/api/health` works with and without the prefix.
|
||||
|
||||
## How books reach the library
|
||||
|
||||
Bindery gives every book to CWA through the ingest folder. CWA makes all changes to the library.
|
||||
|
||||
**Bindery does not make the ingest folder.** If the folder in `INGEST_DIR` does not exist, most often
|
||||
because its volume mount is missing, the log shows an error at the start and each upload fails. The
|
||||
books go to CWA with the next update after you fix the mount.
|
||||
|
||||
1. Bindery puts the ePub of a new story into the ingest folder. CWA imports it as a new book.
|
||||
2. At the next update, Bindery finds the CWA book of the story. If you set `CALIBRE_DB_PATH`, it
|
||||
first looks for the story URL in the library database. Without a single match there, it searches
|
||||
the CWA OPDS feed (using `CWA_URL`, `CWA_USERNAME` and `CWA_PASSWORD`) for the story title. The
|
||||
search ignores the parts of a title in parentheses or brackets, such as tags or "(Book 1 Complete)".
|
||||
Bindery downloads up to 5 ePubs with a matching title, closest title first. The book whose ePub
|
||||
holds the story URL is the match. Bindery saves the id and uuid of the book with the story.
|
||||
3. For the update, Bindery puts two files into the ingest folder: the new ePub, and a sidecar file
|
||||
`<name>.epub.cwa.json` that holds `{"action": "add_format", "book_id": N}`. CWA then replaces the
|
||||
ePub of book N.
|
||||
|
||||
Before each later update, Bindery finds the book by its uuid, first in the library database if you
|
||||
set `CALIBRE_DB_PATH`, then through CWA. If you deleted the book in CWA, Bindery unlinks the story.
|
||||
|
||||
`CALIBRE_DB_PATH` is optional. It is the path of the calibre `metadata.db` inside the container, and
|
||||
it lets Bindery find a book by its story URL instead of its title. To use it, mount the library
|
||||
folder read-only into the container. Mount the whole folder, so that SQLite can read the journal
|
||||
files beside the database. The Compose file has an example. **Bindery does not write to this database.**
|
||||
|
||||
Be sure to use the library that gets the ingest files. A different library can make Bindery update the
|
||||
wrong book. Database lookups also find books that the CWA account of Bindery cannot see. If Bindery
|
||||
cannot read the database, it uses CWA for every lookup. Keep `CWA_URL` and the CWA account set,
|
||||
because library imports and CWA lookups use them.
|
||||
|
||||
A book that the FanFicFare calibre plugin made holds its story URL. If you add its URL, or import it
|
||||
from the library, Bindery links that book and imports its chapters from the ePub.
|
||||
|
||||
### If Bindery cannot find the book
|
||||
|
||||
If library matching is on and configured, an update fails when Bindery cannot find the book.
|
||||
The story page shows the error "Cannot find this story in CWA. Link to library book, or send as new book." The
|
||||
chapters stay stored, and the next successful update includes them. This happens if:
|
||||
|
||||
- CWA did not import the ePub.
|
||||
- The title of the book changed in CWA before Bindery linked it.
|
||||
- The CWA account of Bindery cannot see the book.
|
||||
- You deleted the book in CWA.
|
||||
|
||||
To fix it, go to "Library link" on the story page, and do one of these:
|
||||
|
||||
- Search for the book, and click "Link". Then click "Rebuild book" to send the update.
|
||||
- Click "Send to CWA as a new book". Use this if CWA did not import the first version, or if you
|
||||
deleted the book.
|
||||
|
||||
### Upload settings
|
||||
|
||||
The "Uploads to CWA" section controls library matching and uploads. Bindery still downloads chapters
|
||||
and builds books when uploads stop. The story list marks a built book with "held" only when it waits
|
||||
for approval, library linking is on, and uploads are not paused. Bindery keeps track of all unsent books.
|
||||
|
||||
- Link books to CWA library. This defaults to on when `CWA_URL` or `CALIBRE_DB_PATH` is set.
|
||||
Without either value, it stays off and disabled. Turn it off to skip matching with a configured library connection.
|
||||
Uploads use saved CWA book IDs when available. Without an ID, every build goes to `INGEST_DIR` as a plain ePub.
|
||||
If CWA imports these files, updates will create duplicate books. Settings and the story page show a highlighted warning.
|
||||
A change of this setting sends the unsent books, unless uploads are paused.
|
||||
- **Hold new links for approval.** This setting applies only when library linking is on.
|
||||
Use this to catch a story that is linked to the wrong CWA book.
|
||||
The first upload of a new story waits for your approval. The first upload after a change of the
|
||||
library link also waits. This includes a link that you make, and a book that Bindery finds after
|
||||
the linked book is gone. The story page names the book that gets the update, or says that the
|
||||
story goes to CWA as a new book.
|
||||
- **Pause all uploads.** No book goes to CWA. This is always checked when `INGEST_DIR` is unset.
|
||||
When you turn the pause off, Bindery sends the unsent books. The other upload settings still apply.
|
||||
While paused, use "Approve" or "Approve as a new book instead" to save a decision without sending the book.
|
||||
|
||||
To approve a link, open the story page and do one of these:
|
||||
|
||||
- If the book is right, click "Approve and send".
|
||||
- If the book is wrong, search for the right book under "Library link", and click "Link". Then
|
||||
approve the new link.
|
||||
- If CWA does not have the story, click "Send as a new book instead".
|
||||
|
||||
The book that CWA imports from the first upload of a story needs no approval. After an approval,
|
||||
later updates go to CWA as usual.
|
||||
|
||||
### The CWA account
|
||||
|
||||
Make a CWA user for Bindery with these settings:
|
||||
|
||||
- The download permission. Bindery reads story URLs and chapters from the ePubs.
|
||||
- "Show Publisher Section", in the sidebar settings of the user. CWA shows its list of publishers in
|
||||
OPDS only to users with this setting. "Import from the library" uses the list to find the books of
|
||||
publishing sites, because FanFicFare sets the publisher of a book to the site name.
|
||||
- Access to all books. Bindery finds only the books that this account can see.
|
||||
|
||||
Bindery sends the password of this account with each OPDS request. **Use a password that no other
|
||||
service uses.**
|
||||
|
||||
If CWA allows anonymous browsing, Bindery needs no account. Leave `CWA_USERNAME` and `CWA_PASSWORD`
|
||||
unset, and give the guest user of CWA the settings above.
|
||||
|
||||
## Import an ePub file
|
||||
|
||||
If the publishing site removed chapters before Bindery tracked the story, import an ePub that you made earlier
|
||||
@@ -485,7 +332,7 @@ publishes later go after them.
|
||||
that FanFicFare writes into each ePub. A chapter that Bindery stored already keeps its stored text.
|
||||
The other chapters go in the order of the ePub.
|
||||
|
||||
After the import, Bindery checks the publishing site for new chapters and sends the book to CWA. If you clear
|
||||
After the import, Bindery checks the publishing site for new chapters and builds the book. If you clear
|
||||
"Check the site for new chapters after import", Bindery only rebuilds the book of a story that it
|
||||
downloaded before.
|
||||
|
||||
@@ -522,7 +369,7 @@ Bindery field. Calibre does not have it, and gives empty text for it.
|
||||
| `{pubdate}` | The published date. |
|
||||
| `{timestamp}` | The date when Bindery added the story. |
|
||||
| `{last_modified}` | The date when the chapters last changed. |
|
||||
| `{id}` | The CWA book ID. For a story with no library book, and without CWA, the Bindery story ID. |
|
||||
| `{id}` | The Bindery story ID. With CWA on, the CWA book ID of a story that has a library book. |
|
||||
|
||||
- A field with no value becomes empty text.
|
||||
- `{field:format|text before|text after}` adds the two texts only when the field has a value. Each
|
||||
@@ -558,11 +405,12 @@ does not change the names of the files in `INGEST_DIR`. The browser can ignore t
|
||||
| Recheck all chapters | Downloads every chapter on the publishing site again, and new chapters too. |
|
||||
| Download and replace entire book | Downloads every chapter again. Chapters that the publishing site no longer lists leave the book. They stay stored, and you can put them back. |
|
||||
| Update metadata from site | Updates only the metadata in the ePub. |
|
||||
| Rebuild book | Builds the book again from the stored chapters, and sends it. |
|
||||
| Rebuild book | Builds the book again from the stored chapters. |
|
||||
|
||||
**Bindery builds and sends a book only if the book changed.** A change is a new, changed, moved or
|
||||
renamed chapter, a changed rule, or changed metadata. A job that finds no change answers "No changes"
|
||||
and sends nothing to CWA. Only "Rebuild book" builds and sends a book that did not change.
|
||||
**Bindery builds a book only if the book changed.** A change is a new, changed, moved or renamed
|
||||
chapter, a changed rule, or changed metadata. A job that finds no change answers "No changes". Only
|
||||
"Rebuild book" builds a book that did not change. With CWA on, Bindery sends each book that it
|
||||
builds to CWA.
|
||||
|
||||
### Find edited chapters
|
||||
|
||||
@@ -646,9 +494,36 @@ CWA before, it rebuilds the book after an edit.
|
||||
`DEFAULT_LOCKED_FIELDS` sets the locks for new stories. The default is `title`. The publisher is
|
||||
always the site name.
|
||||
|
||||
**CWA reads the metadata and the cover of an ePub only when it imports a new book.** An update
|
||||
replaces only the ePub file. So edits that you make in CWA stay, and later changes on the site show
|
||||
in the ePub but not in the CWA catalog.
|
||||
## Page count
|
||||
|
||||
The story list shows an estimated page count beside the chapter count. The OPDS catalog shows it in the
|
||||
description of a book. Bindery counts the pages as the last step when it builds an ePub, so the count
|
||||
includes your replacement rules and the line-break setting. A story shows no page count until Bindery
|
||||
builds its book.
|
||||
|
||||
The method comes from "Paragraphs (APNX Accurate)" in the
|
||||
[Count Pages plugin](https://github.com/kiwidude68/calibre_plugins/wiki/Count-Pages) for calibre:
|
||||
|
||||
- A paragraph start is one line. Each 70 characters of text is one more line.
|
||||
- A page is 31 lines.
|
||||
- A page is at most 2400 characters. This limit sets the count of a book with almost no paragraphs.
|
||||
|
||||
The plugin counts either the `<p>` elements or the `<div>` elements of a book as its paragraphs. Bindery
|
||||
counts each paragraph that a reader sees:
|
||||
|
||||
- Text starts a paragraph after a `<br>`, and after the start or the end of a block element such as `<p>`,
|
||||
`<div>`, `<li>` or a heading.
|
||||
- A run of `<br>` elements is one paragraph break.
|
||||
- An element that only holds other blocks is not a paragraph.
|
||||
- The cells of a table row share one line.
|
||||
- A paragraph with no text is not counted. This includes a paragraph that holds only a no-break space,
|
||||
which some stories put between paragraphs as a blank line.
|
||||
|
||||
So the same text gives the same count as `<p>` elements, as `<div>` elements, and as lines with `<br>`
|
||||
between them. For a book that uses `<p>` elements and has no empty paragraphs, the count is within about 1%
|
||||
of the count of the plugin. The plugin counts each empty paragraph as a line, so its count is higher for a
|
||||
book that has many of them.
|
||||
The count is an estimate and differs from the page count of a printed book.
|
||||
|
||||
## Replacement rules
|
||||
|
||||
@@ -671,6 +546,184 @@ stays. This step runs before the global rules. The global setting is off by defa
|
||||
each story that has no setting of its own. To see the new paragraphs in the preview, select "Compare
|
||||
HTML source".
|
||||
|
||||
## Calibre-Web-Automated
|
||||
|
||||
Bindery can send each book that it builds to a [Calibre-Web-Automated](https://github.com/crocodilestick/Calibre-Web-Automated) (CWA) library, and update that book when the
|
||||
story changes. CWA is off by default. To turn it on:
|
||||
|
||||
1. Set `CWA_ENABLED=true`.
|
||||
2. Set `INGEST_DIR` to the CWA ingest folder. Set `CWA_URL`, `CALIBRE_DB_PATH` or both.
|
||||
3. Restart Bindery.
|
||||
|
||||
Use CWA v4.0.1 or later. Older versions can import an update as a new book. See "Limitations".
|
||||
|
||||
An earlier Bindery used CWA unless `CWA_ENABLED` was `false`. After an upgrade, set `CWA_ENABLED=true`
|
||||
to keep the uploads to CWA.
|
||||
|
||||
| Variable | Description | Default value |
|
||||
| --- | --- | --- |
|
||||
| `CWA_ENABLED` | `true` to use CWA | `false` |
|
||||
| `CWA_URL` | The address of CWA, for example `http://calibre-web-automated:8083` | None |
|
||||
| `CWA_USERNAME`, `CWA_PASSWORD` | The CWA account of Bindery. Leave them unset if CWA asks for no login. | None |
|
||||
| `CALIBRE_DB_PATH` | The calibre `metadata.db`, for book ids. See "How books reach the library". | None |
|
||||
| `CWA_DB_PATH` | Calibre-Web's `app.db`, for logins. See "CWA logins". | None |
|
||||
| `CWA_ALLOWED_USERS` | The CWA users that can log in. Empty allows every CWA user. | Empty |
|
||||
|
||||
The example Compose file has these values as comments. The user of the container must be able to write
|
||||
to the CWA ingest folder. Usually this is the same user and group as CWA.
|
||||
|
||||
Settings shows an alert for each missing value:
|
||||
|
||||
- Neither `CWA_URL` nor `CALIBRE_DB_PATH` is set. The alert is under "Uploads to CWA". Library linking
|
||||
is then off, and its checkbox is disabled. Set either value and restart Bindery. Bindery keeps your
|
||||
saved linking preference.
|
||||
- `INGEST_DIR` is not set. The alert is under "Book Downloads". "Pause all uploads" is then checked
|
||||
until you set `INGEST_DIR` and restart Bindery.
|
||||
|
||||
At each start, the log has a warning for each missing value.
|
||||
|
||||
### Without CWA
|
||||
|
||||
Without `CWA_ENABLED=true`:
|
||||
|
||||
- Bindery makes no request to CWA. It ignores `CWA_URL`, `CWA_USERNAME`, `CWA_PASSWORD`,
|
||||
`CWA_DB_PATH` and `CALIBRE_DB_PATH`. If one of them is set, the log has a warning at each start.
|
||||
- The web interface shows nothing about CWA. The pause, the library links and the approvals do not
|
||||
apply.
|
||||
- If `INGEST_DIR` is set, Bindery puts each changed book in that folder as a plain ePub. Use this to
|
||||
import the books into other software. Each file gets a new name, so the earlier files stay in the
|
||||
folder. If `INGEST_DIR` is not set, Settings shows an alert under "Book Downloads".
|
||||
- `AUTH_METHOD=cwa` does not work. Use `local` or `proxy`. On a private network, you can use `none`.
|
||||
|
||||
### How books reach the library
|
||||
|
||||
Bindery gives every book to CWA through the ingest folder. CWA makes all changes to the library.
|
||||
|
||||
**Bindery does not make the ingest folder.** If the folder in `INGEST_DIR` does not exist, most often
|
||||
because its volume mount is missing, the log shows an error at the start and each upload fails. The
|
||||
books go to CWA with the next update after you fix the mount.
|
||||
|
||||
1. Bindery puts the ePub of a new story into the ingest folder. CWA imports it as a new book.
|
||||
2. At the next update, Bindery finds the CWA book of the story. If you set `CALIBRE_DB_PATH`, it
|
||||
first looks for the story URL in the library database. Without a single match there, it searches
|
||||
the CWA OPDS feed (using `CWA_URL`, `CWA_USERNAME` and `CWA_PASSWORD`) for the story title. The
|
||||
search ignores the parts of a title in parentheses or brackets, such as tags or "(Book 1 Complete)".
|
||||
Bindery downloads up to 5 ePubs with a matching title, closest title first. The book whose ePub
|
||||
holds the story URL is the match. Bindery saves the id and uuid of the book with the story.
|
||||
3. For the update, Bindery puts two files into the ingest folder: the new ePub, and a sidecar file
|
||||
`<name>.epub.cwa.json` that holds `{"action": "add_format", "book_id": N}`. CWA then replaces the
|
||||
ePub of book N.
|
||||
|
||||
Before each later update, Bindery finds the book by its uuid, first in the library database if you
|
||||
set `CALIBRE_DB_PATH`, then through CWA. If you deleted the book in CWA, Bindery unlinks the story.
|
||||
|
||||
`CALIBRE_DB_PATH` is optional. It is the path of the calibre `metadata.db` inside the container, and
|
||||
it lets Bindery find a book by its story URL instead of its title. To use it, mount the library
|
||||
folder read-only into the container. Mount the whole folder, so that SQLite can read the journal
|
||||
files beside the database. The Compose file has an example. **Bindery does not write to this database.**
|
||||
|
||||
Be sure to use the library that gets the ingest files. A different library can make Bindery update the
|
||||
wrong book. Database lookups also find books that the CWA account of Bindery cannot see. If Bindery
|
||||
cannot read the database, it uses CWA for every lookup. Keep `CWA_URL` and the CWA account set,
|
||||
because library imports and CWA lookups use them.
|
||||
|
||||
A book that the FanFicFare calibre plugin made holds its story URL. If you add its URL, or import it
|
||||
from the library, Bindery links that book and imports its chapters from the ePub.
|
||||
|
||||
**CWA reads the metadata and the cover of an ePub only when it imports a new book.** An update
|
||||
replaces only the ePub file. So edits that you make in CWA stay, and later changes on the site show
|
||||
in the ePub but not in the CWA catalog.
|
||||
|
||||
### If Bindery cannot find the book
|
||||
|
||||
If library matching is on and configured, an update fails when Bindery cannot find the book.
|
||||
The story page shows the error "Cannot find this story in CWA. Link to library book, or send as new book." The
|
||||
chapters stay stored, and the next successful update includes them. This happens if:
|
||||
|
||||
- CWA did not import the ePub.
|
||||
- The title of the book changed in CWA before Bindery linked it.
|
||||
- The CWA account of Bindery cannot see the book.
|
||||
- You deleted the book in CWA.
|
||||
|
||||
To fix it, go to "Library link" on the story page, and do one of these:
|
||||
|
||||
- Search for the book, and click "Link". Then click "Rebuild book" to send the update.
|
||||
- Click "Send to CWA as a new book". Use this if CWA did not import the first version, or if you
|
||||
deleted the book.
|
||||
|
||||
### Upload settings
|
||||
|
||||
The "Uploads to CWA" section of Settings controls library matching and uploads. Bindery still downloads chapters
|
||||
and builds books when uploads stop. The story list marks a built book with "held" only when it waits
|
||||
for approval, library linking is on, and uploads are not paused. Bindery keeps track of all unsent books.
|
||||
|
||||
- Link books to CWA library. This defaults to on when `CWA_URL` or `CALIBRE_DB_PATH` is set.
|
||||
Without either value, it stays off and disabled. Turn it off to skip matching with a configured library connection.
|
||||
Uploads use saved CWA book IDs when available. Without an ID, every build goes to `INGEST_DIR` as a plain ePub.
|
||||
If CWA imports these files, updates will create duplicate books. Settings and the story page show a highlighted warning.
|
||||
A change of this setting sends the unsent books, unless uploads are paused.
|
||||
- **Hold new links for approval.** This setting applies only when library linking is on.
|
||||
Use this to catch a story that is linked to the wrong CWA book.
|
||||
The first upload of a new story waits for your approval. The first upload after a change of the
|
||||
library link also waits. This includes a link that you make, and a book that Bindery finds after
|
||||
the linked book is gone. The story page names the book that gets the update, or says that the
|
||||
story goes to CWA as a new book.
|
||||
- **Pause all uploads.** No book goes to CWA. This is always checked when `INGEST_DIR` is unset.
|
||||
When you turn the pause off, Bindery sends the unsent books. The other upload settings still apply.
|
||||
While paused, use "Approve" or "Approve as a new book instead" to save a decision without sending the book.
|
||||
|
||||
To approve a link, open the story page and do one of these:
|
||||
|
||||
- If the book is right, click "Approve and send".
|
||||
- If the book is wrong, search for the right book under "Library link", and click "Link". Then
|
||||
approve the new link.
|
||||
- If CWA does not have the story, click "Send as a new book instead".
|
||||
|
||||
The book that CWA imports from the first upload of a story needs no approval. After an approval,
|
||||
later updates go to CWA as usual.
|
||||
|
||||
### The CWA account
|
||||
|
||||
Make a CWA user for Bindery with these settings:
|
||||
|
||||
- The download permission. Bindery reads story URLs and chapters from the ePubs.
|
||||
- "Show Publisher Section", in the sidebar settings of the user. CWA shows its list of publishers in
|
||||
OPDS only to users with this setting. "Import from the library" uses the list to find the books of
|
||||
publishing sites, because FanFicFare sets the publisher of a book to the site name.
|
||||
- Access to all books. Bindery finds only the books that this account can see.
|
||||
|
||||
Bindery sends the password of this account with each OPDS request. **Use a password that no other
|
||||
service uses.**
|
||||
|
||||
If CWA allows anonymous browsing, Bindery needs no account. Leave `CWA_USERNAME` and `CWA_PASSWORD`
|
||||
unset, and give the guest user of CWA the settings above.
|
||||
|
||||
### CWA logins
|
||||
|
||||
`AUTH_METHOD=cwa` needs `CWA_ENABLED=true`, and `CWA_URL`, `CWA_DB_PATH` or both. Without `CWA_DB_PATH`, Bindery sends the
|
||||
user name and password to `CWA_URL/opds` as an HTTP Basic login. This is the OPDS login.
|
||||
|
||||
**The OPDS login works only if CWA refuses a wrong password.** A CWA with anonymous browsing can
|
||||
accept every OPDS login. Bindery tests this with a login that cannot exist. If CWA accepts it,
|
||||
Bindery refuses each OPDS login, and the log names the reason. Set `CWA_DB_PATH` then, or use
|
||||
another `AUTH_METHOD`.
|
||||
|
||||
With `CWA_DB_PATH`, Bindery checks the password against the account in Calibre-Web's `app.db`. A
|
||||
wrong password or an unknown account fails at once. If Bindery cannot read the file or its password
|
||||
hash, it uses the OPDS login.
|
||||
|
||||
`CWA_DB_PATH` is the path of `app.db` inside the container. To use it, mount the CWA config folder
|
||||
read-only into the container. Mount the whole folder, so that SQLite can read the journal files
|
||||
beside the database. **Bindery does not write this database.**
|
||||
|
||||
**Database logins check only the password.** CWA's external login providers and multi-factor
|
||||
authentication do not apply. If they must apply, use the `proxy` mode.
|
||||
|
||||
- The CWA guest user cannot log in.
|
||||
- If `CWA_ALLOWED_USERS` is set, only those users can log in. Bindery checks this list on each
|
||||
request.
|
||||
- `CWA_USERNAME` and `CWA_PASSWORD` are only for the library lookups of Bindery.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Bindery identifies a chapter by its URL.** Most sites, such as RoyalRoad and AO3, keep the URL
|
||||
@@ -679,8 +732,8 @@ HTML source".
|
||||
the same, run "Recheck all chapters". When an author removes a chapter there, the book loses that
|
||||
chapter, and the last chapter appears twice. The removed chapter stays as an older revision of the
|
||||
chapter that took its URL.
|
||||
- "Stop tracking" deletes only the data of Bindery. The book stays in CWA.
|
||||
- **Updates need CWA v4.0.1 or later.** CWA v4.0.0 added the `.cwa.json` sidecar files. But v4.0.0
|
||||
- "Stop tracking" deletes only the data of Bindery. A book in CWA stays there.
|
||||
- **Updates of a CWA library need CWA v4.0.1 or later.** CWA v4.0.0 added the `.cwa.json` sidecar files. But v4.0.0
|
||||
can delete a sidecar file before it imports the ePub, and then imports the update as a new book.
|
||||
Versions before v4.0.0 cannot read sidecar files, so each update becomes a new book.
|
||||
|
||||
@@ -703,14 +756,14 @@ The Python tests use a fake CWA server and temporary folders, so they need no CW
|
||||
sites. The frontend tests use the built-in test runner of Node, and need no npm packages. Also check
|
||||
the pages in a browser.
|
||||
|
||||
### Run locally without CWA
|
||||
### Run locally
|
||||
|
||||
Without CWA, Bindery puts each new build of a story into its ingest folder as an ePub. This is
|
||||
enough to work on downloads, chapters, rules and the web interface.
|
||||
CWA is off by default. Bindery puts each new build of a story into its ingest folder as an
|
||||
ePub. This is enough to work on downloads, chapters, rules and the web interface.
|
||||
|
||||
```bash
|
||||
mkdir -p dev/ingest
|
||||
CWA_ENABLED=false CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp uv run python -m bindery
|
||||
CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp uv run python -m bindery
|
||||
```
|
||||
|
||||
Open `http://127.0.0.1:8085`. The folders go in `dev/`, which git ignores. To look at an ePub, use
|
||||
@@ -741,7 +794,7 @@ not use your real library.**
|
||||
test logins too.
|
||||
|
||||
```bash
|
||||
CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp \
|
||||
CWA_ENABLED=true CONFIG_DIR=dev/config INGEST_DIR=dev/ingest TMP_DIR=dev/tmp \
|
||||
CWA_URL=http://localhost:8083 CWA_USERNAME=admin CWA_PASSWORD=admin123 AUTH_METHOD=cwa \
|
||||
uv run python -m bindery
|
||||
```
|
||||
|
||||
+1
-1
@@ -123,7 +123,7 @@ def config_error() -> str | None:
|
||||
if not _valid_local_hash(config.LOCAL_PASSWORD_HASH):
|
||||
return "BINDERY_PASSWORD_HASH must be a Werkzeug scrypt or PBKDF2 hash of a nonempty password"
|
||||
if method == AUTH_CWA and not config.CWA_ENABLED:
|
||||
return "AUTH_METHOD=cwa needs CWA, but CWA_ENABLED is false"
|
||||
return "AUTH_METHOD=cwa needs CWA_ENABLED=true"
|
||||
if method == AUTH_CWA and not (config.CWA_URL or config.CWA_DB_PATH):
|
||||
return "AUTH_METHOD=cwa needs CWA_URL or CWA_DB_PATH"
|
||||
if method == AUTH_PROXY:
|
||||
|
||||
+8
-5
@@ -74,9 +74,9 @@ CWA_PASSWORD = os.getenv("CWA_PASSWORD", "")
|
||||
# Independent, optional read-only databases. HTTP remains the fallback.
|
||||
CWA_DB_PATH = _optional_path("CWA_DB_PATH") # Calibre-Web app.db, for logins
|
||||
CALIBRE_DB_PATH = _optional_path("CALIBRE_DB_PATH") # calibre metadata.db, for book IDs
|
||||
# false if the user has no CWA. Bindery then ignores the CWA settings above, and shows nothing about CWA. INGEST_DIR
|
||||
# still gets each built book, as a plain epub. Only a clear "false" turns CWA off.
|
||||
CWA_ENABLED = _bool("CWA_ENABLED", True)
|
||||
# true if the user has CWA. Without it, Bindery ignores the CWA settings above, and shows nothing about CWA. INGEST_DIR
|
||||
# still gets each built book, as a plain epub. Only a clear "true" turns CWA on.
|
||||
CWA_ENABLED = _bool("CWA_ENABLED")
|
||||
CWA_SETTINGS = ("CWA_URL", "CWA_USERNAME", "CWA_PASSWORD", "CWA_DB_PATH", "CALIBRE_DB_PATH")
|
||||
IGNORED_SETTINGS = [] if CWA_ENABLED else [name for name in CWA_SETTINGS if os.getenv(name, "").strip()]
|
||||
if not CWA_ENABLED:
|
||||
@@ -141,7 +141,10 @@ def prepare() -> None:
|
||||
logger.info(f"Created the default FanFicFare configuration at {PERSONAL_INI}")
|
||||
if not CWA_ENABLED:
|
||||
if IGNORED_SETTINGS:
|
||||
logger.info(f"CWA_ENABLED is false. Bindery ignores {', '.join(IGNORED_SETTINGS)}.")
|
||||
logger.warning(
|
||||
f"CWA_ENABLED is not true. Bindery ignores {', '.join(IGNORED_SETTINGS)}. To use CWA, set "
|
||||
"CWA_ENABLED=true."
|
||||
)
|
||||
return
|
||||
if CWA_PASSWORD and not CWA_USERNAME:
|
||||
logger.warning("CWA_PASSWORD is set, but CWA_USERNAME is not. Bindery sends no login to CWA.")
|
||||
@@ -149,7 +152,7 @@ def prepare() -> None:
|
||||
for warning in missing:
|
||||
logger.warning(warning)
|
||||
if missing:
|
||||
logger.warning("If you do not use CWA, set CWA_ENABLED=false. Bindery then shows no messages about CWA.")
|
||||
logger.warning("If you do not use CWA, remove CWA_ENABLED. Bindery then shows no messages about CWA.")
|
||||
|
||||
|
||||
def _missing_cwa() -> list[str]:
|
||||
|
||||
+27
-5
@@ -1,9 +1,11 @@
|
||||
"""Read the OPF metadata of an epub that came from CWA."""
|
||||
|
||||
import posixpath
|
||||
import re
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib.parse import unquote
|
||||
from xml.etree.ElementTree import Element
|
||||
|
||||
from defusedxml import ElementTree
|
||||
@@ -17,14 +19,19 @@ class EpubError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _opf(path: Path, z: zipfile.ZipFile) -> tuple[str, Element]:
|
||||
"""The name of the OPF file in the epub, and its content."""
|
||||
container = ElementTree.fromstring(z.read("META-INF/container.xml"))
|
||||
rootfile = container.find(f".//{CONTAINER}rootfile")
|
||||
if rootfile is None or not rootfile.get("full-path"):
|
||||
raise EpubError(f"{path.name} has no OPF file")
|
||||
return rootfile.get("full-path"), ElementTree.fromstring(z.read(rootfile.get("full-path")))
|
||||
|
||||
|
||||
def _metadata(path: Path) -> Element:
|
||||
try:
|
||||
with zipfile.ZipFile(path) as z:
|
||||
container = ElementTree.fromstring(z.read("META-INF/container.xml"))
|
||||
rootfile = container.find(f".//{CONTAINER}rootfile")
|
||||
if rootfile is None or not rootfile.get("full-path"):
|
||||
raise EpubError(f"{path.name} has no OPF file")
|
||||
opf = ElementTree.fromstring(z.read(rootfile.get("full-path")))
|
||||
opf = _opf(path, z)[1]
|
||||
except (zipfile.BadZipFile, KeyError, ElementTree.ParseError) as e:
|
||||
raise EpubError(f"{path.name} is not a readable epub: {e}") from e
|
||||
metadata = opf.find(f"{OPF}metadata")
|
||||
@@ -33,6 +40,21 @@ def _metadata(path: Path) -> Element:
|
||||
return metadata
|
||||
|
||||
|
||||
def spine_documents(path: Path) -> list[str]:
|
||||
"""The text of each document in the spine of an epub, in reading order."""
|
||||
try:
|
||||
with zipfile.ZipFile(path) as z:
|
||||
name, opf = _opf(path, z)
|
||||
hrefs = {item.get("id"): item.get("href") for item in opf.iter(f"{OPF}item")}
|
||||
files = [
|
||||
posixpath.join(posixpath.dirname(name), unquote(hrefs[ref.get("idref")]))
|
||||
for ref in opf.iter(f"{OPF}itemref")
|
||||
]
|
||||
return [z.read(file).decode("utf-8", "replace") for file in files]
|
||||
except (zipfile.BadZipFile, KeyError, ElementTree.ParseError) as e:
|
||||
raise EpubError(f"{path.name} is not a readable epub: {e}") from e
|
||||
|
||||
|
||||
def epub_urls(path: Path) -> list[str]:
|
||||
"""Every URL in the identifiers and the source of an epub's OPF.
|
||||
|
||||
|
||||
@@ -77,6 +77,7 @@ def _languages(value: Any) -> list[str]:
|
||||
def _books() -> list[dict[str, Any]]:
|
||||
"""The books of the catalog, in title order."""
|
||||
books = []
|
||||
page_counts = orchestrator.store.story_settings("page_count")
|
||||
for story in orchestrator.store.catalog():
|
||||
meta = orchestrator.book_metadata(story)
|
||||
title = (meta["title"] or "").strip() or story["url"]
|
||||
@@ -89,6 +90,7 @@ def _books() -> list[dict[str, Any]]:
|
||||
"title": title,
|
||||
"authors": authors,
|
||||
"updated": _updated(story),
|
||||
"page_count": page_counts.get(story["id"]),
|
||||
"cover": hashlib.sha256(cover.encode()).hexdigest()[:16] if cover else None,
|
||||
}
|
||||
)
|
||||
@@ -107,6 +109,8 @@ def _content(book: dict[str, Any]) -> str:
|
||||
index = filename_template.series_index(meta.get("series_index"))
|
||||
lines.append(f"Series: {meta['series']}" + (f", book {index}" if index else ""))
|
||||
lines.append(f"Chapters: {book['chapter_count']}")
|
||||
if book["page_count"] is not None:
|
||||
lines.append(f"Pages: {book['page_count']}")
|
||||
description = "\n\n".join(diffing.html_to_blocks(meta.get("comments") or ""))
|
||||
return "\n".join(lines) + (f"\n\n{description}" if description else "")
|
||||
|
||||
|
||||
+19
-8
@@ -13,7 +13,7 @@ from typing import Any, BinaryIO
|
||||
|
||||
from werkzeug.security import generate_password_hash
|
||||
|
||||
from bindery import config, fff, filename_template
|
||||
from bindery import config, fff, filename_template, pages
|
||||
from bindery.backup import daily_backup
|
||||
from bindery.cwa import Book, BookFinder, CwaClient, CwaError
|
||||
from bindery.diffing import diff_html, text_hash
|
||||
@@ -226,9 +226,11 @@ def list_stories() -> list[dict[str, Any]]:
|
||||
stories = store.list_stories()
|
||||
held = store.story_settings("upload_held")
|
||||
unapproved = store.story_settings("unapproved_link")
|
||||
page_counts = store.story_settings("page_count")
|
||||
show_holds = _link_books() and store.get_setting(None, "hold_new_links") and _can_upload()
|
||||
for s in stories:
|
||||
s["busy"] = job_queue.has_active(s["id"])
|
||||
s["page_count"] = page_counts.get(s["id"])
|
||||
s["upload_held"] = bool(held.get(s["id"]))
|
||||
s["approval_held"] = bool(
|
||||
show_holds
|
||||
@@ -833,6 +835,14 @@ def _build_key(story: dict[str, Any], overrides: dict[str, Any], site: dict[str,
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _count_pages(story_id: int, path: Path) -> None:
|
||||
"""Store the estimated pages of a built epub, for the story list and the OPDS catalog."""
|
||||
try:
|
||||
store.set_setting(story_id, "page_count", pages.epub_pages(path))
|
||||
except EpubError as e:
|
||||
logger.warning(f"Cannot count the pages of story {story_id}: {e}")
|
||||
|
||||
|
||||
def _built_epub(
|
||||
story: dict[str, Any], overrides: dict[str, Any], key: str, force: bool = False, record: bool = True
|
||||
) -> Path:
|
||||
@@ -842,7 +852,8 @@ def _built_epub(
|
||||
"""
|
||||
path = config.TMP_DIR / f"story-{story['id']}-{key[:16]}.epub"
|
||||
with _build_lock:
|
||||
if force or not path.exists():
|
||||
built = force or not path.exists()
|
||||
if built:
|
||||
# A build under another name first, so a download never reads half a book.
|
||||
partial = config.TMP_DIR / f"building-{uuid.uuid4().hex}.epub"
|
||||
try:
|
||||
@@ -853,6 +864,9 @@ def _built_epub(
|
||||
for old in config.TMP_DIR.glob(f"story-{story['id']}-*.epub"):
|
||||
if old != path:
|
||||
old.unlink(missing_ok=True)
|
||||
# A book from before the page count has no count yet.
|
||||
if built or store.get_setting(story["id"], "page_count") is None:
|
||||
_count_pages(story["id"], path)
|
||||
if record:
|
||||
store.set_setting(story["id"], "built_key", key)
|
||||
return path
|
||||
@@ -1182,9 +1196,8 @@ def _run_rebuild(job: Job) -> str:
|
||||
force = bool(job.options.get("force"))
|
||||
if not force and key == _done_key(story["id"]):
|
||||
return "No changes"
|
||||
message, changed = _send(story, overrides, key, story["title"], story["author"], force)
|
||||
changes: dict[str, Any] = {"last_changed": now()} if changed or force else {}
|
||||
store.update_story(story["id"], pinned=pinned, status="ok", message=message, **changes)
|
||||
message, _ = _send(story, overrides, key, story["title"], story["author"], force)
|
||||
store.update_story(story["id"], pinned=pinned, status="ok", message=message)
|
||||
return message
|
||||
|
||||
|
||||
@@ -1436,9 +1449,7 @@ def _run_site_job(job: Job) -> str:
|
||||
return f"No changes. {summary}"
|
||||
|
||||
_progress(job, "Building epub", 1.0)
|
||||
message, book_changed = _send(story, overrides, key, title, snap.author)
|
||||
if book_changed:
|
||||
changes["last_changed"] = now()
|
||||
message, _ = _send(story, overrides, key, title, snap.author)
|
||||
store.update_story(story_id, **changes, message=message, status="error" if failed else "ok")
|
||||
return f"{summary}. {message}"
|
||||
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
"""An estimate of the pages of a built epub, after the "Paragraphs (APNX Accurate)" method of the Count Pages
|
||||
plugin for calibre.
|
||||
|
||||
The method counts lines. A paragraph start is one line, and so is each 70 characters of text. A page is 31
|
||||
lines. A page is also at most 2400 characters, which sets the count of a book with almost no paragraphs.
|
||||
|
||||
The plugin counts either the <p> or the <div> elements of a book as its paragraphs. Bindery counts what a
|
||||
reader sees as a paragraph: text that follows a <br> or the start or end of a block element. So the same text
|
||||
gives the same count as <p> elements, as <div> elements, and as lines with <br> between them. A paragraph with
|
||||
no text, or with only a no-break space, is not counted.
|
||||
"""
|
||||
|
||||
import html as htmllib
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import NamedTuple
|
||||
|
||||
from bindery.epub import spine_documents
|
||||
|
||||
LINE_CHARS = 70
|
||||
LINES_PER_PAGE = 31
|
||||
CHARS_PER_PAGE = 2400
|
||||
# Elements that stay in the line of the text around them. Each other element, and <br>, ends a paragraph.
|
||||
# The cells of a table row share a line.
|
||||
INLINE = frozenset(
|
||||
"a abbr acronym b bdi bdo big cite code data del dfn em font i img ins kbd mark q rp rt ruby s samp small "
|
||||
"span strike strong sub sup td th time tt u var wbr".split()
|
||||
)
|
||||
# Elements whose text is not text of the book.
|
||||
HIDDEN = frozenset(("head", "script", "style"))
|
||||
_TOKEN = re.compile(r"<(/?)([a-zA-Z][a-zA-Z0-9]*)[^>]*>|<[^>]*>|([^<]+)")
|
||||
|
||||
|
||||
class Measure(NamedTuple):
|
||||
paragraphs: int = 0
|
||||
chars: int = 0
|
||||
|
||||
|
||||
def measure(html: str) -> Measure:
|
||||
"""The paragraphs and the characters of text of one document."""
|
||||
paragraphs = chars = 0
|
||||
hidden = False
|
||||
new_paragraph = True
|
||||
for match in _TOKEN.finditer(html):
|
||||
closing, name, text = match.groups()
|
||||
if name:
|
||||
name = name.lower()
|
||||
if name in HIDDEN:
|
||||
hidden = not closing and not match.group().endswith("/>")
|
||||
elif name not in INLINE:
|
||||
new_paragraph = True
|
||||
elif text and not hidden:
|
||||
if "&" in text:
|
||||
text = htmllib.unescape(text)
|
||||
# A no-break space is white space here, so a paragraph that only spaces the text is no paragraph.
|
||||
if text.strip():
|
||||
if new_paragraph:
|
||||
paragraphs += 1
|
||||
new_paragraph = False
|
||||
text = text.lstrip()
|
||||
chars += len(text)
|
||||
elif not new_paragraph:
|
||||
chars += 1 # the space between two inline elements
|
||||
return Measure(paragraphs, chars)
|
||||
|
||||
|
||||
def page_count(total: Measure) -> int:
|
||||
"""The pages of a book, from the sums of its documents. A book with no text has no pages."""
|
||||
if not total.chars:
|
||||
return 0
|
||||
lines = total.paragraphs + total.chars // LINE_CHARS
|
||||
return max(lines // LINES_PER_PAGE, total.chars // CHARS_PER_PAGE + 1)
|
||||
|
||||
|
||||
def epub_pages(path: Path) -> int:
|
||||
"""The estimated pages of an epub, from each document in its spine."""
|
||||
measures = [measure(text) for text in spine_documents(path)]
|
||||
return page_count(Measure(sum(m.paragraphs for m in measures), sum(m.chars for m in measures)))
|
||||
@@ -1,4 +1,4 @@
|
||||
// Settings: the recheck of recent chapters, the uploads to CWA, the names of downloaded books, the global rules,
|
||||
// Settings: the recheck of recent chapters, the names of downloaded books, the uploads to CWA, the global rules,
|
||||
// and FanFicFare's personal.ini.
|
||||
|
||||
import { api, button, checkbox, externalLink, fill, h, showError, toast } from './lib.js';
|
||||
@@ -39,7 +39,7 @@ export function settingsPage(main, _params, config) {
|
||||
const linkBooks = checkbox(true, showLinkState);
|
||||
const pauseUploads = checkbox(false, () => {});
|
||||
const ingestNote = h('p', { class: 'notice', id: 'ingest-note', hidden: true },
|
||||
'INGEST_DIR is not set. Uploads stay paused until you set it and restart Bindery.');
|
||||
'INGEST_DIR is not set. Bindery sends no book to the ingest folder until you set it and restart Bindery.');
|
||||
const showUploadSettings = (settings) => {
|
||||
linkBooks.disabled = settings.linking_available === false;
|
||||
linkBooks.checked = !linkBooks.disabled && (settings.link_books ?? true);
|
||||
@@ -178,41 +178,6 @@ export function settingsPage(main, _params, config) {
|
||||
),
|
||||
saveButton(() => ({ recent_chapters: recent.value === '' ? 0 : Number(recent.value) })),
|
||||
),
|
||||
config.cwa_enabled && card(
|
||||
'Uploads to CWA',
|
||||
h('label', {}, linkBooks, ' Link books to CWA library'),
|
||||
h(
|
||||
'p',
|
||||
{ class: 'hint' },
|
||||
'Find library books through CWA_URL or CALIBRE_DB_PATH. When this is off, uploads use saved CWA book IDs ' +
|
||||
'or send a plain ePub to INGEST_DIR.',
|
||||
),
|
||||
libraryNote,
|
||||
linkNote,
|
||||
h('label', {}, holdNewLinks, ' Hold new links for approval'),
|
||||
h(
|
||||
'p',
|
||||
{ class: 'hint' },
|
||||
'The first upload of a new story waits for your approval. So does the first upload after a change of the ' +
|
||||
'library link, also a link that you make. Check the linked book on the story page, and approve it there. ' +
|
||||
'Later updates of an approved book go to CWA as usual.',
|
||||
),
|
||||
h('label', {}, pauseUploads, ' Pause all uploads'),
|
||||
ingestNote,
|
||||
h(
|
||||
'p',
|
||||
{ class: 'hint' },
|
||||
'Bindery downloads chapters and builds books, but sends no book to CWA. ' +
|
||||
'When you turn the pause off, Bindery sends the books that wait.',
|
||||
),
|
||||
saveButton(
|
||||
() => ({ link_books: linkBooks.checked, hold_new_links: holdNewLinks.checked, pause_uploads: pauseUploads.checked }),
|
||||
(settings) => {
|
||||
showUploadSettings(settings);
|
||||
toast('Saved');
|
||||
},
|
||||
),
|
||||
),
|
||||
card(
|
||||
'Book Downloads',
|
||||
h('label', {}, opdsEnabled, ' Enable OPDS'),
|
||||
@@ -331,6 +296,7 @@ export function settingsPage(main, _params, config) {
|
||||
'. The default gives "May, 2020". A / in a date becomes _, as in Calibre.',
|
||||
),
|
||||
),
|
||||
ingestNote,
|
||||
saveButton(
|
||||
() => ({
|
||||
opds_enabled: opdsEnabled.checked,
|
||||
@@ -346,6 +312,40 @@ export function settingsPage(main, _params, config) {
|
||||
},
|
||||
),
|
||||
),
|
||||
config.cwa_enabled && card(
|
||||
'Uploads to CWA',
|
||||
h('label', {}, linkBooks, ' Link books to CWA library'),
|
||||
h(
|
||||
'p',
|
||||
{ class: 'hint' },
|
||||
'Find library books through CWA_URL or CALIBRE_DB_PATH. When this is off, uploads use saved CWA book IDs ' +
|
||||
'or send a plain ePub to INGEST_DIR.',
|
||||
),
|
||||
libraryNote,
|
||||
linkNote,
|
||||
h('label', {}, holdNewLinks, ' Hold new links for approval'),
|
||||
h(
|
||||
'p',
|
||||
{ class: 'hint' },
|
||||
'The first upload of a new story waits for your approval. So does the first upload after a change of the ' +
|
||||
'library link, also a link that you make. Check the linked book on the story page, and approve it there. ' +
|
||||
'Later updates of an approved book go to CWA as usual.',
|
||||
),
|
||||
h('label', {}, pauseUploads, ' Pause all uploads'),
|
||||
h(
|
||||
'p',
|
||||
{ class: 'hint' },
|
||||
'Bindery downloads chapters and builds books, but sends no book to CWA. ' +
|
||||
'When you turn the pause off, Bindery sends the books that wait.',
|
||||
),
|
||||
saveButton(
|
||||
() => ({ link_books: linkBooks.checked, hold_new_links: holdNewLinks.checked, pause_uploads: pauseUploads.checked }),
|
||||
(settings) => {
|
||||
showUploadSettings(settings);
|
||||
toast('Saved');
|
||||
},
|
||||
),
|
||||
),
|
||||
card(
|
||||
'Jobs list',
|
||||
h(
|
||||
|
||||
@@ -9,6 +9,7 @@ const STORY_COLUMNS = [
|
||||
{ label: 'Author', key: 'author', type: 'text' },
|
||||
{ label: 'Site', key: 'site', type: 'text' },
|
||||
{ label: 'Chapters', key: 'chapter_count', type: 'number' },
|
||||
{ label: 'Pages', key: 'page_count', type: 'number' },
|
||||
{ label: 'Status', key: 'status', type: 'text' },
|
||||
{ label: 'Last change', key: 'last_changed', type: 'date' },
|
||||
{ label: 'Last check', key: 'last_checked', type: 'date' },
|
||||
@@ -21,11 +22,14 @@ function libraryImport(config) {
|
||||
let candidates = [];
|
||||
let selected = new Set();
|
||||
let check = true;
|
||||
let empty = '';
|
||||
// The load of the list, kept for each search, because CWA takes a while to send the list.
|
||||
let list;
|
||||
const results = h('div');
|
||||
|
||||
const render = () => {
|
||||
if (!candidates.length) {
|
||||
fill(results, h('p', { class: 'empty' }, 'Every library book from a story site is tracked already.'));
|
||||
fill(results, h('p', { class: 'empty' }, empty));
|
||||
return;
|
||||
}
|
||||
const toggle = (id, on) => {
|
||||
@@ -81,9 +85,14 @@ function libraryImport(config) {
|
||||
const books = candidates
|
||||
.filter((c) => selected.has(c.id))
|
||||
.map(({ id, uuid, title }) => ({ id, uuid, title }));
|
||||
if (!books.length) return;
|
||||
if (!books.length) {
|
||||
toast('Select books first', true);
|
||||
return;
|
||||
}
|
||||
const jobs = await api('library/import', 'POST', { books, check });
|
||||
toast(`Queued ${jobs.length} imports`);
|
||||
const queued = new Set(books.map((b) => b.id));
|
||||
list = list?.then((all) => all.filter((c) => !queued.has(c.id)));
|
||||
fill(results);
|
||||
refreshJobs();
|
||||
},
|
||||
@@ -92,22 +101,68 @@ function libraryImport(config) {
|
||||
);
|
||||
};
|
||||
|
||||
// The full list starts with each book selected. A search starts with no book selected.
|
||||
const show = (books, all) => {
|
||||
candidates = books;
|
||||
selected = new Set(all ? books.map((c) => c.id) : []);
|
||||
empty = all
|
||||
? 'Every library book from a story site is tracked already.'
|
||||
: 'No library books from a story site match. A book that Bindery tracks already does not show.';
|
||||
render();
|
||||
};
|
||||
const query = h('input', {
|
||||
type: 'search',
|
||||
placeholder: 'Search the library by title, author or site',
|
||||
'aria-label': 'Search the library',
|
||||
});
|
||||
const load = () => {
|
||||
list ??= api('library/candidates').catch((e) => {
|
||||
list = undefined;
|
||||
throw e;
|
||||
});
|
||||
return list;
|
||||
};
|
||||
// A book matches if its title, authors and site hold each word of the search.
|
||||
const search = async () => {
|
||||
const words = query.value.toLowerCase().split(/\s+/).filter(Boolean);
|
||||
if (!words.length) return;
|
||||
const books = await load();
|
||||
show(books.filter((c) => words.every((w) => [c.title, ...c.authors, c.publisher].join(' ').toLowerCase().includes(w))), false);
|
||||
};
|
||||
const submit = h('button', { type: 'submit' }, 'Search');
|
||||
|
||||
return h(
|
||||
'div',
|
||||
{},
|
||||
h(
|
||||
'p',
|
||||
{ class: 'hint' },
|
||||
'Finds library books whose publisher is a story site, for example books from the FanFicFare calibre plugin. ' +
|
||||
'Search the library, or show all the stories in it. Both find only books whose publisher is a story site, ' +
|
||||
'for example books from the FanFicFare calibre plugin, and that Bindery does not track yet. ' +
|
||||
'Importing downloads each ePub from CWA and keeps its chapters, including chapters the site no longer has.',
|
||||
),
|
||||
h(
|
||||
'form',
|
||||
{
|
||||
class: 'library-search',
|
||||
onsubmit: (e) => {
|
||||
e.preventDefault();
|
||||
if (submit.disabled) return;
|
||||
submit.disabled = true;
|
||||
search()
|
||||
.catch(showError)
|
||||
.finally(() => (submit.disabled = false));
|
||||
},
|
||||
},
|
||||
query,
|
||||
submit,
|
||||
),
|
||||
h(
|
||||
'div',
|
||||
{ class: 'controls' },
|
||||
button('Find stories in the library', async () => {
|
||||
candidates = await api('library/candidates');
|
||||
selected = new Set(candidates.map((c) => c.id));
|
||||
render();
|
||||
button('Show all stories in the library', async () => {
|
||||
list = undefined;
|
||||
show(await load(), true);
|
||||
}),
|
||||
h('label', {}, checkbox(true, (on) => (check = on)), ' Check the site for new chapters after import'),
|
||||
),
|
||||
@@ -199,6 +254,8 @@ function storyRow(s) {
|
||||
s.offsite_count > 0 && h('small', { title: 'Chapters the site no longer lists' }, `${s.offsite_count} removed`),
|
||||
s.revised_count > 0 && h('small', { title: 'Chapters the site changed since Bindery added them' }, `${s.revised_count} revised`),
|
||||
),
|
||||
// A story has no page count until Bindery builds its book.
|
||||
h('td', { 'data-label': 'Pages', title: 'Estimated from the built book' }, s.page_count != null && String(s.page_count)),
|
||||
h(
|
||||
'td',
|
||||
{ 'data-label': 'Status' },
|
||||
@@ -244,6 +301,8 @@ export function storiesPage(main, _params, config) {
|
||||
const order = (left - right) * direction;
|
||||
if (order) return order;
|
||||
} else if (sort !== 'title') {
|
||||
// A story with no number goes last in each direction.
|
||||
if (sortType === 'number' && (a[sort] == null) !== (b[sort] == null)) return a[sort] == null ? 1 : -1;
|
||||
const order = sortType === 'number'
|
||||
? a[sort] - b[sort]
|
||||
: (a[sort] || '').localeCompare(b[sort] || '', undefined, { sensitivity: 'base' });
|
||||
|
||||
@@ -68,7 +68,7 @@ const table = (headings, rows, extra = '') =>
|
||||
export function storyPage(main, { storyId }, config) {
|
||||
const id = Number(storyId);
|
||||
const fields = config.metadata_fields;
|
||||
// With CWA_ENABLED=false, the page shows nothing about CWA.
|
||||
// Without CWA_ENABLED=true, the page shows nothing about CWA.
|
||||
const cwa = config.cwa_enabled;
|
||||
const libraryStays = cwa ? ' The library book stays.' : '';
|
||||
let details = null;
|
||||
|
||||
@@ -940,6 +940,11 @@ textarea {
|
||||
color: var(--muted);
|
||||
}
|
||||
|
||||
/* A story with no page count yet shows no line for it. */
|
||||
.stories-table td:empty {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* The chapter list has far more rows than the story list, so each row packs its cells onto
|
||||
three lines instead of one line per cell: checkbox and number, title, then the rest. */
|
||||
.chapters-table tbody tr {
|
||||
|
||||
+13
-4
@@ -273,15 +273,17 @@ class Store:
|
||||
conn: sqlite3.Connection, story_id: int, chapter: dict[str, str], source: str, position: int
|
||||
) -> int:
|
||||
"""Store a chapter with its first revision. The chapter needs url, title, html and text_hash."""
|
||||
changed_at = now()
|
||||
cur = conn.execute(
|
||||
"INSERT INTO chapters (story_id, position, url, title, added_at) VALUES (?, ?, ?, ?, ?)",
|
||||
(story_id, position, chapter["url"], chapter["title"], now()),
|
||||
(story_id, position, chapter["url"], chapter["title"], changed_at),
|
||||
)
|
||||
chapter_id = int(cur.lastrowid)
|
||||
conn.execute(
|
||||
"INSERT INTO revisions (chapter_id, rev, html, text_hash, source, fetched_at) VALUES (?, 1, ?, ?, ?, ?)",
|
||||
(chapter_id, chapter["html"], chapter["text_hash"], source, now()),
|
||||
(chapter_id, chapter["html"], chapter["text_hash"], source, changed_at),
|
||||
)
|
||||
conn.execute("UPDATE stories SET last_changed = ? WHERE id = ?", (changed_at, story_id))
|
||||
return chapter_id
|
||||
|
||||
def add_chapter(
|
||||
@@ -326,6 +328,7 @@ class Store:
|
||||
|
||||
def add_revision(self, chapter_id: int, html: str, text_hash: str, source: str) -> int:
|
||||
"""Store a new revision and make it the current one."""
|
||||
changed_at = now()
|
||||
with self._connect() as conn:
|
||||
rev = conn.execute(
|
||||
"SELECT COALESCE(MAX(rev), 0) + 1 FROM revisions WHERE chapter_id = ?", (chapter_id,)
|
||||
@@ -333,9 +336,15 @@ class Store:
|
||||
conn.execute(
|
||||
"INSERT INTO revisions (chapter_id, rev, html, text_hash, source, fetched_at) "
|
||||
"VALUES (?, ?, ?, ?, ?, ?)",
|
||||
(chapter_id, rev, html, text_hash, source, now()),
|
||||
(chapter_id, rev, html, text_hash, source, changed_at),
|
||||
)
|
||||
conn.execute(
|
||||
"UPDATE chapters SET current_rev = ?, changed_at = ? WHERE id = ?", (rev, changed_at, chapter_id)
|
||||
)
|
||||
conn.execute(
|
||||
"UPDATE stories SET last_changed = ? WHERE id = (SELECT story_id FROM chapters WHERE id = ?)",
|
||||
(changed_at, chapter_id),
|
||||
)
|
||||
conn.execute("UPDATE chapters SET current_rev = ?, changed_at = ? WHERE id = ?", (rev, now(), chapter_id))
|
||||
return int(rev)
|
||||
|
||||
def update_chapter(self, chapter_id: int, **fields: Any) -> None:
|
||||
|
||||
+18
-17
@@ -2,26 +2,14 @@ services:
|
||||
bindery:
|
||||
build: .
|
||||
image: bindery:latest
|
||||
# The user and group that own ./config and can write CWA's ingest folder. Usually the same as CWA's.
|
||||
# The user and group that own ./config.
|
||||
user: "1000:1000"
|
||||
environment:
|
||||
# Without CWA: set this, and remove the CWA values below. INGEST_DIR then gets the built ePubs.
|
||||
# CWA_ENABLED: "false"
|
||||
# Omit this and its volume mount to keep uploads paused.
|
||||
INGEST_DIR: /cwa-book-ingest
|
||||
# The address of CWA, as the Bindery container reaches it, and the CWA account that Bindery uses.
|
||||
# Leave the account out if CWA asks for no login.
|
||||
CWA_URL: http://calibre-web-automated:8083
|
||||
CWA_USERNAME: bindery
|
||||
CWA_PASSWORD: change-me
|
||||
# none, local (one Bindery account), cwa (CWA accounts), or proxy (a trusted user header)
|
||||
AUTH_METHOD: cwa
|
||||
AUTH_METHOD: none
|
||||
# For local logins, put these values in .env. See "Local login" in the README.
|
||||
# BINDERY_USERNAME: ${BINDERY_USERNAME:-}
|
||||
# BINDERY_PASSWORD_HASH: ${BINDERY_PASSWORD_HASH:-}
|
||||
# Optional and independent. app.db is for logins, metadata.db is for book IDs.
|
||||
# CWA_DB_PATH: /cwa-config/app.db
|
||||
# CALIBRE_DB_PATH: /calibre-library/metadata.db
|
||||
# URL_BASE: /bindery
|
||||
# The host names that Bindery answers to. AUTH_METHOD=none needs this for a public name such as this one.
|
||||
# ALLOWED_HOSTS: bindery.example.com
|
||||
@@ -32,6 +20,19 @@ services:
|
||||
# For the cwa and proxy modes: the users that can change the global settings. Without a list, each user can.
|
||||
# BINDERY_ADMIN_USERS: alice, bob
|
||||
# BINDERY_API_KEY: a-long-random-string
|
||||
# Optional: a folder that gets each built ePub. With CWA, this is CWA's ingest folder.
|
||||
# INGEST_DIR: /cwa-book-ingest
|
||||
# Optional: Calibre-Web-Automated. The user above must then be able to write CWA's ingest folder.
|
||||
# Usually it is the same as CWA's.
|
||||
# CWA_ENABLED: "true"
|
||||
# The address of CWA, as the Bindery container reaches it, and the CWA account that Bindery uses.
|
||||
# Leave the account out if CWA asks for no login.
|
||||
# CWA_URL: http://calibre-web-automated:8083
|
||||
# CWA_USERNAME: bindery
|
||||
# CWA_PASSWORD: change-me
|
||||
# Optional and independent. app.db is for logins, metadata.db is for book IDs.
|
||||
# CWA_DB_PATH: /cwa-config/app.db
|
||||
# CALIBRE_DB_PATH: /calibre-library/metadata.db
|
||||
ports:
|
||||
# If a reverse proxy on this host is the only way in, publish the port only on the host: 127.0.0.1:8085:8085
|
||||
- 8085:8085
|
||||
@@ -44,11 +45,11 @@ services:
|
||||
start_period: 30s
|
||||
retries: 3
|
||||
volumes:
|
||||
# CWA's ingest folder. New stories and updates go here. Bindery does not make the folder.
|
||||
- /path/to/cwa/ingest:/cwa-book-ingest
|
||||
# The story database and personal.ini. Create the folder before the first start.
|
||||
- ./config:/config
|
||||
# Optional read-only folders. Include SQLite journal files beside each database.
|
||||
# Optional: the folder of INGEST_DIR. With CWA, CWA's ingest folder. Bindery does not make the folder.
|
||||
# - /path/to/cwa/ingest:/cwa-book-ingest
|
||||
# Optional read-only folders of CWA. Include SQLite journal files beside each database.
|
||||
# Mount the library that receives the ingest files, not a copy or another library.
|
||||
# - /path/to/cwa/config:/cwa-config:ro
|
||||
# - /path/to/calibre/library:/calibre-library:ro
|
||||
|
||||
@@ -10,6 +10,7 @@ for name, sub in [("CONFIG_DIR", "config"), ("INGEST_DIR", "ingest"), ("TMP_DIR"
|
||||
(_tmp / "ingest").mkdir() # Bindery does not make the ingest folder
|
||||
os.environ["BINDERY_START_THREADS"] = "false"
|
||||
os.environ["UPDATE_INTERVAL_HOURS"] = "0"
|
||||
os.environ["CWA_ENABLED"] = "true" # most tests are about a Bindery with CWA
|
||||
for name in (
|
||||
"CWA_URL",
|
||||
"CWA_USERNAME",
|
||||
|
||||
+102
-9
@@ -115,8 +115,8 @@ test('each group of settings is an open card, and the download help starts close
|
||||
const cards = main.children.filter((el) => el.tagName !== 'h1');
|
||||
assert.deepEqual(cards.map((el) => [el.tagName, el.className, el.open, el.children[0].textContent]), [
|
||||
['details', 'card', true, 'Recheck recent chapters'],
|
||||
['details', 'card', true, 'Uploads to CWA'],
|
||||
['details', 'card', true, 'Book Downloads'],
|
||||
['details', 'card', true, 'Uploads to CWA'],
|
||||
['details', 'card', true, 'Jobs list'],
|
||||
['details', 'card', true, 'Global replacement rules'],
|
||||
['details', 'card', true, 'FanFicFare settings (personal.ini)'],
|
||||
@@ -233,11 +233,11 @@ test('dropping a file on a closed epub import card opens it and fills the file f
|
||||
test('library import sends only selected books and handles an empty candidate list', async () => {
|
||||
await mountStories();
|
||||
api.on('GET', 'library/candidates', []);
|
||||
await click(main, 'Find stories in the library');
|
||||
await click(main, 'Show all stories in the library');
|
||||
assert.match(main.textContent, /tracked already/);
|
||||
const books = [{ id: 8, uuid: 'a', title: 'A', authors: ['W'] }, { id: 9, uuid: 'b', title: 'B', authors: [] }];
|
||||
api.on('GET', 'library/candidates', books);
|
||||
await click(main, 'Find stories in the library');
|
||||
await click(main, 'Show all stories in the library');
|
||||
change(field(main, 'Select all'), false);
|
||||
await click(main, 'Import selected');
|
||||
assert.equal(writes().length, 0);
|
||||
@@ -252,6 +252,91 @@ test('library import sends only selected books and handles an empty candidate li
|
||||
assert.deepEqual(api.sent('POST', 'library/import')[0].body, { books: [{ id: 8, uuid: 'a', title: 'A' }], check: false });
|
||||
});
|
||||
|
||||
test('library import search filters the loaded list, selects no book at first and imports the ticked books', async () => {
|
||||
await mountStories();
|
||||
const card = section(main, 'Import from the library');
|
||||
const form = nodes(card).find((el) => el.tagName === 'form');
|
||||
const submit = find(form, 'button', 'Search');
|
||||
assert.equal(form.className, 'library-search');
|
||||
assert.equal(submit.attributes.type, 'submit');
|
||||
const search = async (text) => {
|
||||
field(card, 'Search the library').value = text;
|
||||
emit(form, 'submit');
|
||||
await settle();
|
||||
};
|
||||
const shown = () => ['Ash & Bone', 'Bright Sky', 'Ash Road'].filter((title) => field(card, title));
|
||||
await search(' ');
|
||||
assert.equal(api.requests.some((r) => r.path.startsWith('library/candidates')), false);
|
||||
const books = [
|
||||
{ id: 8, uuid: 'a', title: 'Ash & Bone', authors: ['Wren'], publisher: 'www.royalroad.com' },
|
||||
{ id: 9, uuid: 'b', title: 'Bright Sky', authors: ['Ashley Poe', 'Wren'], publisher: 'archiveofourown.org' },
|
||||
{ id: 10, uuid: 'c', title: 'Ash Road', authors: [], publisher: 'www.royalroad.com' },
|
||||
];
|
||||
api.on('GET', 'library/candidates', books);
|
||||
await search('nothing');
|
||||
assert.match(card.textContent, /No library books from a story site match/);
|
||||
assert.equal(find(card, 'button', 'Import selected'), undefined);
|
||||
// Each word must be in the title, the authors or the site. Capitals do not matter.
|
||||
await search(' ASH ');
|
||||
assert.deepEqual(shown(), ['Ash & Bone', 'Bright Sky', 'Ash Road']);
|
||||
await search('ash royalroad');
|
||||
assert.deepEqual(shown(), ['Ash & Bone', 'Ash Road']);
|
||||
await search('wren sky');
|
||||
assert.deepEqual(shown(), ['Bright Sky']);
|
||||
assert.equal(api.sent('GET', 'library/candidates').length, 1, 'One load serves each search');
|
||||
await search('ash');
|
||||
assert.deepEqual(['Select all', ...shown()].map((label) => field(card, label).checked), [false, false, false, false]);
|
||||
await click(card, 'Import selected');
|
||||
assert.equal(writes().length, 0);
|
||||
change(field(card, 'Bright Sky'), true);
|
||||
api.on('POST', 'library/import', [{ id: 'job' }]);
|
||||
await click(card, 'Import selected');
|
||||
assert.deepEqual(api.sent('POST', 'library/import')[0].body, { books: [{ id: 9, uuid: 'b', title: 'Bright Sky' }], check: true });
|
||||
// A queued book leaves the list, without another load.
|
||||
await search('ash');
|
||||
assert.deepEqual(shown(), ['Ash & Bone', 'Ash Road']);
|
||||
assert.equal(api.sent('GET', 'library/candidates').length, 1);
|
||||
// The full list loads again, and selects each book.
|
||||
await click(card, 'Show all stories in the library');
|
||||
assert.equal(api.sent('GET', 'library/candidates').length, 2);
|
||||
assert.deepEqual(['Select all', ...shown()].map((label) => field(card, label).checked), [true, true, true, true]);
|
||||
});
|
||||
|
||||
test('the library import buttons cannot be used again until the answer comes', async () => {
|
||||
await mountStories();
|
||||
const card = section(main, 'Import from the library');
|
||||
const form = nodes(card).find((el) => el.tagName === 'form');
|
||||
const submit = find(form, 'button', 'Search');
|
||||
const all = find(card, 'button', 'Show all stories in the library');
|
||||
let answer;
|
||||
api.on('GET', 'library/candidates', () => new Promise((resolve) => { answer = resolve; }));
|
||||
field(card, 'Search the library').value = 'a';
|
||||
emit(form, 'submit');
|
||||
await settle();
|
||||
assert.equal(submit.disabled, true);
|
||||
emit(form, 'submit');
|
||||
await settle();
|
||||
assert.equal(api.sent('GET', 'library/candidates').length, 1);
|
||||
answer([{ id: 8, uuid: 'a', title: 'A', authors: [] }]);
|
||||
await settle();
|
||||
assert.equal(submit.disabled, false);
|
||||
assert.ok(field(card, 'A'));
|
||||
emit(all, 'click');
|
||||
await settle();
|
||||
assert.equal(all.disabled, true);
|
||||
answer([]);
|
||||
await settle();
|
||||
assert.equal(all.disabled, false);
|
||||
// A failed load frees the button, and the next search loads again.
|
||||
api.on('GET', 'library/candidates', { error: 'CWA is offline' }, 502);
|
||||
await click(card, 'Show all stories in the library');
|
||||
emit(form, 'submit');
|
||||
await settle();
|
||||
assert.equal(submit.disabled, false);
|
||||
assert.equal(all.disabled, false);
|
||||
assert.equal(api.sent('GET', 'library/candidates').length, 4);
|
||||
});
|
||||
|
||||
test('rules edits, moves, removal and paragraph choices are saved in order', async () => {
|
||||
api.on('GET', 'stories/1/rules', { ...emptyRules, rules: [rule, { ...rule, pattern: 'second' }] });
|
||||
const editor = rulesEditor('stories/1/rules');
|
||||
@@ -336,7 +421,7 @@ test('story list stacks on a narrow screen instead of scrolling sideways', async
|
||||
const wrap = nodes(main).find((el) => el.className === 'table-wrap stories-table');
|
||||
assert.ok(wrap);
|
||||
const dataLabels = nodes(wrap).filter((el) => el.tagName === 'td').map((el) => el.attributes['data-label']);
|
||||
assert.deepEqual(dataLabels, [undefined, 'Author', 'Site', 'Chapters', 'Status', 'Last change', 'Last check']);
|
||||
assert.deepEqual(dataLabels, [undefined, 'Author', 'Site', 'Chapters', 'Pages', 'Status', 'Last change', 'Last check']);
|
||||
});
|
||||
|
||||
test('the story list shows removed and revised chapter counts, only when there are any', async () => {
|
||||
@@ -704,9 +789,17 @@ test('with CWA off, the pages show nothing about CWA', async () => {
|
||||
assert.ok(section(main, 'Add stories'));
|
||||
assert.equal(section(main, 'Import from the library'), undefined);
|
||||
assert.doesNotMatch(main.textContent, /CWA|INGEST_DIR|library/i);
|
||||
settingsPage(main, {}, off);
|
||||
await settle();
|
||||
assert.ok(section(main, 'Jobs list'));
|
||||
assert.equal(section(main, 'Uploads to CWA'), undefined);
|
||||
assert.doesNotMatch(main.textContent, /CWA|INGEST_DIR|library/i);
|
||||
for (const ingest of [true, false]) {
|
||||
api.on('GET', 'settings', { recent_chapters: 3, personal_ini: '[defaults]', ingest_available: ingest });
|
||||
settingsPage(main, {}, off);
|
||||
await settle();
|
||||
assert.ok(section(main, 'Jobs list'));
|
||||
assert.equal(section(main, 'Uploads to CWA'), undefined);
|
||||
assert.doesNotMatch(main.textContent, /CWA|library/i);
|
||||
// The alert about INGEST_DIR is under Book Downloads, also with CWA off.
|
||||
const note = nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note');
|
||||
assert.equal(note.hidden, ingest);
|
||||
assert.equal(note.className, 'notice');
|
||||
assert.match(note.textContent, /INGEST_DIR is not set\. Bindery sends no book to the ingest folder/);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -114,6 +114,12 @@ test('sortable headings show direction and remain usable above story cards on na
|
||||
assert.doesNotMatch(css, /\.stories-table thead\s*\{[^}]*clip-path/);
|
||||
});
|
||||
|
||||
test('on narrow screens a story cell with no value shows no line', () => {
|
||||
const start = css.indexOf('@media (max-width: 600px)', css.indexOf('.stories-table th button'));
|
||||
assert.equal(declarations('.stories-table td:empty', css.slice(start)).display, 'none');
|
||||
assert.doesNotMatch(css.slice(0, start), /\.stories-table td:empty/);
|
||||
});
|
||||
|
||||
test('on narrow screens the headings are sort chips, and only the sorted chip is tinted with an arrow', () => {
|
||||
const start = css.indexOf('@media (max-width: 600px)', css.indexOf('.stories-table th button'));
|
||||
const mobile = css.slice(start);
|
||||
|
||||
@@ -21,7 +21,7 @@ const story = (id, title, changes = {}) => ({
|
||||
id, title, url: `https://example.com/${id}`,
|
||||
sort_title: (title || `https://example.com/${id}`).trim().replace(/^(?:the|an?)\s+/i, '').toLowerCase(),
|
||||
author: 'Reader', site: 'example.com',
|
||||
chapter_count: 42, offsite_count: 0, revised_count: 0, status: 'ok', message: 'No changes',
|
||||
chapter_count: 42, page_count: 120, offsite_count: 0, revised_count: 0, status: 'ok', message: 'No changes',
|
||||
last_changed: null, last_checked: null, ...changes,
|
||||
});
|
||||
const rows = () => {
|
||||
@@ -106,6 +106,39 @@ test('chapter sorting compares counts numerically and keeps equal counts in titl
|
||||
assert.equal(heading('Chapters').attributes['aria-sort'], 'ascending');
|
||||
});
|
||||
|
||||
test('pages show beside the chapters and sort numerically', async (t) => {
|
||||
await mount(t, [
|
||||
story(1, 'Zeta', { page_count: 9 }),
|
||||
story(2, 'Alpha', { page_count: 1200 }),
|
||||
story(3, 'Beta', { page_count: 0 }),
|
||||
story(4, 'Gamma', { page_count: 9 }),
|
||||
]);
|
||||
const cells = nodes(rows()[0]).filter((el) => el.tagName === 'td').map((el) => el.attributes['data-label']);
|
||||
assert.deepEqual(cells.slice(3, 5), ['Chapters', 'Pages']);
|
||||
assert.equal(find(app, 'button', 'Pages').attributes['aria-label'], 'Sort by pages, most first');
|
||||
await click(app, 'Pages');
|
||||
assert.deepEqual(titles(), ['Alpha', 'Gamma', 'Zeta', 'Beta']);
|
||||
await click(app, 'Pages');
|
||||
assert.deepEqual(titles(), ['Beta', 'Gamma', 'Zeta', 'Alpha']);
|
||||
assert.deepEqual(JSON.parse(window.localStorage.getItem(SORT_KEY)), { column: 'page_count', direction: 1 });
|
||||
});
|
||||
|
||||
test('a story with no built book shows no page count and sorts last in each direction', async (t) => {
|
||||
await mount(t, [
|
||||
story(1, 'Alpha', { page_count: null }),
|
||||
story(2, 'Beta', { page_count: 300 }),
|
||||
story(3, 'Gamma', { page_count: 0 }),
|
||||
story(4, 'Delta', { page_count: null }),
|
||||
]);
|
||||
const pages = () => rows().map((row) => cell(row, 'Pages'));
|
||||
assert.deepEqual(pages().map((el) => el.textContent), ['', '300', '', '0']);
|
||||
assert.deepEqual(pages().map((el) => el.children.length), [0, 1, 0, 1], 'An empty cell hides on a narrow screen');
|
||||
await click(app, 'Pages');
|
||||
assert.deepEqual(titles(), ['Beta', 'Gamma', 'Alpha', 'Delta']);
|
||||
await click(app, 'Pages');
|
||||
assert.deepEqual(titles(), ['Gamma', 'Beta', 'Alpha', 'Delta']);
|
||||
});
|
||||
|
||||
for (const [label, key] of [['Author', 'author'], ['Site', 'site'], ['Status', 'status']]) {
|
||||
test(`${label} sorts alphabetically in either direction and uses titles to break ties`, async (t) => {
|
||||
await mount(t, [
|
||||
@@ -248,7 +281,7 @@ test('date columns keep each timestamp on one line, with last change before last
|
||||
const checked = '2026-01-03T05:06:00Z';
|
||||
await mount(t, [story(1, 'Alpha', { last_changed: changed, last_checked: checked })]);
|
||||
const headings = nodes(app).filter((el) => el.tagName === 'th').map((el) => el.textContent);
|
||||
assert.deepEqual(headings, ['Title', 'Author', 'Site', 'Chapters', 'Status', 'Last change', 'Last check']);
|
||||
assert.deepEqual(headings, ['Title', 'Author', 'Site', 'Chapters', 'Pages', 'Status', 'Last change', 'Last check']);
|
||||
for (const [label, value] of [['Last change', changed], ['Last check', checked]]) {
|
||||
const dateCell = cell(rows()[0], label);
|
||||
const time = nodes(dateCell).find((el) => el.tagName === 'time');
|
||||
|
||||
@@ -54,7 +54,7 @@ test('the upload switches load and save on the settings page', async (t) => {
|
||||
assert.equal(warning.hidden, true);
|
||||
assert.equal(link.attributes['aria-describedby'], '');
|
||||
assert.match(warning.textContent, /Updates without a saved CWA book ID create duplicate books if CWA imports them/);
|
||||
assert.equal(nodes(card).find((el) => el.attributes.id === 'ingest-note').hidden, true);
|
||||
assert.equal(nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note').hidden, true);
|
||||
assert.equal(pause.disabled, false);
|
||||
assert.equal(hold.disabled, false);
|
||||
assert.equal(link.disabled, false);
|
||||
@@ -129,7 +129,7 @@ for (const linkBooks of [true, false]) {
|
||||
assert.equal(nodes(card).find((el) => el.attributes.id === 'link-note').hidden, linkBooks);
|
||||
assert.equal(pause.checked, true);
|
||||
assert.equal(pause.disabled, true);
|
||||
const note = nodes(card).find((el) => el.attributes.id === 'ingest-note');
|
||||
const note = nodes(section(main, 'Book Downloads')).find((el) => el.attributes.id === 'ingest-note');
|
||||
assert.equal(note.hidden, false);
|
||||
assert.equal(note.className, 'notice');
|
||||
assert.equal(pause.attributes['aria-describedby'], note.attributes.id);
|
||||
|
||||
+102
-9
@@ -11,7 +11,8 @@ from test_deliver import fake as fake
|
||||
from test_deliver import jobs as jobs
|
||||
from test_deliver import story_epub as story_epub
|
||||
|
||||
from bindery import config, orchestrator
|
||||
from bindery import config, orchestrator, pages
|
||||
from bindery import store as storage
|
||||
from bindery.jobs import DownloadMode, Job
|
||||
|
||||
ONE = [(1, "One", "a")]
|
||||
@@ -44,13 +45,21 @@ def test_an_unchanged_story_is_not_built_or_sent_again(builds, monkeypatch):
|
||||
assert len(ingest_files()) == 2 # one epub and its sidecar
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change", ["chapter", "rule", "global rule", "paragraphs", "metadata", "included"])
|
||||
@pytest.mark.parametrize(
|
||||
"change", ["chapter", "revision", "renamed", "rule", "global rule", "paragraphs", "metadata", "included"]
|
||||
)
|
||||
def test_each_change_builds_and_sends_the_book(builds, monkeypatch, change):
|
||||
story_id = linked_story([(1, "One", "a"), (2, "Two", "b")])
|
||||
site(monkeypatch, [(1, "One", "a"), (2, "Two", "b")])
|
||||
update(story_id)
|
||||
previous = "2020-01-01T00:00:00"
|
||||
orchestrator.store.update_story(story_id, last_changed=previous)
|
||||
if change == "chapter":
|
||||
site(monkeypatch, [(1, "One", "a"), (2, "Two", "b"), (3, "Three", "c")])
|
||||
elif change == "revision":
|
||||
site(monkeypatch, [(1, "One", "a"), (2, "Two", "new text")])
|
||||
elif change == "renamed":
|
||||
site(monkeypatch, [(1, "One", "a"), (2, "Renamed", "b")])
|
||||
elif change == "rule":
|
||||
orchestrator.save_rules(story_id, [{"pattern": "a", "replacement": "b"}])
|
||||
elif change == "global rule":
|
||||
@@ -62,10 +71,14 @@ def test_each_change_builds_and_sends_the_book(builds, monkeypatch, change):
|
||||
else:
|
||||
chapter = orchestrator.store.list_chapters(story_id)[0]
|
||||
orchestrator.store.update_chapter(chapter["id"], included=0)
|
||||
assert "library book 1" in update(story_id)
|
||||
mode = DownloadMode.RECHECK_ALL if change == "revision" else DownloadMode.NEW
|
||||
assert "library book 1" in update(story_id, mode)
|
||||
assert builds.call_count == 2
|
||||
assert len(ingest_files()) == 4
|
||||
changed_at = story(story_id)["last_changed"]
|
||||
assert (changed_at != previous) == (change in ("chapter", "revision"))
|
||||
assert update(story_id).startswith("No changes")
|
||||
assert story(story_id)["last_changed"] == changed_at
|
||||
|
||||
|
||||
def test_a_held_book_that_did_not_change_is_not_built_again(builds, monkeypatch):
|
||||
@@ -110,13 +123,14 @@ def test_a_lost_file_is_built_again_only_to_send_it(builds):
|
||||
assert builds.call_count == 2
|
||||
|
||||
|
||||
def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(builds, client, jobs):
|
||||
@pytest.mark.parametrize("last_changed", [None, "2020-01-01T00:00:00"])
|
||||
def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(builds, client, jobs, last_changed):
|
||||
story_id = add_story(ingested_at="2026-01-01T00:00:00", calibre_id=1, calibre_uuid="uuid-1")
|
||||
job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
|
||||
assert "library book 1" in orchestrator._run_rebuild(job)
|
||||
orchestrator.store.update_story(story_id, last_changed=None, message="before")
|
||||
orchestrator.store.update_story(story_id, last_changed=last_changed, message="before")
|
||||
assert orchestrator._run_rebuild(job) == "No changes"
|
||||
assert (story(story_id)["last_changed"], story(story_id)["message"]) == (None, "before")
|
||||
assert (story(story_id)["last_changed"], story(story_id)["message"]) == (last_changed, "before")
|
||||
assert builds.call_count == 1
|
||||
# The Rebuild button of the story page.
|
||||
queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "rebuild"}, headers=H).json
|
||||
@@ -124,7 +138,7 @@ def test_a_rebuild_without_a_change_does_nothing_unless_the_user_asks_for_it(bui
|
||||
assert "library book 1" in orchestrator._run_rebuild(jobs.get_next())
|
||||
assert builds.call_count == 2
|
||||
assert len(ingest_files()) == 4
|
||||
assert story(story_id)["last_changed"]
|
||||
assert story(story_id)["last_changed"] == last_changed
|
||||
# Other modes take no such option.
|
||||
queued = client.post(f"/api/stories/{story_id}/download", json={"mode": "new"}, headers=H).json
|
||||
assert queued["options"] == {}
|
||||
@@ -152,6 +166,31 @@ def test_a_failed_delivery_is_tried_again_without_a_build(builds, fake, monkeypa
|
||||
assert builds.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change", ["chapter", "revision"])
|
||||
def test_chapter_timestamp_survives_a_failed_build_and_retry(builds, monkeypatch, change):
|
||||
story_id = linked_story(ONE)
|
||||
site(monkeypatch, ONE)
|
||||
update(story_id)
|
||||
previous = "2020-01-01T00:00:00"
|
||||
orchestrator.store.update_story(story_id, last_changed=previous)
|
||||
clock = ["2024-01-01T00:00:00"]
|
||||
monkeypatch.setattr(storage, "now", lambda: clock[0])
|
||||
site(monkeypatch, [*ONE, (2, "Two", "b")] if change == "chapter" else [(1, "One", "revised")])
|
||||
job = Job(mode=DownloadMode.RECHECK_ALL, story_id=story_id)
|
||||
build = orchestrator._build
|
||||
monkeypatch.setattr(orchestrator, "_build", Mock(side_effect=OSError("disk full")))
|
||||
with pytest.raises(OSError, match="disk full"):
|
||||
orchestrator._run_site_job(job)
|
||||
assert story(story_id)["last_changed"] == clock[0]
|
||||
assert job.result["new"] or job.result["changed"]
|
||||
clock[0] = "2024-02-01T00:00:00"
|
||||
monkeypatch.setattr(orchestrator, "_build", build)
|
||||
assert "library book 1" in orchestrator._run_site_job(job)
|
||||
assert story(story_id)["last_changed"] == "2024-01-01T00:00:00"
|
||||
assert update(story_id).startswith("No changes")
|
||||
assert story(story_id)["last_changed"] == "2024-01-01T00:00:00"
|
||||
|
||||
|
||||
def test_a_new_build_removes_the_older_file(builds, monkeypatch):
|
||||
story_id = linked_story(ONE)
|
||||
site(monkeypatch, ONE)
|
||||
@@ -227,6 +266,7 @@ def test_a_download_uses_the_built_book(builds, client, monkeypatch):
|
||||
|
||||
def test_a_download_builds_a_book_that_has_no_file_or_that_changed(builds, client, monkeypatch):
|
||||
story_id = add_story()
|
||||
changed_at = story(story_id)["last_changed"]
|
||||
first = download(client, story_id)
|
||||
download(client, story_id)
|
||||
assert builds.call_count == 1
|
||||
@@ -234,12 +274,65 @@ def test_a_download_builds_a_book_that_has_no_file_or_that_changed(builds, clien
|
||||
assert download(client, story_id) != first
|
||||
assert builds.call_count == 2
|
||||
assert len(list(config.TMP_DIR.iterdir())) == 1
|
||||
# The build of a download is not the build of a job. The job sends the book, and marks the change.
|
||||
# The build of a download is not the build of a job. The job sends the book.
|
||||
assert orchestrator.store.get_setting(story_id, "built_key") is None
|
||||
job = Job(mode=DownloadMode.REBUILD, story_id=story_id)
|
||||
assert "library book 1" in orchestrator._run_rebuild(job)
|
||||
assert builds.call_count == 2
|
||||
assert story(story_id)["last_changed"]
|
||||
assert story(story_id)["last_changed"] == changed_at
|
||||
|
||||
|
||||
# Page counts
|
||||
|
||||
LONG = "A line of the story. " * 20
|
||||
|
||||
|
||||
def listed_pages(story_id):
|
||||
return {s["id"]: s["page_count"] for s in orchestrator.list_stories()}[story_id]
|
||||
|
||||
|
||||
def test_a_story_has_no_page_count_until_its_book_is_built(builds, monkeypatch):
|
||||
story_id = linked_story(ONE)
|
||||
assert listed_pages(story_id) is None
|
||||
site(monkeypatch, ONE)
|
||||
update(story_id)
|
||||
(built,) = config.TMP_DIR.iterdir()
|
||||
assert listed_pages(story_id) == pages.epub_pages(built) == 1
|
||||
|
||||
|
||||
def test_each_build_counts_the_pages_of_the_new_book(builds, client, monkeypatch):
|
||||
story_id = linked_story(ONE)
|
||||
site(monkeypatch, [*ONE, *[(n, f"Chapter {n}", "<br/>".join([LONG] * 31)) for n in range(2, 6)]])
|
||||
update(story_id)
|
||||
long_book = listed_pages(story_id)
|
||||
# Each of the 124 lines is a paragraph and 6 lines of text.
|
||||
assert long_book >= 4 * 31 * 7 // 31
|
||||
# The count is of the built book, so a rule that removes text changes it without a download.
|
||||
orchestrator.save_rules(story_id, [{"pattern": "(A line of the story. ){19}", "replacement": ""}])
|
||||
download(client, story_id)
|
||||
assert builds.call_count == 2
|
||||
assert 0 < listed_pages(story_id) < long_book / 3
|
||||
|
||||
|
||||
def test_a_book_from_before_the_page_count_is_counted_without_a_build(builds, client, monkeypatch):
|
||||
story_id = linked_story(ONE)
|
||||
site(monkeypatch, ONE)
|
||||
update(story_id)
|
||||
orchestrator.store.set_setting(story_id, "page_count", None)
|
||||
download(client, story_id)
|
||||
assert builds.call_count == 1
|
||||
assert listed_pages(story_id) == 1
|
||||
|
||||
|
||||
def test_a_book_that_cannot_be_counted_is_still_built(builds, monkeypatch):
|
||||
def fail(path):
|
||||
raise orchestrator.EpubError("unreadable")
|
||||
|
||||
monkeypatch.setattr(pages, "epub_pages", fail)
|
||||
story_id = linked_story(ONE)
|
||||
site(monkeypatch, ONE)
|
||||
assert "library book 1" in update(story_id)
|
||||
assert listed_pages(story_id) is None
|
||||
|
||||
|
||||
def test_stop_tracking_deletes_the_built_book(builds, client):
|
||||
|
||||
+15
-10
@@ -143,7 +143,7 @@ def test_defaults(monkeypatch, reload_config):
|
||||
]
|
||||
assert config.ALLOWED_HOSTS == []
|
||||
assert config.ADMIN_USERS == []
|
||||
assert config.CWA_ENABLED is True
|
||||
assert config.CWA_ENABLED is False
|
||||
assert config.IGNORED_SETTINGS == []
|
||||
assert config.SESSION_COOKIE_SECURE is False
|
||||
assert config.TRUST_PROXY is False
|
||||
@@ -161,9 +161,12 @@ def test_empty_ingest_directory_disables_uploads(monkeypatch, reload_config, val
|
||||
assert config.INGEST_DIR is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["false", " No ", "0", "off"])
|
||||
@pytest.mark.parametrize("value", [None, "", "false", " No ", "0", "maybe"])
|
||||
def test_cwa_off_ignores_the_cwa_settings_but_not_the_ingest_folder(monkeypatch, reload_config, value):
|
||||
monkeypatch.setenv("CWA_ENABLED", value)
|
||||
if value is None:
|
||||
monkeypatch.delenv("CWA_ENABLED")
|
||||
else:
|
||||
monkeypatch.setenv("CWA_ENABLED", value)
|
||||
monkeypatch.setenv("CWA_URL", "https://cwa.invalid")
|
||||
monkeypatch.setenv("CWA_USERNAME", "bindery")
|
||||
monkeypatch.setenv("CWA_PASSWORD", "secret")
|
||||
@@ -178,8 +181,8 @@ def test_cwa_off_ignores_the_cwa_settings_but_not_the_ingest_folder(monkeypatch,
|
||||
assert str(config.INGEST_DIR) == "/books"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["", "true", "maybe"])
|
||||
def test_only_a_clear_false_turns_cwa_off(monkeypatch, reload_config, value):
|
||||
@pytest.mark.parametrize("value", ["true", " Yes ", "1", "on"])
|
||||
def test_only_a_clear_true_turns_cwa_on(monkeypatch, reload_config, value):
|
||||
monkeypatch.setenv("CWA_ENABLED", value)
|
||||
monkeypatch.setenv("CWA_URL", "https://cwa.invalid")
|
||||
reload_config()
|
||||
@@ -197,17 +200,19 @@ def test_prepare_with_cwa_off_logs_nothing_about_missing_cwa(folders, monkeypatc
|
||||
with caplog.at_level("INFO"):
|
||||
config.prepare()
|
||||
lines = [r.getMessage() for r in caplog.records]
|
||||
expected = ["CWA_ENABLED is false. Bindery ignores CWA_URL, CALIBRE_DB_PATH."] if ignored else []
|
||||
ignores = "CWA_ENABLED is not true. Bindery ignores CWA_URL, CALIBRE_DB_PATH. To use CWA, set CWA_ENABLED=true."
|
||||
expected = [ignores] if ignored else []
|
||||
assert [r.levelname for r in caplog.records if "CWA" in r.getMessage()] == ["WARNING"] * len(expected)
|
||||
assert [line for line in lines if "CWA" in line or "INGEST" in line] == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("ingest", "url", "database", "expected"),
|
||||
[
|
||||
(False, "", None, ["INGEST_DIR is not set", "cannot find the library book of a story.", "CWA_ENABLED=false"]),
|
||||
(True, "", None, ["so each update adds a new book to CWA.", "CWA_ENABLED=false"]),
|
||||
(True, "", Path("/library/metadata.db"), ["CWA_URL is not set. Bindery cannot search", "CWA_ENABLED=false"]),
|
||||
(False, "https://cwa.invalid", None, ["INGEST_DIR is not set", "CWA_ENABLED=false"]),
|
||||
(False, "", None, ["INGEST_DIR is not set", "cannot find the library book of a story.", "remove CWA_ENABLED"]),
|
||||
(True, "", None, ["so each update adds a new book to CWA.", "remove CWA_ENABLED"]),
|
||||
(True, "", Path("/library/metadata.db"), ["CWA_URL is not set. Bindery cannot search", "remove CWA_ENABLED"]),
|
||||
(False, "https://cwa.invalid", None, ["INGEST_DIR is not set", "remove CWA_ENABLED"]),
|
||||
(True, "https://cwa.invalid", None, []),
|
||||
],
|
||||
)
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Last Changed records stored chapters and revisions, not changes to the built book."""
|
||||
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
|
||||
from bindery import store as storage
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def state(tmp_path, monkeypatch):
|
||||
clock = ["2024-01-01T00:00:00"]
|
||||
monkeypatch.setattr(storage, "now", lambda: clock[0])
|
||||
store = storage.Store(tmp_path / "stories.db")
|
||||
sid = store.create_story("story", "site", "Title", "Author", [])
|
||||
return store, sid, clock
|
||||
|
||||
|
||||
@pytest.mark.parametrize("source", ["new", "import"])
|
||||
def test_new_chapters_and_revisions_set_the_timestamp_in_the_store(state, source):
|
||||
store, sid, clock = state
|
||||
other = store.create_story("other", "site", "Other", "Author", [])
|
||||
assert store.get_story(sid)["last_changed"] is None
|
||||
first = store.add_chapter(sid, "one", "One", "first", "h1", source, 0)
|
||||
assert store.get_story(sid)["last_changed"] == clock[0]
|
||||
assert store.get_chapter(first)["added_at"] == clock[0]
|
||||
clock[0] = "2024-02-01T00:00:00"
|
||||
store.add_chapters(sid, [{"url": "two", "title": "Two", "html": "second", "text_hash": "h2"}], source, [])
|
||||
assert store.get_story(sid)["last_changed"] == clock[0]
|
||||
clock[0] = "2024-03-01T00:00:00"
|
||||
store.add_revision(first, "revised", "h3", "recheck")
|
||||
assert store.get_story(sid)["last_changed"] == clock[0]
|
||||
assert store.get_chapter(first)["changed_at"] == clock[0]
|
||||
assert store.list_revisions(first)[-1]["fetched_at"] == clock[0]
|
||||
assert storage.Store(store.path).get_story(sid)["last_changed"] == clock[0]
|
||||
assert store.get_story(other)["last_changed"] is None
|
||||
|
||||
|
||||
def test_changes_without_new_chapters_or_revisions_keep_the_timestamp(state):
|
||||
store, sid, clock = state
|
||||
first = store.add_chapter(sid, "one", "One", "first", "h1", "new", 0)
|
||||
second = store.add_chapter(sid, "two", "Two", "second", "h2", "new", 1)
|
||||
store.add_revision(first, "revised", "h3", "recheck")
|
||||
previous = clock[0]
|
||||
clock[0] = "2024-02-01T00:00:00"
|
||||
store.update_story(sid, title="Edited", pinned={"title": "Edited"})
|
||||
store.set_rules(sid, [{"pattern": "first", "replacement": "1st"}])
|
||||
store.update_chapter(first, title="Renamed", included=False, on_site=False, current_rev=1)
|
||||
store.replace_revision(first, 1, "original site text", "h4")
|
||||
store.reorder_chapters(sid, lambda order: list(reversed(order)))
|
||||
store.add_chapters(sid, [], "import", ["one", "two"])
|
||||
store.delete_revision(first, 2)
|
||||
store.delete_chapters(sid, [second])
|
||||
assert store.get_story(sid)["last_changed"] == previous
|
||||
|
||||
|
||||
def test_a_failed_chapter_transaction_does_not_change_the_timestamp(state):
|
||||
store, sid, clock = state
|
||||
store.add_chapter(sid, "one", "One", "first", "h1", "new", 0)
|
||||
previous = clock[0]
|
||||
clock[0] = "2024-02-01T00:00:00"
|
||||
chapters = [{"url": url, "title": url, "html": "text", "text_hash": "h"} for url in ("two", "one")]
|
||||
with pytest.raises(sqlite3.IntegrityError):
|
||||
store.add_chapters(sid, chapters, "import", [])
|
||||
assert store.get_story(sid)["last_changed"] == previous
|
||||
assert [chapter["url"] for chapter in store.list_chapters(sid)] == ["one"]
|
||||
+12
-1
@@ -48,10 +48,10 @@ def add_book(title="A book", author="Writer", changed=None, url=None, included=T
|
||||
store = orchestrator.store
|
||||
url = url or f"https://example.com/books/{len(store.list_stories()) + 1}"
|
||||
sid = store.create_story(url, "example.com", title, author, [])
|
||||
store.update_story(sid, last_changed=changed, site_values=meta or {})
|
||||
if included:
|
||||
html = "<p>Stored text.</p>"
|
||||
store.add_chapter(sid, url + "/1", "One", html, diffing.text_hash(html), "site", 0)
|
||||
store.update_story(sid, last_changed=changed, site_values=meta or {})
|
||||
return sid
|
||||
|
||||
|
||||
@@ -205,6 +205,17 @@ def test_entry_shows_the_book_metadata_and_cover(client):
|
||||
assert client.get(f"/opds/books/{add_book()}/cover").status_code == 404
|
||||
|
||||
|
||||
def test_entry_shows_the_page_count_of_a_built_book(client):
|
||||
sid = add_book()
|
||||
orchestrator.store.set_setting(sid, "page_count", 321)
|
||||
add_book(title="Not built")
|
||||
contents = {
|
||||
e.findtext("a:title", namespaces=NS): e.findtext("a:content", namespaces=NS)
|
||||
for e in entries(feed(client.get("/opds/title")))
|
||||
}
|
||||
assert contents == {"A book": "Chapters: 1\nPages: 321", "Not built": "Chapters: 1"}
|
||||
|
||||
|
||||
def test_cover_of_an_unsafe_type_is_left_out(client):
|
||||
sid = add_book()
|
||||
orchestrator.store.put_image(sid, "https://example.com/c.svg", "svg", "image/svg+xml", b"<svg/>", is_cover=True)
|
||||
|
||||
@@ -40,7 +40,7 @@ def library(isolated, monkeypatch):
|
||||
monkeypatch.setattr(orchestrator, "_find_book", Mock(return_value=None))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enabled", [False, True])
|
||||
@pytest.mark.parametrize("enabled", [None, False, True])
|
||||
def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
|
||||
env = {
|
||||
**os.environ,
|
||||
@@ -51,7 +51,14 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
|
||||
"URL_BASE": "",
|
||||
"CWA_ENABLED": str(enabled),
|
||||
}
|
||||
for name in ("INGEST_DIR", "CWA_URL", "CWA_DB_PATH", "CALIBRE_DB_PATH"):
|
||||
# CWA is off unless CWA_ENABLED turns it on.
|
||||
for name in (
|
||||
"INGEST_DIR",
|
||||
"CWA_URL",
|
||||
"CWA_DB_PATH",
|
||||
"CALIBRE_DB_PATH",
|
||||
*(["CWA_ENABLED"] if enabled is None else []),
|
||||
):
|
||||
env.pop(name, None)
|
||||
result = subprocess.run(
|
||||
[
|
||||
@@ -60,6 +67,7 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
|
||||
"from bindery.web import app; "
|
||||
"client = app.test_client(); "
|
||||
"assert client.get('/api/health').status_code == 200; "
|
||||
f"assert client.get('/api/config').json['cwa_enabled'] is {bool(enabled)}; "
|
||||
"settings = client.get('/api/settings').json; "
|
||||
"assert settings['pause_uploads'] is True; "
|
||||
"assert settings['ingest_available'] is False; "
|
||||
@@ -73,9 +81,9 @@ def test_app_starts_without_any_cwa_configuration(tmp_path, enabled):
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "ERROR" not in result.stderr
|
||||
# Bindery names what is missing, until the user says that there is no CWA.
|
||||
for text in ("WARNING", "INGEST_DIR is not set", "CWA_URL and CALIBRE_DB_PATH are not set", "CWA_ENABLED=false"):
|
||||
assert (text in result.stderr) is enabled
|
||||
# Bindery names what is missing only for a user who turned CWA on.
|
||||
for text in ("WARNING", "INGEST_DIR is not set", "CWA_URL and CALIBRE_DB_PATH are not set", "remove CWA_ENABLED"):
|
||||
assert (text in result.stderr) is bool(enabled)
|
||||
if not enabled:
|
||||
assert "CWA" not in result.stderr and "INGEST" not in result.stderr
|
||||
|
||||
@@ -134,7 +142,7 @@ def test_cwa_off_shows_in_the_config_and_refuses_cwa_logins(client, monkeypatch,
|
||||
monkeypatch.setattr(config, "CWA_DB_PATH", "/cwa/app.db")
|
||||
response = client.get("/api/stories")
|
||||
assert response.status_code == 503
|
||||
assert response.json["error"].startswith("AUTH_METHOD=cwa needs CWA, but CWA_ENABLED is false.")
|
||||
assert response.json["error"].startswith("AUTH_METHOD=cwa needs CWA_ENABLED=true.")
|
||||
|
||||
|
||||
def test_without_cwa_the_download_template_uses_the_story_id(client, cwa_off):
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
"""The page estimate of a document and of a built epub."""
|
||||
|
||||
import zipfile
|
||||
|
||||
import pytest
|
||||
|
||||
from bindery import pages
|
||||
from bindery.epub import EpubError, spine_documents
|
||||
from bindery.pages import Measure
|
||||
|
||||
TEXT = ["One line of the story.", "Another, longer line of the story that goes on for a while.", "The end."]
|
||||
CHARS = sum(map(len, TEXT))
|
||||
|
||||
|
||||
def test_a_paragraph_is_text_after_a_block_or_a_line_break():
|
||||
assert pages.measure("<p>One</p><p>Two</p>") == Measure(2, 6)
|
||||
assert pages.measure("<div>One<br/><br/>Two<br>Three</div>") == Measure(3, 11)
|
||||
assert pages.measure("<div><div>One</div><div>Two</div></div>") == Measure(2, 6)
|
||||
# Text before, between and after the blocks of an element counts too.
|
||||
assert pages.measure("<div>Intro<p>One</p>Middle<p>Two</p>End</div>") == Measure(5, 20)
|
||||
assert pages.measure("<h3>Title</h3><ul><li>One</li><li>Two</li></ul><blockquote>Three</blockquote>") == Measure(
|
||||
4, 16
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"book",
|
||||
[
|
||||
"".join(f"<p>{line}</p>" for line in TEXT),
|
||||
"".join(f'<p class="a">{line}</p>\n' for line in TEXT),
|
||||
"<div>" + "".join(f"<div>{line}</div>" for line in TEXT) + "</div>",
|
||||
"<div>" + "<br/><br/>".join(TEXT) + "</div>",
|
||||
"<div>" + "<br>\n".join(TEXT) + "</div>",
|
||||
f"<div>{TEXT[0]}<p>{TEXT[1]}</p>{TEXT[2]}</div>",
|
||||
],
|
||||
)
|
||||
def test_the_same_text_gives_the_same_measure_in_each_markup(book):
|
||||
assert pages.measure(book) == Measure(3, CHARS)
|
||||
|
||||
|
||||
def test_inline_elements_and_table_cells_stay_in_the_line():
|
||||
assert pages.measure('<p>ab<b>cd</b><a href="x">ef</a> <span><i>gh</i></span></p>') == Measure(1, 9)
|
||||
assert pages.measure("<table><tr><td>Name</td><td>Value</td></tr><tr><th>a</th><td>b</td></tr></table>") == Measure(
|
||||
2, 11
|
||||
)
|
||||
assert pages.measure('<p>Before<img src="a.png"/>after</p>') == Measure(1, 11)
|
||||
|
||||
|
||||
def test_only_the_text_of_the_book_counts():
|
||||
document = (
|
||||
'<?xml version="1.0"?><html><head><title>Hidden title</title><style>p { color: red }</style>'
|
||||
'<link href="a.css"/></head><body><script>var hidden;</script><script src="x.js"/>'
|
||||
"<!-- a note --><p>Shown & counted</p>\n\n<p> </p><p></p></body></html>"
|
||||
)
|
||||
assert pages.measure(document) == Measure(1, len("Shown & counted"))
|
||||
assert pages.measure("") == Measure(0, 0)
|
||||
assert pages.measure("<p>\n</p>") == Measure(0, 0)
|
||||
|
||||
|
||||
def test_a_paragraph_that_only_spaces_the_text_is_not_counted():
|
||||
spaced = "<p>One</p><p> </p><p>\xa0</p><p>   </p><p><br/></p><p><span> </span></p><p>Two</p>"
|
||||
assert pages.measure(spaced) == pages.measure("<p>One</p><p>Two</p>") == Measure(2, 6)
|
||||
assert pages.measure("<div>One<br/> <br/>Two</div>") == Measure(2, 6)
|
||||
# A no-break space inside a paragraph is a character of its text.
|
||||
assert pages.measure("<p>One <b>two</b>\xa0three</p>") == Measure(1, 13)
|
||||
|
||||
|
||||
def test_a_page_is_31_lines_and_a_line_is_a_paragraph_or_70_characters():
|
||||
assert pages.page_count(Measure(62, 620)) == (62 + 620 // 70) // 31 == 2
|
||||
assert pages.page_count(Measure(31 * 9, 31 * 9)) == 9
|
||||
assert pages.page_count(Measure(1000, 177_000)) == (1000 + 177_000 // 70) // 31
|
||||
|
||||
|
||||
def test_a_short_book_has_one_page_and_a_book_with_no_text_has_none():
|
||||
assert pages.page_count(Measure(1, 10)) == 1
|
||||
assert pages.page_count(Measure(30, 30)) == 1
|
||||
assert pages.page_count(Measure()) == 0
|
||||
# One block of text: 2400 characters start the second page before 31 lines do.
|
||||
assert pages.page_count(Measure(1, 2399)) == 1
|
||||
assert pages.page_count(Measure(1, 2400)) == 2
|
||||
|
||||
|
||||
def epub(path, documents, spine=None):
|
||||
names = [f"file {n}.xhtml" for n in range(len(documents))]
|
||||
items = "".join(f'<item id="f{n}" href="{name.replace(" ", "%20")}"/>' for n, name in enumerate(names))
|
||||
items += '<item id="css" href="stylesheet.css"/>'
|
||||
refs = "".join(f'<itemref idref="f{n}"/>' for n in (spine or range(len(documents))))
|
||||
with zipfile.ZipFile(path, "w") as z:
|
||||
z.writestr(
|
||||
"META-INF/container.xml",
|
||||
'<container xmlns="urn:oasis:names:tc:opendocument:xmlns:container"><rootfiles>'
|
||||
'<rootfile full-path="OEBPS/content.opf"/></rootfiles></container>',
|
||||
)
|
||||
z.writestr(
|
||||
"OEBPS/content.opf",
|
||||
f'<package xmlns="http://www.idpf.org/2007/opf"><metadata/><manifest>{items}</manifest>'
|
||||
f"<spine>{refs}</spine></package>",
|
||||
)
|
||||
z.writestr("OEBPS/stylesheet.css", "p { margin: 0 }")
|
||||
for name, document in zip(names, documents, strict=True):
|
||||
z.writestr(f"OEBPS/{name}", document)
|
||||
return path
|
||||
|
||||
|
||||
def test_an_epub_is_counted_from_the_documents_of_its_spine(tmp_path):
|
||||
chapter = "<html><head><title>T</title></head><body>" + "<p>x</p>" * 31 + "</body></html>"
|
||||
book = epub(tmp_path / "book.epub", [chapter, "<html><body><p>Not in the spine</p></body></html>", chapter], [2, 0])
|
||||
assert spine_documents(book) == [chapter, chapter]
|
||||
# 62 paragraphs of one character.
|
||||
assert pages.epub_pages(book) == 2
|
||||
assert pages.epub_pages(epub(tmp_path / "empty.epub", ["<html><body><img src='a.png'/></body></html>"])) == 0
|
||||
|
||||
|
||||
def test_a_file_that_is_not_an_epub_cannot_be_counted(tmp_path):
|
||||
path = tmp_path / "bad.epub"
|
||||
path.write_bytes(b"not a zip")
|
||||
with pytest.raises(EpubError):
|
||||
pages.epub_pages(path)
|
||||
with zipfile.ZipFile(path, "w") as z:
|
||||
z.writestr("META-INF/container.xml", "<container/>")
|
||||
with pytest.raises(EpubError):
|
||||
pages.epub_pages(path)
|
||||
Reference in New Issue
Block a user