Note schema 13, the new rotation semantics and the closures in the history
This commit is contained in:
1 parent
9be04341f4
commit
6e6157f95e
1 file changed
+28
-28
+28
-28
@@ -8,22 +8,22 @@ Short working handover for the next model. Keep it short.
|
||||
|
||||
## Status
|
||||
|
||||
- **Done and verified:** sections 1–31 and section 18 steps 1–11, plus **section 32.1** (the icon), **32.2** (the UI pass and the retro theme), **32.3** (the alert and page texts), **32.4** (the bug report, rounds 32-A and 32-B) and **32.7** (the top-bar menu and the dashboard clean-up), each with its follow-up round. Schema version 12. **513 green tests.** The StartOS package is built, sideloaded and running on a StartOS 0.4.0 box; the package version is still `0.1.0:0`.
|
||||
- **What is left:** **32.8 the seven points before the documentation round** (the checkboxes in `TODO.md` §32.8), then **32.6 the documentation drift**, and then **32.5 the raise to `1.0.0:0`** together with section 18 step 12. 32.8 comes first, 32.6 has to come after the rounds it documents, and 32.5 last. One item inside 32.4 is left to the user: **BR-003**, a manual end-to-end pass — it is now point 7 in `TODO.md` §32.8; read the note under 32.4 before acting on it, because the report's own wording for it is out of date. The user's manual pass over the finished §32 work is outstanding as well.
|
||||
- **Done and verified:** sections 1–31 and section 18 steps 1–11, plus **section 32.1** (the icon), **32.2** (the UI pass and the retro theme), **32.3** (the alert and page texts), **32.4** (the bug report, rounds 32-A and 32-B) and **32.7** (the top-bar menu and the dashboard clean-up), each with its follow-up round, and then **32.8 stages A, B and C** (the login lockout, the date limit, the sticky top bar, the package task, the new key row per rotation with the frozen history, and the security source taken from the row). Schema version 13. **526 green tests.** The StartOS package is built, sideloaded and running on a StartOS 0.4.0 box; the package version is still `0.1.0:0`.
|
||||
- **What is left:** **32.8 point 7**, the user's manual end-to-end pass (BR-003), which no agent can do; then **32.6 the documentation drift**, and then **32.5 the raise to `1.0.0:0`** together with section 18 step 12. 32.6 has to come after the rounds it documents, and 32.5 last. The user's manual pass over the finished §32 work is outstanding as well, and it is what point 7 is about: read the note under 32.4 before acting on it, because the report's own wording for it is out of date.
|
||||
- **A round stops when it is done.** Do not start the next item unless the user asks.
|
||||
- **Verified how:** the automated suite, the user's manual browser testing, and the install and test round on a StartOS 0.4.0 box. The user runs the suite; the agent has no shell.
|
||||
- **Tests:** `python3 -m unittest discover -s tests`. They run on a Windows copy at `C:\Users\<brukarnamn>\Downloads\helsesjekk`, so they must pass there as well as on Linux.
|
||||
- **Tests:** `python3 -m unittest discover -s tests`. They run on a Windows copy at `C:\Users\<brukarnamn>\Downloads\helsesjekk`, so they must pass there as well as on Linux. Stage C went from 519 to **526** tests, and the user confirms the number.
|
||||
- **Keep this note short.** The finished §32 sub-rounds are one short line each in `TODO.md` §32, and the reasoning is in the git history. A finished round does not need its narrative back, and the two documents should not be rewritten in full every round.
|
||||
|
||||
## Section 32 — what is left
|
||||
|
||||
- **32.8 The seven points before 32.6 and 32.5.** Listed with checkboxes in `TODO.md` §32.8, in three stages. **Stage A** is points 3, 4 and 2: no check date in the future, a top bar that sticks to the top, and a login lockout (five failures, fifteen minutes, the new `429`). **Stage B** is point 1: the package holds the service until the admin password is set, through a required StartOS task that runs `maintenance.py set-admin-password` — the password is still generated **inside** the container, and the web create-admin form stays as the local bootstrap. **Stage C** is points 6 and 5 together, because both need schema **13**: `ai_security_checks.key_id` so a security alert's source can never flip on a rename, software-sourced closures listed in the history and the CSV, and a rotation that inserts a **new `keys` row** with the same `key_number` plus a name/type/model snapshot on every check, so older history can never change. **Point 7** is the user's manual end-to-end pass (BR-003) and cannot be done by an agent. Stage C is the big one and is split from the rest for that reason: it is a schema change with a migration test, and it touches the storage layer, the alert rules, the export and the history.
|
||||
- **32.6 The documentation drift.** `README.md`, `docs/application.md`, `instructions.md`, `UPDATING.md`, `AGENTS.md` and the package's own i18n strings are all behind the code, and `TODO.md` §32.6 lists what is wrong in each. It answers the rest of BR-004, and it must be done before 32.5, so the documents describe the released state. In one line, what the documents still miss: the login and session endpoints and the gating/CSRF rule, the CSV formula guard, the canonical path comparison, the local secret guard, the two summary fields, the moved field reminder, the added `model` field in the check detail, the icon link and the two PNGs, the top-bar menu, the removed dashboard cards, the export section and its note on the history page, the `#history-detail` anchor, and the robots rule. After 32.8 the schema is 13 and the documents also miss the admin step and the package task, the `429` lockout, the date limit, the sticky top bar, the new key row per rotation with a frozen history, and software closures in the history and the CSV.
|
||||
- **32.8 The seven points before 32.6 and 32.5.** Listed with checkboxes in `TODO.md` §32.8, in three stages. **Stage A** is points 3, 4 and 2: no check date in the future, a top bar that sticks to the top, and a login lockout (five failures, fifteen minutes, the new `429`). **Stage B** is point 1: the package holds the service until the admin password is set, through a required StartOS task that runs `maintenance.py set-admin-password` — the password is still generated **inside** the container, and the web create-admin form stays as the local bootstrap. **Stage C** is points 6 and 5 together, because both need schema **13**: `ai_security_checks.key_id` so a security alert's source can never flip on a rename, software-sourced closures listed in the history and the CSV, and a rotation that inserts a **new `keys` row** with the same `key_number` plus a name/type/model snapshot on every check, so older history can never change. Stages A, B and C are done; only **point 7** is left, which is the user's manual end-to-end pass (BR-003) and cannot be done by an agent.
|
||||
- **32.6 The documentation drift.** `README.md`, `docs/application.md`, `instructions.md`, `UPDATING.md`, `AGENTS.md` and the package's own i18n strings are all behind the code, and `TODO.md` §32.6 lists what is wrong in each. It answers the rest of BR-004, and it must be done before 32.5, so the documents describe the released state. In one line, what the documents still miss: the login and session endpoints and the gating/CSRF rule, the CSV formula guard, the canonical path comparison, the local secret guard, the two summary fields, the moved field reminder, the added `model` field in the check detail, the icon link and the two PNGs, the top-bar menu, the removed dashboard cards, the export section and its note on the history page, the `#history-detail` anchor, and the robots rule. After 32.8 the schema is 13 and the documents also miss the admin step and the package task, the `429` lockout, the date limit, the sticky top bar, the new key row per rotation with a frozen history, the snapshot as the source of the name and the type on a check, and software closures in the history and the CSV.
|
||||
- **32.5 The version raise.** `0.1.0:0` → `1.0.0:0`, once 32.6 is done and the user says so. No schema change, no new endpoint, no new feature. `startos/versions/` stays empty, because the app migrates its own schema.
|
||||
|
||||
### BR-003 — the one open item in 32.4
|
||||
|
||||
The report says the manual end-to-end pass is still open, but it quotes a §31 text that no longer exists, and §16 and §17 record that the manual browser pass and the box tests were run and verified through §31. What is outstanding is the **new** manual work §32 added — the icon, the retro theme on a narrow phone, the alert texts, the top-bar menu, the cleaned dashboard, the export section on the history page, the PWA with the two PNG icons, the seven points in §32.8, and the whole box test again after 32.5 — plus the user's confirmation that it has been run. Do not re-verify what §16 and §17 already record as done. The other verdicts are settled: BR-001 is a false positive, BR-004 falls away, BR-005 is verified with no behaviour change, and BR-002, BR-006, BR-007 and BR-008 are fixed with regression tests. The report was written against a different file set and asks for eight documents that do not exist here; do not create them.
|
||||
The report says the manual end-to-end pass is still open, but it quotes a §31 text that no longer exists, and §16 and §17 record that the manual browser pass and the box tests were run and verified through §31. What is outstanding is the **new** manual work §32 added — the icon, the retro theme on a narrow phone, the alert texts, the top-bar menu, the cleaned dashboard, the export section on the history page, the PWA with the two PNG icons, the six finished points in §32.8, and the whole box test again after 32.5 — plus the user's confirmation that it has been run. Do not re-verify what §16 and §17 already record as done. The other verdicts are settled: BR-001 is a false positive, BR-004 falls away, BR-005 is verified with no behaviour change, and BR-002, BR-006, BR-007 and BR-008 are fixed with regression tests. The report was written against a different file set and asks for eight documents that do not exist here; do not create them.
|
||||
|
||||
## The wallet-secret guard (round 32-B)
|
||||
|
||||
@@ -42,13 +42,13 @@ The report says the manual end-to-end pass is still open, but it quotes a §31 t
|
||||
The rules are in `TODO.md` §7; these are the traps that have already cost time.
|
||||
|
||||
- **The acknowledgement stores the key.** `alert_acknowledgements.key_id` is nullable and holds the key the alert belonged to, or `NULL` for a software alert raised by the settings list. An alert with a key is acknowledged and hidden by `(code, key_id)`; an alert without one stays on `(code, source)`. This closed a real bug: a rename — or a `rotate` with `keep_name=false` — used to reopen a closed alert, because `source` carries the name the key has *now*.
|
||||
- **A key-sourced security alert carries `Key N: <name>`.** `build_alerts()` reads the key checks into two lookups — `key_ids` (name → id) and `key_sources` (name → `Key N: <name>`, built with `_key_source()`) — and `_security_alerts()` takes both. When the provider is the key's own configured name, the key is the source and the alert belongs to it, so it is closed by a rotation; otherwise the source is the plain software name and it is closed by removing that software from the settings. **Because the acknowledgement hides a key alert by `key_id`, not by `source`, an alert closed before this change stays closed** — `test_a_key_named_security_alert_is_hidden_by_its_key` pins it. `rotation_alert_targets()`, `software_alert_targets()` and `newest_alert_check_id()` all read the same `build_alerts()`, so they see the same source, which is what keeps an alert closable.
|
||||
- **A key-sourced security alert carries `Key N: <name>`, and the row decides.** `build_alerts()` reads the key checks into a lookup by key id, and `_security_alerts()` looks the row's own `key_id` up in it. When the row carries the checked key, the key is the source — the same `Key N: <name>` label every other key alert uses — and the alert belongs to it, so it is closed by a rotation; otherwise the source is the plain software name and it is closed by removing that software from the settings. **The provider name is never compared with the names that are configured now**, so a rename or a rotation cannot turn a key alert into a software one (section 32.8 point 6). `rotation_alert_targets()`, `software_alert_targets()` and `newest_alert_check_id()` all read the same `build_alerts()`, so they see the same source, which is what keeps an alert closable.
|
||||
- **The contact alert belongs to the checked key.** It gets `key_id` and `source` = `Key N: <name>`, and its message says "this key" and does not repeat the name. The severity stays `info` and the action stays `contact_updated`. It blocks its key like every other key alert, because `blocking_alert()` and the `blockingAlerts` filter in the form only look at `key_id`. A record without a key check raises no contact alert at all.
|
||||
- **`software_incompatible` has two forms.** A software from the settings list keeps `source` = the software name, `key_id` = `None`, actions `[remove_software]`, and carries `software`. A wallet software that is the name of a configured cloud key can never be removed from the settings, so that form belongs to the checked key instead: `key_id` = the checked key, `source` = `Key N: <name>`, actions `[replace_hardware]`, and **no** `software` value, so a settings removal can never close it. Only that form names the software in its message, because its source is the key.
|
||||
- **`alerts.py` needs the cloud key names.** `build_alerts()` and `collect_alerts()` take `cloud_names=()`, threaded through `rotation_alert_targets()`, `software_alert_targets()`, and the four call sites in `app.py` (`_send_alerts`, `_blocking_alert`, `_rotate_key`, `_close_removed_software`), plus `newest_alert_check_id()`. `app.cloud_key_names(keys)` builds the tuple. If `newest_alert_check_id()` does not get exactly the same names as `collect_alerts()`, the alert's `source` differs, the acknowledgement stores `check_id = None`, `_is_acknowledged()` refuses to hide an alert without a check id, and the alert can never be closed. Keep the default `()` on `build_alerts()`. `_send_health_check_list()` passes the same tuple for its `needs_follow_up` field, for the same reason. **Section 32.8 point 6 removes the name-matching half of this trap for security alerts**: once each `ai_security_checks` row carries its own `key_id`, the source no longer depends on the current names. The `cloud_names` threading still matters for `software_incompatible`.
|
||||
- **The closures that belong to a key are shown.** `app.closed_alerts()` and `database._grouped_closures()` both filter on `key_id IS NOT NULL`, so a rotation, a replaced hardware, an updated contact and an ignored key alert are listed, while `remove_software` and a software-sourced `ignore` are not. `GET /api/health-checks/<id>` returns them as `closed_alerts`, and the CSV has one last column, `Follow-up closed`, with `code: action (YYYY-MM-DD)`, hanging on the check it closed through `check_id`. The history table shows `Alert`, `Action` and `Closed` only — the source is not a column. **Section 32.8 point 6 removes that filter on purpose**, so software closures are listed too.
|
||||
- **`alerts.py` still needs the cloud key names, but only for `software_incompatible`.** `build_alerts()` and `collect_alerts()` take `cloud_names=()`, threaded through `rotation_alert_targets()`, `software_alert_targets()`, and the four call sites in `app.py` (`_send_alerts`, `_blocking_alert`, `_rotate_key`, `_close_removed_software`), plus `newest_alert_check_id()`. `app.cloud_key_names(keys)` builds the tuple. If `newest_alert_check_id()` does not get exactly the same names as `collect_alerts()`, the alert's `source` differs, the acknowledgement stores `check_id = None`, `_is_acknowledged()` refuses to hide an alert without a check id, and the alert can never be closed. Keep the default `()` on `build_alerts()`. `_send_health_check_list()` passes the same tuple for its `needs_follow_up` field, for the same reason. Section 32.8 point 6 removed the name-matching half of this trap for security alerts: each `ai_security_checks` row carries its own `key_id`, so the source no longer depends on the current names.
|
||||
- **Every closure of a check is shown, key-sourced or not.** `app.closed_alerts()` and `database._grouped_closures()` list every acknowledgement that carries a `check_id`, so a rotation, a replaced hardware, an updated contact, an ignored key alert, and a software alert closed by removing it are all shown; only an acknowledgement without a check id is left out. `GET /api/health-checks/<id>` returns them as `closed_alerts`, and the CSV has one last column, `Follow-up closed`, with `code: action (YYYY-MM-DD)`, hanging on the check it closed through `check_id`. The history table shows `Alert`, `Action` and `Closed` only — the source is not a column. The filter on `key_id IS NOT NULL` was removed in section 32.8 point 6, on purpose.
|
||||
- **The key card shows the next due date with the alert note.** `createKeyCard()` in `static/app.js` always renders `Next check due` and adds the alert note on top. `Last checked` was dropped in 32.2, but the note still comes in addition to the date, never instead of it — that is what cost the dates once before, for a key whose only alert was the informational contact one.
|
||||
- **The type is part of a key's identity.** The setup locks it, the settings never change it, and only a rotation may pick the other one: `POST /api/keys/<id>/rotate` takes an optional `type`, and leaving it out keeps the current type. `save_key_settings(..., key_type=None)` uses `COALESCE(?, type)`, so the settings and a rotation share one statement. **Section 32.8 point 5 changes what a rotation does**: instead of updating the row in place, it inserts a new row for the same `key_number` and retires the old one, so the statement above becomes an insert plus a retire.
|
||||
- **The type is part of a key's identity.** The setup locks it, the settings never change it, and only a rotation may pick the other one: `POST /api/keys/<id>/rotate` takes an optional `type`, and leaving it out keeps the current type. `save_key_settings(..., key_type=None)` uses `COALESCE(?, type)`, so the settings and a rotation share one statement — but since section 32.8 point 5 a rotation does **not** use it: `rotate_key()` inserts a new row for the same `key_number` and retires the old one, so the settings write the active row in place and the rotation does not.
|
||||
- **A key that becomes a cloud key must not take a taken name.** A cloud key offers its own name as a wallet software choice, and the settings list holds the other choices, so a name the list already holds is refused with `409` through `WalletSoftwareNameInUse` — a `ValidationError` subclass, so it has to be caught before the general `ValidationError` in `_rotate_key`, or the `409` turns into a `400`. `validate_rotation()` therefore takes the wallet software list as a fourth argument.
|
||||
- **A rename needs no rotation and no confirmation.** In the rotation section the name can be changed on its own through `POST /api/settings/keys`, for any key, and the address path of a key with **no** saved check can be changed there the same way. Marking a key with `Rotate this key` and ticking the confirmation box sends it through the rotate endpoint instead, together with the type and the new path. The single button is `Save keys`.
|
||||
|
||||
@@ -76,43 +76,43 @@ The rules are in `TODO.md` §7; these are the traps that have already cost time.
|
||||
- **The navigation lives in the top bar, not on the page.** `static/session.js` builds the menu, the panel and the `.topbar-actions` wrapper on the five application pages; `/login` has none. The dashboard markup therefore carries no `href` for `/health-check`, `/settings`, `/history` or `/api/export`, and the export link and its note are on the history page. The page the user is already on is a `<span>` with `aria-current="page"`, not a link. Do not add a navigation link back to the dashboard markup: `tests/test_app.py`'s `test_the_dashboard_carries_no_navigation_link_of_its_own` pins it.
|
||||
- **The Attention card is the only summary card.** Its detail line carries the overdue list when something is overdue and the earliest due date otherwise; `nextCheckTitle` and `nextCheckDetail` no longer exist, and `.overview-grid` is one column at every width.
|
||||
- **An empty alert list renders nothing.** The count beside the heading says `No alerts`; do not put placeholder text back into `#alerts-list`.
|
||||
- **The two summary fields are additive and mean what they say.** `key_names` follows `key_numbers`; `needs_follow_up` is a property of the saved check, so it does not change when an alert is closed or a key is renamed. Do not "fix" it to follow `collect_alerts()`, which hides acknowledged alerts and would rewrite history. **Section 32.8 point 5 changes where `key_names` comes from**: with the snapshot on the check, it carries the name recorded on that check, not the name the active key has now.
|
||||
- **The two summary fields are additive and mean what they say.** `key_names` follows `key_numbers` and carries the name the check was recorded with, read from the snapshot on the check, so a rotation or a rename cannot rewrite an older summary; `needs_follow_up` is a property of the saved check, so it does not change when an alert is closed or a key is renamed. Do not "fix" it to follow `collect_alerts()`, which hides acknowledged alerts and would rewrite history.
|
||||
- **The prohibition lists** in sections 1 and 13 of `TODO.md` stay unticked. They are standing prohibitions, to be re-confirmed before release — with the one amendment for the installable PWA.
|
||||
- **Whether a date is overdue is decided in the browser**, so `/api/key-status` stays independent of the current date.
|
||||
- **`/api/previous-values` still returns a `security` map** that nothing consumes any more, and its `address_path` per key is no longer used by the form, which pre-fills the path from `keys.address_path`. Both are deliberate: removing them would shrink the endpoint, but nothing breaks while they stay.
|
||||
- **`delete_all_data()` deletes the checks before the keys.** `key_checks.key_id` is `ON DELETE RESTRICT`, so the other order fails. The acknowledgements are deleted explicitly, because they have no foreign key at all. **Section 32.8 point 5 keeps this order**, and the delete has to clear every row of `keys`, active and retired alike.
|
||||
- **`delete_all_data()` deletes the checks before the keys.** `key_checks.key_id` is `ON DELETE RESTRICT`, so the other order fails. The acknowledgements are deleted explicitly, because they have no foreign key at all. The order is unchanged by section 32.8 point 5, and the delete has to clear every row of `keys`, active and retired alike.
|
||||
|
||||
## Data model (schema version 12)
|
||||
## Data model (schema version 13)
|
||||
|
||||
- `keys`: `key_number` (1–`multisig_key_count`), `name`, `type` (`hardware`/`cloud`), `model`, `address_path`, `created_at`. There is no `device` column. The name is editable in the settings; the type and the address path are the identity of the key, the path must be unique across the rows, and the type only changes through a rotation. `model` is optional documentation and is not locked.
|
||||
- `keys`: `key_number` (1–`multisig_key_count`), `name`, `type` (`hardware`/`cloud`), `model`, `address_path`, `active`, `rotated_at`, `created_at`. There is no `device` column. The name is editable in the settings; the type and the address path are the identity of the key, the path must be unique across the **active** rows, and the type only changes through a rotation. `model` is optional documentation and is not locked. `active` is `1` on the row the app reads as "the key", and `rotated_at` records when a row was retired.
|
||||
- `health_checks`: `checked_at`, `next_check_due`, `wallet_software`, `software_version`, `software_compatibility_status`, `software_comments`, `contact_info_verified`, `comments`, plus the legacy `sparrow_version` / `nunchuk_version`. `wallet_software` and `contact_info_verified` are free text.
|
||||
- `key_checks`: one row per submission — `available`, `hardware_works`, `firmware_version`, `seed_backup_verified`, `address_path_verified`, `address_path`, `comments`. The two cloud columns are gone.
|
||||
- `ai_security_checks`: `provider` (free text), `status`, `comments`, with `UNIQUE (health_check_id, provider)`.
|
||||
- `key_checks`: one row per submission — `key_id`, `key_name`, `key_type`, `key_model`, `available`, `hardware_works`, `firmware_version`, `seed_backup_verified`, `address_path_verified`, `address_path`, `comments`. The two cloud columns are gone. The three snapshot columns are a frozen copy of the key the check was recorded for.
|
||||
- `ai_security_checks`: `provider` (free text), `key_id` (nullable), `status`, `comments`, with `UNIQUE (health_check_id, provider)`. `key_id` is the checked key when the provider is that key's own configured name, and `NULL` otherwise.
|
||||
- `wallet_software`: `name`, `created_at`. Stored whole, seeded with `DEFAULT_WALLET_SOFTWARE` (`Sparrow Wallet`) when the list is empty, never emptied through the API. `delete_all_data()` empties it, and the seed puts the default back.
|
||||
- `alert_acknowledgements`: `code`, `source`, `key_id`, `check_id`, `reason`, `acknowledged_at`. `reason` holds the action name that closed the alert, and `key_id` holds the key the alert belonged to, or `NULL` for a software alert from the settings list.
|
||||
- `app_settings`: `name` (PRIMARY KEY), `value`. Holds `multisig_threshold`, `multisig_key_count`, `admin_username`, `admin_password_hash` and `session_secret`. `delete_all_data()` removes the two multisig rows, so the 2-of-3 default applies again, and keeps the three credential rows. **Section 32.8 point 2 adds two more rows here**: `failed_login_count` and `failed_login_until`.
|
||||
- `app_settings`: `name` (PRIMARY KEY), `value`. Holds `multisig_threshold`, `multisig_key_count`, `admin_username`, `admin_password_hash`, `session_secret`, `failed_login_count` and `failed_login_until`. `delete_all_data()` removes the two multisig rows, so the 2-of-3 default applies again, and keeps the credential rows and the two lockout rows.
|
||||
- `tool_checks` no longer exists.
|
||||
- Startup migration rebuilds `keys` when it still has `device` or the old `key_number` CHECK, `key_checks` when it uses the old layout or still holds the two cloud columns, and `health_checks` and `key_checks` when their `CHECK` constraints still allow the old values. It then drops `tool_checks` if it is there, and adds the newer columns through `OPTIONAL_COLUMNS`. **Section 32.8 point 5 and 6 add schema 13**: the `UNIQUE` on `keys.key_number` becomes a partial unique index over the active row, `keys` gets an active flag and a `rotated_at`, `key_checks` gets the name/type/model snapshot, and `ai_security_checks` gets a nullable `key_id`.
|
||||
- Startup migration rebuilds `keys` when it still has `device`, the old `key_number` CHECK, or no `active` flag; `key_checks` when it uses the old layout, still holds the two cloud columns, or still allows the old values; and `health_checks` when its `CHECK` constraints still allow the old values. It then drops `tool_checks` if it is there, adds the newer columns through `OPTIONAL_COLUMNS`, backfills the snapshot on every check from the key it belongs to, and creates the partial unique index over the active row. **Schema 13 is done (section 32.8 points 5 and 6):** the `UNIQUE` on `keys.key_number` is a partial unique index over the active row (`idx_keys_active_key_number … WHERE active = 1`), `keys` carries `active` and `rotated_at`, `key_checks` carries the name/type/model snapshot, and `ai_security_checks` carries a nullable `key_id`. A row written before the flag is active, and a security row written before the column keeps a `NULL` key.
|
||||
- The table definitions are `{table}` templates, so a rebuild can create the same table under a temporary name. `health_checks` is a parent table, so the whole migration runs with `PRAGMA foreign_keys = OFF`, checks `PRAGMA foreign_key_check` before committing, and rolls back on any failure.
|
||||
- `tests/test_database.py` carries `LEGACY_SCHEMA`, `SCHEMA_VERSION_5`, `SCHEMA_VERSION_8`, `SCHEMA_VERSION_9`, `SCHEMA_VERSION_10` and `SCHEMA_VERSION_11` as literal older schemas. Copy that pattern for a new migration test: write the old schema, run `initialize_database()`, then assert on the migrated rows. The schema-13 migration needs a literal `SCHEMA_VERSION_12` beside them.
|
||||
- `tests/test_database.py` carries `LEGACY_SCHEMA`, `SCHEMA_VERSION_5`, `SCHEMA_VERSION_8`, `SCHEMA_VERSION_9`, `SCHEMA_VERSION_10`, `SCHEMA_VERSION_11` and `SCHEMA_VERSION_12` as literal older schemas. Copy that pattern for a new migration test: write the old schema, run `initialize_database()`, then assert on the migrated rows. The schema-13 migration has its own test there, with the literal `SCHEMA_VERSION_12`, covering the snapshot backfill and the partial index.
|
||||
- There is **no data migration** for an older test database: the user deletes the `data` folder and configures the setup again. An acknowledgement written before version 11 has no key, and the older contact alert used the source `Contact information`, which no longer matches, so the alert it closed is raised again. Section 29 renamed the database file for the same reason, and the old file is discarded with the rest.
|
||||
|
||||
## API
|
||||
|
||||
Pages: `/`, `/setup`, `/settings`, `/health-check`, `/history`, `/login`. Endpoints: `GET /health`, `GET /api/session`, `POST /api/admin`, `POST /api/login`, `POST /api/logout`, `GET /api/setup`, `POST /api/setup`, `GET /api/keys`, `GET /api/key-status`, `GET /api/previous-values`, `GET /api/settings`, `POST /api/settings/keys`, `POST /api/settings/wallet-software`, `POST /api/keys/<id>/rotate`, `GET /api/health-checks`, `GET /api/health-checks/<id>`, `GET /api/alerts`, `POST /api/alerts/acknowledge`, `GET /api/export`, `POST /api/health-checks`. Field details are in `docs/application.md` — which is behind the code, see the gap under Status.
|
||||
|
||||
`GET /api/health-checks` returns each summary with `id`, `checked_at`, `next_check_due`, `software_compatibility_status`, `wallet_software`, `contact_info_verified`, `comments`, `key_numbers`, and the two added in 32.2: `key_names` and `needs_follow_up`.
|
||||
`GET /api/health-checks` returns each summary with `id`, `checked_at`, `next_check_due`, `software_compatibility_status`, `wallet_software`, `contact_info_verified`, `comments`, `key_numbers`, and the two added in 32.2: `key_names` (the name the check was recorded with, read from the snapshot) and `needs_follow_up`.
|
||||
|
||||
Maintenance commands, run inside the container by the StartOS actions and usable by hand: `maintenance.py delete-all-data` and `maintenance.py set-admin-password`. Neither is an HTTP endpoint.
|
||||
|
||||
The rotate payload is `replaced` + `keep_name` + `name` + `type` + `address_path`, and `type` is optional. `GET /api/health-checks/<id>` also returns `closed_alerts`, and each of its `key_checks` rows carries `model` as well — the field was added in the follow-up round of 32.1 so the history can show the model and the firmware version on one line, and the fields that were already there keep their names. The settings page has two sections, and `POST /api/settings/keys` is what a rename alone goes through. The `#history-detail` anchor and the `?id=` query on `/history` are client-side only: they never reach the server, so no route changed for them.
|
||||
The rotate payload is `replaced` + `keep_name` + `name` + `type` + `address_path`, and `type` is optional. A rotation inserts a new row and retires the old one, and the acknowledgements are written against the **old** key id, which is the row the alerts and their checks belong to. `GET /api/health-checks/<id>` also returns `closed_alerts` (every closure of the check), and each of its `key_checks` rows carries `model` as well — the field was added in the follow-up round of 32.1 so the history can show the model and the firmware version on one line, and the fields that were already there keep their names. The settings page has two sections, and `POST /api/settings/keys` is what a rename alone goes through. The `#history-detail` anchor and the `?id=` query on `/history` are client-side only: they never reach the server, so no route changed for them.
|
||||
|
||||
Five things that are easy to break:
|
||||
|
||||
- New static files must be added to `STATIC_FILES` in `app.py`, or they return `404`. That includes `/settings`, `/settings.js`, `/login`, `/login.js`, `/session.js`, `/sw.js`, `/manifest.webmanifest`, `/icon.svg`, the two raster icons `/icon-192.png` and `/icon-512.png`, and `/robots.txt`, which is `text/plain; charset=utf-8` and is deliberately absent from the service worker's shell list.
|
||||
- `/api/setup`, `/api/settings`, `/api/alerts`, `/api/key-status`, `/api/previous-values` and `/api/session` are extra endpoints that are not in the minimum list in section 11.
|
||||
- `STORAGE_ERRORS` in `app.py` includes `RuntimeError`, because `list_keys()` raises it for a partial setup. Keep it there, or a broken database answers with a traceback instead of a `500`. It also includes `ValueError`, so `validate_admin_credentials()` has to be called in its own `try` and mapped to a `400` before the storage handler — and it is why a refused secret has to be a `ValidationError` rather than a bare `ValueError`.
|
||||
- The settings, rotation and acknowledgement endpoints keep the same error handling as the existing ones: a bad payload is a `400`, an unknown key or alert is a `404`, a locked or duplicate address path is a `409`, and a broken database is a `500`. `DuplicateAddressPath` and `WalletSoftwareNameInUse` must be caught before the general `ValidationError` — in `_save_key_settings` and `_rotate_key` respectively — or the `409` turns into a `400`. A missing or invalid session is a `401`, a missing CSRF token is a `403`, and a locked login is the new `429` from §32.8 point 2.
|
||||
- The settings, rotation and acknowledgement endpoints keep the same error handling as the existing ones: a bad payload is a `400`, an unknown key or alert is a `404`, a locked or duplicate address path is a `409`, and a broken database is a `500`. `DuplicateAddressPath` and `WalletSoftwareNameInUse` must be caught before the general `ValidationError` — in `_save_key_settings` and `_rotate_key` respectively — or the `409` turns into a `400`. A missing or invalid session is a `401`, a missing CSRF token is a `403`, and a locked login is the `429` from §32.8 point 2.
|
||||
- `newest_alert_check_id()` in `app.py` derives the check id for the acknowledgement, the rotation and the software removal, so the same comparison rule is used in all three places. `blocking_alert()` is the module-level helper the `409` is built on, and `_blocking_alert()` reads the key id straight from the payload, so the rule applies before validation.
|
||||
|
||||
## The StartOS package — the parts that bite
|
||||
@@ -122,10 +122,10 @@ The package is one service: one daemon, one container, one SQLite database on th
|
||||
- **The application and the package share one repository, at the root.** A deliberate deviation from the guide's "a package repo is not a fork of the application": the rule assumes a published upstream image or a git submodule, and VardKey has neither. `AGENTS.md` states the deviation so nobody adds an `upstream-project/` submodule to "fix" it.
|
||||
- **Anything the `Dockerfile` does not `COPY` is not in the image.** The modules go in with `COPY *.py ./` and the assets with `COPY static ./static`; `.dockerignore` keeps `tests`, `data`, `node_modules`, `javascript` and the build output out of the context. `static/` must be copied: `app.py` answers `/health` from code, but every page is read from `ROOT/static`, so an image without it reports a healthy service whose whole UI answers `404`. `static/robots.txt` is served the same way, so it is in the image too.
|
||||
- **Three things must agree.** `startos/utils.ts` holds `uiPort` (8080); `startos/main.ts` passes it as `PORT` and uses it in the readiness URL; `startos/interfaces.ts` binds it. Change one alone and the readiness check never goes green. `DATABASE_PATH` in the same file is what puts the database on the volume. The readiness check is `checkWebUrl` against `http://127.0.0.1:8080/health`, and `/health` must stay open, or StartOS reports the service as unhealthy. A red `ECONNREFUSED 127.0.0.1:8080` blip before `VardKey listening on port 8080` is just the first poll landing before Python has bound the socket; nothing to fix.
|
||||
- **Two actions**, registered through `Actions.of()` in `startos/actions/index.ts`, with their strings in `startos/i18n/dictionaries/`: `deleteAllData.ts` (`delete-all-data`) and `setAdminPassword.ts` (`set-admin-password`). Both run `maintenance.py` in a `withTemp` subcontainer, behind a confirmation the handler enforces. The generated password never travels into the container: `maintenance.py` generates it there and prints it, which is the packaging guide's own password-reset pattern. **Section 32.8 point 1 adds a required task** that holds the service until the admin password exists, and it has to keep that same property: no password may travel from the host into the container.
|
||||
- **Two actions**, registered through `Actions.of()` in `startos/actions/index.ts`, with their strings in `startos/i18n/dictionaries/`: `deleteAllData.ts` (`delete-all-data`) and `setAdminPassword.ts` (`set-admin-password`). Both run `maintenance.py` in a `withTemp` subcontainer, behind a confirmation the handler enforces. The generated password never travels into the container: `maintenance.py` generates it there and prints it, which is the packaging guide's own password-reset pattern. **Section 32.8 point 1 adds a required task** that holds the service until the admin password exists, in `startos/init/requireAdmin.ts`, and it keeps that same property: no password travels from the host into the container.
|
||||
- **Backups.** `startos/backups.ts` is `sdk.Backups.ofVolumes('main')`. StartOS stops the service for the duration of a backup, so the database file is quiescent while it is copied — no snapshot hook is needed, and it is not a known limitation.
|
||||
- **No CI.** The guide assumes GitHub Actions; this repository is on Gitea, so there are no `.github/workflows/` and there must not be any. The `.s9pk` is built locally with `make`.
|
||||
- **The skeleton** was written to match the Start9 template rather than generated by `s9pk init-workspace`, because that command was not available where it was written. The successful build and install confirm they are equivalent; `@start9labs/start-sdk` is pinned to `3.0.4`. The task API used by §32.8 point 1 has to be verified against that pinned version, because `node_modules/` is not in the repository.
|
||||
- **The skeleton** was written to match the Start9 template rather than generated by `s9pk init-workspace`, because that command was not available where it was written. The successful build and install confirm they are equivalent; `@start9labs/start-sdk` is pinned to `3.0.4`. The task API used by §32.8 point 1 was verified against that pinned version (`sdk.action.createOwnTask` / `createTask` / `sdk.action.clearTask`, `TaskSeverity`, and `createTask` answering `null`), because `node_modules/` is not in the repository.
|
||||
- **The root `icon.svg` is the package artwork**, read from the repository root by the build. It is the same retro drawing as `static/icon.svg`, and it must stay under 40 KiB.
|
||||
- **Documentation layout.** `README.md` is the StartOS package README, with the guide's fixed heading set. The application's own reference is `docs/application.md`, which `instructions.md` links to. `AGENTS.md` is the repo-specific agent context, and `CLAUDE.md` is the one-line `@AGENTS.md` import. Both `README.md` and `docs/application.md` are behind the code — that is 32.6.
|
||||
|
||||
@@ -159,11 +159,11 @@ The package is one service: one daemon, one container, one SQLite database on th
|
||||
- **A static help line that the script overwrites is dead text.** `#software-help` in `health-check.html` was emptied in round 32.3, because `lockSoftwareFields()` fills it before the user can read it. `static/health-check.js` must keep setting it — the wording depends on the selected key. A test pins both halves: the empty element in the markup and `softwareHelp.textContent` in the script.
|
||||
- **`aria-labelledby` may point at the eyebrow.** `login.html`'s first form has no `h2` any more; its `aria-labelledby="login-form-title"` points at the eyebrow `<p>` that carries the same `id`. Do not add a heading back just to have one, and do not drop the `id`.
|
||||
- **Do not assert on the quoting style of the JavaScript.** `static/health-check.js` uses single quotes for constant selectors and template literals where a `${...}` substitution is needed, and `static/settings.js` uses double quotes. Assert on the selector text, e.g. `[name="address_path"]`, and on quote-free substrings such as `/api/settings` and `/settings?rotate=`.
|
||||
- **`clean_record()` in `tests/test_alerts.py` has a fixed provider list and carries `key_id` and `wallet_software`.** A test that needs another provider must replace the entry itself, and the recorded software is `Sparrow Wallet`, which is what a software alert names. A test that exercises the cloud-key form of `software_incompatible` has to pass `cloud_names` to `build_alerts()`. The alert texts are asserted there, and **not** in `tests/test_app.py`, which checks the code, the severity, the source, the actions and the `key_id` only — so a message change touches `tests/test_alerts.py` alone.
|
||||
- **`clean_record()` in `tests/test_alerts.py` has a fixed provider list and carries the key on the security rows.** Each `ai_security_checks` entry is a dict with `provider`, `key_id` and `status`: the entry whose provider is the key's own name carries the key number, and the others carry `None`, which is what the storage writes. A test that needs another provider must replace the entry itself, and it has to keep the `key_id` it wants, or the alert's source changes. The recorded software is `Sparrow Wallet`, which is what a software alert names. A test that exercises the cloud-key form of `software_incompatible` has to pass `cloud_names` to `build_alerts()`. The alert texts are asserted there, and **not** in `tests/test_app.py`, which checks the code, the severity, the source, the actions and the `key_id` only — so a message change touches `tests/test_alerts.py` alone.
|
||||
- **`sample_payload()` / `unavailable_payload()` are duplicated in `tests/test_validation.py`, `tests/test_database.py`, `tests/test_app.py`, `tests/test_maintenance.py` and `tests/test_secret_guard.py`.** They build the labels from the key's name and the selected software, they always send `contact_info_verified`, and for a hardware key `hardware_works`. They must send the key's **own** `address_path`, because the server compares it with the stored one.
|
||||
- **Every test fixture uses one address path per key** (`ADDRESS_PATH`, `SECOND_ADDRESS_PATH`, `THIRD_ADDRESS_PATH`), because the setup refuses duplicates. `tests/test_app.py` also has `ROTATION_PATHS` and the `rotate()` helper, which picks an unused path per key so a test can rotate several keys in one run. With schema 13 a rotation retires the key row, so a test that rotates and then records a check has to use the **new** key id.
|
||||
- **Every test fixture uses one address path per key** (`ADDRESS_PATH`, `SECOND_ADDRESS_PATH`, `THIRD_ADDRESS_PATH`), because the setup refuses duplicates. `tests/test_app.py` also has `ROTATION_PATHS` and the `rotate()` helper, which picks an unused path per key so a test can rotate several keys in one run. With schema 13 a rotation retires the key row, so a test that rotates and then records a check has to use the **new** key id — `rotate()` hands it back under `key_id`, read from the settings in its own answer.
|
||||
- **`validate_health_check()` takes a third argument** (the wallet software list), and `validate_rotation()` takes a fourth for the same reason. `tests/test_validation.py` and `tests/test_database.py` wrap the first one in a small `validate()` helper.
|
||||
- **`tests/test_app.py` has seven test classes.** `ServerTestCase` starts the server and keeps the session and CSRF cookies a sign-in returns; `AppHandlerTests`, `RenamedSetupTests`, `SettingsApiTests`, `RotationApiTests` and `AcknowledgementApiTests` store the default or renamed setup and call `authenticate()`; `AuthApiTests` starts from a database with **no admin** and covers the gating, the first-run admin, the login and logout, the CSRF check, the robots header on a redirect and — after §32.8 point 2 — the login lockout; and `KeySetupApiTests` creates its admin through `POST /api/admin` and otherwise starts from an empty database. A new test must go in the class whose setup it needs. Every request helper sends the cookies and the CSRF header, and `assert_status()` is the one place an error status is expected.
|
||||
- **`tests/test_app.py` has seven test classes.** `ServerTestCase` starts the server and keeps the session and CSRF cookies a sign-in returns; `AppHandlerTests`, `RenamedSetupTests`, `SettingsApiTests`, `RotationApiTests` and `AcknowledgementApiTests` store the default or renamed setup and call `authenticate()`; `AuthApiTests` starts from a database with **no admin** and covers the gating, the first-run admin, the login and logout, the CSRF check, the robots header on a redirect and the login lockout; and `KeySetupApiTests` creates its admin through `POST /api/admin` and otherwise starts from an empty database. A new test must go in the class whose setup it needs. Every request helper sends the cookies and the CSRF header, and `assert_status()` is the one place an error status is expected.
|
||||
- **The other server test modules share `tests/server_session.py`.** `test_export_filename.py`, `test_key_model.py`, `test_key_status.py`, `test_multisig_setup.py`, `test_pwa.py` and `test_secret_guard.py` mix in `SessionMixin`, which signs in through the API and carries the cookies; each class calls `authenticate()` at the end of its own `setUp`. **A test that switches `DATABASE_PATH` to another file has to call `authenticate()` again**, because the session secret lives in the database, so the old cookie is signed with the wrong secret and every request answers `401`. The mixin resolves the path with `app.get_database_path()`, so the admin lands in the file the server actually reads.
|
||||
- **A login lockout test has to clear the counter.** `AuthApiTests` shares one database per test, and the lockout lives in `app_settings`, so a test that trips it leaves the account locked for the tests that follow unless it resets the two rows or uses its own database file.
|
||||
- **The export link and its note live on the history page, and four tests moved with them.** `tests/test_export_filename.py` reads `static/history.html` and asserts the dashboard has no `/api/export` link; `tests/test_app.py`'s dashboard test became `test_the_dashboard_carries_no_navigation_link_of_its_own`; `test_the_history_page_offers_the_export_in_its_own_section` in `tests/test_health_check_form.py` pins the `Export` section, the title bar, the note under the button and the absent `.page-heading > .secondary-button` rule; and `test_the_history_links_to_the_detail_section` pins both anchors. A future round that adds a link to the dashboard has to update these, not delete them.
|
||||
@@ -173,4 +173,4 @@ The package is one service: one daemon, one container, one SQLite database on th
|
||||
|
||||
## Product clarification
|
||||
|
||||
The key's configured name is its product label, and a cloud key's name is also a wallet-software choice. A hardware key name is never a wallet-software choice, and a cloud key records its own name as its software. A key is identified by its type and its address path, while the name is only a label that the user may change. The address path is unique per key, and a rotation replaces the key with a new one that must carry a new address path and may carry the other type. A rotation creates a new wallet, so the funds have to be moved from the old multisig setup to the new one before the old keys are discarded. An alert is closed by the user with the action that belongs to it, and a key with an open alert cannot be checked until the alert is closed. **After §32.8 point 5, a rotation also retires the old key row**: the multisig slot number stays, the old checks keep the name and type they were recorded with, and the new row starts with an empty history.
|
||||
The key's configured name is its product label, and a cloud key's name is also a wallet-software choice. A hardware key name is never a wallet-software choice, and a cloud key records its own name as its software. A key is identified by its type and its address path, while the name is only a label that the user may change. The address path is unique per key, and a rotation replaces the key with a new one that must carry a new address path and may carry the other type. A rotation creates a new wallet, so the funds have to be moved from the old multisig setup to the new one before the old keys are discarded. An alert is closed by the user with the action that belongs to it, and a key with an open alert cannot be checked until the alert is closed. **Since §32.8 point 5, a rotation also retires the old key row**: the multisig slot number stays, the old checks keep the name and type they were recorded with, and the new row starts with an empty history.
|
||||
Reference in new issue
Block a user