Files
skuld/docs/TODO.md
T

354 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Skuld — work plan
Read `docs/spec.md` first. It is the contract; this file is the order of work. If this plan and
the spec disagree, the spec wins and the disagreement is a bug in this file.
## Goal
A StartOS service package named **Skuld**: a self-hosted web app that keeps a Bitcoin ledger and
computes the Norwegian tax figures for a tax year (formue, gevinst, tap, inntekt). Python
standard library, SQLite on the `main` volume, admin-password protected, UI in Norwegian
Nynorsk. After the last step the package builds, installs on StartOS, and produces the same
figures as the owner's spreadsheet.
## Not in this plan
Everything in `docs/spec.md` §11. Specifically: no other assets than BTC, no price feeds or
exchange import, no multi-user, no database encryption, no charts, no NOSTR, no PHP version.
Do not add abstraction, configuration or dependencies for any of them. The PHP version (§12) is
a later project; the only concession to it is that the DDL lives in `schema.sql`.
## Rules for every step
- Code, identifiers, comments and commit messages in English. User-facing text in Nynorsk.
- One step, then the repository is consistent: it imports, the tests pass, and nothing is left
half-written. Never leave a page without its script or a module without its caller.
- Commit at the end of each step, message in English, one line about what changed.
- Run the step's verification before moving on. If it fails, fix it in the same step.
- Do not reorder. Phase B2 must be finished before Phase C starts.
- When the spec is silent, choose the smaller option and record the choice in the commit
message. Ask the owner only for the four open questions in the spec (§13).
## Decisions and why
| # | Decision | Reason |
| --- | --- | --- |
| D1 | Python standard library + SQLite, same shape as the owner's VardKey app | No dependencies to install or patch, one file to back up, and the logic stays plain SQL and arithmetic so the PHP port is mechanical |
| D2 | Dates as `TEXT 'YYYY-MM-DD'` | Sorts and compares correctly in SQLite, no timezone handling anywhere |
| D3 | Money as integer øre, BTC as integer sats | No floating point in the ledger, so no rounding drift in cost bases |
| D4 | Lot-based ledger, explicit lot picking plus FIFO/LIFO/HIFO | The owner asked for UTXO-like picking; Norway has no FIFO requirement, so both must exist |
| D5 | `ledger.rebuild()` recomputes lots and auto consumptions on every write | Simplest model that cannot be corrupted by an edit or a delete, and fast enough for thousands of transactions |
| D6 | `schema.sql` at the repository root is the only DDL source, applied by `database.py` | One place to change, and phase 2 can apply the same file |
| D7 | Server returns JSON only; every page is a static file under `static/`; no JS framework, no build step | Same pattern as VardKey, and it is the pattern that ports to PHP unchanged |
| D8 | Prices entered by hand, seeded with Skatteetaten's published year-end BTC rates | A StartOS service should not need an external price API, and the owner already enters values by hand |
| D9 | GPL-3.0 | The auth pattern and the packaging shape are adapted from VardKey, which is GPL-3.0 |
| D10 | Wealth tax off by default, threshold and rate configurable | Thresholds change every year; a hardcoded number would be wrong within a year |
| D11 | A transfer consumes lots at the source and mirrors them at the destination, and generates the `transfer_in` row | The owner's spec requires the auto-generated row; mirroring is what carries the acquisition date and the cost basis |
| D12 | Non-taxable accounts are real accounts with `taxable = 0`, plus `custody_market_id` for the `TOT` rows | Reproduces the spreadsheet's main sheet, including the reconciliation rows |
| D13 | No secret-scanning module, unlike VardKey | Skuld has no field where a seed phrase or a key would belong; a guard would be decoration |
| D14 | The 22 % rate is a setting, not a constant | Rates change; the report shows which rate produced the figures |
| D15 | BTC is shown in sats everywhere, in the UI and in the CSV; the only `10^8` conversion is inside the price and formue arithmetic | The owner reads sats; a BTC amount with eight decimals is unreadable in a sheet |
| D16 | NOK is displayed in whole kroner, stored and computed in øre; decimals are accepted only in input fields | The owner does not want to see decimals, but rounding in the ledger would lose øre and let sums drift |
| D17 | `admin_cost` has no account (`market_id` null), takes no fee, and consumes no lots | The owner wants it to stand on its own; it changes no holdings, so nothing else can be derived from it |
| D18 | The transaction price is derived on the fly and never stored, and is called `btc_price_ore` in the API on both transactions and lots | It is a function of legs the owner already entered, so storing it would let it drift; the owner's word for it is "bitcoin pris", so the API uses one name for one unit |
| D19 | The stored rate column keeps its name `prices.nok_per_btc_ore`, and the settings table keeps the heading `NOK per BTC` | It is Skatteetaten's year-end rate, not a transaction price; renaming a stored column buys a migration and nothing else |
| D20 | The Avgift section is hidden for `gift_received` and `admin_cost` | The owner's field list gives those two no fee, and the schema now refuses one |
## Phase A — application core (steps 1-16) — DONE
Implemented and committed: `LICENSE`, `.gitignore`, `schema.sql`, `database.py`, `rates.py`,
`auth.py`, `validation.py`, `ledger.py`, `tax.py`, `export.py`, `maintenance.py`, `app.py` (in
three steps) and `tests/test_ledger.py`, `tests/test_tax.py`.
`python3 -m unittest discover -s tests -v` is green.
## Phase B — user interface (steps 17-26) — DONE
Implemented and committed: `static/style.css`, `static/session.js`, `static/app.js`, and the
page pairs `login.*`, `index.*`, `transactions.*`, `transaction.*`, `markets.*`, `report.*` and
`settings.*`.
## Phase B2 — corrections the owner asked for (steps 27-44)
Do these before Phase C. They change the ledger schema, the field set of one transaction type,
the way amounts are shown, and add one derived figure. Nothing here is optional, and nothing
here may spill into Phase C.
### 27. `schema.sql`
Change `transactions.market_id` from `INTEGER NOT NULL REFERENCES markets(id)` to
`INTEGER REFERENCES markets(id)`, with a comment that it is null only for `admin_cost`. Add the
two CHECK constraints of `docs/spec.md` §4: `(type = 'admin_cost' OR market_id IS NOT NULL)` and
`(type NOT IN ('gift_received', 'admin_cost') OR (fee_sats = 0 AND fee_ore = 0))`. `lots` keeps
its `NOT NULL`.
**Verify:** on a fresh database, an `INSERT` of an `admin_cost` row without a market succeeds,
the same insert for a `trade` fails, and an `admin_cost` with a fee fails.
### 28. `rates.py`
Replace `parse_btc_to_sats(text)` with `parse_sats(text)`: whole sats only, space, narrow space
and non-breaking space allowed as thousands separators, a decimal separator rejected with
`ValueError`. Replace `ore_to_nok(ore)` with `kroner_whole(ore)`: whole kroner, halves rounded
up, grouped with spaces, no unit and no decimals. Delete `sats_to_btc()`. Keep `nok_to_ore`,
`group_int`, `SATS_PER_BTC`, `ORE_PER_NOK` and the rate functions untouched.
**Verify:** `parse_sats("1 500 000") == 1500000`, `parse_sats("1500000") == 1500000`,
`parse_sats("0,5")` raises, `kroner_whole(106037489) == "1 060 375"`.
### 29. `validation.py`
`validate_sats` uses `rates.parse_sats` and its messages talk about sats: "Antal må fyllast ut.",
"Antal må vere eit heiltal i sats.", "Antal må vere større enn null." `_ALLOWED_LEGS` for
`admin_cost` becomes `("leg_out_sats",)` and `_REQUIRED["admin_cost"]` becomes
`(("leg_out_sats", "Fyll ut kostnaden i sats."),)`. `_check_admin_cost` requires exactly one
`leg_out_sats` and rejects `leg_out_ore`. `validate_transaction` must not require an account for
`admin_cost` (store `market_id` as `None`), must still require one for every other type, must
reject a fee for `admin_cost` and `gift_received`, and must default the taxable flag to 1 for
`admin_cost` since there is no account to take it from.
**Verify:** a manual check that an `admin_cost` payload without `market_id` validates clean, and
that the same payload with `leg_out_ore` gives a Nynorsk error on `leg_out_sats`.
### 30. `ledger.py`
Remove `admin_cost` from `CONSUMING_TYPES`. Rename the derived field of `_lot_view` from
`nok_per_btc_ore` to `btc_price_ore` (`docs/spec.md` §8), keeping the arithmetic identical.
**Verify:** `python3 -m unittest discover -s tests -v`; an `admin_cost` row leaves every lot
untouched after `ledger.rebuild()`.
### 31. `tax.py`
Add `btc_price_ore(tx)`: the bitcoin price of `docs/spec.md` §7.3 as øre per whole BTC, or
`None`. The kroner amount is `leg_out_ore` or `leg_in_ore`; the sats amount is the sats leg; for
`income` and `gift_received` the kroner amount is `value_ore` with the incoming sats; for
`spend` and `loss` it is `value_ore` with the outgoing sats; `transfer_out`, `transfer_in`,
`gift_given` and `admin_cost` give `None`. Round halves up.
Handle `admin_cost` with no account: it is no longer looked up among the accounts, it is counted
when its own `taxable` flag is set, its sats are summed into a new totals field
`admin_cost_sats`, and `admin_cost_ore` becomes `admin_cost_sats × the year-end rate ÷ 10^8`
when `deduct_admin_costs` is on and `0` otherwise. Keep every other figure as it is.
**Verify:** the golden price case — a buy of 1 000 sats for 10 kr gives
`btc_price_ore == 100000000` (1 000 000 kr) — and `report(2024)` still returns every field of §6.
### 32. `export.py`
`JOIN markets m ON m.id = t.market_id` becomes `LEFT JOIN` in `transactions_csv`. Rename `_btc()`
to `_sats()` and use `rates.group_int`; make `_amount()` use `rates.kroner_whole`. Headings
become `Antall inn (sats)`, `Antall ut (sats)`, `Avgift (sats)`, `Antall (sats)` and
`Antall (sats)` for the TOT block. Add a `Pris (kr)` column to the transaction block, after
`Til konto`, carrying `tax.btc_price_ore(row)`. Change the report line
`Forvaltningskostnad (kr)` into `Forvaltningskostnad (sats)` from `admin_cost_sats`, and keep a
`Forvaltningskostnad (kr)` line from `admin_cost_ore`.
**Verify:** download the CSV for a year with an `admin_cost` row and open it; the row is present,
the price column is filled for the trades, and no cell carries a decimal.
### 33. `app.py`
Make the account join a `LEFT JOIN markets m ON m.id = t.market_id` in `_load_transaction` and
in `transactions_list`, so a row with no account is not dropped. Add `btc_price_ore` to both
payloads by calling `tax.btc_price_ore`. Confirm that `transactions_get` builds no candidate
lots for `admin_cost`, which step 30 should already have settled.
**Verify:** `GET /api/transactions` returns the `admin_cost` row with `market_id: null`,
`market_name: null` and `btc_price_ore: null`, and a buy returns the golden price.
### 34. `tests/test_ledger.py`
Update the fixtures to whole sats and the renamed lot field. Add: an `admin_cost` transaction
consumes nothing and leaves `holdings_at` unchanged; a rebuild after it still produces the same
lots.
**Verify:** `python3 -m unittest discover -s tests -v` — green.
### 35. `tests/test_tax.py`
Update the fixtures and add: an `admin_cost` with `market_id` null and `taxable = 1` lands in
`admin_cost_sats` and leaves every account row untouched; `admin_cost_ore` is 0 with the setting
off, and the sats converted at the year-end rate with it on; `btc_price_ore` for the golden case
(1 000 sats for 10 kr → 1 000 000 kr) and for a sale (sats out, kroner in); `btc_price_ore` is
None for `transfer_out` and `admin_cost`.
**Verify:** `python3 -m unittest discover -s tests -v` — green.
### 36. `static/app.js`
`formatSats(sats)` returns the grouped whole number of sats and nothing else, and its doc comment
says so. `formatKroner(ore)` returns whole kroner with spaces and the unit, rounding halves up,
with no decimals. `formatOre(ore)` stays exactly as it is and stays exported: it is what fills a
form from a stored value. `parseSats(text)` parses whole sats with spaces allowed and returns
null for anything carrying a decimal separator.
**Verify:** in the browser console, `Skuld.formatSats(1500000)` is `"1 500 000"`,
`Skuld.formatKroner(106037489)` is `"1 060 375 kr"`, `Skuld.formatOre(123450)` is
`"1 234,50"`, and `Skuld.parseSats("1 500 000")` is `1500000`.
### 37. `static/transactions.html`
Add `<th class="numeric">Pris</th>` between the `Til konto` and `Inn` headings.
**Verify:** the list renders with the new column in the right place.
### 38. `static/transactions.js`
`amount()` returns `Skuld.formatSats(sats) + " sats"` instead of `" BTC"`. Add the `Pris` cell
between the `Til konto` and `Inn` cells, showing `Skuld.formatKroner(transaction.btc_price_ore)`
or an empty string.
**Verify:** a buy shows its price, a transfer shows an empty cell, and every sats cell ends in
`sats`.
### 39. `static/transaction.html`
Rename the lot table heading `Kostnad per BTC` to `Bitcoin pris`. Give the Marknad field a
`data-group="market"` wrapper so it can be hidden, and do the same for the whole Avgift
`fieldset` with `data-group="fee"`.
**Verify:** the form still renders, and nothing else moved.
### 40. `static/transaction.js`
In `TYPE_FIELDS`: `admin_cost` becomes `out: {label: "Kostnad", currencies: ["sats"]}` with
`consumes: false`, `market: false` and `fee: false`; `gift_received` gets `fee: false`. Every
other type gets `market: true, fee: true`. `applyType()` hides the market group and the fee
group when the type says so, sets the `market_id` select's `required` flag to match (a hidden
required field blocks the submit), and clears the value it hides. `payload()` sends
`market_id: null` for `admin_cost`. Keep `Skuld.formatOre` for the NOK inputs and use
`Skuld.parseSats` for the sats inputs and the lot amounts, which are now whole sats. Rename the
`nok_per_btc_ore` reads in `autoPick` and `buildLotRow` to `btc_price_ore`.
**Verify:** choose Forvaltningskostnad: the Marknad and Avgift sections disappear, the Valuta
selector is gone and fixed to sats, and saving writes a row with no account. Choose Gåve
(motteke): the Avgift section is gone.
### 41. `static/index.html`
Change the `Antall` heading to `Antall (sats)`.
**Verify:** the main sheet heading matches the values.
### 42. `static/report.html`
Change all three `Antall` headings to `Antall (sats)`.
**Verify:** the report headings match the values.
### 43. `static/report.js`
The `Forvaltningskostnad` figure shows sats:
`Skuld.formatSats(totals.admin_cost_sats) + " sats"`, and when `totals.admin_cost_ore` is not
zero a second figure shows the kroner that were deducted. The year-end rate line keeps
`formatKroner`, which now prints whole kroner.
**Verify:** a year with an `admin_cost` shows it in sats; with the setting on, the kroner line
appears too.
### 44. `static/markets.html`
Change the `Antall` heading to `Antall (sats)`. `static/markets.js` needs no change: it already
calls `Skuld.formatSats`, which now prints sats.
**Verify:** the account table shows whole sats with spaces.
## Phase C — StartOS package (steps 45-60)
### 45. `package.json`, `tsconfig.json`, `.prettierrc`
The build tooling for the packaging TypeScript, copied in shape from the owner's VardKey
repository and the StartOS packaging guide. Start from the packaging workspace created by
`start-cli s9pk init-workspace` and read the recipe index before copying anything.
**Verify:** `npm install` succeeds and `npx tsc --noEmit` reports no errors.
### 46. `Dockerfile`, `.dockerignore`
Slim Python base; copy `app.py`, `auth.py`, `database.py`, `ledger.py`, `tax.py`, `rates.py`,
`export.py`, `validation.py`, `maintenance.py` and `schema.sql` into `/app`; copy `static/` into
`/app/static`. No `USER` line, because the `main` volume must be writable without a `chown`.
No packages installed: the app is standard library only.
**Verify:** `docker build -t skuld-test .` succeeds and
`docker run --rm -e PORT=8080 -e DATABASE_PATH=/tmp/s.sqlite3 skuld-test python3 -c "import app"`
exits 0.
### 47. `Makefile`
The packaging targets of the StartOS template, pointing at `startos/index.ts`.
**Verify:** `make` produces `skuld.s9pk`.
### 48. `icon.svg`
The app icon, under 40 KiB, same artwork as any icon the app serves. A simple mark is enough; do
not add a second icon file.
**Verify:** the file is valid SVG and under the size limit.
### 49. `startos/sdk.ts`, `startos/utils.ts`
`sdk` built with the package id `skuld`; the constants `uiPort = 8080`, `volumeId = 'main'`,
`dataMountpoint = '/data'`, `databasePath = '/data/skuld.sqlite3'`, `imageId`, `subcontainerName`.
**Verify:** `npx tsc --noEmit` passes.
### 50. `startos/manifest/index.ts`, `startos/manifest/i18n.ts`
The manifest: id `skuld`, Nynorsk title and description, `dockerBuild: {}`, architectures
`x86_64` and `aarch64`, the `main` volume, the `ui` interface, no dependencies, the donation link
if the owner wants one.
**Verify:** `make` still builds and the manifest text is Nynorsk.
### 51. `startos/versions/index.ts`
The version graph with the first version and its `main` entry.
**Verify:** `npx tsc --noEmit` passes and `make` produces a package whose version matches.
### 52. `startos/interfaces.ts`, `startos/backups.ts`, `startos/dependencies.ts`
The `ui` interface on port 8080 with `preferredExternalPort: 80`;
`sdk.Backups.ofVolumes('main')`; no dependencies.
**Verify:** `make` builds; the built package lists one interface and one volume.
### 53. `startos/i18n/index.ts`, `startos/i18n/dictionaries/nb_NO.ts`
The i18n wrapper and the Nynorsk dictionary for everything StartOS displays: the readiness
messages, the action names and descriptions, the task text.
**Verify:** `make` builds and no user-facing string in `startos/` is English.
### 54. `startos/actions/index.ts`, `startos/actions/setAdminPassword.ts`, `startos/actions/deleteAllData.ts`
The two actions of `docs/spec.md` §9, each running `maintenance.py` in a `withTemp` subcontainer
mounted on the main volume. `set-admin-password` shows the generated credentials once;
`delete-all-data` refuses to run unless its confirmation box is ticked.
**Verify:** both actions appear under Actions & Config on a test box; the generated password
signs in; the wipe returns the app to the empty state while the admin still works.
### 55. `startos/init/index.ts`, `startos/init/requireAdmin.ts`
The init script registered in `sdk.setupInit()` after the actions, raising the recommended task
"Set the admin password before Skuld can be used." while no admin exists and clearing it
afterwards, with a stored replay id. The probe reads the SQLite database readonly in a `withTemp`
subcontainer and fails open.
**Verify:** on a fresh install the task shows as recommended and the service still starts; after
setting the password the task disappears.
### 56. `startos/main.ts`
The single daemon `primary` in `skuld-sub`: image, the `main` volume mounted writable at `/data`,
`python3 app.py` with `PORT` and `DATABASE_PATH`, and the readiness check against
`http://127.0.0.1:8080/health` displayed as "Web Interface". The port must be the same number in
`utils.ts`, `main.ts` and `interfaces.ts`.
**Verify:** `make` builds and installing the package on a test box turns the health check green.
### 57. `.github/workflows/build.yml`
The build workflow of the StartOS template: build and release the `.s9pk` on a tag.
**Verify:** the workflow file is valid YAML and the lint job (if the template has one) passes.
### 58. `README.md`
The technical reference for an AI support agent or an administering agent, in the section order of
the owner's VardKey README: image and container runtime, volume and data layout, file models,
dependencies, network access and interfaces, installation and first-run flow, actions, tasks,
health checks, backups and restore, limitations and differences, and a quick reference block for
AI consumers.
**Verify:** every claim in it matches `startos/` and the Dockerfile.
### 59. `instructions.md`
The end-user documentation, in Nynorsk: what Skuld does, how to set the admin password, how to
create accounts, how to enter each transaction type, how to read the main sheet, how to get the
tax figures, and the limitations — including that the database is not encrypted, that amounts are
shown in sats and whole kroner, and that Skuld gives figures, not tax advice.
**Verify:** a reader who has never seen the app can go from install to a tax figure.
### 60. `AGENTS.md`
The repository mechanics: that the code is English and the UI Nynorsk, that `schema.sql` is the
only DDL source, that the ledger is rebuilt rather than patched, that amounts are sats and whole
kroner, that `docs/spec.md` outranks this plan, and the build and test invocations.
**Verify:** it contradicts nothing in `docs/spec.md`.
## Phase D — verification
### 61. Build, install and run through
1. `python3 -m unittest discover -s tests -v` — all green.
2. `make` — builds `skuld.s9pk` without warnings.
3. Install on a StartOS box: the service starts, the health check goes green, the interface opens,
the first-run flow creates the admin.
4. Enter the golden example of `docs/spec.md` §6.4 through the UI and compare every figure with
the spec, for all three methods. Every BTC quantity must read as whole sats with spaces, and
every kroner figure as whole kroner.
5. Enter a buy of 1 000 sats for 10 kr and confirm `Pris` reads `1 000 000 kr` on
`/transaksjonar`.
6. Enter a Forvaltningskostnad and confirm it has no account, no fee section, and its own line in
the report and the CSV.
7. Create a child account with the exchange as custodian and confirm the `TOT` row and the
exclusion from the totals.
8. Do a transfer between two accounts, edit it, and delete it: the lots and the report must return
to their earlier state.
9. Back up, restore, and confirm the figures are identical.
10. Download the CSV and open it in a spreadsheet.
## Verification commands
| Command | Checks |
| --- | --- |
| `python3 -m unittest discover -s tests -v` | the ledger and the tax rules |
| `python3 app.py` + `curl -s localhost:8080/health` | the server starts and answers |
| `curl -s -o /dev/null -w '%{http_code}' localhost:8080/api/markets` | 401 without a session |
| `sqlite3 data/skuld.sqlite3 ".schema"` | the schema was applied |
| `npx tsc --noEmit` | the packaging TypeScript |
| `make` | the `.s9pk` builds |
| install on a StartOS box | the package really runs |
`make` and `npx tsc` need the StartOS packaging workspace (`start-cli s9pk init-workspace`) and
Docker. If the toolchain is not available, say so in the step's commit message instead of skipping
the step silently.