brianmcgee
b5fea257d7
buildbot/nix-eval Build done.
buildbot/nix-build Build done.
PR Size Review Check / pr-size-review-gate (pull_request) Successful in 35s
sizelint / sizelint (pull_request) Successful in 59s
gitea-mq/buildbot/nix-eval Build done.
gitea-mq/buildbot/nix-build Build done.
gitea-mq Merge queue passed
Switch Markdown formatting from prettier (which ignores printWidth for prose) to mdformat with the GFM plugin. Configure 80-column word wrap and consecutive list numbering. Prettier remains for CSS, HTML, JS, JSON, and YAML with *.md excluded. Reformat all existing markdown files.
1.1 KiB
1.1 KiB
3. Store state in a JSON file
Date: 2025-02-27
Status
Accepted
Context
As the name indicates, Data Mesher will need to store data persistently.
The volume of data is expected to be low, on the order of megabytes, not gigabytes. More reads are expected than writes, with expected throughput on the order of 10/s.
Initially, our state will be relatively simple. Eventually, it is intended to become more flexible with a dynamic schema.
Decision
Several options were considered, such as Bolt DB and sqlite but in the end, we have decided to store state in-memory, with changes flushed to a JSON file when changes need to be made persistent.
Consequences
This has several benefits:
- simpler to reason about.
- avoids third-party dependencies, as this can be implemented with the Go standard library.
- makes it easy to introspect our state on disk, and to manipulate it offline if required.
- amenable to dynamic schema in the future.
- file-format can be re-used for transport.