data-mesher/README.md

data-mesher

A daemon for distributing cryptographically signed files across mesh-based networks where peers communicate directly.

Important

Status: EXPERIMENTAL

We have decided to move in a different direction with Data Mesher and are currently exploring a new architecture.

The previous version is available in the v0.1.0 release.

Basic Features

• Securely exchanges files signed by ED25519 keys
• No central server - peers independently verify file signatures
• Self-healing network with automatic file propagation
• Conflict resolution via signature timestamps (newer versions win)

Architecture

The data model is simple:

• Files are identified by name and must be signed by an authorized ED25519 key
• Nodes form a libp2p cluster and automatically sync files with each other periodically
• Authorization is configured per-file: each file has a list of public keys that can sign it

When a file is uploaded:

1. The client signs the file content + filename + timestamp with their private key
2. The server verifies the signature against the configured allowed signers
3. The file is stored and its signature propagates to other nodes in the cluster
4. Other nodes request and verify the file, storing it locally

Quick Start

Generate a Keypair

❯ nix run .#data-mesher -- generate keypair

  Public Key:  ./key.pub
  Private Key:  ./key.priv

# Outputs: keys are in OpenSSH format

Upload a File

data-mesher file update ./myfile.txt \
  --url http://localhost:7331 \
  --key-path /path/to/private.key

Configuration (dm.toml)

# Log level: fatal, error, warn, info, debug
log_level = "info"

# State directory for file storage
state_dir = "/var/lib/data-mesher"

[cluster]
# Port for libp2p connections
port = 7946

# Bootstrap peers to join on startup
bootstrap_peers = [
    "/dns/alpha.clan/tcp/7946/p2p/12D3KooWHs5NXo2CPFvFT6cdsu8eSkfggNtfqTtKBVojHY7En86i",
    "/dns/beta.clan/tcp/7946/p2p/12D3KooWDSrQ6xLqFmS44bZKXDujBmTDNu7BQS9HmDz6FRS52CyN"
]

# How often to sync state with random peers
push_pull_interval = "30s"

[http]
# Port for HTTP API
port = 7331

# File authorization: map of filename -> list of allowed signer public keys (base64)
[files]
"config.json" = ["P6AE0lukf9/qmVglYrGPNYo5ZnpFrnqLeAzlCZF0lTk="]
"data.bin" = [
    "ZasdhiAVJTa5b2qG8ynWvdHqALUxC6Eg8pdn6RVXuQE=",
    "1ru2QQ1eWV7yDlyfTTDEml3xTiacASYn0KprzknN8Pc="
]

NixOS Configuration

Add Data Mesher as a flake input:

{
    inputs.data-mesher.url = "git+https://git.clan.lol/clan/data-mesher";
}

Configure the service:

{ config, ... }:
{
  services.data-mesher = {
    enable = true;
    openFirewall = true;

    settings = {
      log_level = "info";

      cluster = {
        bootstrap_peers = [
            "/dns/alpha.clan/tcp/7946/p2p/12D3KooWHs5NXo2CPFvFT6cdsu8eSkfggNtfqTtKBVojHY7En86i"
            "/dns/beta.clan/tcp/7946/p2p/12D3KooWDSrQ6xLqFmS44bZKXDujBmTDNu7BQS9HmDz6FRS52CyN"
        ];
      };

      http = {
        interface = "tailscale0";
      };

      # Configure which files can be distributed and who can sign them
      files = {
        "app-config" = [
          "P6AE0lukf9/qmVglYrGPNYo5ZnpFrnqLeAzlCZF0lTk="  # admin key
        ];
        "shared-data" = [
          "P6AE0lukf9/qmVglYrGPNYo5ZnpFrnqLeAzlCZF0lTk="  # admin key
          "ZasdhiAVJTa5b2qG8ynWvdHqALUxC6Eg8pdn6RVXuQE="  # user key
        ];
      };
    };
  };
}