clan/clan-core
12
47
Fork 113
Code Issues 155 Pull Requests 35 Actions 1 Packages Projects 3 Releases Wiki Activity

yggdrasil: multicast peer discovery by default, per-transport dial protocols #7877

Merged
kenji merged 2 commits from yggdrasil-multicast into main 2026-08-26 14:11:06 +00:00
Conversation 9 Commits 2 Files Changed 4
+99 -41
DavHau commented 2026-08-11 09:33:39 +00:00
Owner
Copy Link
Copy Source

Two changes to the yggdrasil service:

yggdrasil: enable multicast peer discovery by default

  • multicastInterfaces now defaults to discovery on all interfaces, with overlay/VPN interfaces (zt|wg|tailscale|mycelium|tinc|tun|tap|ygg) degraded to priority 10 so direct links win.
  • Multicast listener ports are opened in the firewall automatically.
  • Static peers get ?priority=20, worse than any multicast-discovered link, so locally discovered peerings carry the traffic and static routes act as fallback. Priority only tie-breaks between multiple links to the same peer, so nodes reachable only statically are unaffected.

yggdrasil: dial per-transport link protocols, disable quic/ws listeners by default

  • Exports from VPN transports (zerotier, wireguard, ...) are dialed with a single plain tcp:// link: the transport already encrypts and doesn't filter, so extra protocols add connections but no reachability.
  • Exports from the internet service may cross filtered networks, so every enabled protocol is dialed to maximize the chance that one gets through.
  • ports.quic and ports.ws are now nullOr port and default to null (listeners disabled, firewall ports closed, not dialed).
Two changes to the yggdrasil service: **yggdrasil: enable multicast peer discovery by default** - `multicastInterfaces` now defaults to discovery on all interfaces, with overlay/VPN interfaces (`zt|wg|tailscale|mycelium|tinc|tun|tap|ygg`) degraded to priority 10 so direct links win. - Multicast listener ports are opened in the firewall automatically. - Static peers get `?priority=20`, worse than any multicast-discovered link, so locally discovered peerings carry the traffic and static routes act as fallback. Priority only tie-breaks between multiple links to the same peer, so nodes reachable only statically are unaffected. **yggdrasil: dial per-transport link protocols, disable quic/ws listeners by default** - Exports from VPN transports (zerotier, wireguard, ...) are dialed with a single plain `tcp://` link: the transport already encrypts and doesn't filter, so extra protocols add connections but no reachability. - Exports from the internet service may cross filtered networks, so every enabled protocol is dialed to maximize the chance that one gets through. - `ports.quic` and `ports.ws` are now `nullOr port` and default to `null` (listeners disabled, firewall ports closed, not dialed).
CODEOWNERS rules requested review from pinpox 2026-08-11 09:33:40 +00:00
CODEOWNERS rules requested review from kenji 2026-08-11 09:33:40 +00:00
DavHau added 1 commit 2026-08-11 09:43:10 +00:00
yggdrasil: enable multicast peer discovery by default 434482b8a7 
The multicastInterfaces default of [] disabled multicast entirely,
contradicting the option docs. Default is now the all-interfaces entry
with Port 5400; the firewall opens configured multicast listener ports
automatically (the link-local listener is TCP; the previous UDP 5400
rule was ineffective).

Link priorities ensure direct local links always carry the traffic when
multiple links to the same peer exist: physical-interface multicast 0,
overlay/VPN-interface multicast 10, static peers from exports 20
(appended as ?priority= to generated peer URIs). Without this, ygg
routing tie-breaks equal-priority sub-millisecond links by age, letting
yggdrasil-over-zerotier win over a direct LAN link.

Adds VM test regression assertions for multicast interfaces and fe80
peering.
DavHau force-pushed yggdrasil-multicast from 02b33a6f1c to ef9f41c580 2026-08-11 09:43:10 +00:00 Compare
lassulus commented 2026-08-11 09:56:40 +00:00
Owner
Copy Link
Copy Source

hmm, what are the security implications by having this on by default? can strangers in the same network can connect now to my "private" yggdrasil network?

hmm, what are the security implications by having this on by default? can strangers in the same network can connect now to my "private" yggdrasil network?
DavHau added 1 commit 2026-08-12 07:21:01 +00:00
yggdrasil: dial per-transport link protocols, disable quic/ws listeners by default
PR Size Review Check / pr-size-review-gate (pull_request) Successful in 35s
Details
buildbot/nix-eval evaluation succeeded
Details
buildbot/nix-build 150 attributes built
Details
sizelint on each commit / sizelint (pull_request) Successful in 39s
Details
PR Size Review Check / pr-size-review-gate (pull_request_review) Successful in 25s
Details
gitea-mq/buildbot/nix-eval evaluation succeeded (3 warnings)
Details
gitea-mq/buildbot/nix-build 150 attributes built
Details
gitea-mq Merge queue passed
Details
13b50f388e
DavHau force-pushed yggdrasil-multicast from ef9f41c580 to 13b50f388e 2026-08-12 07:21:01 +00:00 Compare
pinpox commented 2026-08-12 07:22:34 +00:00
Owner
Copy Link
Copy Source

hmm, what are the security implications by having this on by default? can strangers in the same network can connect now to my "private" yggdrasil network?

We should be doing access control by allowing only our keys to peer with our hosts. Local discovery should have been enabled all along.

clan-core/clanServices/yggdrasil/default.nix Lines 217 to 230 in 33a9c3dcd9
allowedPublicKeys = lib.filter (key: key != "") (
map (
name:
lib.trim (
clanLib.getPublicValue {
flake = config.clan.core.settings.directory;
machine = name;
generator = "yggdrasil";
file = "publicKey";
default = "";
}
)
) (lib.attrNames roles.default.machines)
);

> hmm, what are the security implications by having this on by default? can strangers in the same network can connect now to my "private" yggdrasil network? We should be doing access control by allowing only our keys to peer with our hosts. Local discovery should have been enabled all along. https://git.clan.lol/clan/clan-core/src/commit/33a9c3dcd95f5f92657f74b0e13122ded1b3a405/clanServices/yggdrasil/default.nix#L217-L230
DavHau commented 2026-08-12 07:24:23 +00:00
Author
Owner
Copy Link
Copy Source

good question, went trough the yggdrasil source to check. Answer is: they can peer, but they get nothing useful out of it. AllowedPublicKeys explicitly does NOT apply to link-local multicast peers (multicast listeners go through ListenLocal which skips teh check). So yes, a stranger on the same LAN can establish a peering with your node.

But peering != access: yggdrasil is e2e encrypted and this module already firewalls the ygg interface to only accept traffic from clan member IPs. So the stranger can use your node as transit and sees your pubkey, but cant reach any services or hosts in the clan.

If we want to close even that, multicast supports a Password option (only nodes with the same password discover each other), we could derive one from a clan-wide secret via vars. Would do that in a follow up PR tho.

good question, went trough the yggdrasil source to check. Answer is: they can peer, but they get nothing useful out of it. AllowedPublicKeys explicitly does NOT apply to link-local multicast peers (multicast listeners go through ListenLocal which skips teh check). So yes, a stranger on the same LAN can establish a peering with your node. But peering != access: yggdrasil is e2e encrypted and this module already firewalls the ygg interface to only accept traffic from clan member IPs. So the stranger can use your node as transit and sees your pubkey, but cant reach any services or hosts in the clan. If we want to close even that, multicast supports a Password option (only nodes with the same password discover each other), we could derive one from a clan-wide secret via vars. Would do that in a follow up PR tho.
DavHau changed title from yggdrasil: multicast peer discovery by default, single link protocol per peer to yggdrasil: multicast peer discovery by default, per-transport dial protocols 2026-08-12 07:24:38 +00:00
DavHau commented 2026-08-12 08:49:27 +00:00
Author
Owner
Copy Link
Copy Source

The password option doesn't seem to be sound from a security perspective. Enabling it would just prevent connecting with foreigners by accident, but doesn't guarantee it.

The password option doesn't seem to be sound from a security perspective. Enabling it would just prevent connecting with foreigners by accident, but doesn't guarantee it.
lassulus commented 2026-08-24 08:05:09 +00:00
Owner
Copy Link
Copy Source

hmm, but they could route via my peers to another peer which maybe is not firewalled off? maybe that's obscure enough that we don't need to think too hard about it?

hmm, but they could route via my peers to another peer which maybe is not firewalled off? maybe that's obscure enough that we don't need to think too hard about it?
kenji commented 2026-08-26 08:00:32 +00:00
Owner
Copy Link
Copy Source

with overlay/VPN interfaces (zt|wg|tailscale|mycelium|tinc|tun|tap|ygg) degraded to priority 10 so direct links win.

wg and most tap/tun devices don't have any multicast support. Is there anything we lose/gain from disabling that for them anyway?

hmm, but they could route via my peers to another peer which maybe is not firewalled off? maybe that's obscure enough that we don't need to think too hard about it?

That is a good point. Other than that I think this looks reasonable.

> with overlay/VPN interfaces (zt|wg|tailscale|mycelium|tinc|tun|tap|ygg) degraded to priority 10 so direct links win. wg and most tap/tun devices don't have any multicast support. Is there anything we lose/gain from disabling that for them anyway? > hmm, but they could route via my peers to another peer which maybe is not firewalled off? maybe that's obscure enough that we don't need to think too hard about it? That is a good point. Other than that I think this looks reasonable.
kenji approved these changes 2026-08-26 13:56:23 +00:00
kenji scheduled this pull request to auto merge when all checks succeed 2026-08-26 13:56:28 +00:00
kenji merged commit 95675a2a70 into main 2026-08-26 14:11:06 +00:00
kenji deleted branch yggdrasil-multicast 2026-08-26 14:11:07 +00:00
brianmcgee commented 2026-08-27 09:25:02 +00:00
Owner
Copy Link
Copy Source

How does this affect data center based deployments? Will Hetzner be mad?

How does this affect data center based deployments? Will Hetzner be mad?
kenji commented 2026-08-27 13:57:39 +00:00
Owner
Copy Link
Copy Source

I looked at it from that side and I doubt it:

AllowedEncryptionPublicKeys = allowedPublicKeys

first we actually would only peer with peers we already have added.
Second I could not find any actual multicast specific wording in the hetzner tos, I assume they just block it and thats it.

Also I'm the canary in the coalmine I cheery picked this and have been running it for 4 days now actually on Hetzner and my homelab.

I looked at it from that side and I doubt it: ``` AllowedEncryptionPublicKeys = allowedPublicKeys ``` first we actually would only peer with peers we already have added. Second I could not find any actual multicast specific wording in the hetzner tos, I assume they just block it and thats it. Also I'm the canary in the coalmine I cheery picked this and have been running it for 4 days now actually on Hetzner and my homelab.
Sign in to join this conversation.
Reviewers
No Reviewers
pinpox
kenji
Labels
Clear labels
Type/Regression
Breaking change in behavior
Architecture
Change
Fundamental change spanning across components
Component
API
Issues regarding the clan api
Component
App
Issues regarding the clan app
Component
Backup
Component
CICD
Component
CLI
Component
Clan Services
Component
Clan module system
Issues regarding the clan module system including inventory.
Component
Docs
Component
Flake
Component
Install & Update
Component
Networking
Component
Select
Issues regarding select - the caching layer
Component
Templates
Issues regarding templates.
Component
Testing
Issues regarding testing
Component
VARS
Issues regarding vars.
Component
Virtualisation
Issues regarding virtual machines.
Good first issue
Help wanted
OS
Darwin
OS
Linux
Priority
Critical
1
Priority
High
2
Priority
Medium
3
Priority
Low
4
Status
Blocked
1
Status
On Hold
2
Status
Needs More Info
3
Status
Needs Review
4
Status
In Progress
5
Status
Won't Fix
6
Status
Available
Status
Duplicate
Type
Community Contribution
1
contributed by members of the Clan community
Type
Discussion
1
Type
Bug
2
Type
Question
3
Type
Epic
4
Type
Feature Request
5
Type
User Experience
5
Type
Enhancement
6
Type
Maintenance
7
Type
Documentation
Type
Question
Question to be resolved
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
admin brianmcgee clan-bot DavHau Enzime (Michael Hoang) hsjobeki kenji (Kenji Berthold) lassulus Mic92 nixbot pinpox Qubasa (Luis Hebendanz) renovate
No Assignees
5 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: clan/clan-core#7877