0.9

2026-09-30 · Full Changelog

clabernetes release

0.9.00.89.0

anewc9sera

• direct device runtime
• routed management mesh
• carrier-aware links

C9s v0.9.0 changes how labs run, connect, and recover. Devices have become native Kubernetes containers, management networking has gained configurable addressing and a routed mesh, and links propagate carrier loss across workers. Saved device configuration can survive Pod replacement, while reusable planners and startup admission controls support larger labs.

The direct device runtime

This is the crux of the major refactoring.

Every network device now runs as a regular application container in a c9s-managed Pod. The launcher image, the Docker daemon inside the Pod, the containerlab executable, image import, and nested container discovery are all gone. What this changes in practice:

• kubectl logs, kubectl exec, resource limits, and Pod status address the actual device container. Chassis cards and grouped nodes are visible application containers of one Pod.
• The kubelet performs the only image pull, with native imagePullPolicy and imagePullSecrets. c9s resolves only registry metadata (manifests and config blobs) and fails readiness if the running image diverges from the planned digest. No more pull through pains and/or docker daemon configuration juggling.
• C9s still leverages Containerlab's knowledge of NOS kind behavior. This knowledge comes from the pinned containerlab 0.79.0 Go module. Four reusable planner worker pods contain the containerlab executable and "ask Containerlab" how to run and configure each network device. See Planning and startup at scale.
• Each device Pod boots through a preparation init container that stages and digest-verifies every generated and payload artifact, and a restartable privileged connectivity init-sidecar that gates device start on the accepted cold-start interface set and applies live Link changes.

Links, carrier loss, and recovery

Like before, the point-to-point links between the nodes are laid out. However, in this release the transport of these p2p links has been changed from the VXLAN to the UDP-based tunnels. The network device always sees a plain veth interface and has not recollection of the tunnel semantics. The wiring mechanics (L2 point-to-point, MTU intent, carrier propagation, live rewires, cleanup, rescheduling) are handled by the c9s controller.

Link loss detection

Shutting down or removing an interface at one end makes the peer see loss of carrier (NO-CARRIER / LOWERLAYERDOWN), while its interface stays administratively up. Protocols can react to the cable failure instead of waiting for their hold timers. If a Pod dies, heartbeats take its peer links carrier-down within a few seconds; carrier returns when the replacement endpoint converges. Carrier messages, heartbeats, and frames use the same UDP socket and network path.

Fragmentation and MTU

To allow the default 9500-byte link MTU cross a typical 1500-byte cluster pod networks without lowering the lab's link MTUs we fragment the jumbo frames to fit the Pod network and either reassemble the complete frame or drop it. VLAN tags are preserved; full-size QinQ payloads need four extra bytes of link MTU. Losing any fragment loses the frame, so large frames amplify underlay loss.

Changing links in a running lab

Kinds whose plans support Live link changes receive connectivity updates in place. For non-live Restart and Recreate changes, c9s replaces the Pod directly, avoiding devices such as cEOS getting stuck after a stop signal. Every device sharing that Pod restarts with it.

See Link wire semantics for carrier, loss, MTU, and timing limits.

Controllable management networking and a routed mesh

c9s now allocates management addresses before devices start. Choose IPv4 and IPv6 subnets, gateways, and automatic allocation ranges through the containerlab mgmt: block in a Topology, or spec.mgmt on a NodeProfile. Pin individual nodes with mgmt-ipv4 / mgmt-ipv6; automatic allocation works around pinned addresses and rejects duplicates or gateway collisions. Without an override, IPv4 uses 172.20.20.0/24; IPv6 allocation requires an IPv6 subnet.

For example, this block inside a containerlab definition selects a subnet and limits automatic allocation to part of it:

mgmt:
  ipv4-subnet: 10.20.30.0/24
  ipv4-gw: 10.20.30.1
  ipv4-range: 10.20.30.128/25

Devices in the namespace reach each other's management addresses across Pods and workers, with peer source addresses preserved. Node names, aliases, and chassis component names resolve to management addresses inside devices, so telemetry collectors, syslog targets, and tools can use the same names and addresses as a local containerlab lab. Membership and Pod-address changes propagate through a shared peer directory without restarting the other devices.

The mesh routes traffic over a dedicated management tunnel. Proxy ARP and IPv6 neighbor discovery provide peer reachability without flooding traffic or using DNS for discovery.

See the management network guide and management opt-out for details.

Planning and startup at scale

The Helm chart starts four reusable planner workers by default.

Recall, that planners are the pods that contain the "containerlab's knowledge" about the network kinds.

The four planner pods will be enough for most labs, where each planner pod can handle multiple device plans concurrently.

With plannerPool.replicas Helm variable you control how much planners you need; plannerPool.enabled: false selects disposable planning Pods.

Two optional controls pace large deployments through Config.spec.rollout installation-wide and Topology.spec.rollout per lab:

See Measuring large topology startup for conditions and limits.

Registry access

The controller can resolve image metadata through registry mirrors, including Harbor proxy projects with path rewriting, using Config.spec.imagePull.registryMetadataMirrors. registryMetadataTrust configures private CAs or HTTP for those metadata connections. Image references retain their original identity; worker container-runtime mirror and trust settings still control kubelet pulls. A failed configured mirror fails planning without falling back to the public registry.

Missing or unreadable pull Secrets now produce explicit Node conditions, and planning recovers when the Secret becomes readable. See Image pulling for more details.

Service exposure

LoadBalancer Services can request either pinned or automatically allocated management addresses, so devices can be exposed at their management IP without pinning every node. This changes the behavior of useNodeMgmtIpv4Address / useNodeMgmtIpv6Address.

Expanded node vocabulary

The Node spec and Topology compiler accept more of the native containerlab vocabulary: startup-delay, restart-policy (always/unless-stopped), image-pull-policy, cpu and memory (device-container limits), healthcheck (merged over the image OCI healthcheck), aliases (extra same-namespace Service names for the node's Pod), link-apply-mode, group, and topology.groups inheritance.

See Differences from containerlab.

Compatibility and clearer failure reporting

• Topology compilation sanitizes names Kubernetes cannot carry, such as uppercase node names, while rejecting collisions. Conflicting child resource names are detected before creation, and reserved container mount paths fail with a diagnostic naming the path.
• Failed topology exec commands are logged without putting an otherwise running device into a crash loop. Grouped nodes can share identical payloads and image-derived ports; explicit port collisions still fail planning.
• Topology status uses a status subresource, so reporting progress no longer overwrites spec defaults. status.observedGeneration lets clients distinguish readiness for the current definition, and Node readiness is invalidated when relevant inputs change.
• Deployment apply failures, rejected plans, and artifact-name conflicts are visible on the Node instead of only in manager logs. SSH probe passwords that coincide with declared or generated device defaults no longer cause false plan rejection; other Secret values remain screened from plan artifacts.
• A new Cumulus VX example and opt-in e2e suite cover deployment and connectivity with a Linux peer.

Breaking API changes

The LauncherProfile CRD is renamed to NodeProfile (Node.spec.profileRef, Node.status.appliedProfile); it carries reusable direct-workload policy (expose, scheduling, resources, persistence, probes, image pull). Device privilege and kind-owned container requirements come from the imported package plan.

Launcher-era fields are removed from Config, NodeProfile, and Topology: launcher image/log/env settings, privilegedLauncher, containerlab debug/timeout/version, Docker config and daemon config, insecure registries, pull-through, CRI overrides, by-kind resource defaults, and Docker management-network fields. Link.status.error is replaced by the Accepted condition, Link.status.tunnelID becomes Link.status.wireID, and Node.status.probeStatuses is removed. Per-node wire discovery Services use <node>-wire instead of <node>-vx.

disableExpose is removed from Topology and NodeProfile. Replace disableExpose: true with spec.expose.exposeType: None; wire and alias Services are unaffected. See #333.

Two global settings that no longer controlled anything are also gone: the Topology and Config naming fields and the Config inClusterDNSSuffix field, together with the globalConfig.naming and globalConfig.inClusterDNSSuffix Helm values. Generated resources are named after the containerlab node inside the Topology namespace, and services resolve without a configurable suffix. Drop the fields from manifests and values files before installing; a manifest that still sets them is rejected by the new schemas. See #335.

Two node-definition fields change meaning:

• ports entries are destination ports only (22/tcp). The Docker-style host:container form is rejected on Nodes, and two-sided entries inside a Topology definition: are normalized to their destination port. c9s allocates the service side and records both halves in Node.status.exposedPorts.
• labels in a Topology definition: become Kubernetes labels on the emitted Node, its Deployment, and Pods; there is no Node.spec.labels. c9s-owned labels move to the qualified c9s.run/ namespace. The old clabernetes/... label keys are no longer written or selected, while annotation keys keep their existing names.

useNodeMgmtIpv4Address and useNodeMgmtIpv6Address now request the node's management address as the LoadBalancer IP whether the address is pinned or allocated. In 0.8, containerlab allocated unpinned addresses inside the launcher Pod, so these options read only a pinned mgmt-ipv4 or mgmt-ipv6, and unpinned nodes got an IP chosen by the LoadBalancer provider. c9s now allocates every management address itself, and the Service requests the address reported in Node.status.directManagement. If you enable either option, pin only some nodes, and your address pool covers only the pinned addresses, pin the remaining nodes or set ipv4-range so allocated addresses fall inside the pool; otherwise their Services stay pending. See Service exposure.

Removed node vocabulary

The Node spec carries only the vocabulary the direct runtime can realize. Unknown fields are rejected at apply time, and a Topology definition: fails compilation with a diagnostic naming each unsupported field and its location.