Install Kubernetes on Jetson (Master Node) with Docker or containerd

• Rasim Mahmudov • 8 min read • 137 views

Install Container Runtime (Before Kubernetes)

Kubernetes requires a container runtime. You can use Docker or containerd. Choose one of the following:

Section 1.a Installing Docker before Kubernetes

Note: Select only one of these options (Docker itself uses containerd internally): Section 1.a or Section 1.b.

Important: Before starting, check if docker.io is already installed. On many boards it comes pre-installed after flashing. To verify, run:

sudo systemctl status docker

If the output shows Active: active (running), Docker is already installed and you can skip this section and continue with Section 2.

sudo apt update
sudo apt install docker.io
sudo systemctl stop docker
sudo nano /etc/docker/daemon.json

Create or edit /etc/docker/daemon.json and ensure it contains the following:

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  },
  "data-root": "/nvme/docker"
}

Explanation: This configures Docker to use systemd for cgroups (required by Kubernetes), sets up log rotation, enables the NVIDIA GPU runtime, and moves Docker’s data directory to NVMe storage for better performance.

sudo mkdir -p /nvme/docker
sudo chown -R root:root /nvme/docker
sudo chmod 700 /nvme/docker

These commands create the NVMe storage directory for Docker and set proper ownership and permissions.

Section 1.b Installing containerd before Kubernetes

sudo apt install containerd
sudo mkdir -p /nvme/containerd
sudo chown -R root:root /nvme/containerd
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml
sudo ls /etc/containerd/
sudo nano /etc/containerd/config.toml

Edit /etc/containerd/config.toml and update the following lines:

root = "/nvme/containerd"
state = "/nvme/containerd/state"

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
  SystemdCgroup = true

Explanation: This moves containerd’s data to NVMe storage and configures it to use systemd as the cgroup driver, ensuring compatibility with Kubernetes.

sudo systemctl restart containerd
sudo systemctl status containerd

Restart containerd and confirm that it is running correctly.

Section 2. Installing Kubernetes (kubeadm, kubectl, kubelet)

sudo swapoff -a

Disables swap (required, otherwise kubelet will not start). This is temporary; to disable swap permanently, edit /etc/fstab.

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

Installs dependencies required for the Kubernetes repository.

sudo mkdir -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

Adds the Kubernetes GPG key to verify package signatures.

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

Adds the Kubernetes apt repository.

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Installs kubelet (node agent), kubeadm (cluster bootstrap), and kubectl (command-line client). Marking them on hold prevents accidental version upgrades.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

Initializes the Kubernetes control plane with a Pod network CIDR (required for Flannel or other CNI plugins).

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Configures kubectl for your user so you can manage the cluster without using sudo.

Install Metrics Server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Deploys Metrics Server to collect CPU and memory usage from nodes and pods.

Check metrics with:

kubectl top nodes
kubectl top pods --all-namespaces

Fix x509 errors

If you see TLS errors when Metrics Server scrapes kubelets, add this argument to its deployment:

- --kubelet-insecure-tls

This disables certificate verification (safe for testing, but not recommended for production environments).

Install jtop

On the Jetson device, run:

sudo apt-get install python3-pip
sudo -H pip3 install -U jetson-stats
sudo jtop

jtop provides a terminal-based UI that shows CPU, GPU, memory, and temperature stats in real time.

Conclusion

Metrics Server allows Kubernetes to make better scheduling decisions based on resource usage. jtop provides insight into Jetson hardware utilization. Using both helps you avoid bottlenecks and overheating.

Related