TrueNas Scale uses k3s, which is a lightweight Kubernetes distribution. It’s intended to be used only as platform for APPS, but I don’t see why you can’t use it as your playground.

Connecting to TrueNas k3s locally

This way you connect to k3s locally, which is more secure than exposing your Kubernetes API. Also, if you don’t want to install anything, you can use sudo k3s kubectl as a regular user. However, this way:

• you have to enter sudo password all the time
• editing in k9s doesn’t work as it invokes kubectl edit under the hood

I think that more convenient solution is to install kubectl, k9s and configure user kubeconfig. Here’s how you can do this:

1. Make sure that shell knows about ~/bin path. For fish, it’s:

#To persist changes, $PATH in fish can be changed in `~/.config/fish/config.fish`
set -a PATH "$HOME/bin/"

2. Download kubectl

mkdir ~/bin
cd ~/bin
curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.25.0/bin/linux/amd64/kubectl
chmod +x kubectl

3. Configure kubeconfig

mkdir ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown -R <USER> ~/.kube/config

4. Check that kubectl configured properly:

sab@truenas ~> kubectl version -o yaml
clientVersion:
  buildDate: "2022-10-18T16:38:44Z"
  compiler: gc
  gitCommit: 7d66e419556e1be16c1a6cc3f9178f3bd8a95b54
  gitTreeState: dirty
  gitVersion: v1.24.4+k3s-7d66e419-dirty
  goVersion: go1.19.1
  major: "1"
  minor: "24"
  platform: linux/amd64
kustomizeVersion: v4.5.4
serverVersion:
  buildDate: "2022-10-18T16:38:44Z"
  compiler: gc
  gitCommit: 7d66e419556e1be16c1a6cc3f9178f3bd8a95b54
  gitTreeState: dirty
  gitVersion: v1.24.4+k3s-7d66e419-dirty
  goVersion: go1.19.1
  major: "1"
  minor: "24"
  platform: linux/amd64

5. Install k9s

curl -LO https://github.com/derailed/k9s/releases/download/v0.26.7/k9s_Linux_x86_64.tar.gz
tar xvf k9s_Linux_x86_64.tar.gz
mv k9s ~/bin/

6. Run k9s to check that it’s working

Connecting to TrueNas k3s remotely

Exposing k3s can be risky, please make sure that you understand what you’re doing.

As of now, k3s in TrueNas listens on all ports:

sab@truenas ~> ss -tlen | grep :6443
LISTEN 0      4096                                    *:6443             *:*    ino:22275 sk:2073 cgroup:/system.slice/k3s.service

But it’s protected by the IPTABLES.

So, to grant access to the Kubernetes API:

1. remove DROP rule from the IPTABLES: This can be done via the Truetool, which is a collection of useful bash scripts

./truetool.sh  --kubeapi-enable

Or directly through the iptaples command:

iptables -D INPUT -p tcp -m tcp --dport 6443 -m comment --comment "iX Custom Rule to drop connection requests to k8s cluster from external sources" -j DROP

2. copy kubeconfig from /etc/rancher/k3s/k3s.yaml to your local machine
3. configure KUBECONFIG environment variable to point to this file:

sab@mbp13 ~> export KUBECONFIG=/Users/sab/.kube/truenas
sab@mbp13 ~> kubectl get ns
NAME              STATUS   AGE
default           Active   6d12h
kube-system       Active   6d12h
kube-public       Active   6d12h
kube-node-lease   Active   6d12h
openebs           Active   6d12h
ix-pihole         Active   6d12h
metallb-system    Active   41h
tc-system         Active   41h
ix-grafana        Active   38h
ix-loki           Active   23h

Also worth noticing, that this is not persistent, and after reboot iptables will be reverted to initial state. To persist those changes, add your command/script to the Init/Shutdown Scripts