# On Kubernetes

> Installing · https://spoletta.com/docs/install-kubernetes

Run Spoletta on a Kubernetes cluster: MySQL in a StatefulSet, and the webapp and the worker in Deployments, from images you build and push to a registry the cluster pulls from.

## What you need

-   `kubectl` pointed at the cluster, and a namespace you may create Deployments, Secrets and volumes in.
-   A container registry the cluster can pull from, and Docker or Podman to build the images.
-   A storage class with **ReadWriteMany** volumes, such as NFS, CephFS, EFS or Azure Files. The worker writes the files runs make and the webapp serves them, so both mount one volume. On a single node, ReadWriteOnce does, as both pods then run on that node.
-   An API key for at least one model provider.

## Install

1.  **Build and push the images**
    
    Spoletta builds into two images: the webapp, and the worker, built from the top of the repository as it takes the authoring guide in `docs/` too.
    
    ```sh
    git clone https://github.com/Nucc/spoletta.git
    cd spoletta
    REGISTRY=registry.example.com/spoletta
    TAG=$(git rev-parse --short HEAD)
    docker build -t $REGISTRY/webapp:$TAG webapp
    docker build -t $REGISTRY/worker:$TAG -f worker/Containerfile .
    docker push $REGISTRY/webapp:$TAG
    docker push $REGISTRY/worker:$TAG
    ```
    
    With Podman, the same with `podman`. Building on an ARM Mac for an x86 cluster, add `--platform linux/amd64` to both builds.
    
2.  **Make the Secrets**
    
    Three of them, so each part holds only what it needs:
    
    ```sh
    kubectl create namespace spoletta
    kubectl config set-context --current --namespace=spoletta
    
    DB_PASSWORD=$(openssl rand -hex 16)
    kubectl create secret generic spoletta-mysql \
      --from-literal=MYSQL_ROOT_PASSWORD=$(openssl rand -hex 16) \
      --from-literal=MYSQL_PASSWORD=$DB_PASSWORD
    kubectl create secret generic spoletta \
      --from-literal=DB_PASSWORD=$DB_PASSWORD \
      --from-literal=SECRET_KEY=$(openssl rand -base64 32)
    kubectl create secret generic spoletta-worker \
      --from-literal=OPENAI_API_KEY=...
    ```
    
    `spoletta` is the webapp's: the database password, the secret key that seals the secrets it keeps and, later, sign-in. `spoletta-worker` holds the model keys and any tokens scripts use, such as `GIT_TOKEN` or `SENTRY_AUTH_TOKEN`. Sentry processes run in the webapp, so they need `SENTRY_AUTH_TOKEN` (and `SENTRY_URL` for a self-hosted Sentry) in `spoletta` too. Keep the two apart: agents and scripts can read everything in the worker's environment.
    
    Copy `SECRET_KEY` to wherever you keep your backups (`kubectl get secret spoletta -o jsonpath='{.data.SECRET_KEY}' | base64 -d`): the webapp won't start without the key that sealed its secrets. If a secrets manager keeps your Secrets, see [Rotating the secret key](https://spoletta.com/docs/sign-in.md#rotating).
    
3.  **Apply the manifests**
    
    Save this as `spoletta.yaml`, set the two `image:` lines to what you pushed, and the models in the ConfigMap to those agents may use:
    
    spoletta.yaml:
    
    ```yaml
    # The models agents may use, as pi's settings.json names them;
    # models.json adds providers pi doesn't know.
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: spoletta-pi
    data:
      settings.json: |
        {
          "defaultProvider": "openai",
          "defaultModel": "gpt-5.5",
          "enabledModels": ["openai/gpt-5.5"]
        }
      models.json: |
        {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: spoletta-mysql
    spec:
      selector:
        app: spoletta-mysql
      ports:
        - port: 3306
    ---
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: spoletta-mysql
    spec:
      serviceName: spoletta-mysql
      replicas: 1
      selector:
        matchLabels:
          app: spoletta-mysql
      template:
        metadata:
          labels:
            app: spoletta-mysql
        spec:
          containers:
            - name: mysql
              image: docker.io/library/mysql:8.4
              env:
                - name: MYSQL_DATABASE
                  value: app
                - name: MYSQL_USER
                  value: app
              envFrom:
                - secretRef:
                    name: spoletta-mysql
              ports:
                - containerPort: 3306
              readinessProbe:
                tcpSocket:
                  port: 3306
              volumeMounts:
                - name: data
                  mountPath: /var/lib/mysql
      volumeClaimTemplates:
        - metadata:
            name: data
          spec:
            accessModes: [ReadWriteOnce]
            resources:
              requests:
                storage: 10Gi
    ---
    # Files runs make: the worker writes them, the webapp serves them.
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: spoletta-files
    spec:
      accessModes: [ReadWriteMany]
      resources:
        requests:
          storage: 5Gi
    ---
    # The runs' git checkouts, kept between their steps.
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: spoletta-workspaces
    spec:
      accessModes: [ReadWriteOnce]
      resources:
        requests:
          storage: 20Gi
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: spoletta-worker
    spec:
      selector:
        app: spoletta-worker
      ports:
        - port: 8090
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: spoletta-worker
    spec:
      replicas: 1
      strategy:
        type: Recreate
      selector:
        matchLabels:
          app: spoletta-worker
      template:
        metadata:
          labels:
            app: spoletta-worker
        spec:
          containers:
            - name: worker
              image: registry.example.com/spoletta/worker:TAG
              env:
                - name: PORT
                  value: "8090"
                - name: PI_SKIP_VERSION_CHECK
                  value: "1"
                - name: PI_TELEMETRY
                  value: "0"
              envFrom:
                - secretRef:
                    name: spoletta-worker
              ports:
                - containerPort: 8090
              volumeMounts:
                - name: pi
                  mountPath: /pi-host
                  readOnly: true
                - name: files
                  mountPath: /data/files
                - name: workspaces
                  mountPath: /data/workspaces
          volumes:
            - name: pi
              configMap:
                name: spoletta-pi
            - name: files
              persistentVolumeClaim:
                claimName: spoletta-files
            - name: workspaces
              persistentVolumeClaim:
                claimName: spoletta-workspaces
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: spoletta-webapp
    spec:
      selector:
        app: spoletta-webapp
      ports:
        - port: 8080
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: spoletta-webapp
    spec:
      replicas: 1
      strategy:
        type: Recreate
      selector:
        matchLabels:
          app: spoletta-webapp
      template:
        metadata:
          labels:
            app: spoletta-webapp
        spec:
          containers:
            - name: webapp
              image: registry.example.com/spoletta/webapp:TAG
              env:
                - name: DB_HOST
                  value: spoletta-mysql
                - name: DB_PORT
                  value: "3306"
                - name: DB_NAME
                  value: app
                - name: DB_USER
                  value: app
                - name: PORT
                  value: "8080"
                # Without sign-in the webapp listens on 127.0.0.1 only, which a
                # Service can't reach.
                - name: HOST
                  value: 0.0.0.0
                - name: WORKER_URL
                  value: http://spoletta-worker:8090
                # Where the worker reaches the proxy agents use MCP servers through.
                - name: MCP_PROXY_URL
                  value: http://spoletta-webapp:8080
              envFrom:
                - secretRef:
                    name: spoletta
              ports:
                - containerPort: 8080
              readinessProbe:
                tcpSocket:
                  port: 8080
              volumeMounts:
                - name: files
                  mountPath: /data/files
                  readOnly: true
          volumes:
            - name: files
              persistentVolumeClaim:
                claimName: spoletta-files
    ```
    
    ```sh
    kubectl apply -f spoletta.yaml
    kubectl rollout status statefulset/spoletta-mysql
    kubectl rollout status deployment/spoletta-worker
    kubectl rollout status deployment/spoletta-webapp
    ```
    
    The webapp restarts a few times while MySQL starts the first time. That is expected; it settles once MySQL is ready.
    
4.  **Open it**
    
    ```sh
    kubectl port-forward svc/spoletta-webapp 8080:8080
    ```
    
    Then go to `http://localhost:8080`.
    

> **No Ingress until people sign in:** Without sign-in, anyone who reaches the webapp can do anything in it, running scripts on the worker included. Reach it with `kubectl port-forward` until sign-in is set up.

## Opening it up

Register `https://spoletta.example.com/api/auth/callback` with your identity provider, Google or both, add sign-in to the webapp's Secret, and restart it. For Google, add `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET` too. [Sign-in and secrets](https://spoletta.com/docs/sign-in.md) says what each setting is.

```sh
kubectl patch secret spoletta --type merge -p '{"stringData":{
  "OIDC_ISSUER":"https://login.example.com",
  "OIDC_CLIENT_ID":"spoletta",
  "OIDC_CLIENT_SECRET":"...",
  "ADMIN_EMAILS":"you@example.com",
  "PUBLIC_URL":"https://spoletta.example.com"}}'
kubectl rollout restart deployment/spoletta-webapp
```

Then route to it with an Ingress, here with ingress-nginx and cert-manager:

spoletta-ingress.yaml:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: spoletta
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  ingressClassName: nginx
  tls:
    - hosts: [spoletta.example.com]
      secretName: spoletta-tls
  rules:
    - host: spoletta.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: spoletta-webapp
                port:
                  number: 8080
```

## Updating

Build and push both images with a new tag, then:

```sh
kubectl set image deployment/spoletta-worker worker=$REGISTRY/worker:$TAG
kubectl set image deployment/spoletta-webapp webapp=$REGISTRY/webapp:$TAG
```

A changed ConfigMap reaches the worker when it restarts: `kubectl rollout restart deployment/spoletta-worker`.