Installing on a shared cluster

Use this page when other teams share the cluster, or when the cluster already runs Prometheus, Loki or cert-manager. An Octostar install creates some objects that exist once for the whole cluster (ClusterRoles, admission webhooks, IngressClasses, CRDs). On a shared cluster, these can collide with existing objects, or the installer account may not be allowed to create them.

install.sh checks this before it changes anything. When it finds a problem, it stops with one line per problem and writes nothing to the cluster.

What you need

• An account that can create cluster-scoped objects. Cluster-admin is the simple choice.
• The namespaces of the install: namespace, monitoring.namespace, ingress.namespace, cert-manager, and t3k.namespace with T3K. The account needs full rights in them, and create on Namespaces.
• Rights in kube-system, because the install changes the CoreDNS configuration.

Without cluster-admin, the account also needs escalate and bind on clusterroles, or every permission that the rendered ClusterRoles grant.

1. Run the install

./bin/install.sh --context <context>

If the preflight stops, read each line, fix the cause, and run it again. For two cases you can use a flag:

Flag	Use it when	Risk
--adopt-crds	An earlier Octostar install put the CRDs there before the installer applied them itself	The install replaces them with its pinned versions
--ignore-preflight	A check is wrong for your cluster, for example an admission policy refuses the dry runs	A real problem now fails the install at that release, or takes over the object

Existing monitoring

On a cluster that already runs Prometheus, turn off the cluster-wide stack and report to the existing one. Set this before the first install:

# local-env.yaml
monitoring:
  clusterScoped:
    enabled: false
  external:
    prometheus:
      url: "http://prometheus-operated.monitoring.svc:9090"
      # Only if the Prometheus selects monitors by label:
      remoteWriteUrl: "http://prometheus-operated.monitoring.svc:9090/api/v1/write"
    loki:
      url: "http://loki-gateway.monitoring.svc"   # empty: run Octostar's own Loki

The install then runs no Prometheus, Alertmanager, node-exporter or kube-state-metrics. The servicemonitors, podmonitors and prometheusrules CRDs must already exist, or the install stops. Grafana, Alloy and (if you do not set a Loki URL) Loki still run.

Then, on the existing side:

1. Metrics. Look at kubectl get prometheus -A -o yaml.
   • If serviceMonitorSelector and podMonitorSelector are open ({}), leave remoteWriteUrl empty. Rancher Monitoring works this way. If the Prometheus URL is not of the form <service>.<namespace>.svc or <service>.<namespace>, set apiNetworkPolicy.prometheusNamespace.
   • If they select on a label, set remoteWriteUrl and turn on enableRemoteWriteReceiver in that Prometheus. The vLLM PodMonitor does not reach it this way. Its only label is app.kubernetes.io/name: vllm-monitoring. Let the podMonitorSelector of that Prometheus select it, or the vLLM dashboard stays empty.
   • Do not use both ways. The existing Prometheus refuses the second copy of each series as out of order, and remote write breaks.
2. Alert rules. The existing Prometheus must select the PrometheusRule kube-prometheus-stack-octostar (in monitoring.namespace) and cnpg-cluster-alert-rules (in namespace). Its kube-state-metrics needs --metric-labels-allowlist=pods=[*].
3. Alert delivery. Set it up on the existing Alertmanager. monitoring.alerting is refused here.

⚠️ Switching an existing install deletes metrics history — If you set clusterScoped.enabled: false on an install that ran with it on, the next update deletes this install's Prometheus, Alertmanager, node-exporter and kube-state-metrics, and their metrics history. Octostar did not yet run this path on a cluster. Setting certManager.enabled: false or monitoring.external.loki.url leaves the old release in place. Remove it by hand with helm uninstall.

A second Octostar install

⚠️ Not fully supported yet — The ClickHouse, CloudNativePG and Octostar operators, octostar-api, SeaweedFS and the ClusterIssuer use fixed cluster-wide names. The preflight names each one, and it refuses to upgrade the shared releases of the first install.

Point the second install at the first install's monitoring:

namespace: octostar-b
monitoring:
  namespace: octostar-b
  clusterScoped:
    enabled: false
  external:
    prometheus:
      url: "http://kube-prometheus-stack-prometheus.octostar-main.svc:9090"
    loki:
      url: "http://loki.octostar-main.svc:3100"

If it fails

• "An object with the same name exists". Another Helm release or tool owns it. Use another name, or remove the old object if it is yours.
• "A Prometheus already runs". Set monitoring.clusterScoped.enabled: false (see above).
• "The API server would refuse a ClusterRole". The account holds fewer rights than the role grants. Use escalate and bind, or cluster-admin.
• A read is denied. The preflight reports it and does not assume "nothing there". Give the account get and list rights on the kind it names.