OpenShift Production Deployment

This guide walks you through deploying the complete FileGrant infrastructure on OpenShift using a Helm chart. It covers architecture, prerequisites, step-by-step installation, DNS configuration, scaling, monitoring, security best practices, and uninstallation.

Architecture Overview

The Helm chart deploys three groups of components: core application services, supporting observability services, and OpenShift-specific infrastructure resources.

Core Services

• FileGrant Frontend — Web application frontend, pulled from Azure Container Registry.
• CGSignalR — SignalR service for real-time communication.
• RabbitMQ — Message broker with AMQPS support.
• FileGrantAPI — Main FileGrant REST API.
• FileGrantPdfProcessing — PDF processing service.
• FGLicense — License management service.

Supporting Services

• Loki — Centralized log aggregation.
• Alloy — Log collection agent deployed as a DaemonSet. Alloy replaces Promtail, which has reached end-of-life.
• Grafana — Dashboard for log visualization.

OpenShift Infrastructure

• Routes — HTTP/HTTPS traffic management with TLS edge termination.
• SecurityContextConstraints (SCC) — Permissions for the Alloy DaemonSet.
• NetworkPolicies — Security rules governing inter-pod communication.

Prerequisites

Before you begin, make sure you have the following in place:

• OpenShift Cluster — version 4.x or higher.
• oc CLI — configured and authenticated against your cluster.
• Helm 3.x — installed locally.
• DNS Domains — configured and pointing to the OpenShift Router IP.
• Azure Container Registry access — a service principal with pull permissions (used to create an imagePullSecret).

Verify the Cluster

Run the following commands to confirm your cluster is reachable and the ingress controller is operational:

oc status
oc version
oc get pods -n openshift-ingress

Get the OpenShift Router IP

You will need the Router IP for DNS configuration later. Retrieve it with:

oc get svc -n openshift-ingress router-default -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

Installation

The installation requires 3 phases due to sequential dependencies:

1. FileGrantAPI, RabbitMQ, and FGLicense must start first (core infrastructure).
2. RabbitMQ must be configured (virtual host and first user) before other services can connect.
3. FGLicense must perform the initial EEVMS variable setup in storage — without these variables, EEVMS certificates cannot be authorized.
4. Only after the initial setup can you generate certificates for CGSignalR and PDFProcessing and authorize the EEVMS keys.

Phase 1: Bootstrap — FileGrantAPI + RabbitMQ + FGLicense
  │
  ├─ Prepare values-production.yaml (without EEVMS certificates)
  ├─ helm install (only FileGrantAPI, RabbitMQ, FGLicense)
  ├─ Wait for pods to be Running
  ├─ Configure RabbitMQ (virtual host + first user)
  └─ FGLicense: initial EEVMS variable setup in storage
  │
Phase 2: Generate certificates and authorize EEVMS
  │
  ├─ Generate PFX certificates for CGSignalR and PDFProcessing
  ├─ Update values-production.yaml with certificates
  └─ Authorize EEVMS keys
  │
Phase 3: Enable all services
  │
  ├─ helm upgrade (CGSignalR and PDFProcessing enabled)
  ├─ CGSignalR and PDFProcessing start and load secrets from EEVMS
  └─ Final verification

Configure Azure Container Registry (ACR) Access

All service images are hosted on Azure Container Registry. Create a project and an imagePullSecret so OpenShift can pull the images:

oc new-project filegrant

oc create secret docker-registry acr-secret \
  --docker-server=cybergrantregistry-cuash9ckhse5gcha.azurecr.io \
  --docker-username=<SERVICE_PRINCIPAL_ID> \
  --docker-password=<SERVICE_PRINCIPAL_PASSWORD> \
  -n filegrant

Create the RabbitMQ TLS Secret

RabbitMQ TLS is enabled by default (rabbitmq.tls.enabled: true). You must create a TLS secret before installing the chart.

oc create secret tls rabbitmq-tls \
  --cert=path/to/tls.crt \
  --key=path/to/tls.key \
  -n filegrant

openssl req -x509 -nodes -days 365 \
  -newkey rsa:2048 \
  -keyout rabbitmq-tls.key \
  -out rabbitmq-tls.crt \
  -subj "/CN=rabbitmq.filegrant.svc.cluster.local"

oc create secret tls rabbitmq-tls \
  --cert=rabbitmq-tls.crt \
  --key=rabbitmq-tls.key \
  -n filegrant

Installation Steps

1. 1

   Prepare values-production.yaml (initial)

   Copy the default values file and customize it for your environment. In this phase, EEVMS certificates for CGSignalR and PDFProcessing are not needed yet — they will be added in Phase 2.

   Copy

   cp values.yaml values-production.yaml

   Edit the following fields in values-production.yaml:

   values-production.yaml (initial fields)

   Copy

   global:
     domain:
       base: "yourdomain.com"
     editorApiKey: "<editor-api-key>"
   
   grafana:
     auth:
       adminPassword: "secure-password"
   
   filegrantapi:
     secrets:
       masterCertificateBase64: "<base64-encoded-certificate>"
       masterCertificatePassword: "<certificate-password>"
       secretsAzureConnectionString: "<azure-blob-connection-string>"
   
   fglicense:
     secrets:
       monitoringApiKey: "<monitoring-api-key>"
       grafanaToken: "<grafana-service-account-token>"
   
   filegrantfe:
     env:
       pdfProcessingBaseUrl: "<value>"
       officeExcelIconUrl: "<value>"
       officeWordIconUrl: "<value>"
       officeWopisourceUrl: "<value>"
       officeWopitestUrl: "<value>"
       officePowerpointUrl: "<value>"
       officeExcelUrl: "<value>"
       officeWordUrl: "<value>"
       stripePublicKey: "<value>"
       cryptoKey: "<value>"
       remotegrantAgentBaseUrl: "<value>"
       apiBaseUrl: "<value>"
       azureResourcesContainer: "<value>"
       websocketUrl: "<value>"
       reactPdfLicenseKey: "<value>"

   Do not commit secrets

   Never commit values-production.yaml with real passwords or secrets to version control.
2. 2

   Assign the SCC for Alloy

   Alloy (the log collector) requires a privileged SecurityContextConstraints because it needs access to node logs at /var/log. The SCC is automatically created by the chart. Verify it has been applied:

   Copy

   oc get scc alloy-scc

3. 3

   Install the Chart (Phase 1 — Bootstrap)

   Install with only FileGrantAPI, RabbitMQ, and FGLicense enabled. CGSignalR and PDFProcessing are disabled because the EEVMS variables do not exist yet.

   Phase 1: Install core services

   Copy

   helm install filegrant . \
     --namespace filegrant \
     --create-namespace \
     -f values-production.yaml \
     --set cgsignalr.enabled=false \
     --set filegrantpdfprocessing.enabled=false

   Wait for filegrantapi, rabbitmq, and fglicense to reach the Running state with READY 1/1:

   Copy

   oc get pods -n filegrant -w

4. 4

   Configure RabbitMQ

   RabbitMQ requires the virtual host and the first application user to be configured before other services can connect.

   Access the RabbitMQ Management UI:

   Access RabbitMQ Management

   Copy

   # Via Route (if enabled)
   open https://rabbitmq.yourdomain.com
   
   # Or via port-forward
   oc port-forward -n filegrant svc/rabbitmq 15672:15672
   # Then open http://localhost:15672

   Log in with the credentials from values-production.yaml (default username: admin-rmq), then:

   1. Create the Virtual Host: go to Admin > Virtual Hosts, create the vhost required by FileGrant (e.g., /filegrant).
   2. Create the application user: go to Admin > Users, create the user that services will use to connect to RabbitMQ.
   3. Set permissions: assign the user permissions on the virtual host (configure, write, read: .*).
5. 5

   FGLicense — Initial EEVMS Variable Setup

   FGLicense must perform the initial setup to create all required variables in EEVMS storage. Without this step, EEVMS certificates cannot be authorized because the variables do not exist yet.

   Access FGLicense:

   Access FGLicense

   Copy

   # Via Route (if enabled)
   open https://license.yourdomain.com
   
   # Or via port-forward
   oc port-forward -n filegrant svc/fglicense 5003:8080
   # Then open http://localhost:5003

   Complete the initial setup. FGLicense will:

   • Create all required variables in EEVMS storage.
   • Set initial configuration values (connection strings, JWT keys, etc.).
   • Prepare the storage for EEVMS certificate authorization.

   Mandatory step

   This step is mandatory. If skipped, the EEVMS authorization script (Step 8) will fail with not_found errors for all variables.
6. 6

   Generate EEVMS Certificates

   Now that the variables exist in EEVMS storage, generate PFX certificates for CGSignalR and PDFProcessing. These certificates are used for RSA authentication with FileGrantAPI.

   Generate certificates

   Copy

   ./scripts/generate-eevms-certs.sh --password "YOUR_SECURE_PASSWORD"

   This creates the following files inside the certs/ directory:

   • cgsignalr.pfx / cgsignalr.b64 / cgsignalr-pubkey.pem
   • filegrantpdfprocessing.pfx / filegrantpdfprocessing.b64 / filegrantpdfprocessing-pubkey.pem
7. 7

   Update values-production.yaml with Certificates

   Add the EEVMS client certificate sections to values-production.yaml:

   Add to values-production.yaml

   Copy

   cgsignalr:
     eevms:
       apiUrl: "http://filegrantapi:8080"
       apiSecret: "<contents-of-certs/cgsignalr.b64>"
       apiSecretPassword: "YOUR_SECURE_PASSWORD"
   
   filegrantpdfprocessing:
     eevms:
       apiUrl: "http://filegrantapi:8080"
       apiSecret: "<contents-of-certs/filegrantpdfprocessing.b64>"
       apiSecretPassword: "YOUR_SECURE_PASSWORD"

8. 8

   Authorize EEVMS Keys

   Authorize the public keys so CGSignalR and PDFProcessing can read the secrets from EEVMS:

   Authorize EEVMS public keys

   Copy

   ./scripts/authorize-eevms-keys.sh \
     --api-url "https://filegrantapi.yourdomain.com" \
     --api-key "<editor-api-key>"

   Verify the output shows all variables as authorized. You should see CGSignalR with 11 authorized variables and PDFProcessing with 10.

   Variables not found?

   If some variables show as not_found, go back to Step 5 and verify that FGLicense has completed the initial setup.
9. 9

   Enable All Services (Phase 3)

   Now that the keys are authorized and the certificates are in values-production.yaml, enable CGSignalR and PDFProcessing:

   Phase 3: Enable remaining services

   Copy

   helm upgrade filegrant . \
     --namespace filegrant \
     -f values-production.yaml

   This starts the remaining services:

   • CGSignalR — connects to FileGrantAPI via EEVMS and loads 11 secrets.
   • PDFProcessing — connects to FileGrantAPI via EEVMS and loads 10 secrets.
10. 10

    Verify the Deployment

    Run the following commands to confirm all pods are running, all routes are created, and the services are logging as expected:

    Verify deployment health

    Copy

    oc get pods -n filegrant
    oc get routes -n filegrant
    oc get scc alloy-scc
    oc logs -n filegrant -l app=cgsignalr --tail=10
    oc logs -n filegrant -l app=filegrantpdfprocessing --tail=10
    oc logs -n filegrant -l app=fglicense --tail=10

    All pods should be in Running state after a few minutes.

Accessing Services

Public Services (via Route)

The following services are exposed externally through OpenShift Routes with TLS edge termination. Replace <domain> with your configured base domain.

Internal Services (Port-Forward)

Grafana and FGLicense (when its Route is disabled) are accessible through port-forward:

# Grafana (logging)
oc port-forward -n filegrant svc/grafana 3000:3000
# http://localhost:3000

# FGLicense (if Route is disabled)
oc port-forward -n filegrant svc/fglicense 5003:8080
# http://localhost:5003

Default Credentials

DNS Configuration

Configure A-type DNS records pointing to the OpenShift Router IP for each of the following subdomains:

• filegrant.<domain>
• signalr.<domain>
• rabbitmq.<domain>
• filegrantapi.<domain>
• pdfprocessing.<domain>

Updates and Rollback

Upgrading the Release

To apply configuration changes or deploy new image versions, run a Helm upgrade:

helm upgrade filegrant . -f values-production.yaml

Update a Specific Service

To update the image tag for a single service without changing anything else:

helm upgrade filegrant . \
  --set signalrserviceapi.image.tag=v1.2.3 \
  -f values-production.yaml

Rolling Back

If an upgrade causes issues, roll back to the previous revision or to a specific revision number:

# Roll back to the previous revision
helm rollback filegrant

# Roll back to a specific revision (e.g., revision 2)
helm rollback filegrant 2

Scaling

Manual Scaling

Scale any deployment manually using oc scale or via Helm:

# Via oc
oc scale deployment signalrserviceapi -n filegrant --replicas=3

# Or via Helm
helm upgrade filegrant . \
  --set signalrserviceapi.replicaCount=3 \
  -f values-production.yaml

Autoscaling (HPA)

You can enable Horizontal Pod Autoscaling for any service through the values file. For example, to autoscale the SignalR service:

signalrserviceapi:
  autoscaling:
    enabled: true
    minReplicas: 2
    maxReplicas: 10
    targetCPUUtilizationPercentage: 80

Monitoring and Logging

Accessing Logs with Grafana

The chart deploys Loki for log aggregation, Alloy for log collection, and Grafana for visualization. Access Grafana via port-forward (see above), then navigate to Explore and select Loki as the data source.

Here are some useful LogQL queries to get started:

# FileGrantAPI logs
{job="filegrantapi"}

# Error logs
{namespace="filegrant"} |~ "(?i)error"

# Specific pod logs
{pod="filegrantapi-xxx-yyy"}

OpenShift Metrics

# Pod CPU and memory
oc adm top pods -n filegrant

# Namespace events
oc get events -n filegrant --sort-by='.lastTimestamp'

Backup and Disaster Recovery

RabbitMQ Backup

Export the RabbitMQ definitions (users, vhosts, queues, exchanges, bindings) for backup:

oc exec -n filegrant rabbitmq-0 -- \
  rabbitmqctl export_definitions /tmp/definitions.json

oc cp filegrant/rabbitmq-0:/tmp/definitions.json ./rabbitmq-backup.json

Security

Best Practices

• Change all default passwords before going to production.
• Do not commit values-production.yaml with real passwords or secrets to version control.
• NetworkPolicies are enabled by default and restrict inter-pod communication to only what is necessary.
• SecurityContextConstraints are configured for the minimum required privileges.

Secrets Management

The following table summarizes the Kubernetes secrets created by the chart and the keys they contain. All secret values default to CHANGE_ME in values.yaml. Override them using --set or a separate values-production.yaml.

SecurityContextConstraints

• alloy-scc — Allows the Alloy DaemonSet to access node logs in privileged mode.
• All other services run with OpenShift's default restricted SCC (random UID, no privilege escalation).

Container Ports

All application services use port 8080 (not 80) to comply with OpenShift's restricted SCC, which does not allow binding to ports below 1024.

Troubleshooting

Routes Are Not Working

# Check Route status
oc describe route -n filegrant

# Check the OpenShift Router
oc get pods -n openshift-ingress

# Check Router logs
oc logs -n openshift-ingress -l ingresscontroller.operator.openshift.io/deployment-ingresscontroller=default

Pods Are Not Starting

# Check pod logs
oc logs -n filegrant <pod-name>

# Describe the pod to see events
oc describe pod -n filegrant <pod-name>

# Check available resources
oc adm top nodes

SCC Errors (SecurityContextConstraints)

# Verify SCCs assigned to the service account
oc get scc

# Verify that alloy-scc exists
oc describe scc alloy-scc

# Check Alloy pod events
oc describe pod -n filegrant -l app=alloy

Alloy Is Not Collecting Logs

# Check Alloy logs
oc logs -n filegrant -l app=alloy

# Verify that Alloy can see targets
oc exec -n filegrant -l app=alloy -- \
  wget -qO- http://localhost:12345/targets

# Verify that Loki is reachable
oc exec -n filegrant loki-0 -- wget -qO- http://localhost:3100/ready

Uninstallation

To completely remove the FileGrant deployment and clean up all associated resources:

# Remove the release
helm uninstall filegrant -n filegrant

# (Optional) Remove the project
oc delete project filegrant

oc delete pvc --all -n filegrant