Architecture

Citeck Launcher is a single Go binary (~24 MB) with no external dependencies (pure Go, no CGO). It combines a CLI client and a background daemon that interacts with Docker Engine to manage platform containers.

General overview

+------------------+       Unix socket       +------------------+
|                  | ----------------------> |                  |
|   CLI (citeck)   |       HTTP API          |   Daemon         |
|                  | <---------------------- |   (HTTP server)  |
+------------------+                         +--------+---------+
                                                      |
                                               Docker SDK
                                                      |
                                             +--------v---------+
                                             |                  |
                                             |   Docker Engine  |
                                             |                  |
                                             +--------+---------+
                                                      |
                              +------------------------------------------+
                              |                  |                       |
                       +------v------+    +------v------+    +----------v--+
                       |  postgres   |    |  keycloak   |    |    eapps    |
                       +-------------+    +-------------+    +-------------+
                       |  zookeeper  |    |  rabbitmq   |    |    emodel   |
                       +-------------+    +-------------+    +-------------+
                       |   proxy     |    |   pgadmin   |    |    uiserv   |
                       +-------------+    +-------------+    +-------------+

Components:

  • CLI – Cobra commands, output formatting (text/JSON), interactive TUI prompts

  • Daemon – HTTP server on a Unix socket serving the REST API and SSE event stream

  • Docker SDK – direct interaction with Docker Engine via the official Go SDK

  • Containers – Docker containers with Citeck platform components

CLI and daemon interaction

The CLI and daemon communicate via a Unix socket (/run/citeck/daemon.sock). All CLI commands send HTTP requests to the daemon:

citeck status  ->  GET  /api/v1/namespace
citeck start   ->  POST /api/v1/namespace/start
citeck stop    ->  POST /api/v1/namespace/stop
citeck reload  ->  POST /api/v1/namespace/reload
citeck logs    ->  GET  /api/v1/apps/{name}/logs

For real-time monitoring (citeck status --watch, citeck start), SSE (Server-Sent Events) is used instead of WebSocket.

Container lifecycle

Each application goes through the following states:

READY_TO_PULL --> PULLING --> DEPS_WAITING --> STARTING --> RUNNING
     ^              |                             |
     |              v                             v
     |         PULL_FAILED                   START_FAILED
     |                                            |
     +--------------------------------------------+
               (reconciler retry: 1m -> 30m)

State descriptions:

State

Description

READY_TO_PULL

Initial state, ready to pull the image

PULLING

Pulling the Docker image

PULL_FAILED

Image pull failed

DEPS_WAITING

Waiting for dependencies to start (e.g. postgres before keycloak)

STARTING

Container created and starting

RUNNING

Container is running, liveness check passed

START_FAILED

Container failed to start

STOPPED

Container is stopped

On start failure, the reconciler automatically retries with exponential backoff: 1 minute, 2 minutes, 4 minutes, …, up to a maximum of 30 minutes.

Startup order

Applications start in dependency order:

Фаза 1:  postgres
Фаза 2:  keycloak, zookeeper, rabbitmq
Фаза 3:  Веб-приложения (eapps, emodel, gateway, ...)
Фаза 4:  proxy

Each application waits for its dependencies to be ready (DEPS_WAITING state) before starting.

The shutdown order is the reverse:

Фаза 1:  proxy
Фаза 2:  Веб-приложения
Фаза 3:  keycloak, zookeeper, rabbitmq
Фаза 4:  postgres

Deployment hash

When the reload command is run, the launcher uses a mechanism similar to docker-compose up:

  1. A deployment hash is computed for each application based on:

    • Docker image (image + tag)

    • Environment variables

    • Mounted volumes

    • Ports

    • Resource limits

    • Dependencies

  2. The hash is compared with the hash of the running container.

  3. Only changed containers are recreated – unchanged ones continue running without interruption.

Note

The stability of the GetHashInput function is a strict compatibility contract between launcher versions. Changing the hashing algorithm requires a migration.

Updating the CLI and daemon

The CLI and daemon are updated by the same install.sh script used for initial installation – it detects the presence of a previous version and switches to update mode:

curl -fsSL https://github.com/Citeck/citeck-launcher/releases/latest/download/install.sh | bash

When updating the launcher binary:

  1. A backup of the current binary is created (/usr/local/bin/citeck.bak).

  2. The daemon stops in detach mode: the daemon process exits, but Docker containers continue running.

  3. The new binary is copied to /usr/local/bin/citeck.

  4. A new daemon starts and picks up running containers by comparing deployment hashes.

Старый демон                     Новый демон
+-----------+                    +-----------+
| daemon v1 |  --detach-->       | daemon v2 |
+-----------+  (containers       +-----------+
     |          stay running)         |
+----v----+                      +----v----+
| Docker  |  =================>  | Docker  |
| Engine  |  (контейнеры         | Engine  |
+---------+   сохранены)         +---------+

What happens to applications depends on whether their deployment hash has changed (see the “Smart regenerate” section above):

  • Application hash unchanged (the new CLI version does not change the generator format for this application type) – the container continues running without a restart. This is the typical zero-downtime update scenario.

  • Application hash changed (e.g. the new version added an environment variable to the generator, the volume mount format changed, or the image version in the bundle was updated) – the affected containers are recreated. Expected downtime per application: 1 to 5 minutes (depends on service type; Java applications take longer).

Note

To find out in advance which applications will be recreated, run citeck reload --dry-run after updating the binary. The command will show the list of containers with a changed hash before you actually restart them.

If the update failed, use the rollback:

citeck install --rollback

This will restore the binary from .bak and restart the daemon. Docker containers are not affected.

systemd integration

The launcher automatically creates a systemd service on installation:

Unit file: /etc/systemd/system/citeck.service

Key characteristics:

  • Restart=on-failure – automatic restart on daemon failure

  • Docker containers continue running when the daemon restarts

  • A new daemon instance picks up running containers via hash matching

  • During update, a runtime drop-in (/run/systemd/system/citeck.service.d/no-restart.conf) is used to prevent systemd from restarting the old daemon while the binary is being replaced

Management via systemd:

# Статус сервиса
systemctl status citeck

# Логи демона через journald
journalctl -u citeck -f

# Перезапуск (контейнеры сохраняются)
systemctl restart citeck

Warning

Do not use systemctl stop citeck to stop the platform. This will stop only the daemon, not the containers. Use citeck stop --shutdown.

Secrets management

Secrets are stored with AES-256-GCM encryption in the /opt/citeck/conf/secrets/ directory.

Secret types:

  • System – generated automatically:

    • JWT key for token signing

    • OIDC client secret for Keycloak integration

    • Administrator password — for user login (ecos-app realm)

    • Password for the citeck service account — for internal Keycloak operations

    • Passwords for RabbitMQ and PgAdmin

  • User-defined – set via citeck setup:

    • SMTP password

    • S3 secret key

    • Docker registry credentials

Each secret is encrypted individually. The master password is used to derive encryption keys (PBKDF2-HMAC-SHA256).

Reconciler

The reconciler is a background process running in the daemon that ensures the desired and actual state of containers match:

  1. Periodic check (every 60 seconds by default): compares the configuration against the actual Docker state.

  2. Liveness checks (every 30 seconds by default): performs an HTTP request to the application’s health endpoint. After 3 consecutive failures – restart.

  3. Auto-recovery: when a crashed container is detected, initiates a restart with exponential backoff.

Configuration via /opt/citeck/conf/daemon.yml:

reconciler:
  interval: 60
  livenessPeriod: 30000
  livenessEnabled: true