Skip to main content

Appliance Backup & Disaster Recovery

CallTelemetry Appliance Backup (.cta) provides bare-metal and virtual machine disaster recovery for enterprise deployments. Unlike tenant-level JSON export or raw database dumps, an Appliance Archive (.cta) captures everything necessary to rebuild or migrate a complete CallTelemetry appliance onto fresh infrastructure in minutes.

Appliance Archive vs. Tenant Export
  • Appliance Archive (ct appliance export): Comprehensive appliance disaster recovery. Includes PostgreSQL/TimescaleDB data, AES-256-GCM encrypted secrets (CLOAK_KEY, database passwords), enterprise TLS certificates, reverse proxy configuration, media assets, and SeaweedFS object storage.
  • Tenant Export (Customer Data Import/Export): Application UI workflow for moving a single customer organization configuration between appliances (excludes secrets and global system state).
  • Database Backup (ct db backup / ct backup run): Standard compressed SQL dump (.sql.gz) of database tables.

Anatomy of a .cta Archive Bundle​

An Appliance Archive is a compressed tarball (.cta) containing the following cryptographically validated components:

calltelemetry-appliance-2026-09-04-150405.cta
├── manifest.json              # Version metadata, SHA-256 catalog, row counts, excluded tables
├── secrets/secrets.env.enc    # AES-256-GCM encrypted credentials vault (CLOAK_KEY, etc.)
├── database/
│   └── calltelemetry_prod.dump # PostgreSQL custom archive format (-Fc) for multi-core restore
├── certs/certs.tar.gz         # Enterprise TLS certificates, private keys, and trusted CAs
├── configs/configs.tar.gz     # Appliance reverse-proxy (Caddyfile) & SeaweedFS S3 configs
├── media/media.tar.gz         # Audio greetings (WAV, MP3, G.711) and custom branding assets
└── storage/seaweedfs-data.tar.gz # S3 object storage volume (metadata and file chunks)

Encryption & Security​

All cryptographic application secrets (including CLOAK_KEY used for encrypting Cisco credentials and LDAP passwords in the database, SECRET_KEY_BASE, and POSTGRES_PASSWORD) are encrypted inside secrets.env.enc using AES-256-GCM authenticated encryption with keys derived via PBKDF2-SHA256 (64,000 iterations).

Preflight & Dry-Run​

Before exporting an appliance archive, you can perform a non-destructive dry-run inspection to estimate backup size and verify component availability:

sudo ct appliance export --dry-run

Sample output:

=== Appliance Export Dry-Run Inspection ===
Appliance Version:   0.8.7-rc483
Database Size:       14.2 GB
Estimated Records:   ~84.2M records (from pg_class catalog)
Configurations:      2 files (seaweedfs-s3.json, Caddyfile)
Media Assets:        18 files (4.2 MB)
SeaweedFS Storage:   Ready (128.4 MB)
Debug & Activity:    Included (all diagnostic tables and user activity logs)
✓ Dry-run completed. All preflight checks passed.

Exporting an Appliance Archive​

To export a full appliance archive to local disk:

# Export with optional custom encryption passphrase (interactive masked prompt if omitted)
sudo ct appliance export /var/backups/

# Or supply passphrase via flag or secure file
sudo ct appliance export /var/backups/ --passphrase "YourSecurePassphrase"
sudo ct appliance export /var/backups/ --passphrase-file /etc/calltelemetry/backup.key
Passphrase Is Fully Optional
  • Interactive Terminal: When --passphrase or --passphrase-file is omitted, the CLI presents a masked input prompt (*) where you can enter a custom passphrase or simply press Enter to skip.
  • Default Appliance Key: If skipped or run non-interactively without a passphrase, secrets are automatically encrypted using CallTelemetry's internal default vault key.
  • Zero-Touch Restore: Archives created with the default vault key restore seamlessly without prompting for a passphrase.

Excluding Debug Tables and Activity Logs (--skip-logs)​

For appliances with high-volume debugging history, you can omit diagnostic tracing and audit events while preserving full call history:

sudo ct appliance export /var/backups/ --skip-logs
What --skip-logs Omits

--skip-logs uses PostgreSQL's --exclude-table-data directive to omit ephemeral logs while preserving table definitions, constraints, and indexes:

  • CURRI Debugs: curri_event_debugs, curri_event_responses, curri_event_post_call_processing_complete
  • CUBE / XCC Logs: xcc_mirror_logs, cube_webhook_logs, cube_recovery_failures, cube_connection_health_events
  • CDR Debugs & Alerts: cdr_webhook_logs, cdr_event_logs, cdr_alert_logs
  • MS Teams Debug & Logs: teams_log_events, ms_teams_webhook_events, teams_workflow_logs, teams_emergency_resolver_logs, teams_app_runs
  • Audit & Sync: user_activity_logs, config_audit_events, config_sync_audit_events, discovery_run_logs, discovery_phone_logs

Retains: Primary call records and routing events (curri_events, cdrcalls, cmr_records, ms_teams_call_records).

Configuration-Only Archive (--skip-data)​

If you want a lightweight, portable backup for lab staging, rapid migration, or disaster recovery without historical call records:

sudo ct appliance export /var/backups/ --skip-data
Configuration-Only Archive

--skip-data (or --config-only, --skip-telemetry) excludes both debug logs and voluminous operational call data:

  • Call Routing Events: curri_events, cube_event_logs, call_disconnect_metrics
  • Cisco CDR & CMR: cdrcalls, cmr_records, ios_cdrs, dime_cdr_downloads
  • Microsoft Teams Call Data: ms_teams_call_records, ms_teams_pstn_usage_records, ms_teams_call_sessions, teams_user_events, teams_chats, teams_chat_messages
  • Historical Rollups: policy_history_*, policy_analytics_*, ms_teams_call_queue_*, ms_teams_voice_apps_*

Automatic Cascade for Relational Integrity: Because child diagnostic and webhook tables (e.g. curri_event_debugs, curri_event_responses, teams_log_events, cube_webhook_logs) contain foreign keys and direct relational bindings to parent call and routing records (curri_events, ms_teams_call_records), exporting child logs without the parent data would violate PostgreSQL foreign key constraints on restoration. Therefore, --skip-data automatically includes --skip-logs, cascading to exclude all 29 related debug, audit, and webhook tables alongside the 31 operational data tables.

Preserves: 100% of all configurations, organizations, users, policies, rules, CUBEs, CallManagers, phones, switches, MS Teams configs, enterprise TLS certificates, reverse proxy configuration, and encrypted secrets vault.

Additional Export Options​

FlagDescription
--skip-logs / --no-logsExclude diagnostic debug tables, webhooks, and user activity logs (retains call records)
--skip-data / --config-onlyExclude all call records, CURRI events, CDRs, and Teams data (creates config-only archive)
--skip-mediaExclude audio greetings and custom branding assets
--skip-storageExclude SeaweedFS S3 object storage volume data
--clean-localRemove local archive file after successful upload to remote SFTP/FTP server
--dry-runSizing estimate and preflight validation without writing files

Exporting Directly to Remote SFTP / FTP​

Appliance archives can be streamed directly to remote SFTP or FTP backup servers:

# Export to remote SFTP server with auto-cleanup of local staging
sudo ct appliance export sftp://backupuser@storage.example.com:22/backups/calltelemetry/ \
  --passphrase-file /etc/calltelemetry/backup.key \
  --clean-local

# Export to remote FTP server
sudo ct appliance export ftp://backupuser@ftp.example.com/backups/ \
  --passphrase-file /etc/calltelemetry/backup.key

Inspecting an Archive (ct appliance inspect)​

You can inspect the contents, checksums, and metadata of any local or remote .cta archive without restoring it:

sudo ct appliance inspect /var/backups/calltelemetry-appliance-2026-09-04.cta

Sample output:

=== CallTelemetry Appliance Archive Inspection ===
Archive:           calltelemetry-appliance-2026-09-04.cta
Created At:        Fri, 04 Sep 2026 15:04:05 UTC
Appliance Version: 0.8.7-rc483
Format Version:    1.0

Component Summary:
  - Database:      database/calltelemetry_prod.dump (14.2 GB, ~84.2M estimated records)
    Format:        pg_dump_custom (multi-core parallel restore compatible)
    SHA-256:       e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
    Excluded Data: 14 debug/activity log tables (--skip-logs)
    Key Tables (from pg_class catalog):
      cdrcalls                 42,500,000
      cmr_records              41,200,000
      curri_events             450,000
      orgs                     2
      users                    14
  - Secrets Vault: secrets/secrets.env.enc (AES-256-GCM, PBKDF2-SHA256)
    Keys Secured:  CLOAK_KEY, SECRET_KEY_BASE, GUARDIAN_SECRET_KEY, POSTGRES_PASSWORD, GRAFANA_PASSWORD
  - Certificates:  certs/certs.tar.gz (4 files, 12.8 KB)
  - Config Files:  configs/configs.tar.gz (seaweedfs-s3.json, Caddyfile, 3.4 KB)
  - Media Assets:  media/media.tar.gz (18 files, 4.2 MB)
  - S3 Storage:    storage/seaweedfs-data.tar.gz (128.4 MB)

✓ Archive structure and manifest verified successfully.

You can also inspect remote archives directly:

sudo ct appliance inspect sftp://backupuser@storage.example.com/backups/calltelemetry-appliance-2026-09-04.cta

Disaster Recovery & Appliance Restore (ct appliance import)​

Restoring an appliance replaces the current database and adopts encrypted credentials into the host configuration.

# Restore archive (zero-prompt decryption if exported with default key)
sudo ct appliance import /var/backups/calltelemetry-appliance-2026-09-04.cta --yes

# Or provide custom passphrase directly (if custom encrypted)
sudo ct appliance import /var/backups/calltelemetry-appliance-2026-09-04.cta \
  --passphrase "YourSecurePassphrase" --yes
Automatic Vault Decryption

If the archive was created without a custom passphrase, ct appliance import decrypts the configuration secrets vault automatically using the default appliance key with zero prompts. If the archive was created with a custom passphrase, the CLI interactively prompts for the passphrase with masked input (*).

The Restoration Pipeline​

The ct appliance import command follows an automated, crash-resilient 9-stage restoration pipeline:

  1. Early Secret Validation: Decrypts the secrets vault payload before stopping any services. If the passphrase is wrong or corrupted, restore aborts immediately with zero downtime.
  2. Version Compatibility Verification: Validates the archive version against the current appliance image.
  3. Service Quiescing: Gracefully stops dependent consumers (web, nats, jtapi-sidecar, ct-media) to prevent partial database writes.
  4. Connection Termination: Drops lingering connections via pg_terminate_backend().
  5. TimescaleDB Catalog Preparation: Calls timescaledb_pre_restore() to pause background hypertables workers and compression policies.
  6. Multi-Core Parallel Restore: Runs pg_restore with auto-detected multi-core parallelism (-j <vcpus>), restoring millions of CDR and audit records in minutes.
  7. TimescaleDB Finalization: Calls timescaledb_post_restore() to reactivate chunks, compression jobs, and continuous aggregates.
  8. Credential Adoption: Decrypted secrets (including CLOAK_KEY and database passwords) are atomically synchronized into /home/calltelemetry/.env and /etc/calltelemetry/.env.
  9. Component Extraction & Service Bringup: Unpacks TLS certificates, Caddy reverse-proxy rules, custom media greetings, and S3 object storage volumes. Services are brought online and verified via pg_isready and health endpoints.

Non-Interactive Automation​

For scripted disaster recovery pipelines or infrastructure-as-code automation:

sudo ct appliance import sftp://backupuser@storage.example.com/backups/latest.cta \
  --passphrase-file /etc/calltelemetry/backup.key \
  --yes

PostgreSQL Major Version Upgrade​

PostgreSQL major version upgrades (e.g. PostgreSQL 14 → 15, 16, 17, 18) are fully automated through the ct CLI. CallTelemetry handles target image preflight, compressed volume backup, container orchestration, schema restore, and TimescaleDB extension catalog hooks automatically. Operators never need to manually edit docker-compose.yml or run raw Docker commands.

In-Place Upgrade Workflow (3 Phases)​

For on-box upgrades with sufficient disk space (≥ 2.5x database size), CallTelemetry provides a first-class, structured workflow:

Phase 1: Preflight Validation (--dry-run)​

Always run preflight validation before upgrading. This inspects the target PostgreSQL image, probes TimescaleDB extension compatibility, and validates disk space without stopping any running services:

# Validate PostgreSQL 15 upgrade readiness
sudo ct postgres upgrade 15 --dry-run

(Also available via sudo ct appliance upgrade-postgres 15 --dry-run or sudo ct db upgrade 15 --dry-run)

The preflight check:

  1. Target Image Verification: Validates that the target version is officially supported (14, 15, 16, 17, 18) and pulls the image to verify repository reachability.
  2. Disk Sizing & Headroom: Queries database catalog size and confirms that the host filesystem has ≥ 2.5x database free headroom to create the pre-upgrade dump and new volume.
  3. Extension Probe: Connects to the database and probes required extensions (including TimescaleDB) to ensure catalog binary compatibility.

Phase 2: Fully Automated Execution​

Once preflight checks pass, execute the upgrade:

# Standard automated upgrade (includes interactive countdown prompt)
sudo ct postgres upgrade 15

# Accelerated upgrade (excludes verbose debug tables, skips prompt)
sudo ct postgres upgrade 15 --skip-logs --yes
FlagDescription
--dry-runValidates target image, installed extensions, and disk headroom without taking down services
--skip-logs, --no-logsExcludes debug event tables and user activity logs from the upgrade backup to accelerate completion
--yes, -yConfirms execution without interactive countdown prompt

Under the hood, CallTelemetry executes the complete migration pipeline:

  1. Configuration Snapshot: Downloads the official target compose override (e.g. postgres-15.yaml) into docker-compose.override.yml, safely snapshotting any existing override to postgres-override.v<oldVer>-<ts>.bak.
  2. Compressed Database Dump: Takes a compressed SQL dump of the database into /var/backups/postgres-upgrade-<ver>-<date>.sql.gz.
  3. Service Quiescing: Gracefully stops dependent services (web, nats, jtapi-sidecar, ct-media).
  4. Volume Isolation: Moves postgres-data to postgres-data.v<oldVer>-<ts>.bak. The previous data is preserved untouched for instant rollback.
  5. Fresh Engine Boot: Boots the target PostgreSQL engine container on a clean volume.
  6. TimescaleDB Pre-Restore Hook: Invokes SELECT timescaledb_pre_restore(); to prepare hypertables and pause compression policies.
  7. Multi-Core Parallel Restore: Restores the database schema and data using parallel workers.
  8. TimescaleDB Post-Restore Hook: Invokes SELECT timescaledb_post_restore(); to reactivate background workers and compression jobs.
  9. Service Bringup & Health Gate: Restarts all appliance services and monitors pg_isready until the database is fully operational.

Phase 3: Rollback Plan (ct postgres rollback)​

CallTelemetry incorporates self-healing failure recovery. If any error occurs during execution or health verification, the upgrade engine automatically aborts and restores the original volume.

If you ever need to manually revert back to the previous version after completing an upgrade:

# Interactive rollback
sudo ct postgres rollback

# Instant non-interactive rollback
sudo ct postgres rollback --yes

(Also available via sudo ct appliance rollback-postgres or sudo ct db rollback)

The rollback engine:

  1. Automatically identifies the latest preserved backup volume (postgres-data.v<oldVer>-<ts>.bak).
  2. Quiesces running services (docker compose down).
  3. Safely preserves the aborted volume as postgres-data.v<curVer>-aborted-<ts>.bak so no newly written data is lost.
  4. Restores the preserved backup volume back to postgres-data.
  5. Restores the previous docker-compose.override.yml configuration.
  6. Restarts all services (docker compose up -d) and verifies database connectivity.

All original data and services are restored in under 60 seconds with zero manual configuration.

Cross-Server Migration & Fresh OVA Deployment (ct appliance)​

For migrating to a new virtual machine, deploying a fresh OVA, or upgrading severely disk-constrained appliances (< 2.5x free disk headroom), use the Appliance Archive Workflow (ct appliance export → ct appliance import).

This workflow streams data directly to remote SFTP/FTP storage, requiring zero additional local disk space on the appliance:

Step 1: Export Source Appliance to Remote Storage​

# Export full appliance archive directly to SFTP with local cleanup
sudo ct appliance export sftp://backupuser@storage.example.com/backups/migration-pg15.cta \
  --clean-local

Step 2: Deploy Fresh OVA or Target Server​

Deploy a fresh CallTelemetry virtual machine running the target PostgreSQL version.

Step 3: Import Archive on Destination Host​

# Restore full database, secrets vault, media, and certificates on new host
sudo ct appliance import sftp://backupuser@storage.example.com/backups/migration-pg15.cta --yes

The import automatically adopts secrets into .env, restores TimescaleDB hypertables, unpacks enterprise certificates and media assets, and boots all services online. Zero manual file editing is ever required.

Automated Recurring Backups (ct backup schedule)​

CallTelemetry appliances support fully automated, recurring disaster recovery backups (.cta) managed entirely through the native CLI. Powered by native Linux systemd timers, scheduled backups persist across reboots, catch up if the appliance was powered off during the scheduled window, include bounded execution jitter, and automatically rotate older backups according to your retention policy.

Operators never need to configure crontabs, edit system files, or write custom shell scripts.

Checking Current Schedule & Status​

View the active schedule, timer status, target destination, and last/next execution times:

sudo ct backup schedule

(Also accessible via sudo ct appliance schedule-backup)

Example output:

Scheduled Appliance Backups (.cta)
===================================
Status:       Enabled
Schedule:     Daily at 02:00 UTC
Calendar:     *-*-* 02:00:00
Target:       sftp://backupuser:***@storage.corp.lan/backups/
Flags:        --skip-logs --clean-local
Retention:    Keep newest 7 backups
Next Run:     tomorrow at 02:00:00 UTC
Last Backup:  2026-09-04T02:00:38Z (SUCCESS)

Commands:
  Run now:      ct backup schedule run
  Show logs:    ct backup schedule log
  Disable:      ct backup schedule disable
  Reconfigure:  ct backup schedule enable [cadence] [flags]

Enabling or Reconfiguring Recurring Backups​

1. Daily Backups to Local Storage​

Schedule a daily full .cta archive at 02:00 UTC, retaining the newest 7 archives:

sudo ct backup schedule enable daily --time 02:00 --target /var/backups/ --retention 7

2. Weekly Backups to Remote SFTP Storage with Log Filtering​

Schedule a weekly backup every Sunday at 03:00 UTC, uploading directly to remote SFTP, excluding ephemeral debug logs to save bandwidth, and cleaning up local staging files:

sudo ct backup schedule enable weekly --day sun --time 03:00 \
  --target sftp://backupuser:mypassword@storage.corp.lan/backups/ \
  --retention 4 \
  --skip-logs \
  --clean-local

3. Monthly Backups​

Schedule a monthly archive on the 1st of each month at 01:00 UTC:

sudo ct backup schedule enable monthly --time 01:00 --target /var/backups/ --retention 12

4. Custom Calendar Schedule​

Specify any systemd OnCalendar expression for complex schedules:

sudo ct backup schedule enable --calendar "Mon,Wed,Fri *-*-* 04:00:00" --retention 5

Command Options for ct backup schedule enable​

OptionDefaultDescription
daily | weekly | monthlydailyRecurring frequency
--time <HH:MM>02:00Time of day to run the backup in UTC (24-hour format)
--day <sun..sat>sunDay of week for weekly backups
--target <path|url>/var/backups/Destination directory or remote URL (sftp:// / ftp://)
--retention <N>7Number of newest archives to retain (older ones are pruned automatically)
--skip-logsfalseExclude ephemeral debug and audit log tables from the archive
--skip-datafalseConfiguration-only archive (exclude operational call records and logs)
--skip-mediafalseExclude audio greetings and custom branding assets
--clean-localfalseRemove local archive file after successful remote SFTP/FTP upload
--passphrase <secret>(default key)Encrypt secrets vault with a custom passphrase
--passphrase-file <file>—Read custom encryption passphrase from file

Testing and Running On Demand​

To test your scheduled backup configuration immediately without waiting for the timer:

sudo ct backup schedule run

Viewing Execution History & Journal Logs​

Inspect recent scheduled backup runs and systemd journal logs:

sudo ct backup schedule log

Disabling Scheduled Backups​

To suspend automated backups:

sudo ct backup schedule disable

Troubleshooting & Diagnostics​

If an error occurs during export or import, detailed logs are automatically preserved:

✗ Restore failed: pg_restore exited with code 1
Diagnostic log: /home/calltelemetry/dumps/restore-debug-2026-09-04-153022.log
To inspect the failure details:
  cat /home/calltelemetry/dumps/restore-debug-2026-09-04-153022.log

Common diagnostic checks:

  • Verify passphrase correctness using ct appliance inspect <archive>.
  • Check database container logs: sudo ct logs db.
  • Validate container health and storage permissions: sudo ct doctor.
Last updated on by Call Telemetry