self hosting
n8n Queue Mode Setup: Redis, Postgres, Workers, and Encryption Key Checklist
Queue mode splits n8n into a main process, Redis-backed queue, database, and worker processes so executions can scale beyond a single process; treat it as a production architecture change, not just a checkbox.
Match your incident first
Start with the symptom you can prove
Webhook request reaches n8n but becomes slow or times out after queue mode
First check: Match the webhook access log timestamp to the queued execution and worker pickup time.
Wrong fix to avoid: Do not add more webhook processors before proving workers, Redis, and database throughput can drain the queue.
Verify: A fresh webhook request creates a queued execution, the worker starts it promptly, and the caller receives the expected response.
Worker starts but does not process jobs
First check: Compare queue mode, Redis host/db, DB_TYPE, and n8n image tag between main and worker.
Wrong fix to avoid: Do not scale more workers before proving one worker shares the same queue and database.
Verify: A new queued execution changes from waiting to running on a worker and finishes.
Redis connection refused or ENOTFOUND redis
First check: From the worker network, run a Redis ping using the same host and port configured for n8n.
Wrong fix to avoid: Do not point workers at localhost unless Redis is inside the same container.
Verify: redis-cli ping returns PONG and worker logs stop reconnecting.
- Use when
- self-hosted, scaling, queue mode
- First check
- Confirm main and worker processes share Redis, database, and N8N_ENCRYPTION_KEY.
- Time to check
- 5-10 minutes
- Next step
- Match the symptom, then run the verification checks.
Independent third-party notes. n8n is a trademark of its owner and is referenced only for compatibility and troubleshooting context.
Quick Answer
Queue mode splits n8n into a main process, Redis-backed queue, database, and worker processes so executions can scale beyond a single process; treat it as a production architecture change, not just a checkbox.
Does this match your symptom?
Queue mode or workers are stuck
Jobs stay queued, workers look healthy but do not process executions, or Redis/database settings drift.
First check: Confirm main and worker processes share Redis, database, and N8N_ENCRYPTION_KEY.
Problem Pattern
The user has outgrown a single n8n process or is considering queue mode, but needs to understand the moving parts before enabling it.
Queue launch gates
Do not enable queue mode until these gates pass
Queue mode changes execution ownership. Prove the shared infrastructure path before sending production traffic through workers.
| Gate | Pass condition | Stop if | Proof to keep |
|---|---|---|---|
| Shared secrets | Main, worker, and any webhook processor use the same N8N_ENCRYPTION_KEY without printing the raw value. | Credentials decrypt on main but fail only on worker. | Private fingerprint or yes/no parity note for each process. |
| Redis and database | Each process reaches the same Redis and Postgres/database settings from its own network namespace. | Redis returns ENOTFOUND/refused, DB settings differ, or SQLite is still the distributed backend. | Redis PONG, DB host/db name parity, and a successful queued execution ID. |
| Worker readiness | A worker reports ready, picks up a fresh execution, and finishes it before more workers are added. | Jobs remain waiting, workers reconnect, or concurrency was changed without checking database pool limits. | Worker logs, readiness response if enabled, concurrency value, and completion timestamp. |
| Event log uniqueness | Every main, worker, and webhook processor that shares writable storage has its own event log .log path. | Several processes write to the same configured event log file or recovery logs mention parse/corruption failures. | Safe path list with process names, not raw secrets or payload data. |
| Webhook routing | Production webhook paths route to the intended main or webhook processor pool, while test webhook paths stay on main. | The main UI process receives heavy production webhook load or /webhook-test/* is routed away from main. | Load balancer route notes and one external webhook request matched to one queued execution. |
| Binary data | Workers can read and write the same binary data storage expected by production workflows. | A workflow depends on local filesystem binary data that workers cannot access. | A file write/read workflow execution handled by a worker. |
| Manual and runner path | Manual execution offloading is intentional and every Code node execution path has a matching task runner sidecar when external runners are enabled. | Manual tests pass on main while production workers fail, or worker Code nodes cannot reach a task runner. | OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS value, runner sidecar map, and one Code node smoke test on a worker. |
If any gate fails, keep the single-process deployment stable and fix the shared infrastructure path before scaling worker count.
Version awareness
Last reviewed 2026-06-23
Key Facts
- Best scalability
- n8n describes queue mode as the best scalability option.
- Execution flow
- The main process creates execution IDs, Redis holds the queue, workers process executions, and the database stores workflow data and results.
- Required mode
- Set EXECUTIONS_MODE=queue on the main instance and workers.
- Database note
- For distributed queue mode, Redis and a shared database are required; n8n documents that this setup over SQLite is not supported.
- Webhook processors
- Webhook processors are optional, but they add a separate layer for scaling incoming production webhook requests.
- Manual execution path
- Manual executions run on the main process unless OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS is enabled, so editor tests may not prove worker readiness.
- Task runners
- When task runners are used with queue mode, each worker needs its own runner sidecar; if manual executions are not offloaded, the main process needs one too.
- Per-process event logs
- When multiple n8n processes share a writable filesystem, give each process a unique N8N_EVENTBUS_LOGWRITER_LOGFULLPATH value so event recovery logs do not collide.
Production Diagnostic Matrix
Turn checks into a brief| Exact symptom or log | Likely cause | First check | Wrong fix to avoid | Verification |
|---|---|---|---|---|
| Webhook request reaches n8n but becomes slow or times out after queue mode | The main or webhook process receives the HTTP request, then the execution is queued for a worker, adding latency or exposing worker backlog. | Match the webhook access log timestamp to the queued execution and worker pickup time. | Do not add more webhook processors before proving workers, Redis, and database throughput can drain the queue. | A fresh webhook request creates a queued execution, the worker starts it promptly, and the caller receives the expected response. |
| Worker starts but does not process jobs | Worker is connected to a different Redis, different database, or wrong queue settings than the main instance. | Compare queue mode, Redis host/db, DB_TYPE, and n8n image tag between main and worker. | Do not scale more workers before proving one worker shares the same queue and database. | A new queued execution changes from waiting to running on a worker and finishes. |
| Redis connection refused or ENOTFOUND redis | Redis service name, port, network, password, or startup order is wrong. | From the worker network, run a Redis ping using the same host and port configured for n8n. | Do not point workers at localhost unless Redis is inside the same container. | redis-cli ping returns PONG and worker logs stop reconnecting. |
| Jobs stay waiting or active forever | Workers are offline, concurrency is saturated, Redis state is stale, or a workflow step never completes. | Check worker logs, queue depth, CPU/memory, and the oldest waiting execution. | Do not delete production Redis data until you have a backup and know the impact. | A fresh test execution starts promptly and the old stuck execution is handled intentionally. |
| Database timeouts or connection errors start after adding more workers | Worker count, per-worker concurrency, main process, webhook processors, and other app connections exceed the database pool or managed database limit. | Count active n8n processes and worker concurrency before changing Redis or adding more workers. | Do not keep scaling worker count when the bottleneck is database connections or slow workflow steps. | A smaller worker pool with measured concurrency drains a new queue job without new Postgres connection errors. |
| Credentials fail only on worker | Main and worker do not share the same N8N_ENCRYPTION_KEY or credential-related env vars. | Compare only safe variable names and fingerprints, not raw secret values, across main and worker. | Do not recreate all credentials before checking encryption key consistency. | The same credential works in an execution handled by a worker. |
| Binary data unavailable on worker | Binary data mode, filesystem volume, or external storage settings differ between main and worker. | Check binary data mode and whether the same storage path/bucket is mounted for every process. | Do not switch binary modes during an incident without a migration plan. | A workflow that writes and reads a file succeeds when handled by a worker. |
| Worker logs version mismatch or unknown node type | Main and worker run different n8n images or custom node installations. | Compare image tags and custom node directories between all n8n containers. | Do not upgrade only one worker in a queue-mode cluster. | All containers report the same n8n version and the failing node type loads on the worker. |
| Event recovery or log streaming errors appear after adding workers on a shared volume | Multiple n8n processes are writing to the same event log file path. | List the configured N8N_EVENTBUS_LOGWRITER_LOGFULLPATH value for each main, worker, and webhook processor. | Do not delete event logs during an incident before preserving enough evidence to understand what was not forwarded. | Each process writes to a distinct .log file and new recovery/log-streaming errors stop. |
| Manual run succeeds in the editor, but production queue execution fails | Manual executions are still running on main, while production executions run on workers with different network, env, binary data, or task runner access. | Check OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS, then run a production-mode smoke test that must be picked up by a worker. | Do not mark queue mode ready from editor-only manual tests. | A production execution and, if enabled, an offloaded manual execution both complete on the expected worker path. |
| Code node works on main but fails or hangs on worker | External task runner sidecars, runner auth token, broker URI, allowed modules, or runner image version differ from the worker path. | Confirm the worker has a matching runner sidecar and that n8n and runner images use the same tested version. | Do not relax runner security settings or expose runner tokens before proving the sidecar topology. | A safe Code node smoke test completes on a queued worker execution without printing secrets. |
Still blocked after these checks?
Use the brief to decide whether to keep fixing this setup, move the workload to n8n Cloud, or rebuild the self-hosted path on cleaner infrastructure.
Recommended Steps
- Set a shared N8N_ENCRYPTION_KEY for the main process and all workers.
- Set EXECUTIONS_MODE=queue.
- Start and configure Redis.
- Configure n8n Redis host, port, and optional credentials, then test Redis reachability from the worker path, not only from the Redis container.
- Start one or more worker processes.
- Decide whether manual executions should be offloaded to workers and set OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS intentionally.
- If Code node task runners are enabled, attach runner sidecars to every worker and any main process that still runs manual executions.
- Keep all main and worker processes on the same n8n version.
- If workers share a writable volume, set a unique event log full path for each main, worker, and webhook processor process.
- If webhook traffic is high, add webhook processors and route production webhook paths to them instead of the editor/UI main process.
Verification
- Workers connect to Redis and the database.
- Production executions are processed by workers.
- Worker health or readiness endpoints pass if enabled.
- Manual and production smoke tests run on the expected process path, especially when OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS is false.
- Each process has a unique event log path when shared writable storage is used.
- One production webhook request reaches the webhook or main process, creates a queued execution, and completes on a worker.
First Commands / Checks
docker compose logs worker --tail=100 docker compose exec worker node -e "const net = require('node:net'); const host = process.env.QUEUE_BULL_REDIS_HOST || 'redis'; const port = Number(process.env.QUEUE_BULL_REDIS_PORT || 6379); const socket = net.createConnection({ host, port }, () => { console.log('redis tcp reachable from worker'); socket.end(); }); socket.setTimeout(5000, () => { console.error('redis tcp timeout from worker'); socket.destroy(); process.exit(1); }); socket.on('error', (error) => { console.error(error.code || error.message); process.exit(1); });" docker compose exec n8n printenv EXECUTIONS_MODE OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS QUEUE_BULL_REDIS_HOST DB_TYPE N8N_DEFAULT_BINARY_DATA_MODE N8N_RUNNERS_ENABLED N8N_RUNNERS_MODE
docker compose exec worker printenv EXECUTIONS_MODE OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS QUEUE_BULL_REDIS_HOST DB_TYPE N8N_DEFAULT_BINARY_DATA_MODE N8N_RUNNERS_ENABLED N8N_RUNNERS_MODE docker compose exec worker printenv QUEUE_HEALTH_CHECK_ACTIVE QUEUE_HEALTH_CHECK_PORT
docker compose exec worker node -e "fetch('http://127.0.0.1:' + (process.env.QUEUE_HEALTH_CHECK_PORT || 5678) + '/healthz/readiness').then(r => { console.log(r.status); process.exit(r.ok ? 0 : 1); })" docker compose ps --services | grep -E '^(n8n|worker|webhook)' || true
docker compose exec worker printenv EXECUTIONS_MODE DB_TYPE QUEUE_BULL_REDIS_HOST
docker compose logs worker --tail=50 for service in n8n worker webhook; do
docker compose ps --services | grep -qx "$service" || continue
echo "[$service]"
docker compose exec "$service" printenv N8N_EVENTBUS_LOGWRITER_LOGFULLPATH N8N_EVENTBUS_LOGWRITER_MAXTOTALMESSAGESPERFILE
done Safe Copyable Config
x-n8n-env: &n8n-env
DB_TYPE: postgresdb
DB_POSTGRESDB_HOST: postgres
DB_POSTGRESDB_DATABASE: n8n
DB_POSTGRESDB_USER: n8n_app
DB_POSTGRESDB_PASSWORD: REDACTED_CHANGE_ME
EXECUTIONS_MODE: queue
QUEUE_BULL_REDIS_HOST: redis
QUEUE_HEALTH_CHECK_ACTIVE: "true"
N8N_ENCRYPTION_KEY: REDACTED_GENERATE_ONCE
WEBHOOK_URL: https://automation.example.com/
x-n8n-image: &n8n-image docker.n8n.io/n8nio/n8n:REPLACE_WITH_TESTED_VERSION
services:
redis:
image: redis:7-alpine
restart: unless-stopped
n8n:
image: *n8n-image
restart: unless-stopped
environment:
<<: *n8n-env
N8N_EVENTBUS_LOGWRITER_LOGFULLPATH: /home/node/.n8n/n8nEventLog-main.log
ports:
- "127.0.0.1:5678:5678"
worker:
image: *n8n-image
restart: unless-stopped
command: worker --concurrency=5
environment:
<<: *n8n-env
N8N_EVENTBUS_LOGWRITER_LOGFULLPATH: /home/node/.n8n/n8nEventLog-worker-1.log services:
postgres:
image: postgres:16-alpine
restart: unless-stopped
environment:
- POSTGRES_DB=n8n
- POSTGRES_USER=n8n_app
- POSTGRES_PASSWORD=REDACTED_CHANGE_ME
volumes:
- postgres_data:/var/lib/postgresql/data
n8n:
image: docker.n8n.io/n8nio/n8n:REPLACE_WITH_TESTED_VERSION
restart: unless-stopped
depends_on:
- postgres
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_DATABASE=n8n
- DB_POSTGRESDB_USER=n8n_app
- DB_POSTGRESDB_PASSWORD=REDACTED_CHANGE_ME
- N8N_ENCRYPTION_KEY=REDACTED_GENERATE_ONCE
- WEBHOOK_URL=https://automation.example.com/
volumes:
- n8n_data:/home/node/.n8n
volumes:
postgres_data:
n8n_data: Warnings
- Queue mode with filesystem binary storage is not supported; use external storage such as S3 when persistent binary data is needed.
- Too many workers with very low concurrency can pressure the database connection pool.
- A successful manual editor execution may still run on main; do not use it as proof that workers, runner sidecars, or worker-only network paths are healthy.
- Do not point multiple workers at the same explicit event log file on a shared writable volume; use unique .log paths per process.
Best For
- Self-hosted deployments that need to scale execution processing.
- Workloads where main UI/API responsibilities should be separated from workers.
- Teams already comfortable running Redis, Postgres, and multiple n8n processes.
Not For
- Small single-user instances with light execution volume.
- Deployments still using SQLite.
- Teams without monitoring for workers, Redis, database connections, and failed jobs.
Common Mistakes
- Using different N8N_ENCRYPTION_KEY values between main and worker processes.
- Forgetting to set EXECUTIONS_MODE=queue consistently.
- Running too many workers without considering database connection limits.
- Using filesystem binary storage in a queue-mode setup that needs shared binary access.
- Assuming a manual editor run proves workers are healthy when manual executions still run on main.
- Enabling external task runners for the main process but forgetting runner sidecars next to workers.
- Putting the main editor process in the webhook load balancer pool when webhook processors should absorb production webhook traffic.
- Sharing one explicit event log full path across several workers on a shared writable volume.
- Leaving examples on a floating latest image tag instead of promoting a tested n8n image tag with rollback notes.
Examples
EXECUTIONS_MODE=queue
N8N_ENCRYPTION_KEY=use-the-same-secret-everywhere
QUEUE_BULL_REDIS_HOST=redis
QUEUE_BULL_REDIS_PORT=6379
DB_TYPE=postgresdb Main n8n process -> creates execution IDs
Redis -> holds execution queue
Workers -> process queued executions
Postgres -> stores workflow data and results FAQ
Do I need Redis for queue mode?
Yes. Redis is the queue broker that connects the main process and worker processes.
Can I use SQLite with queue mode?
n8n does not recommend SQLite for queue mode. Plan on Postgres for serious queue-mode deployments.
Why do workers fail to decrypt credentials?
A common cause is mismatched N8N_ENCRYPTION_KEY values between the main process and workers.
Why does a manual run pass when production queue jobs fail?
Manual executions can still run on the main process unless OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS is enabled, so run a production-mode smoke test that proves worker pickup.
Do Code node task runners need special queue-mode handling?
Yes. In external task runner mode, each queue worker needs its own runner sidecar; the main process also needs one if it still runs manual executions.