Job Queue Plugin

Job Queue Plugin

The job-queue plugin (CS_10, port 8213) provides a Redis-backed persistent job queue for ɳSelf backends. It supports at-least-once delivery, configurable worker pools per queue, exponential backoff retry, and a dead-letter queue visible in the Admin UI.

Installation

nself plugin install job-queue
nself build && nself start

Requires Redis (included in the optional services bundle).

Configuration

JOBQUEUE_CONCURRENCY=5        # worker goroutines per queue type
JOBQUEUE_MAX_ATTEMPTS=8       # max delivery attempts before DLQ
JOBQUEUE_QUEUES=default,email,ai,media   # queue names
JOBQUEUE_PORT=8213            # HTTP API port

Enqueuing jobs from any plugin

Use the Go SDK:

import "github.com/nself-org/nself-sdk"

// Enqueue an email send job
if err := sdk.Enqueue(ctx, "email", "email:send", map[string]string{
    "to":      "user@example.com",
    "subject": "Welcome to nSelf",
}); err != nil {
    return err
}

Jobs appear in np_jobs within 100ms and in the Admin UI job list immediately.

Registering handlers (in your service)

sdk.RegisterHandler("email:send", func(ctx context.Context, payload []byte) error {
    var params struct { To, Subject string }
    json.Unmarshal(payload, &params)
    return sendEmail(ctx, params.To, params.Subject)
})

Queue names

Queue	Use for
default	General background work
email	Email delivery (mux plugin)
ai	AI inference jobs
media	Media processing (ɳTV bundle)

Custom queues can be added via JOBQUEUE_QUEUES.

Dead-letter queue

Failed jobs (after JOBQUEUE_MAX_ATTEMPTS retries) move to np_job_dlq. They are visible in the Admin UI under Jobs → DLQ.

Re-queue a failed job:

# Via Admin UI: Jobs → DLQ → Re-queue button
# Via API:
curl -X POST http://localhost:8213/dlq/requeue \
  -H "Content-Type: application/json" \
  -d '{"job_id": "abc123"}'

Retry schedule

Jobs retry with exponential backoff: 2, 4, 8, 16, 32, 64, 128, 256 seconds (capped at 1 hour). After 8 attempts the job moves to the DLQ.

Monitoring

Queue depths are exposed at http://localhost:8213/metrics in Prometheus format:

nself_jobqueue_queue_depth{queue="email"} 42
nself_jobqueue_queue_depth{queue="ai"} 0

Import the Job Queue Grafana dashboard from the monitoring plugin for a full depth, throughput, and error rate overview.