Deployment

Pidgey workers run as long-running processes. Since serverless platforms (like Vercel) don’t support this, deploy your worker to a platform that does (Railway, Render, Fly.io, etc.).

┌─────────────────────┐      ┌──────────────┐      ┌─────────────────┐
│   Next.js App       │────▶ │   Database   │ ◀────│  Worker Process │
│  (Enqueues jobs)    │      │              │      │ (Processes jobs)│
└─────────────────────┘      └──────────────┘      └─────────────────┘
     Vercel, etc.                                 Railway, Render, etc.

Worker Startup Process

When the worker starts:

1. Syncs scheduled jobs — Registers cron schedules from job definitions
2. Begins processing — Polls for and executes jobs

Running Migrations

Best practice: Run migrations in your CI/CD pipeline before deploying:

npm run build
npx pidgey migrate

Deployment Order

• With migrations handled, you can deploy app and worker in any order.
• Jobs enqueued while the worker is updating will wait in the database.
• Scheduled jobs are synced when the worker starts. Changes to cron expressions require redeploying the worker.

Prerequisites

Ensure before deploying:

1. Pidgey packages installed
2. pidgey.config.ts exists
3. Jobs directory populated
4. Database accessible from both app and worker

Deploying to Railway

Railway supports long-running processes natively.

1. Install Railway CLI

brew install railway
railway login

2. Create a Railway project

railway init --name my-worker

3. Add railway.toml

[build]
builder = "nixpacks"
buildCommand = "npm install && npm run build"
 
[deploy]
startCommand = "npx pidgey worker start --concurrency 50"

4. Set environment variables

railway variables set DATABASE_URL="postgres://user:pass@host:5432/db"
railway variables set NODE_ENV=production

5. Deploy

railway up

6. Verify

railway logs

Expected output:

🔍 Discovering jobs in ./jobs
✅ Found 3 jobs: send-email, process-payment, generate-report
🚀 Worker started (concurrency: 50)

Worker Configuration Options

Concurrency

Adjust based on job type:

[deploy]
startCommand = "npx pidgey worker start --concurrency 100"

• I/O-bound jobs (API calls, emails): 50–100
• CPU-bound jobs (image processing): 5–10
• Mixed workload: Start with 20–30

Using pidgey.config.ts

import { defineConfig } from '@pidgeyjs/core';
 
export default defineConfig({
  adapter: 'postgres',
  connection: process.env.DATABASE_URL!,
 
  worker: {
    jobsDir: 'jobs',
    concurrency: 50,
    pollInterval: 100,
  },
});

Useful Railway Commands

# View logs
railway logs
 
# Open dashboard
railway open
 
# Set additional environment variables
railway variables set MY_API_KEY="..."
 
# Check current status
railway status

Troubleshooting

”No jobs found”

• Ensure jobs directory is included in build
• Verify jobs are exported correctly
• Check directory path matches config

Database connection errors

• Confirm DATABASE_URL is correct
• Ensure database accepts connections from Railway
• For Neon/Supabase, SSL is usually required

Worker keeps restarting

• Missing environment variables
• DB connection issues
• Syntax errors in job files

Check logs:

railway logs

Scaling

• Increase concurrency or deploy multiple workers:

# Additional worker instance
railway init --name my-worker-2
railway up

• Deploy multiple services with different names to scale horizontally

Other Platforms

Next Steps

• Worker Configuration — Fine-tune worker settings
• Adapters — Configure storage backends
• CLI Reference — All CLI commands