Deployment

Pidgey workers need to run as a separate long-running process. Since most serverless platforms (like Vercel) don’t support long-running processes, you’ll deploy your worker to a platform that does.

┌─────────────────────┐      ┌──────────────┐      ┌─────────────────┐
│   Next.js App       │────▶ │   Database   │ ◀────│  Worker Process │
│  (Enqueues jobs)    │      │              │      │ (Processes jobs)│
└─────────────────────┘      └──────────────┘      └─────────────────┘
     Vercel, etc.                                 Railway, Render, etc.

This guide shows you how to deploy a Pidgey worker to Railway.

What Happens on Worker Startup

When the worker starts, it:

1. Syncs scheduled jobs — Registers cron schedules from your job definitions
2. Starts processing — Begins polling for and executing jobs

Running Migrations

Recommended: Run migrations in your CI/CD pipeline before deploying:

npm run build
npx pidgey migrate

This ensures the database is ready before your app or worker starts, and catches migration failures early.

Deployment Order

With migrations handled in CI, deploy your app and worker in any order. Jobs enqueued while the worker is updating simply wait in the database.

Scheduled Jobs

Scheduled jobs are synced when the worker starts, not from your app. When you change a cron expression or add a new scheduled job, redeploy the worker to pick up the changes.

Prerequisites

Before deploying the worker, make sure your main app is set up:

1. Pidgey packages installed in your app
2. Pidgey config file (pidgey.config.ts) created
3. Jobs directory with your job definitions
4. Database accessible from both your app and Railway (same DATABASE_URL)

Deploy to Railway

Railway natively supports long-running processes, making it ideal for Pidgey workers.

1. Install Railway CLI

brew install railway
railway login

2. Create a Railway project

railway init --name my-worker

3. Add a railway.toml

Create this file in your project root:

[build]
builder = "nixpacks"
buildCommand = "npm install && npm run build"
 
[deploy]
startCommand = "npx pidgey worker start --concurrency 50"

4. Set environment variables

railway variables set DATABASE_URL="postgres://user:pass@host:5432/db"
railway variables set NODE_ENV=production

5. Deploy

railway up

6. Verify it’s working

railway logs

You should see:

🔍 Discovering jobs in ./jobs
✅ Found 3 jobs: send-email, process-payment, generate-report
🚀 Worker started (concurrency: 50)

Configuration Options

Concurrency

Adjust worker concurrency based on your workload:

[deploy]
startCommand = "npx pidgey worker start --concurrency 100"

• I/O-bound jobs (API calls, emails): 50-100
• CPU-bound jobs (image processing): 5-10
• Mixed workload: Start with 20-30

Using a config file

If you have a pidgey.config.ts, the worker will use it automatically:

import { defineConfig } from '@pidgeyjs/core';
 
export default defineConfig({
  adapter: 'postgres',
  connection: process.env.DATABASE_URL!,
 
  worker: {
    jobsDir: 'jobs',
    concurrency: 50,
    pollInterval: 100,
  },
});

Useful Commands

# View logs
railway logs
 
# Open dashboard
railway open
 
# Set more environment variables
railway variables set MY_API_KEY="..."
 
# View current status
railway status

Troubleshooting

”No jobs found”

Make sure your jobs directory is included in your build. Check that:

• Jobs are exported correctly
• The directory path matches your config

Database connection errors

• Verify DATABASE_URL is set correctly
• Ensure the database accepts connections from Railway’s IP range
• For Neon/Supabase, SSL is usually required automatically

Worker keeps restarting

Check logs for errors:

railway logs

Common causes:

• Missing environment variables
• Database connection issues
• Syntax errors in job files

Scaling

To handle more jobs, increase concurrency or deploy multiple workers:

# Deploy additional worker instances in Railway dashboard
# Or run multiple services with different names
railway init --name my-worker-2
railway up

Other Platforms

The core pattern is the same on any platform that supports long-running processes:

Next Steps

• Worker Configuration — Fine-tune worker settings
• Adapters — Configure storage backends
• CLI Reference — All CLI commands