Cloudflare + Directus Proxy Debugging

Next.js static generation randomly failing during npm run build with no server logs.

Problem: Silent Build Failures

$ npm run build

> reelabilities@1.0.0 build
> next build

- info Linting and checking validity of types
- info Creating an optimized production build
- info Compiled successfully
- info Collecting page data  .
  × Failed to collect page data for /toronto/films/[slug]
    Error: fetch failed
        at Object.fetch (node:internal/deps/undici/undici:11457:11)
        at async fetchFilmById (/app/.next/server/chunks/123.js:1:234)

Symptoms:

• Development: works fine
• Runtime: works fine
• Build time: random failures
• Server logs: empty
• CDN logs: empty

Diagnosis

# Directus container logs - nothing suspicious
docker logs directus-container -f --tail=100

# Caddy reverse proxy logs - clean
docker logs caddy-container -f --tail=100

# Cloudflare dashboard - security events empty
# Analytics dashboard - no unusual traffic patterns

Root cause: Cloudflare's bot protection silently blocking build-time API requests.

Evidence:

• Build-time requests are concurrent and automated
• CDN security treats this as suspicious traffic
• No logs because blocks happen at edge level

Solution: /etc/hosts Bypass

Bypass Cloudflare during builds while keeping it for runtime traffic.

# /etc/hosts
YOUR_SERVER_IP cmsdomain.com

Server configuration:

# Caddyfile - Handle direct HTTPS with IP restrictions
cmsdomain.com:443 {
    # Restrict access to build server IP only
    @allowed remote_ip YOUR_BUILD_SERVER_IP
    handle @allowed {
        reverse_proxy directus:8055
    }
    
    # Deny all other IPs
    handle {
        respond "Access Denied" 403
    }
    
    # TLS handled by Caddy with Let's Encrypt
    tls your-email@domain.com
    
    # CORS headers for API access
    header Access-Control-Allow-Origin *
    header Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE, OPTIONS"
    header Access-Control-Allow-Headers "Authorization, Content-Type, Accept"
    
    # Handle preflight requests
    @cors_preflight method OPTIONS
    respond @cors_preflight 204
}

# Still handle Cloudflare traffic on standard port (no IP restrictions for public access)
cmsdomain.com:80 {
    reverse_proxy directus:8055
}

Build environment:

# Build environment variables
NODE_ENV=production
CI=true
DIRECTUS_URL=https://cmsdomain.com

# Critical: Allow insecure SSL connections for direct server access
# This bypasses SSL certificate validation during builds
NODE_TLS_REJECT_UNAUTHORIZED=0

// Build-time API configuration
const DIRECTUS_URL = process.env.NODE_ENV === 'production' && process.env.CI
  ? 'https://cmsdomain.com'     // Direct to server during builds
  : process.env.DIRECTUS_URL;        // Regular runtime configuration

export const directus = createDirectus(DIRECTUS_URL)
  .with(authentication())
  .with(rest());

Results

$ npm run build

> reelabilities@1.0.0 build
> next build

✓ Linting and checking validity of types
✓ Creating an optimized production build
✓ Compiled successfully
✓ Collecting page data
✓ Generating static pages (247/247)
✓ Finalizing page optimization

Build completed successfully!

100% build success rate.

Production Setup

# CI/Build server /etc/hosts
YOUR_SERVER_IP cmsdomain.com

# Build environment variables
NODE_ENV=production
CI=true
DIRECTUS_URL=https://cmsdomain.com

# Critical for direct server access
NODE_TLS_REJECT_UNAUTHORIZED=0

Caddy config:

cmsdomain.com {
    # IP restriction for build server access
    @build_server remote_ip YOUR_BUILD_SERVER_IP
    
    handle @build_server {
        # Handle build server traffic (bypassing Cloudflare)
        reverse_proxy directus:8055 {
            # Optimize for build-time requests
            timeout 30s
            header_up Host {upstream_hostport}
            header_up X-Forwarded-Proto {scheme}
        }
    }
    
    handle {
        # Handle regular traffic through Cloudflare
        reverse_proxy directus:8055 {
            timeout 30s
        }
    }
    
    # Compression
    encode gzip
    
    # CORS
    header Access-Control-Allow-Origin *
    header Access-Control-Allow-Methods *
    header Access-Control-Allow-Headers *
}

Final Results

Over 3+ months: 100% build success, faster builds (no CDN latency), security maintained (IP restrictions), and runtime performance unchanged.

Debug Methodology

1. Eliminate layers systematically
2. Use /etc/hosts for CDN bypass testing
3. Check build vs runtime contexts
4. Monitor patterns, not just individual failures
5. Test in isolated environments

Key Takeaways

• CDN bot protection can be invisible and inconsistent
• Build-time traffic patterns differ from user traffic
• /etc/hosts is the fastest way to isolate CDN issues
• NODE_TLS_REJECT_UNAUTHORIZED=0 works for controlled environments
• Dual-stack approaches let you optimize per context

Common Signs of CDN Interference

• Works in development, fails in CI
• Intermittent failures with no server logs
• Concurrent requests failing more than sequential
• Different behavior across CDN edge locations