Production Readiness

Performance and scalability

Expected data volumes

Plan your infrastructure for the following transaction volumes:

Metric	Value	Notes
Average transaction size	4-6 KB	Filtered, enriched data
Peak rate	50 transactions/minute	During busy service periods
Daily volume	~1,000 transactions	Per location
Burst capacity	10 transactions/second	Short-term spikes

Endpoint requirements

Your webhook endpoint must support:

Requirement	Specification	Purpose
Request capacity	100 requests/second minimum	Handle burst traffic
Connection pool	50 concurrent connections	Multiple simultaneous deliveries
Response time	Under 5 seconds (30 seconds maximum)	Prevent delivery timeouts
Availability	99.9% uptime	Minimise missed events

Scaling considerations

For high-volume implementations:

• Load balancing: Distribute incoming webhooks across multiple servers

• Queue-based processing: Use message queues (RabbitMQ, AWS SQS) for async processing

• Database optimisation: Index transaction IDs for fast duplicate checking

• Caching: Cache lookup data (clerks, locations, PLUs) to reduce API calls

• Monitoring: Track processing times and error rates

---

Error handling and resilience

Automatic retry behaviour

RELAY implements exponential backoff retry logic for failed deliveries:

Retry aspect	Configuration	Description
Retry mode	Exponential	Increasing delays between retries
Maximum retries	3 attempts	After initial delivery failure
Initial retry delay	0.8 seconds	First retry attempt
Maximum delivery attempts	10 total	Including all retries
Dead letter queue	After 10 failures	Failed events moved to DLQ
Message expiry	14 days	Undelivered messages expire

HTTP status code handling

RELAY interprets HTTP responses as follows:

Status code	Interpretation	RELAY action
200-299	Success	Event marked as delivered
400-499	Client error	No retry, sent to dead letter queue
500-599	Server error	Automatic retry with backoff
Timeout	No response in 30 seconds	Automatic retry with backoff

Error recovery best practices

Implement robust error handling in your webhook:

app.post('/relay-webhook', async (req, res) => {
  try {
    // Validate event structure
    if (!isValidEvent(req.body)) {
      // Return 400 for malformed data
      return res.status(400).json({ 
        error: 'Invalid event structure' 
      });
    }
    
    // Acknowledge receipt immediately
    res.status(200).json({ received: true });
    
    // Process with error handling
    try {
      await processEventAsync(req.body);
    } catch (processingError) {
      // Log but don't fail the webhook
      logger.error('Processing error:', processingError);
      
      // Consider dead letter queue for failed processing
      await sendToDeadLetterQueue(req.body, processingError);
    }
  } catch (error) {
    // Return 500 for server errors to trigger retry
    logger.error('Webhook error:', error);
    res.status(500).json({ error: 'Internal server error' });
  }
});

Monitoring and alerting

Implement monitoring for:

• Failed deliveries: Track 4xx and 5xx responses

• Processing errors: Monitor async processing failures

• Duplicate rate: Track idempotency check hits

• Processing time: Measure end-to-end latency

• Queue depth: Monitor backlog for async processing

---

Security specifications

Network security

All RELAY communications use enterprise-grade security:

• HTTPS required: All webhook endpoints must use HTTPS

• TLS 1.2 minimum: TLS 1.3 recommended for enhanced security

• IP whitelisting: Available upon request for additional security

• Certificate validation: Valid, non-expired TLS certificates required

Data security

Implement appropriate data handling:

• Data encryption: Encrypt sensitive data at rest

• Access controls: Limit access to transaction data

• Audit logging: Log all access to transaction information

• Data retention: Implement appropriate retention policies

• PCI compliance: Follow PCI DSS if storing payment card data

Authentication security

For OAuth 2.0 implementations:

• Secure credential storage: Store client secrets encrypted

• Token rotation: Implement automatic token refresh

• Scope limitation: Request minimum required scopes

• Certificate pinning: Consider pinning for additional security

For API key implementations:

• Secure key storage: Never hardcode keys in source code

• Key rotation: Regularly rotate API keys

• Environment separation: Use different keys for test and production

• Access logging: Monitor API key usage patterns

---

Testing and validation

Integration testing phases

Follow this recommended testing sequence:

Phase 1: Single event test

Test basic connectivity and processing with a single transaction.

Objectives:

• Verify webhook endpoint accessibility

• Confirm authentication works correctly

• Validate event structure parsing

• Test acknowledgement response

Success criteria:

• Webhook receives and returns 200 status

• Event data parses without errors

• Transaction ID stored for idempotency

Phase 2: Batch test

Test concurrent event handling with multiple transactions.

Objectives:

• Handle 10 concurrent event deliveries

• Verify idempotency logic prevents duplicates

• Confirm async processing works under load

• Test database connection pooling

Success criteria:

• All 10 events process successfully

• No duplicate processing occurs

• Response times remain under 5 seconds

• No connection pool exhaustion

Phase 3: Load test

Validate performance under sustained traffic.

Objectives:

• Sustain 100 events/minute for 10 minutes

• Monitor system resource usage

• Identify bottlenecks or scaling issues

• Verify error handling under load

Success criteria:

• 100% delivery acknowledgement rate

• Average response time under 2 seconds

• No memory leaks or resource exhaustion

• Queue processing keeps pace with delivery

Phase 4: Failure test

Test resilience and error handling.

Objectives:

• Simulate timeout conditions

• Test retry logic with 500 responses

• Verify dead letter queue handling

• Confirm monitoring and alerting

Success criteria:

• Retries occur with exponential backoff

• Events eventually deliver or move to DLQ

• Alerts trigger for persistent failures

• No data loss occurs

Test environment

Access Hospitality provides a test environment for integration development:

• Test webhook configuration available

• Sample transaction data for validation

• Simulation tools for load testing

• Support team assistance for troubleshooting

Contact the EPOS Product Team to request test environment access.

---

Production deployment

Pre-deployment checklist

Before going live with RELAY integration:

• Webhook endpoint deployed to production environment

• HTTPS certificate valid and not expiring soon

• Authentication credentials configured and tested

• Idempotency logic implemented and verified

• Error handling and retry logic in place

• Monitoring and alerting configured

• Load testing completed successfully

• Failure scenarios tested and documented

• Data security controls implemented

• Team trained on operational procedures

Deployment process

1. Development phase:

   • Implement webhook endpoint

   • Test with Access Hospitality test environment

   • Complete all integration testing phases

2. Approval phase:

   • Submit integration for review

   • Complete testing with EPOS Product Team

   • Receive production credentials

3. Production deployment:

   • Deploy to production environment

   • Configure production webhook URL

   • Verify initial deliveries

   • Monitor for 24 hours

4. Ongoing operations:

   • Monitor delivery success rates

   • Review error logs regularly

   • Update documentation as needed

   • Participate in quarterly reviews

---