Guides
Troubleshooting

ZoopFlow Troubleshooting Guide

This guide provides comprehensive troubleshooting techniques, common problems, and solutions for ZoopFlow. Use this as a reference when encountering issues in development or production environments.

Introduction to Troubleshooting in ZoopFlow

ZoopFlow combines several technologies and paradigms, including:

  • Temporal.io for workflow orchestration
  • TypeScript-based flow and step definitions
  • Schema validation with ZSchema
  • Signal handling for flow interaction
  • State management across distributed workflows

Troubleshooting effectively requires understanding how these components interact. This guide organizes common issues by component area and provides structured approaches to diagnosing and resolving problems.

Common Issues and Solutions

Flow Definition Issues

Invalid Flow Structure

Symptoms:

  • Error message: Invalid flow structure: Missing required nodes
  • Flow fails to register or execute

Causes:

  • Missing start/end nodes
  • Disconnected nodes in the flow graph
  • Circular dependencies between steps

Solution:

  1. Validate flow definition structure: 
    import { validateFlowDefinition } from '@zoopflow/core';
 
const validationResult = validateFlowDefinition(flowDefinition);
if (!validationResult.valid) {
  console.error(validationResult.errors);
}
  2. Ensure every flow has exactly one start and one end node
  3. Check that all nodes are connected in a directed acyclic graph
  4. Visualize your flow graph to identify structural issues

Step Reference Errors

Symptoms:

  • Error message: Unknown step reference: "stepName"
  • Flow fails during registration

Causes:

  • Referenced step not registered in step registry
  • Typo in step reference name
  • Step exists but in a different namespace

Solution:

  1. Verify step registration before flow definition: 
    import { getStepRegistry } from '@zoopflow/core';
 
const registry = getStepRegistry();
console.log(registry.listRegisteredSteps());
  2. Ensure consistent step naming between definition and registration
  3. Check for namespace prefixes if using namespaced steps

Step Execution Failures

Step Timeouts

Symptoms:

  • Error message: Step execution timeout: "stepName" exceeded 60000ms
  • Flow stops progressing

Causes:

  • Step execution takes longer than configured timeout
  • External service calls not completing
  • Resource contention or bottlenecks

Solution:

  1. Increase timeout for specific steps: 
    defineStep({
  name: 'longRunningStep',
  // ...other config
  timeout: '5 minutes', // Increase timeout for this step
});
  2. Optimize step implementation for performance
  3. Implement retry logic for external service calls
  4. Consider breaking long-running steps into smaller steps

Input/Output Validation Failures

Symptoms:

  • Error message: Schema validation failed: expected string, received number
  • Step fails during execution

Causes:

  • Mismatched input/output types
  • Missing required fields
  • Invalid data format

Solution:

  1. Review schema definitions for accuracy
  2. Add explicit type conversions where needed
  3. Implement data transformation between incompatible steps
  4. Use debug mode to inspect input/output at each step: 
    import { configureDebugger } from '@zoopflow/core';
 
configureDebugger({
  logInputOutput: true,
  detailedValidation: true,
});

Temporal Integration Problems

Workflow Registration Issues

Symptoms:

  • Error message: Failed to register workflow: "flowName"
  • Worker fails to start

Causes:

  • Duplicate workflow registrations
  • Incompatible workflow options
  • Malformed flow definition

Solution:

  1. Ensure unique workflow IDs across your application
  2. Check Temporal worker configuration: 
    import { Worker } from '@temporalio/worker';
 
const worker = await Worker.create({
  workflowsPath: require.resolve('./workflows'),
  activities: { /* your activities */ },
  taskQueue: 'your-task-queue',
});
 
await worker.run();
  3. Validate flow definitions before registration
  4. Check Temporal server logs for detailed error information

Workflow Execution Failures

Symptoms:

  • Error message: Workflow execution failed: NonDeterministicError
  • Workflows fail unpredictably

Causes:

  • Non-deterministic code in workflow implementation
  • Temporal version mismatches
  • Incorrect workflow versioning

Solution:

  1. Follow Temporal determinism guidelines:
    • Avoid random numbers, current time calls, or non-deterministic operations
    • Use Temporal's versioning APIs for workflow changes
  2. Implement proper versioning for workflows: 
    import { workflowVersions } from '@zoopflow/core';
 
// When making changes to existing workflows
workflowVersions.setVersion('my-change', 1);
  3. Update Temporal client and worker dependencies
  4. Implement replay tests for workflows

Schema Validation Errors

Complex Schema Validation Failures

Symptoms:

  • Error message: Validation failed at path: data.nested.property
  • Validation errors in nested properties

Causes:

  • Complex nested schemas
  • Dynamic or conditional schemas
  • Incorrect schema definitions

Solution:

  1. Break down complex schemas into smaller, reusable components
  2. Use schema debugging to inspect validation: 
    import { setSchemaDebugLevel } from '@zoopflow/core';
 
setSchemaDebugLevel('verbose');
  3. Validate inputs before passing to steps: 
    import { validateAgainstSchema } from '@zoopflow/core';
 
const validationResult = validateAgainstSchema(inputData, stepSchema);
if (!validationResult.valid) {
  // Handle validation errors
}
  4. Use schema coercion for type conversion where appropriate

Custom Validation Errors

Symptoms:

  • Error message: Custom validation failed: "validationName"
  • Step fails with custom validation error

Causes:

  • Custom validator implementation error
  • Invalid validation logic
  • Missing error handling in custom validators

Solution:

  1. Debug custom validator implementation: 
    import { registerCustomValidator } from '@zoopflow/core';
 
registerCustomValidator('myValidator', (data, params) => {
  console.log('Validating:', data, 'with params:', params);
  // Validation logic
  return true;
});
  2. Test custom validators independently before integration
  3. Ensure error messages from custom validators are descriptive
  4. Implement unit tests for custom validators

State Management Issues

State Corruption

Symptoms:

  • Unexpected state values
  • Workflow produces incorrect results
  • Intermittent failures

Causes:

  • Direct mutation of state objects
  • Missing checkpoint calls
  • Race conditions in state updates

Solution:

  1. Use immutable state patterns: 
    import { updateState } from '@zoopflow/core';
 
// Instead of: flowContext.state.value = newValue;
updateState(flowContext, 'value', newValue);
  2. Add checkpoints after critical state changes: 
    import { checkpoint } from '@zoopflow/core';
 
await updateState(flowContext, 'value', newValue);
await checkpoint(flowContext, 'State updated');
  3. Implement state validation before and after critical operations
  4. Review state access patterns across async operations

State Persistence Issues

Symptoms:

  • Error message: Failed to persist state: "reason"
  • State lost between workflow restarts

Causes:

  • Storage backend failures
  • Serialization errors for complex objects
  • Temporal history size limits exceeded

Solution:

  1. Ensure all state is serializable: 
    // Check if state is serializable
const isSerializable = (obj) => {
  try {
    JSON.parse(JSON.stringify(obj));
    return true;
  } catch (e) {
    return false;
  }
};
  2. Implement state size monitoring
  3. Use external state storage for large state objects
  4. Break up long-running workflows into smaller, chained workflows

Signal Handling Issues

Signal Registration Problems

Symptoms:

  • Error message: Unknown signal: "signalName"
  • Signals not being processed

Causes:

  • Signal not registered with workflow
  • Incorrect signal handler implementation
  • Signal name mismatches

Solution:

  1. Verify signal registration: 
    import { defineSignal } from '@zoopflow/core';
 
const mySignal = defineSignal({
  name: 'processOrder',
  inputSchema: z.object({
    orderId: z.string(),
    action: z.enum(['approve', 'reject']),
  }),
});
 
// Register in flow definition
defineFlow({
  // ...
  signals: [mySignal],
  // ...
});
  2. Check signal name consistency between sender and receiver
  3. Implement signal logging for debugging: 
    import { configureSignalDebug } from '@zoopflow/core';
 
configureSignalDebug({
  logSignalReceived: true,
  logSignalHandled: true,
});

Signal Handling Failures

Symptoms:

  • Error message: Signal handler exception: "errorMessage"
  • Signal processed but workflow state not updated

Causes:

  • Exceptions in signal handler code
  • Race conditions between signals
  • State mutation issues in signal handlers

Solution:

  1. Implement error handling in signal handlers: 
    import { defineSignalHandler } from '@zoopflow/core';
 
defineSignalHandler('processOrder', async (payload, context) => {
  try {
    // Signal handling code
  } catch (error) {
    context.logger.error('Signal handler error', error);
    // Handle error appropriately
  }
});
  2. Use workflow mutations for state updates in signal handlers
  3. Implement signal queuing for concurrent signals
  4. Add signal validation before processing

Performance Problems

Slow Workflow Execution

Symptoms:

  • Workflows take longer than expected
  • Increasing latency over time
  • Timeouts during execution

Causes:

  • Inefficient step implementations
  • External service bottlenecks
  • Resource constraints
  • Large workflow histories

Solution:

  1. Implement performance monitoring: 
    import { enablePerformanceTracking } from '@zoopflow/core';
 
enablePerformanceTracking({
  detailedStepMetrics: true,
  slowStepThreshold: 500, // ms
});
  2. Optimize steps with high execution times
  3. Implement caching for repeated operations
  4. Use continuation workflows for long-running processes
  5. Scale Temporal workers appropriately

Memory Leaks

Symptoms:

  • Increasing memory usage over time
  • Worker process crashes with out-of-memory errors
  • Degrading performance

Causes:

  • Accumulating state without cleanup
  • Large object retention
  • Event listener leaks

Solution:

  1. Monitor memory usage in workers
  2. Implement state cleanup for completed workflows
  3. Use external storage for large data objects
  4. Properly dispose resources and event listeners
  5. Implement worker restarts on a schedule

Deployment Problems

Environment Configuration Issues

Symptoms:

  • Error message: Missing configuration: "configName"
  • Inconsistent behavior between environments

Causes:

  • Missing environment variables
  • Inconsistent configuration between environments
  • Hardcoded development values

Solution:

  1. Implement configuration validation on startup: 
    import { validateConfig } from '@zoopflow/core';
 
const configValidation = validateConfig(process.env);
if (!configValidation.valid) {
  console.error('Invalid configuration:', configValidation.errors);
  process.exit(1);
}
  2. Use configuration schemas to validate environment variables
  3. Implement environment-specific defaults with override capability
  4. Document required configuration for each environment

Version Compatibility Issues

Symptoms:

  • Error message: Incompatible version: expected >=2.0.0, got 1.8.0
  • Unexpected behavior after updates

Causes:

  • Mismatched package versions
  • Breaking changes in dependencies
  • Incompatible client/worker versions

Solution:

  1. Lock dependencies with exact versions
  2. Implement version compatibility checks: 
    import { checkVersionCompatibility } from '@zoopflow/core';
 
const compatibility = checkVersionCompatibility();
if (!compatibility.compatible) {
  console.error('Version incompatibility:', compatibility.issues);
}
  3. Test upgrades in isolation before deployment
  4. Maintain version compatibility matrix documentation

Debugging Techniques

Using Logging Effectively

ZoopFlow provides multi-level logging for troubleshooting:

import { configureLogging } from '@zoopflow/core';
 
// Configure global logging
configureLogging({
  level: 'debug', // 'error', 'warn', 'info', 'debug', 'trace'
  format: 'json', // 'text', 'json'
  destination: 'console', // 'console', 'file'
  filePath: '/var/log/zoopflow.log', // Only if destination is 'file'
  includeTimestamp: true,
});

Effective logging strategies:

  1. Contextual Logging: Include relevant context in logs

    context.logger.debug('Processing step', {
  stepName: step.name,
  inputSize: JSON.stringify(input).length,
  flowId: context.flowId,
});

  2. Correlation IDs: Track requests across components

    // In client code
const correlationId = generateUUID();
 
// Pass to context
const context = createFlowContext({
  correlationId,
  // other context properties
});
 
// Log with correlation ID
context.logger.info('Operation completed', { correlationId });

  3. Structured Logging: Use consistent structured format

    context.logger.info('Step completed', {
  step: 'processPayment',
  duration: 120,
  status: 'success',
  metadata: {
    paymentId: 'pay_123',
    amount: 100.50,
  }
});

  4. Log Levels: Use appropriate levels for different information

    • ERROR: Failures requiring immediate attention
    • WARN: Issues that don't stop execution but indicate problems
    • INFO: Normal operation information
    • DEBUG: Detailed information for troubleshooting
    • TRACE: Very verbose data for deep debugging

Debugging Flows

  1. Flow Visualization: Visualize flow execution paths

    import { visualizeFlow } from '@zoopflow/core/debug';
 
// Generate flow visualization
const visualization = visualizeFlow(flowDefinition);
 
// Save visualization to file
writeFileSync('flow-visualization.html', visualization);

  2. Step-by-Step Execution: Enable step-by-step execution mode

    import { enableStepByStepExecution } from '@zoopflow/core/debug';
 
enableStepByStepExecution(flowId, {
  breakOnSteps: ['criticalStep1', 'criticalStep2'],
  logStateTransitions: true,
});

  3. Execution Trace: Capture detailed execution traces

    import { captureExecutionTrace } from '@zoopflow/core/debug';
 
const trace = await captureExecutionTrace(flowId);
console.log(JSON.stringify(trace, null, 2));

  4. State Snapshots: Capture state at different execution points

    import { captureStateSnapshot } from '@zoopflow/core/debug';
 
// In step implementation
const snapshot = await captureStateSnapshot(context);
context.logger.debug('State snapshot', { snapshot });

Debugging Steps

  1. Step Isolation: Test steps in isolation

    import { executeStepInIsolation } from '@zoopflow/core/debug';
 
const result = await executeStepInIsolation({
  step: myStep,
  input: testInput,
  mockContext: {
    // Mock context properties
  },
});
 
console.log('Step output:', result);

  2. Input/Output Inspection: Log input and output

    import { logStepIO } from '@zoopflow/core/debug';
 
logStepIO(stepName, true); // Enable for specific step

  3. Step Performance Profiling: Profile step execution time

    import { profileStep } from '@zoopflow/core/debug';
 
const profile = await profileStep(stepName, testInput);
console.log('Step profile:', profile);

  4. Mock Dependencies: Test with mocked dependencies

    import { mockStepDependencies } from '@zoopflow/core/debug';
 
mockStepDependencies(myStep, {
  'dependencyName': mockImplementation,
});

Debugging Schema Validation

  1. Schema Visualization: Generate visual representation

    import { visualizeSchema } from '@zoopflow/core/debug';
 
const visualization = visualizeSchema(mySchema);
writeFileSync('schema-visualization.html', visualization);

  2. Validation Tracing: Trace validation steps

    import { traceValidation } from '@zoopflow/core/debug';
 
const trace = traceValidation(data, schema);
console.log('Validation trace:', trace);

  3. Schema Comparison: Compare schemas for compatibility

    import { compareSchemas } from '@zoopflow/core/debug';
 
const comparison = compareSchemas(schemaA, schemaB);
console.log('Schema differences:', comparison.differences);

  4. Test Data Generation: Generate test data from schema

    import { generateTestData } from '@zoopflow/core/debug';
 
const testData = generateTestData(schema);
console.log('Test data:', testData);

Remote Debugging

  1. Remote Inspector: Connect to running workflows

    import { enableRemoteInspector } from '@zoopflow/core/debug';
 
enableRemoteInspector({
  port: 9229,
  flowId: 'flow-to-debug',
});

  2. Live State Inspection: Inspect workflow state remotely

    import { enableLiveStateInspection } from '@zoopflow/core/debug';
 
enableLiveStateInspection(workflowId);
 
// Connect with: zoopflow-inspector http://localhost:9229

  3. Remote Command Execution: Execute commands in running workflows

    import { enableRemoteCommands } from '@zoopflow/core/debug';
 
enableRemoteCommands({
  allowedCommands: ['inspect', 'pause', 'resume'],
  authToken: 'secure-token',
});

  4. Distributed Tracing: Implement distributed tracing

    import { configureDistributedTracing } from '@zoopflow/core/debug';
 
configureDistributedTracing({
  provider: 'jaeger',
  endpoint: 'http://jaeger-collector:14268/api/traces',
});

Diagnostic Tools and Utilities

ZoopFlow provides several built-in diagnostic tools:

Flow Analyzer

Analyzes flow definitions for potential issues:

import { analyzeFlow } from '@zoopflow/tools';
 
const analysis = analyzeFlow(flowDefinition);
console.log('Flow analysis:', analysis);

The analysis includes:

  • Structural validation
  • Step dependency checks
  • Potential bottlenecks
  • Error handling coverage
  • Schema compatibility warnings

Performance Monitor

Monitors and reports on flow performance:

import { monitorPerformance } from '@zoopflow/tools';
 
monitorPerformance({
  flowIds: ['flow1', 'flow2'],
  metrics: ['executionTime', 'stateSize', 'stepLatency'],
  interval: 60000, // 1 minute
  callback: (metrics) => {
    console.log('Performance metrics:', metrics);
  },
});

Workflow Replay Tool

Replays workflows from history for debugging:

import { replayWorkflow } from '@zoopflow/tools';
 
const replay = await replayWorkflow(workflowId, {
  stopAtFailure: true,
  detailedStateTracking: true,
});
 
console.log('Replay results:', replay);

Schema Inspector

Analyzes schema definitions:

import { inspectSchema } from '@zoopflow/tools';
 
const inspection = inspectSchema(schemaDefinition);
console.log('Schema inspection:', inspection);

The inspection includes:

  • Validation rules
  • Required vs optional fields
  • Type constraints
  • Custom validators
  • Potential validation issues

Performance Tuning and Optimization

Workflow Optimization

  1. State Size Reduction:

    • Keep state minimal by storing only essential data
    • Use external storage for large objects
    • Implement state cleanup for completed steps
    import { optimizeState } from '@zoopflow/core/performance';
 
optimizeState(flowContext);

  2. Step Parallelization:

    • Identify and parallelize independent steps
    • Use parallel execution patterns for suitable tasks
    import { parallel } from '@zoopflow/core';
 
await parallel([
  () => executeStep1(input1),
  () => executeStep2(input2),
  () => executeStep3(input3),
]);

  3. Caching Strategies:

    • Implement caching for repeated operations
    • Cache external service responses
    import { enableCaching } from '@zoopflow/core/performance';
 
enableCaching({
  ttl: 300, // seconds
  maxSize: 100, // entries
});

  4. Workflow Splitting:

    • Split large workflows into child workflows
    • Implement continuation-based workflows for long-running processes
    import { continueAsNew } from '@zoopflow/core';
 
if (isWorkflowTooLarge(context)) {
  await continueAsNew(context, remainingSteps);
}

Worker Optimization

  1. Worker Scaling:

    • Scale workers based on queue depth
    • Implement worker auto-scaling
    import { configureWorkerScaling } from '@zoopflow/core/performance';
 
configureWorkerScaling({
  minWorkers: 5,
  maxWorkers: 50,
  targetQueueDepth: 10,
});

  2. Resource Allocation:

    • Allocate resources based on workflow complexity
    • Implement worker resource limits
    import { configureWorkerResources } from '@zoopflow/core/performance';
 
configureWorkerResources({
  memoryLimit: '1Gi',
  cpuLimit: '500m',
});

  3. Connection Pooling:

    • Implement connection pooling for external services
    • Reuse connections where possible
    import { configureConnectionPool } from '@zoopflow/core/performance';
 
configureConnectionPool({
  service: 'database',
  maxConnections: 20,
  idleTimeout: 10000,
});

  4. Batch Processing:

    • Implement batch processing for similar operations
    • Group external service calls
    import { batchProcess } from '@zoopflow/core/performance';
 
await batchProcess(items, processItem, {
  batchSize: 100,
  concurrency: 5,
});

Recovery Strategies

Flow Recovery

  1. Checkpoint Recovery:

    • Implement regular checkpoints in workflows
    • Recover from last checkpoint on failure
    import { createCheckpoint, recoverFromCheckpoint } from '@zoopflow/core';
 
// Create checkpoint
await createCheckpoint(context, 'after-payment-processing');
 
// Recover from checkpoint
if (isRecoveryNeeded) {
  await recoverFromCheckpoint(context, 'after-payment-processing');
}

  2. Step Retry Strategies:

    • Configure retry policies for steps
    • Implement exponential backoff
    import { configureRetryPolicy } from '@zoopflow/core';
 
configureRetryPolicy('processPayment', {
  maxAttempts: 5,
  initialInterval: 1000,
  backoffCoefficient: 2,
  maximumInterval: 60000,
});

  3. Compensation Transactions:

    • Implement compensation for failed operations
    • Rollback changes on failure
    import { registerCompensation } from '@zoopflow/core';
 
// Register compensation
registerCompensation(context, 'processPayment', async () => {
  await refundPayment(paymentId);
});
 
// Execute compensations if needed
if (flowFailed) {
  await executeCompensations(context);
}

  4. State Recovery:

    • Implement state backup and recovery
    • Recover from state snapshots
    import { backupState, recoverState } from '@zoopflow/core';
 
// Backup state
await backupState(context);
 
// Recover state if needed
if (stateCorrupted) {
  await recoverState(context);
}

System Recovery

  1. Worker Recovery:

    • Implement worker health checks
    • Auto-restart failed workers
    import { configureWorkerRecovery } from '@zoopflow/core';
 
configureWorkerRecovery({
  healthCheckInterval: 30000,
  autoRestart: true,
  maxRestarts: 5,
});

  2. Disaster Recovery:

    • Implement multi-region deployment
    • Configure failover procedures
    import { configureDisasterRecovery } from '@zoopflow/core';
 
configureDisasterRecovery({
  regions: ['us-east-1', 'us-west-2'],
  failoverThreshold: 3,
  dataReplication: true,
});

  3. Data Recovery:

    • Implement state backup and recovery
    • Configure data backup schedules
    import { configureDataBackup } from '@zoopflow/core';
 
configureDataBackup({
  schedule: '0 0 * * *', // Daily at midnight
  retention: 7, // days
  location: 's3://backup-bucket/zoopflow',
});

How to Get Help and Report Issues

Community Resources

Reporting Issues Effectively

When reporting issues, include:

  1. Environment Information:

    ZoopFlow Version: x.y.z
Temporal Version: a.b.c
Node.js Version: d.e.f
OS: Windows/macOS/Linux

  2. Reproduction Steps:

    • Clear steps to reproduce the issue
    • Minimal code example if possible

  3. Actual vs Expected Behavior:

    • What happened
    • What you expected to happen

  4. Logs and Error Messages:

    • Relevant logs with appropriate log level
    • Complete error stack traces

  5. Context:

    • When did the issue start occurring
    • Any recent changes to your system

Support Channels

Troubleshooting Checklist

Use this checklist as a systematic approach to troubleshooting:

Initial Assessment

  • Identify the specific error message or symptom
  • Determine which component is failing (flow, step, schema, signals, etc.)
  • Review recent changes that might have introduced the issue
  • Check environment configuration for misconfigurations

Data Collection

  • Gather logs at appropriate detail level
  • Capture error stack traces
  • Record reproduction steps
  • Identify patterns (intermittent vs consistent, specific conditions)

Analysis

  • Review flow definition for structural issues
  • Validate schemas against actual data
  • Check step implementations for logical errors
  • Verify Temporal configuration
  • Test signal handlers in isolation

Resolution Steps

  • Implement targeted fixes based on root cause
  • Test in isolation before full system testing
  • Validate fix with original reproduction steps
  • Update documentation if issue relates to undocumented behavior
  • Add regression tests to prevent recurrence

Prevention

  • Update monitoring to detect similar issues
  • Share learnings with team members
  • Implement safeguards against similar issues
  • Review error handling in related components

By following this guide, you should be able to diagnose and resolve most issues encountered when working with ZoopFlow. If you still face challenges after trying these solutions, reach out to the community or support channels for additional assistance.

Testing FrameworkProject Structure