Problem Statement
Operations teams in businesses repeatedly request small workflow automations from engineering: "when a new order is submitted, send an approval email", "when a payment is received, update the contract status and notify the account manager". These requests are individually simple but collectively consume significant engineering time. Building each automation as a one-off script creates unmaintainable fragmentation. The goal was a general automation platform where business users could build and maintain these workflows themselves, without requiring engineering involvement for standard automation patterns.
Key Challenges:
- Designing a trigger-action model expressive enough for real workflows but simple enough for non-technical users
- Reliable, ordered execution of multi-step action chains
- Conditional logic and branching without requiring users to write code
- Audit trail for compliance — who created which workflow, when it ran, what it did
- Preventing infinite loops from circular trigger-action relationships
System Architecture
Workflows are defined through a UI as a trigger event + optional filter conditions + an ordered list of actions. Definitions are stored in the database and loaded by a workflow engine that subscribes to the application's event bus. When a matching event fires, the engine evaluates filters and executes the action chain via background workers.
Workflow Definition Model
Each workflow has a trigger event type (e.g., "order.created", "payment.received"), optional filter conditions (e.g., "order.amount > 10000"), and an action sequence. Action types are a predefined catalogue: send email, send SMS, update record field, call webhook, create task, notify user.
Event Engine
Application services publish typed events to an internal event bus when significant state changes occur. The workflow engine subscribes to all event types and evaluates which active workflows match each incoming event, dispatching matched workflows for execution.
Action Execution
Action chains execute via background workers supporting parallel or sequential ordering per workflow design. Each action result is logged, and conditional branches are evaluated against action outputs or event data using the filter expression engine.
Workflow Builder UI
Drag-and-drop interface for composing trigger-condition-action workflows with a live preview of what data is available at each step. Configuration fields for each action type are rendered dynamically based on the action catalogue definition.
Key Engineering Challenges
Infinite Loop Prevention
Challenge: A workflow that updates a record can trigger itself if the same record update fires the trigger event, creating an infinite execution loop.
Solution: Execution context propagation — each workflow run carries an originating event ID and workflow ID chain, and the engine rejects triggering a workflow from within its own execution chain or from events it has already produced in the current run.
Safe User-Defined Conditions
Challenge: Allowing users to write arbitrary filter expressions risks execution of malicious code or expressions that crash the engine.
Solution: Restricted expression language — a safe subset of condition operators (comparisons, logical AND/OR, string matching, numeric ranges) evaluated by a sandboxed interpreter with no access to the file system, network, or application internals.
Action Reliability
Challenge: A multi-step action chain may partially fail — completing the first two actions but failing the third — leaving the workflow in an inconsistent intermediate state.
Solution: Each action step is idempotent and checkpointed individually. Failed actions are retried from the failed step; successfully completed steps are not re-executed on resume, preventing duplicate side effects.
Workflow Versioning
Challenge: Editing an active workflow mid-execution can change the logic for a run that has already started, producing inconsistent behaviour.
Solution: Workflow definitions are versioned on every save. In-flight executions use the version snapshot captured at trigger time; new versions activate only for subsequent trigger events.
Solutions Implemented
- Event-Driven Execution: Application event bus triggering workflow evaluation on every significant state change across the platform.
- Safe Expression Engine: Restricted condition evaluator preventing arbitrary code execution while supporting flexible filter logic.
- Idempotent Checkpointed Actions: Per-step execution checkpoints enabling reliable retry from failure point without duplicating completed actions.
- Workflow Versioning: Snapshot-at-trigger execution ensuring workflow edits don't affect in-flight runs.
- Execution Audit Log: Complete log per workflow run — trigger event, filter evaluation result, each action outcome, timing, and errors — accessible to admins for debugging and compliance review.
Outcome & Impact
Business users build automations
Structural prevention enforced
Every run fully logged
Idempotent retry from failure point