Critical lessons from implementing a highly complex workflow system

Transaction Boundaries are Sacred: When spawning child tasks, the parent MUST commit its transaction before children become visible to workers. Using wait_for_event() properly triggers execution system's SleepError, which causes the orchestrator to commit the transaction and enter SLEEPING state.

Event Uniqueness Matters: Events must be unique per workflow execution, not just per task ID. Old events from failed runs can cause wait_for_event() to return immediately without sleeping, breaking the fork/join pattern. Solution: Include run_id in event names.

PostgreSQL JSONB Auto-Parsing: In psycopg3, jsonb columns return Python dicts directly—no json.loads() needed! Attempting to parse them causes TypeError: the JSON object must be str, bytes or bytearray, not dict.