From the Trenches: Onboarding as Code

From the Trenches is a series on real distributed systems built under real production load where I’ve been fortunate enough to be a part of the team. Our first post is about building a system that allows developers and PMs to spin up new onboarding workflows with simple JSON based configs.

This story comes from a payments company I used to work at where we operated multiple products across different geographies, each with variations in their onboarding flows. The system needed to support shared components across journeys while allowing region-specific customizations. The bigger constraint came from our product managers who wanted the ability to reorder onboarding steps and run A/B tests without waiting for engineering cycles.

Onboarding as Code — why this exists

Hard-coding workflows meant every experiment required developer time and deployment overhead. Hence, we built a JSON-based configuration system where PMs could modify component order, swap in different variants, and adjust copy through config files. This moved most onboarding changes from the engineering backlog to self-service configuration.

Design goals (constraints we held the line on)

  • Change config, not code. Flip a template, not a cluster.
  • Deterministic & testable. Every transition is explicit; no “magic ifs” buried in handlers.
  • Blame-free retries. Idempotent writes; zero duplicate KYC calls.
  • Small blast radius. Versioned templates, canary rollout, instant rollback.
  • Observability by default. Per-component funnels, latency SLIs, audit events.

System at a glance

A JSON-templated onboarding engine backed by a reusable SDK:

  • Versioned templates define workflows, components, and milestones; the SDK compiles them into a state machine.
  • Explicit navigation (next, next_if) keeps paths clear and verifiable.
  • Actions via outbox (workers, retries, timeouts) keep side-effects off the request path.
  • Separation of concerns. The onboarding service owns flow/state; domain services own verification.
  • First-class telemetry. Consistent events/metrics to spot drop-offs and regressions fast.

Architecture

Architecture

Happy-path flow

  1. Client → LB → PGOS (API service embedding the OBS SDK).
  2. SDK loads a workflow template, creates an instance, persists state in PGOS DB.
  3. PUT saves data, validates, computes next component, enqueues side-effects (if any).
  4. Workers execute outbox actions (e.g., KYC/bank verification) against domain services.
  5. Repeat until all milestones complete; audit + metrics emitted throughout.

Core concepts

  • Workflow: Named template parameterized by metadata (country, product, locale, version), composed of milestones and components.
  • Component: Smallest interactive unit (form/upload/review/custom). Holds fields, validation, optional meta (defaults/options), and steps that trigger actions when completed.
  • Milestone: Labeled group of components used for progress and gating.
  • Navigation: start, next, or next_if (conditional) control transitions.
  • Actions: event, service_call, or job executed via outbox.
  • Sync: sync_to_account_service toggles writes to external profile systems.

Final API Endpoints

Endpoint Method Description
/api/workflows POST Creates a new workflow and returns the initial state.
/api/workflows/{workflow_id} GET Retrieves the current state of a workflow, including progress and the current component.
/api/workflows/{workflow_id} PUT Saves the state of the current component, validates fields, and returns the next component or errors.

SDK flow with initialization (tiny samples)

1) Init

1
2
3
4
5
6
7
8

// Load templates once at startup. Fail fast if any template is invalid.
configs := map[string]string{
    "payment_gateway_in": "{...JSON...}",
    "payment_gateway_us": "{...JSON...}",
}
if err := sdk.Init(configs); err != nil {
    log.Fatal(err)
}

2) Save workflow (pseudocode)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

func saveWorkflow(workflowID string, submittedData map[string]interface{}) (WorkflowResponse, error) {
    // Step 1: Fetch Workflow State
    workflow := db.getWorkflowByID(workflowID)
    if workflow == nil {
        return WorkflowResponse{}, fmt.Errorf("workflow not found")
    }

    // Step 2: Update Component Fields
    currentComponent := workflow.getCurrentComponent()
    if err := currentComponent.updateFields(submittedData); err != nil {
        return WorkflowResponse{}, fmt.Errorf("validation error: %v", err)
    }

    // Step 3: Validate Fields
    if err := currentComponent.validateFields(); err != nil {
        return WorkflowResponse{}, fmt.Errorf("validation error: %v", err)
    }

    // Step 4: Update Workflow State
    workflow.updateState(currentComponent)
    workflow.calculateProgress()

    // Save updated state in DB
    if err := db.updateWorkflow(workflow); err != nil {
        return WorkflowResponse{}, err
    }

    // Step 5: Log Workflow Changes
    db.logWorkflowChange(workflowID, currentComponent.name, "update", submittedData, nil)

    // Step 6: Determine Next Component
    nextComponent := workflow.getNextComponent()

    // Step 7: Return Response
    return WorkflowResponse{
        WorkflowID:       workflow.id,
        CurrentComponent: nextComponent,
        Progress:         workflow.progress,
    }, nil
}

Example workflow config (trimmed)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121

{
  "workflow_name": "curlec_onboarding",
  "metadata": {
    "product": "payment_gateway",
    "country": "IN",
    "type": "modular",
    "org_id": "12345",
    "version": "1.0"
  },
  "milestones": [
    {
      "milestone_name": "account_setup",
      "components": [
        {
          "component_name": "user_profile",
          "fields": {
            "email": {
              "label": "Email Address",
              "type": "text",
              "required": true,
              "default_value": "",
              "validation_rules": {
                "regex": "^[\\w.%+-]+@[\\w.-]+\\.[a-zA-Z]{2,}$",
                "error_message": "Invalid email format"
              }
            },
            "business_name": {
              "label": "Business Name",
              "type": "text",
              "required": true,
              "default_value": ""
            }
          },
          "meta": {
            "field_defaults": {
              "country": "IN",
              "currency": "INR"
            }
          },
          "steps": [
            {
              "step_name": "email_verification",
              "action_on_complete": "send_verification_email"
            }
          ],
          "sync_to_account_service": true
        },
        {
          "component_name": "business_details",
          "fields": {
            "registration_number": {
              "label": "Registration Number",
              "type": "text",
              "required": true,
              "default_value": ""
            },
            "industry": {
              "label": "Industry",
              "type": "dropdown",
              "required": true,
              "default_value": "IT",
              "options": [
                { "value": "IT", "label": "Information Technology" },
                { "value": "finance", "label": "Finance" },
                { "value": "retail", "label": "Retail" }
              ]
            }
          },
          "meta": {
            "dropdown_options": {
              "industry": [
                { "value": "IT", "label": "Information Technology" },
                { "value": "finance", "label": "Finance" },
                { "value": "retail", "label": "Retail" }
              ]
            }
          },
          "sync_to_account_service": true
        }
      ]
    },
    {
      "milestone_name": "payment_setup",
      "components": [
        {
          "component_name": "bank_account",
          "fields": {
            "account_number": {
              "label": "Account Number",
              "type": "text",
              "required": true,
              "default_value": ""
            },
            "ifsc_code": {
              "label": "IFSC Code",
              "type": "text",
              "required": true,
              "default_value": "",
              "validation_rules": {
                "regex": "^[A-Z]{4}0[A-Z0-9]{6}$",
                "error_message": "Invalid IFSC Code format"
              }
            }
          },
          "meta": {
            "field_defaults": {
              "account_type": "savings"
            }
          },
          "steps": [
            {
              "step_name": "account_verification",
              "action_on_complete": "verify_bank_account"
            }
          ],
          "sync_to_account_service": false
        }
      ]
    }
  ]
}

Sample GET workflow response

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

{
  "workflow_id": "wf-201",
  "workflow_name": "curlec_onboarding",
  "metadata": {
    "product": "payment_gateway",
    "country": "IN",
    "type": "modular",
    "org_id": "12345",
    "version": "1.0"
  },
  "current_component": {
    "component_name": "user_profile",
    "fields": {
      "email": {
        "label": "Email Address",
        "type": "text",
        "required": true,
        "default_value": "[email protected]",
        "validation_rules": {
          "regex": "^[\\w.%+-]+@[\\w.-]+\\.[a-zA-Z]{2,}$",
          "error_message": "Invalid email format"
        }
      },
      "business_name": {
        "label": "Business Name",
        "type": "text",
        "required": true,
        "default_value": "Acme Corp"
      }
    },
    "meta": {
      "field_defaults": {
        "country": "IN",
        "currency": "INR"
      }
    },
    "steps": [
      {
        "step_name": "email_verification",
        "action_on_complete": "send_verification_email"
      }
    ],
    "sync_to_account_service": true
  },
  "completed_milestones": [],
  "remaining_milestones": [
    "account_setup",
    "payment_setup"
  ],
  "progress": 10
}

Sample PUT workflow response

Scenario: Saving the state of the current component

Request Payload:

1
2
3
4
5
6
7

{
  "component_name": "user_profile",
  "fields": {
    "email": "[email protected]",
    "business_name": "Acme Corp"
  }
}

Response

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

{
  "workflow_id": "wf-12345",
  "current_component": {
    "component_name": "business_details",
    "fields": {
      "registration_number": {
        "label": "Registration Number",
        "type": "text",
        "required": true,
        "default_value": ""
      },
      "industry": {
        "label": "Industry",
        "type": "dropdown",
        "required": true,
        "default_value": "IT",
        "options": [
          { "value": "IT", "label": "Information Technology" },
          { "value": "finance", "label": "Finance" },
          { "value": "retail", "label": "Retail" }
        ]
      }
    },
    "meta": {
      "dropdown_options": {
        "industry": [
          { "value": "IT", "label": "Information Technology" },
          { "value": "finance", "label": "Finance" },
          { "value": "retail", "label": "Retail" }
        ]
      }
    },
    "sync_to_account_service": true
  },
  "completed_milestones": [
    "account_setup"
  ],
  "remaining_milestones": [
    "payment_setup"
  ],
  "progress": 50
}

Runtime behavior & extensibility (quick notes)

  • SDK enforces required fields/regex, applies meta.defaults, and resolves meta.options dynamically.
  • On PUT success, SDK computes next or evaluates next_if to decide transitions.
  • Transitions update progress and emit component.saved, component.validated, milestone.completed.
  • Failures return validation_errors and keep the user on the same component.
  • Add a new component by implementing it in the SDK and referencing its id/component_name in the workflow JSON.
  • Introduce a new action type by adding an adapter (e.g., to call a domain service) and referencing it in steps[].action_on_complete.
  • Version templates via metadata.version and keep multiple templates hot-loaded for A/B or country variants.