DataCollector: Debugging & Advanced Features

🔄 Step-by-Step Debugging Workflow

Strategy/System Type Debugging

1. Open System Type or System edit page
2. Configure connector calls, produces, expressions as needed
3. Click Test in header (or in Debugging tab empty state)
4. TestConfigModal: Select system (system-type) or confirm (system)
5. Optionally select Sample Input (Full, Delta 1h ago) or edit JSON
6. Click Run Test
7. Switch to Debugging tab—see streaming state, then results
8. Use badge trail: Connector Calls tab → row → section → expression
9. Click expression badge or Templates tab "Debug" to open ExpressionEditorDrawer
10. View evaluation grid; optionally enable Sandbox to experiment
11. Download events (JSON/YAML) if needed
12. Clear or run another test

Schema Mapping Debugging

1. Open Schema Edit page
2. Navigate to Debugging tab
3. Select system from dropdown
4. Click Run; wait for connector calls to stream
5. Click a connector call row → SchemaDebugDrawer opens
6. Click cell in transformed grid → Expression editor shows field expression
7. Click Check Reliability to verify roundtrip
8. Edit expressions; click Evaluate to test
9. Use Columns chooser to focus; Add/Remove/Rename fields as needed

🐛 Debugging Overview

DC-UI provides a unified debugging experience across every config page that has something runnable. The Debugging tab and the Test button look and behave the same whether you're editing a Universal-strategy system type, a specific-strategy system type (LDAP Delta, SAP IAS, SAP LDAP VDS, IDM Reference Data VDS, Fieldglass), a system instance, or a schema mapping.

Unified Debugging Tab

The Debugging tab was unified across all entry points. Previously, Universal strategy had UniversalDebuggingTab while other strategies used inline implementations. Now all use the same component with consistent behavior.

Tab Restructuring

The orders below are the page-level tab order — i.e., what the user sees in the browser. The page (e.g., SystemTypeEditPage) inserts General, Used By (existing only), and YAML around what the strategy designer's get*DesignerTabs() function returns.

Debugging Tab States

Test Button Everywhere

• System Type Edit: Test button for ANY system type—simplified popup with system selection only
• System Edit: Test button for ANY system—empty popup (system known), immediate test
• Schema Edit: Test button navigates to Debugging tab (no popup—schema has its own system selector)

The strategy_name in the request body is always builtin.strategies.universal.UniversalStrategy — Universal is the strategy that implements the debug operations. The operation is what differs by entry point: System / System Type tests use run_full_inventory, Schema tests use debug_connector_calls (see "Test buttons → debug operations" further down). Credentials and variables are loaded from the selected system.

DebugState Interface

interface DebugState {
  results: { success: boolean | null; error?: string; events: any[]; inventory_run_id?: string } | null;
  isStreaming: boolean;
  eventCount: number;
  onStop: () => void;
  onClear: () => void;
  onTest?: () => void;
  saveCount?: number;  // Incremented on save to trigger auto-clear
}

Props Flow for Universal Strategy

Auto-Clear on Save

When user saves configuration, the debug session is cleared. saveCount is incremented in handleSave; when it changes, the debugging tab resets to empty state.

Keyboard Shortcuts (Debugging Context)

Relevant shortcuts: Ctrl+S Save (where applicable); Escape Close modals/drawers; Ctrl+R Refresh (e.g., Admin Events).

Streaming API (useDebugStream)

Debug sessions use Server-Sent Events (SSE). The useDebugStream hook provides:

const {
  startStream,    // (params) => Promise - POST to debug/strategy/stream
  stopStream,    // () => void - abort connection
  isStreaming,   // boolean
  events,        // any[] - accumulated events
  error,         // string | null
  clearEvents    // () => void
} = useDebugStream({ onComplete?: (e: DebugCompleteEvent) => void });

Request body: { strategy_name, request: { operation, system_name, system_type_yaml?, inventory_input? } }. Response: SSE stream of monitoring events (step_start, connector_call, template_evaluation, change_dispatch, etc.).

🐰 Debug Navigation Trail ("Follow the Rabbit")

DC-UI introduced animated debug badges at every navigation level, creating a visual trail from tabs down to expression fields. You can "follow the rabbit" to navigate directly to expressions that have debug data.

Navigation Hierarchy with Badges

Contextually Meaningful Counts

Badges show different metrics at different levels—not expression evaluations everywhere:

ConnectorCallDrawer Badge Implementation

Drawer receives callName and uses debug context:

// Entity Production section header
const sectionCount = debugContext?.getChangeDispatchCountForCall(callName) ?? 0;

// Per produce row
const changeType = produce.confirm ? 'confirm' : produce.delete ? 'delete' : ...;
const entityType = produce.confirm ?? produce.delete ?? ...;
const rowCount = debugContext?.getChangeDispatchCount(callName, changeType, entityType) ?? 0;

// Follow-up section: dispatch count from this call
const followUpCount = debugContext?.getFollowUpDispatchCount(callName);
// Per next call row
const targetCount = debugContext?.getFollowUpDispatchCount(callName, targetCallName);

Event Correlation

Monitoring events don't include call_name directly. Correlation requires:

1. Build step_id → call_name map from step_add events where added_step_name === 'execute' and parameters.call
2. Use labels.step_id on other events to look up the call name

// step_add event structure (source for step_id→call map)
event_type: step_add
labels:
  step_id: 745ad347-...        # parent step that created this
  status: another_step_run     # for follow-ups only
added_step_id: 6248daa5-...   # the NEW step's ID
added_step_name: execute       # "execute" = connector call step
parameters:
  call: fetch_accounts        # THE CALL NAME

// change_dispatch - status maps to change_type
labels:
  step_id: 332146a8-...
  entity_type: group
  status: entity_confirmed_exact  # → change_type: confirm
  // entity_deleted → delete
  // entities_purge_unseen → purge_unseen
  // entities_purge_by_relation → purge_by_relation

📝 Expression Editor Drawer

The ExpressionEditorDrawer is the primary interface for viewing and editing expressions across connectors, produces, and next calls. It integrates tightly with debug context.

Always Expanded — split layout

The drawer always renders at 90vw with a fixed split:

• Left panel — Debug evaluation grid (or empty state with Test button when no debug session has captured this expression yet); takes the remaining width
• Right panel — Expression editor, fixed at 500px

Selection modes

Above the evaluation grid sits a Radio.Group (button style) that controls which rows the action buttons operate on:

• Selected (N) — only rows the user has explicitly checked. Auto-selected when any row checkbox is ticked.
• Random 10 — Fisher-Yates shuffle of the current page; useful for spot-checking large debug sessions
• All (N) — every row in the current grid

Two action buttons

The left panel exposes two parallel buttons; both run with concurrency 10 via Promise.all:

• Check Reliability — re-runs each selected row's saved template_yaml. Surfaces drift between what's stored and what re-evaluation produces (e.g., context shape changes after a connector update).
• Evaluate — runs the current expression in the right panel against each selected row's captured context. Use this to iterate on a fix before saving.

Stats cards

Two summary cards sit above the grid; each tracks one of the action buttons:

• Reliability — matched / total + percentage from the last Check Reliability run. Tinted green at 100%, orange at ≥95%, red below 95%.
• Evaluation — same shape, separate counters, driven by the last Evaluate run.

Mixed mode

The drawer can be opened with no specific path — typically from the Templates tab's Debug All button. In that mode, the grid shows every captured template_evaluation with a Path column visible; selecting a row loads that row's template_yaml into the editor (parsed back into the right type by detectExpressionType + getExpressionDisplayValue). Lets you sweep across the entire run instead of focusing on a single field.

Read-Only vs Editable

When opened from Debugging → Templates → Debug, expressions are read-only:

• Info alert: "View Only - This expression cannot be edited from the debugging tab"
• Radio.Group (expression type selector) disabled
• TextArea readOnly={true} with subtle background
• Type hints hidden
• Reason: Drawer doesn't know where in YAML the expression lives—can't save

Sandbox Mode

In read-only mode, a Sandbox toggle enables experimentation without saving:

Sandbox resets to OFF on row change in the evaluation grid and when the drawer is reopened — there's no carry-over of unsaved sandbox state across sessions.

Local State

The drawer uses local state instead of deriving from props on every keystroke:

const [localType, setLocalType] = useState<ExpressionType>('value');
const [localContent, setLocalContent] = useState<string>('');

// Initialize from props when drawer opens
useEffect(() => {
  if (open) {
    const detectedType = detectExpressionType(value);
    const displayContent = getExpressionDisplayValue(value, detectedType);
    setLocalType(detectedType);
    setLocalContent(displayContent);
  }
}, [open, value]);

// Save on close only
const handleClose = useCallback(() => {
  if (!readOnly) {
    const reconstructed = reconstructValue(localContent, localType);
    onSave(reconstructed);
  }
  onCancel();
}, [readOnly, localContent, localType, onSave, onCancel]);

Validation

reconstructValue validates content—throws errors for invalid input. UI shows validation error state and Alert. No heuristics: invalid JSON for Literal Value shows error; invalid YAML for YAML Object shows error.

🔤 Expression Types

DC-UI exposes expressions in two distinct surfaces, and each surface accepts a different subset of the expression-type catalog. Don't mix the two — what works in the schema-mapping editor is a strict subset of what works in the Universal-designer drawer.

Schema-mapping expressions (3 types)

Used by SchemaFieldEditor and MappingTable in the schema editor and the schema debug drawer. The variable in scope is result — the raw item being transformed.

Universal-designer expressions (6 types)

Used by ExpressionEditorDrawer on every expression field in the Universal-strategy designer (params, success_check, pre_transform, produces, next_connector_calls, finish.produces). The variable in scope depends on the field — see the Universal Strategy Connector Calls section in day 27.

reconstructValue Strict Rules

export const reconstructValue = (content: string, type: ExpressionType): any => {
  const trimmed = content.trim();
  if (trimmed === '') return undefined;

  switch (type) {
    case 'expression': return `[[ ${trimmed} ]]`;
    case 'expr_tag': return { __yaml_type: 'expr', __yaml_value: trimmed };
    case 'yaml_tag': return { __yaml_type: 'yaml', __yaml_value: trimmed };
    case 'jinja': return trimmed;
    case 'object': return parseYaml(trimmed);  // STRICT: no fallback
    case 'value': return JSON.parse(trimmed);   // STRICT: must be valid JSON
    default: throw new Error(`Unknown expression type: ${type}`);
  }
};

📡 Debug Events Context

DebugEventsProvider provides debug events and counting functions to deeply nested components. Wraps SystemTypeEditPage, SystemEditPage content.

Context Interface

interface DebugEventsContextValue {
  events: any[];
  getEvaluationCount: (debugPath: string) => number;
  getEvaluationCountByPrefix: (prefix: string) => number;
  getTotalEvaluationCount: () => number;
  getConnectorCallCount: (callName?: string) => number;
  getChangeDispatchCount: (callName: string, changeType: string, entityType: string) => number;
  getChangeDispatchCountForCall: (callName: string) => number;
  getFollowUpDispatchCount: (sourceCall: string, targetCall?: string) => number;
  onTest?: () => void;
  isStreaming?: boolean;
}

Function Descriptions

Provider Placement

DebugEventsProvider
  events={streamingEvents}
  onTest={() => setTestModalVisible(true)}
  isStreaming={isStreaming}
  >
  <UniversalStrategyDesigner ... />
</DebugEventsProvider>

Pre-computed Counts (Implementation)

Context pre-computes maps for O(1) lookup:

// Step 1: Build step_id → call_name from step_add
const stepIdToCallName = new Map<string, string>();
events.forEach((event) => {
  if (event.event_type === 'step_add' &&
      event.added_step_name === 'execute' &&
      event.parameters?.call) {
    stepIdToCallName.set(event.added_step_id, event.parameters.call);
  }
});

// Step 2: Count step_start per call
const stepStartCounts = new Map<string, number>();
events.forEach((event) => {
  if (event.event_type === 'step_start') {
    const callName = stepIdToCallName.get(event.labels?.step_id);
    if (callName) {
      stepStartCounts.set(callName, (stepStartCounts.get(callName) || 0) + 1);
    }
  }
});

// Step 3: Count change_dispatch with compound key callName/changeType/entityType
const statusToChangeType = (status: string): string | null => {
  if (status?.startsWith('entity_confirmed')) return 'confirm';
  if (status === 'entity_deleted') return 'delete';
  if (status === 'entities_purge_unseen') return 'purge_unseen';
  if (status === 'entities_purge_by_relation') return 'purge_by_relation';
  return null;
};

useDebugEvents Hook

Components use useDebugEvents() to access context. Returns full value; components destructure getEvaluationCount, getConnectorCallCount, etc. as needed.

🏷️ Debug Badges

All debug badges use consistent styling via custom CSS classes in GlobalThemeOverrides.

Badge Styling

/* Solid orange background with BLACK text */
.dc-debug-badge.ant-tag {
  background: var(--warning-color, #faad14) !important;
  border-color: var(--warning-color, #faad14) !important;
  color: #000000 !important;
}

.dc-debug-badge.ant-tag * {
  color: #000000 !important;
}

/* Pulsing glow animation when streaming */
@keyframes dc-debug-pulse {
  0%, 100% { box-shadow: 0 0 0 0 rgba(250, 173, 20, 0.4); }
  50% { box-shadow: 0 0 0 4px rgba(250, 173, 20, 0.2); }
}

.dc-debug-badge-streaming {
  animation: dc-debug-pulse 1.5s ease-in-out infinite !important;
}

Badge Usage

// Expression input badge
<Tag
  color="gold"
  className={`dc-debug-badge ${isStreaming ? 'dc-debug-badge-streaming' : ''}`}
>
  {isStreaming ? <SyncOutlined spin /> : <BugOutlined />}
  {' '}{evalCount}
</Tag>

• Icon: Spinning SyncOutlined when streaming, BugOutlined otherwise
• Animation: Apply dc-debug-badge-streaming class when isStreaming=true
• Expression type badge: dc-expression-badge—solid dark background, light text

🗺️ Schema Mapping Debugging

The Schema Edit page has its own debugging tab, powered by ConnectorCallsDebugPanel and SchemaDebugDrawer.

ConnectorCallsDebugPanel

• Schema mapping badges showing counts per schema
• Connector calls grid: Connector, Schema, Status, Duration, Items
• Row selection shows detail panel with transformed data
• Raw connector response JSON viewer
• Expression context: { result: <source item> }

Schema Edit Debugging Tab Flow

1. Open Schema Edit page /dc-ui/config/schemas/:name
2. Navigate to Debugging tab
3. Select system from dropdown (auto-loads systems using this schema via listSystems({ schemaMapping: schemaName }))
4. Click Run to start streaming
5. Backend uses debug_connector_calls operation with system_name, system_type_name
6. View connector calls filtered by schema_mapping
7. Click row to open SchemaDebugDrawer

connector_call Event Structure

event_type: connector_call
event_id: uuid
connector: fetch_accounts
credential_pointer: secrets://...
schema_mapping: account_sapias    # Filter key for Schema tab
duration_ms: 234
labels:
  status: success | error
response_transformed_data:        # Transformed items array
  - { id: "123", name: "..." }
response_correlations:
  raw_connector_response: {...}
  transformed_pairs: [{raw, transformed}, ...]
response_error_data: {...}        # If failed

SchemaDebugDrawer

Opens when clicking a connector call row. Split layout — left panel is the transformed data grid, right panel is the expression editor for the selected column.

Per-store switching

When the captured connector call dispatched to multiple stores, the drawer adds a store switcher above the grid. Each store has its own baseline rows (from the matching store's transformed_data) and its own schema mapping, so switching stores re-keys the grid columns and re-anchors the expression editor against the right schema.

Three-state cell display

Each value cell carries up to three states, each comparable side-by-side:

• Original — the baseline transformed_data the engine produced during the captured run
• Reliability — re-evaluated with the saved schema (catches drift between stored data and what the saved expressions produce now)
• Evaluated — re-evaluated with the current right-panel expression (the in-progress edit)

Click a cell to open a popover showing all three states with type badges. Strikethrough on Original when Reliability disagrees, so visual scanning surfaces drift fast.

Immediate onChange

Edits inside the drawer call onChange(newMappings) immediately, propagating to the schema page state and the YAML tab in real time — same as the wider immediate-editing pattern across DC-UI. There's no internal Save button on the drawer; saving happens at the schema page's header.

Left panel — transformed data grid

• Cell selection: Click cell → row + column highlighted
• Reliability check: re-apply saved mappings, compare, show ✓ (green) or ✗ (red)
• Evaluate: test edited expressions, show new values alongside originals with strikethrough
• Dynamic columns: from server data only — Original / Reliability / Evaluated columns
• Columns chooser: "Show All" / "Minimal" buttons, checkbox per column

Right panel — expression editor

• Mapping expression for the selected column
• Context display: raw data for the selected row (proper JSON: {"result": {...}})
• Field management: Add Field, Remove Field, Rename Field

CSS Specificity Workarounds

Global theme overrides require custom classes:

.schema-debug-table .ant-table-tbody > tr.schema-debug-row-selected > td {
  background: rgba(24, 144, 255, 0.1) !important;
}
.schema-debug-table .ant-table-tbody > tr > td.schema-debug-col-selected {
  background: rgba(24, 144, 255, 0.08) !important;
}
.schema-debug-table .reliability-pass path { fill: rgba(82, 196, 26, 0.8) !important; }
.schema-debug-table .reliability-fail path { fill: rgba(255, 77, 79, 0.8) !important; }
.schema-debug-table .value-original { color: rgba(150, 150, 150, 0.7) !important; }

Type Badges in Popup

SchemaDebugDrawer UI Mockup

The debug API

DC exposes two debug endpoints. The body shape is the same for both: a strategy_name plus an opaque request dict whose schema is defined by the strategy. The Universal strategy implements four operations: RUN_FULL_INVENTORY, EVALUATE_TEMPLATE, DEBUG_CONNECTOR_CALLS, and APPLY_SCHEMA_MAPPING.

POST /debug/strategy            # batch mode — returns all events on completion
POST /debug/strategy/stream     # SSE — events stream as they happen, 5s heartbeats during idle

{
  "strategy_name": "builtin.strategies.universal.UniversalStrategy",
  "request": {
    "operation": "apply_schema_mapping",
    "system_name": "sapias-prod",
    "system_type_name": "sapias",
    "schema_name": "account_sapias",
    "schema_yaml": null,           // optional: override schema YAML inline
    "raw_data": [ ... ]
    // ... operation-specific fields
  }
}

Other strategies can implement their own debug() / debug_streaming() static methods with their own request shapes — DC routes the opaque request to the strategy without interpreting it.

Auth: in PDP mode, debug endpoints require op:dc-ui:strategy.manage (same blast radius as editing strategy code). In scope mode, they require datacollector:debug.

Test buttons → debug operations

The Test button in DC-UI is the same control everywhere, but the operation it sends to POST /debug/strategy/stream depends on which page hosts it:

• System and System Type debugging tabs use operation: run_full_inventory — runs an end-to-end DETACHED_READONLY inventory with the unsaved YAML / form state, dispatching every connector call the strategy would normally make. This is the broad "does my whole config work?" test.
• Schema Mapping debugging tab uses operation: debug_connector_calls — narrows the run to only the connector calls that produce items for this schema, with the schema-mapping engine attached so each captured call has full response_transformed_data ready for the SchemaDebugDrawer. This is the "did my schema-mapping changes apply correctly?" test.

Both POST to /api/datacollector/debug/strategy/stream over SSE; both stream the same connector_call / step_* / template_evaluation events. The difference is operation-side scope, not transport.

🔗 Understanding Step Dependencies

DataCollector strategies execute as directed acyclic graphs (DAGs) of steps. Each step can depend on one or more parent steps, creating a dependency chain that determines execution order.

Dependency Resolution

The step dependency system allows for parallel execution of independent steps while ensuring that dependent steps wait for their prerequisites. This is crucial for:

• Data availability: Ensure parent steps complete before child steps access their data
• Performance optimization: Allow independent steps to run concurrently
• Error handling: Prevent execution of dependent steps if prerequisites fail
• Resource management: Control the order of connector calls and API requests

Common Dependency Patterns

Viewing Dependencies in Debug UI

The debug UI shows step dependencies through:

• Performance Gantt chart: Visual timeline showing when steps execute relative to each other
• Step details: depends_on field shows parent step names
• Follow-up badges: Count of steps dispatched by each connector call
• Step status: Shows which steps are waiting vs executing vs completed

📶 Debug Event Badges & Streaming

Custom CSS classes and animation keyframes for streaming state.

Expression Input Badges

Animation Keyframes

@keyframes dc-debug-pulse {
  0%, 100% { box-shadow: 0 0 0 0 rgba(250, 173, 20, 0.4); }
  50% { box-shadow: 0 0 0 4px rgba(250, 173, 20, 0.2); }
}

📍 Debug Path Reference

Complete table of every expression location and its debug path. Paths must match backend evaluation.

Health check params have no debugPath—evaluated outside inventory context.

Event Matching Logic

Template evaluation events have event_type: 'template_evaluation' and labels.status set to the debug path. ExpressionInput matches via event.labels.status === debugPath. The backend emits these paths during strategy execution.

💾 Download Debug Events

DC-UI adds JSON/YAML export of captured events from the Debugging tab.

Implementation

• Download dropdown between Stop and Clear buttons
• Options: "Download as JSON", "Download as YAML"
• Filename: debug-events-{inventory_run_id}.json or .yaml
• Disabled when no events

Output Format

// JSON
{
  "inventory_run_id": "abc-123",
  "events": [
    { "event_type": "connector_call", ... },
    { "event_type": "template_evaluation", ... }
  ]
}

// YAML
inventory_run_id: abc-123
events:
  - event_type: connector_call
    ...
  - event_type: template_evaluation
    ...

📥 Sample Inventory Inputs

DC-UI adds predefined inventory input options to the Test Config Modal.

Backend Integration

• API: GET /configs/plugins/strategies/{name}/sample-inputs
• SampleInventoryInput: title, input, description_markdown
• Universal strategy returns: "Full" ({}), "Delta 1h ago" ({started_at, full_completed_at})

TestConfigModal Updates

• Added strategyName prop for fetching sample inputs
• onTest signature: (systemName: string, inventoryInput: Record<string, any>) => void
• Fetches sample inputs when modal opens
• Shows selectable preset tiles (Full, Delta 1h ago, etc.)
• Markdown description for selected input
• JSON editor for customizing the input

UI Layout

The debug modal shows: (1) System instance tiles, (2) Sample inventory input tiles, (3) Markdown description, (4) JSON editor for custom input.

SampleInventoryInput Interface

interface SampleInventoryInput {
  title: string;           // e.g., "Full", "Delta 1h ago"
  input: Record<string, any>;  // e.g., {} or {started_at, full_completed_at}
  description_markdown?: string;
}

Universal Strategy Sample Inputs

⚙️ Test Config Modal

DC-UI introduced a simplified common TestConfigModal for all strategies.

Modes

Removed from Old Modal

Credential pointer input, variables table, "specify manually" toggle. Backend loads credentials from selected system.

TestConfigModal Props

interface TestConfigModalProps {
  visible: boolean;
  onCancel: () => void;
  onTest: (systemName: string, inventoryInput?: Record<string, any>) => void;
  loading?: boolean;
  mode: 'system-type' | 'system';
  systemTypeName?: string;  // system-type: filter systems
  systemName?: string;     // system: pre-known
  strategyName?: string;   // for fetching sample inputs
}

API Request

await startStream({
  strategy_name: 'builtin.strategies.universal.UniversalStrategy',
  request: {
    operation: 'run_full_inventory',
    system_name: systemName,
    system_type_yaml: currentYaml,  // Include unsaved changes
    // NO system_type_name when using system_type_yaml
  },
});

📊 DebugResultsPanel - 9 Tabs

DebugResultsPanel displays all captured events across nine sub-tabs: Overview, Steps, Connector Calls, Changes, Logs, Templates, Diffs, Input Changes, Diagrams. Each tab carries a count badge driven by the events stream.

Overview Tab

Summary of run: inventory_run_id, total events, success/failure indicators.

Expandable Row Data

Steps tab rows expand to show contextual data:

• step_start, step_add: Parameters panel (JSON)
• step_finish: Timing breakdown: actions_duration_ms, idle_duration_ms, commit_duration_ms, total_duration_ms, kafka_wait_ms, file_wait_ms
• inventory_finish: Next Inventory Input (if present)
• All step events: IDs panel: step_id, step_run_id, added_step_id, unique_key (copyable)

Steps Tab

Columns: Time, Event, Step, #, Outcome, Duration, Message. Expandable rows with parameters, timing breakdown, IDs.

Connector Calls Tab

Connector call events: connector name, schema, status, duration, item count. Badge: green if all OK, red if any failed.

Changes Tab

Change dispatch events. Blue badge.

Templates Tab

Template evaluation events. Debug buttons open ExpressionEditorDrawer (read-only). Red badge if template errors, blue otherwise.

Diffs Tab

Entity diffs. Blue badge whose count combines both change_decision events (CP-emitted decisions) and diffs_publish events (DiffPublisher-routed diffs) — DebugResultsPanel collapses what Past Run Details splits into separate "Diffs" and "Diff Publications" tabs into a single panel here.

Input Changes Tab

NII change events captured during the debug session. Each row shows: timestamp, target stream, command (when the write was triggered by a command rather than finish_inventory), command_id, step name, plus the old / new value presented as a collapsible diff. Useful for verifying that finish_inventory(next_inventory_input=...) and command-driven writes produce the expected next-run state.

Diagrams Tab

Flow visualization. Blue badge.

Logs Tab

Always shows badge. Color: red if errors, orange if warnings, blue otherwise. "View Details" button opens modal with full log record.

Badge Overflow

All badges use overflowCount={9999999} to show real numbers instead of "99+".

Complete Event Type Reference

🔧 Advanced Configuration

Circuit Breaker

The circuit breaker pattern prevents cascading failures by automatically disabling connector calls when a threshold of failures is reached. This protects both the DataCollector and the target system.

Inheritance: Global defaults → System Type → System. Empty fields inherit.

Storage Settings

StorageConfigEditor: checkbox matrix with entity types as rows, storage options as columns:

• Disable Changes: Disable change storage (StepRunChange) for entity type
• Enable Entities Transformed: Store in Entity.transformed_data
• Enable Changes Transformed: Store in StepRunChange.transformed_data

System Type: required. System: optional override.

Monitoring Exclusions

GeneralSettingsPanel fetches schema from GET /api/datacollector/monitoring/schema. MonitoringDisableRule: event_type, field, destination, decision_unchanged, change_type, entity_type, template_use_case_regex. Contextual fields based on event type.

Monitoring Schema Response

{
  "event_types": [
    {"value": "connector_call", "label": "Connector Call", "fields": ["request_params", "..."]}
  ],
  "monitoring_destinations": [
    {"value": "kafka", "label": "Kafka"}
  ],
  "change_types": [
    {"value": "entity_confirmed_exact", "label": "Entity Confirmed Exact"}
  ]
}

StorageConfig Type

interface StorageConfig {
  disable_changes: string[];              // Entity types to skip change storage
  enable_entities_transformed_data: string[];  // Store in Entity.transformed_data
  enable_changes_transformed_data: string[];   // Store in StepRunChange.transformed_data
}

Circuit Breaker YAML Output

circuit_breaker:
  enabled: true
  threshold: 5
  timeout: 30
  success_threshold: 2
  window_seconds: 60
  security_mode: false

🏗️ Development Patterns

From datacollector/docs/strategies-development/040-development-patterns.md.

Strategy Development

• Use unit and E2E tests where DC runs headless with --execute
• Monitoring events output to files; tests assert against outputs
• Plugin code hot-reloads with 5-second polling

Connector Design

Multi-Method Connectors: Single class, multiple operations via params['method']. Example: LDAP connector with search, open_connection, paged_search, close_connection.

SDK Boundary

Credentials schema is connector-specific. DC-UI credentials designer adheres to that schema. Strategies define what to collect; connectors define protocol and credentials.

Connector execute() Pattern

async def execute(self, params: Dict, credentials: Dict, timeout_ms: float) -> Any:
    method: str = params['method']
    if method == 'search':
        return await self._execute_search(params, credentials, timeout_ms)
    elif method == 'paged_search':
        return await self._execute_paged_search(params, timeout_ms)
    else:
        raise ConnectorCallException(
            ConnectorCallStatus.FATAL_ERROR,
            {"error": f"Unknown method: {method}"}
        )

Plugin Hot-Reload

Changes to custom plugin files (strategies, connectors) are detected automatically with 5-second polling. No DC restart needed. Use unit tests for verification.

✅ Best Practices

Debugging Workflow

• Run Test with Detached Readonly before deploying config changes
• Use Sample Inventory Inputs (Full, Delta 1h ago) for common scenarios
• Follow badge trail to navigate to expressions with issues
• Use Sandbox mode in Templates tab to experiment without saving
• Download events (JSON/YAML) for offline analysis or team sharing

Schema Mapping Debugging

• Run Reliability Check after loading—identifies serialization mismatches
• Use "Minimal" columns first to focus on key fields
• Evaluate after Add/Remove/Rename to see server result—columns come from server data only
• Context panel shows { result: rawItem }—use for expression writing

Expression Editor

• Use Literal Value for static values (quoted JSON: "hello", 42, true)
• Use YAML Object for nested structures; Expression for dynamic values
• Fix validation errors before saving—reconstructValue is strict
• From debugging tab: read-only by default; enable Sandbox for experimentation

⚡ Performance Optimization

Query Patterns

• Use pagination for large result sets (Entities Explorer, Admin Events)
• Server-side filtering reduces payload
• Avoid N+1: batch fetches for related data

Caching Strategies

• Plugin list (strategies, connectors) cached with invalidation on config change
• Monitoring schema cached per session
• Debug events in memory until cleared or page unload

🔬 Advanced Debugging Techniques

Master these advanced techniques to diagnose complex issues and optimize your DataCollector strategies.

Using Debug Badges to Trace Execution

Debug badges provide a visual breadcrumb trail through your strategy execution. Each badge shows contextually relevant metrics at different navigation levels:

• Tab-level badges: Show aggregate counts across all data (e.g., "Connector Calls: 42" means 42 step_start events with added_step_name === 'execute')
• Row-level badges: Show counts for specific calls, entities, or templates (e.g., "fetch_users [🔄 15]" means that connector was called 15 times)
• Section-level badges: Show grouped metrics (e.g., Entity Production section shows total change_dispatch events for that call)
• Field-level badges: Show expression evaluation counts (e.g., "when [🔄 3]" means that expression was evaluated 3 times with different contexts)

Comparing Original vs Re-evaluated Expressions

The Expression Editor Drawer lets you modify and re-evaluate expressions using captured debug context. This is invaluable for testing fixes without redeploying:

1. Click a field badge with debug data to open the Expression Editor Drawer
2. The left panel shows all captured evaluations with their context and results
3. In Sandbox mode, edit the expression in the right panel
4. Click Re-evaluate to test your changes
5. Compare results: original values appear with strikethrough, new values in color
6. Verify your fix works across all captured contexts before saving

Reliability Checks for JSON Serialization

The Schema Debug Drawer includes a Reliability Check feature that verifies your schema mappings survive JSON serialization/deserialization:

1. Open Schema Debug Drawer after running a connector call
2. Click Reliability Check button
3. System re-applies original mappings to deserialized JSON (simulating database round-trip)
4. Results show ✓ (green) for matching values, ✗ (red) for mismatches

Common serialization issues detected:

• Date precision loss: Python datetime with microseconds → JSON → loses precision
• Type coercion: Numbers become strings, booleans become 0/1
• Null vs empty: null vs "" vs [] handling differences
• NaN/Infinity: Not valid JSON, become null
• Timezone ambiguity: Timestamps without explicit timezone

Download Debug Events for Offline Analysis

The Download button in the Debug Results Panel exports all captured monitoring events in JSON or YAML format:

• JSON format: Machine-readable, easy to process with scripts or tools like jq
• YAML format: Human-readable, better for documentation and code review

Use Cases for Downloaded Events

Example: Extracting Connector Timings with jq

# Get all connector call durations
cat debug-events.json | jq -r '.[] | 
  select(.event_type == "connector_call") | 
  "\(.connector): \(.duration_ms)ms"'

# Find slowest connectors
cat debug-events.json | jq -r '.[] | 
  select(.event_type == "connector_call") | 
  {connector, duration_ms}' | 
  jq -s 'sort_by(.duration_ms) | reverse | .[0:5]'

Using Sample Inventory Inputs for Testing

Sample inventory inputs let you test strategies with predefined data instead of making real API calls. This is essential for development and testing:

When to Use Sample Inputs

• Offline development: Test without network access or credentials
• Edge cases: Inject unusual data to test error handling
• Performance testing: Test with large datasets without rate limiting
• Regression testing: Ensure changes don't break existing behavior
• Demo environments: Show functionality without real integrations

Creating Sample Inputs

Sample inputs are YAML/JSON structures that override the entry step output:

# Sample input for user sync strategy
users:
  - id: "user123"
    email: "john.doe@example.com"
    first_name: "John"
    last_name: "Doe"
    department: "Engineering"
    active: true
  - id: "user456"
    email: "jane.smith@example.com"
    first_name: "Jane"
    last_name: "Smith"
    department: "Sales"
    active: false

groups:
  - id: "group789"
    name: "Developers"
    member_ids: ["user123"]

Sample Inputs from Production

1. Run a test with real credentials
2. Download debug events (YAML)
3. Extract the inventory_finish event's next_inventory_input field
4. Save as sample input for future tests
5. Sanitize sensitive data (emails, IDs, etc.) before sharing

📊 Performance Analysis

The Performance tab provides a Gantt chart visualization showing the execution timeline of all steps, connector calls, and database operations. Master this tool to identify and eliminate bottlenecks.

Reading the Performance Gantt Chart

The Gantt chart displays concurrent execution across multiple dimensions:

• X-axis (time): Relative milliseconds from inventory start. Zoom and pan to focus on specific time ranges.
• Y-axis (steps): Each row represents a step. Grouped by step name with instance numbers for repeated steps.
• Bar length: Duration of operation. Longer bars indicate potential bottlenecks.
• Bar color: Operation type (connector call, transformation, database operation, etc.)
• Gaps: Idle time waiting for dependencies or database commits

Identifying Bottlenecks

Connector Call Bottlenecks

Connector calls to external APIs are often the slowest operations. Look for:

• Long bars: Individual calls taking >1 second indicate slow API responses
• Sequential patterns: Calls happening one after another instead of parallel
• Repeated calls: Same connector called many times (pagination or lack of batching)
• Timeouts: Bars that end abruptly at the timeout threshold

Solutions:

• Enable parallel execution by removing unnecessary dependencies
• Implement batching to reduce number of API calls
• Add pagination to handle large datasets efficiently
• Use circuit breakers to fail fast on slow/unavailable services
• Cache frequently accessed data between runs

Transformation Bottlenecks

Schema mappings and template expressions can become CPU-intensive with large datasets:

• Long transformation bars: Complex expressions or large item counts
• Repeated evaluations: Same expression evaluated many times unnecessarily
• Type conversion overhead: Excessive parsing or serialization

Solutions:

• Simplify complex expressions by breaking into smaller pieces
• Move repeated calculations to reusable functions or step outputs
• Use disable_changes storage config for high-volume entity types you don't need to store
• Optimize schema mappings to avoid redundant field transformations

Database Operation Bottlenecks

Database commits happen at step completion and can block progress:

• Long commit bars: Writing many entities or changes in single transaction
• Lock contention: Multiple steps trying to commit simultaneously
• Index overhead: Slow inserts due to missing or excessive indexes

Solutions:

• Batch entity production across fewer steps instead of many small steps
• Use disable_changes for entity types where you only need final state
• Review database indexes for Entity and StepRunChange tables
• Consider enable_entities_transformed_data only for types you need to query

Parallel Step Execution Patterns

Effective use of parallelism is key to fast inventory runs. The Gantt chart makes parallel execution visible:

Optimal Parallel Pattern

Suboptimal Sequential Pattern

Optimization Strategies

Reading Performance Metrics

The Steps tab provides detailed timing breakdown for each step:

Interpreting Timing Patterns

• High actions, low idle: Step is CPU/API bound, optimize the work itself
• Low actions, high idle: Step is waiting on dependencies, review step order
• High commit relative to actions: Database bottleneck, reduce change volume
• Similar duration across repeated steps: Consistent performance, good baseline
• Increasing duration in repeated steps: Memory leak or resource exhaustion

🛠️ Development Workflow

A systematic workflow for developing, testing, and debugging DataCollector strategies. The DC project ships several test layers; this section covers the patterns you'll actually use, the debug API protocol behind the UI's Test button, and a few engine details that turn into footguns if you don't know about them.

Hot-reload during development

DC polls the configured plugin directories for file changes. Custom strategies (Python files in the custom-strategies path) and connectors hot-reload — the polling interval is configurable (plugins.reload_interval_seconds in config.yaml). The DC-UI Test button runs against whatever the latest reload picked up.

What hot-reloads:

• Custom strategy and connector Python files
• Schema mapping YAML files
• System and system-type YAML files

What requires a restart:

• DC core code changes (src/...)
• Database migrations
• Environment variables
• The DC config.yaml itself

End-to-end test pattern (tests_e2e/)

The canonical e2e harness is python -m src.main --execute <system>: DC runs headless, executes a single inventory for the named system, writes monitoring events and diffs to files, and exits. Tests assert against those files.

# Run a single inventory headless
python -m src.main --execute test_36_universal_basic --config tests_e2e/config/config.yaml

# Output ends up in (configured via monitoring.file and diff.file in config.yaml):
#   tests_e2e/output/monitoring/<run_id>.yaml
#   tests_e2e/output/diffs/<system_name>/<destination>.yaml

The pattern in pytest:

# tests_e2e/test_e2e_NN_yourtest.py
def test_my_strategy(run_inventory, read_monitoring_events, read_diffs):
    events = run_inventory("my_test_system")          # invokes --execute, returns events
    assert any(e["event_type"] == "step_finish" and e["outcome"] == "success" for e in events)

    diffs = read_diffs("my_test_system", "eid")
    assert len(diffs) == expected_count

The conftest.py in tests_e2e/ provides the fixtures. Real PostgreSQL via docker-compose. No mocked connectors — connectors talk to test stub services (echo, flatfile) configured in the test system YAML.

ChangesProcessor integration tests (tests_integration/)

For tests that need to exercise the changes-processing pipeline directly (without going through a full inventory), the integration test pattern uses three DC config knobs:

• kafka.mode = DIVERT_TO_FILE — replace Kafka destinations with file destinations
• empowerid.mode = DIVERT_TO_FILE — replace EmpowerID destinations with file destinations
• mirror_all_stores_to_file.enabled = True — file shadow of every store

Tests construct a list of ChangeDispatchEvent objects, call ChangesProcessor.process_changes() directly with a real WritableEntitiesRepository, then read the resulting diff files. Diff file output is the contract — that's what the consumer sees.

Sample inventory inputs for fast iteration

Strategies provide named sample inputs via get_sample_inventory_inputs() for the UI's "Sample Input" picker. Use them during development to avoid waiting for real API state to converge.

@staticmethod
async def get_sample_inventory_inputs(base_engine: BaseEngine) -> List[SampleInventoryInput]:
    now = await base_engine.now()
    one_hour_ago = now - timedelta(hours=1)
    return [
        SampleInventoryInput(
            stream="users",
            title="Force Delta",
            input={
                "force_delta": True,
                "started_at": one_hour_ago,
                "full_completed_at": one_hour_ago,
            },
            description_markdown="Force a **delta** sync starting from one hour ago.",
        ),
    ]

Each sample is tagged with its stream so the UI offers it for the right stream's trigger flow.

Step statistics patterns

engine.get_step_statistics() queries real-time step state during step execution. Three patterns dominate:

Wait for prerequisite

stats = await engine.get_step_statistics(name="fetch_data")
finished = sum(c for s, c in stats.running_states if s == RunningState.FINISHED)
if finished == 0:
    return StepExecutionConfig(delay=True)  # not yet ready

Check for failures before purging

failures = await engine.get_step_statistics(
    outcomes=[Outcome.EXPECTED_FAILURE, Outcome.FATAL_FAILURE],
)
if failures.count > 0:
    engine.finish_step(Outcome.EXPECTED_FAILURE, "Purge canceled due to failures")
    return
# safe to purge

This is the canonical pattern for _purge-style cleanup steps — never delete entities if any prior step failed.

Track progress

success = await engine.get_step_statistics(outcome=Outcome.SUCCESS)
engine.log_message("progress", f"Completed {success.count} sub-steps")

Eager dispatch — the footguns

engine.dispatch_step(name, immediate=True) starts the dispatched step before the parent commits. Useful for latency-sensitive fan-out, but it has hard rules. From strategies-development/080-strategy-ground-rules.md:

• The child must not declare a step-name dependency on the parent — the dependency check would block the child until the parent commits, silently degrading immediate=True to normal timing.
• The child must not depend on the parent's entity changes being in the DB.
• The child must not depend on NII written by the parent (cross-stream NII writes happen during commit).
• The child must not rely on step statistics reflecting the parent's outcome.
• If the parent fails, eagerly-dispatched children get the DISCARDED outcome — connector calls already happened, but entity processing and DB commit don't run. Side effects from the connector call cannot be undone.
• immediate=True cannot be combined with unique_key — raises ValueError. Unique-key merge logic requires durable Step rows that don't exist during eager dispatch's transient phase.

Unique-key behavior — MERGE vs IDEMPOTENCY

get_unique_key_logic() picks the dedup behavior per step name. Two modes; their late-dispatch semantics differ:

Strategies that fan out and re-merge across long-running scenarios usually want IDEMPOTENCY — late dispatches are common.

Visibility-rule debugging

When entities go invisible (because an AttributeCheck matched, or a referenced entity in a ForeignKeyCheck is no longer alive), the Diffs tab shows ChangeDecisionEvents with explicit decision tags:

• "BecameInvisible" — direct V→I transition, DELETED diff produced
• "BecameVisible" — direct I→V transition, CREATED diff produced
• "CascadeBecameInvisible" — cascaded V→I along the FK DAG
• "CascadeBecameVisible" — cascaded I→V

If you expect a cascade and don't see it, check that your ForeignKeyCheck rule's relation_key_number matches the relation{N}_key1 the strategy actually dispatches.

Throttle-group debugging

The Performance tab shows step durations on a Gantt chart; combined with the Steps tab, you can see:

• Slot occupancy — how many steps are running concurrently in a group at each point
• Backoff delays — gaps between failed steps and their retries (visible as idle stretches in the Gantt chart)
• Backoff vs delay_until — backoff is per-slot and computed from RetryBackoff; delay_until is per-instance and explicit. The Gantt chart distinguishes them only by the explicit-delay marker on the step row.

JSON trimming for large events

Monitoring events are serialized to Kafka under two configurable thresholds — a per-event total (default ~256KB, matching Kafka's typical message limit) and a per-item cap (default ~100KB) that triggers field-level trimming before the total ever blows. DC's JsonTrimmer walks the event tree depth-first: oversized strings get truncation markers, oversized arrays get sampled, and large dict values get recursively trimmed. File destinations are unaffected — they receive the full untrimmed event.

If you're debugging via Kafka and seeing truncation markers, switch to file output for full fidelity, or compare against a debug-API streaming run (which captures events to memory without going through Kafka).

Before you build a connector — the universal questionnaire

Building a connector against a new source API is a 79-question exercise documented in strategies-development/060-universal-questionnaire.md. It covers token lifecycle, pagination semantics, ID stability, deletion lifecycle, consistency model, throttling, schema evolution, and edge cases. Work through it before writing any connector code — most connector bugs are unanswered questions in disguise.

Strategy and connector ground rules

The headlines from strategies-development/070-connector-ground-rules.md and 080-strategy-ground-rules.md — read these before merging strategy or connector code:

🔍 Troubleshooting Guide

Trigger validation: system_type_name not permitted

When passing system_type_yaml in a debug request to override the system type's YAML inline, omit system_type_name. The two are mutually exclusive — DC validates that you specify either a name to load from disk, or an inline YAML, not both.

"Resource not found" errors that you expect to fail gracefully

If the source API returning 404 is a normal operational state (e.g., looking up an entity that might not exist), use finish_step(Outcome.EXPECTED_FAILURE, ...) on NOT_FOUND_ERROR. The step's changes will still persist (unlike FATAL_FAILURE), and it won't trigger retry.

Step timing on the Performance tab looks wrong

Each step timing has four components: actions_duration_ms (work), idle_duration_ms (waiting), commit_duration_ms (database write), and total_duration_ms (overall). High idle without high actions usually points at dependency contention — the step is waiting for prerequisites. High commit without high actions points at heavy storage config (consider trimming enable_changes_transformed_data).

Cascade decisions don't appear in Diffs tab

Visibility cascades produce "CascadeBecameInvisible" / "CascadeBecameVisible" ChangeDecisionEvents. If you expect a cascade and don't see one, check that:

• The ForeignKeyCheck rule's relation_key_number matches the relation key the strategy actually populates on the dependent entity.
• The dependent entity actually has the relation key populated — DC can't cascade from a NULL relation key.
• The store_name on the rule matches the store you're inspecting in the Diffs tab.

Throttle group not throttling

If steps don't appear to respect a throttle group's max_parallelism:

• Check that get_step_execution_config returns throttle_groups=["..."] for the steps you expect to be in the group, with the same group name as get_throttle_group_config.
• Throttle group membership must be stable across attempts — if a step's group set varies between attempts, slot accounting breaks.
• Steps with throttle_groups=[] fall back to a hardcoded 1000 cap, not the group cap.

Authentication failures (401 / 403)

See system-admins/06-authentication-and-authorization.md for the full troubleshooting guide. The shape: 401 = bad token or no token; 403 = good token, missing scope or PermissionGrant. 401 actor_arn_required specifically means a trusted-service token arrived without an X-Actor-ARN header — the calling service forgot to forward the user identity.

Inventory won't start

• Stream disabled? Auto-scheduling skips disabled streams. Triggered runs still work — but check the streams.status column in the DB if a scheduled run isn't firing.
• Stream GONE? A stream that disappeared from the strategy's get_streams() result is marked GONE; triggered runs are rejected.
• Lease still held? If a previous instance crashed without releasing the lease, the new instance waits up to 60s before stealing. Check leases.heartbeat_at if a stream sits idle.
• Existing inventory in DB (writable mode)? Writable inventories can't be triggered if a non-completed inventory exists for that stream. Use EXISTING_READONLY_INVENTORY to inspect, or cleanup-ongoing-inventory to abandon.

Replay produces no diffs

Replay only re-publishes entities whose attribute_transformed_data is stored. If the profile's store doesn't have enable_entities_transformed_data for the entity types being replayed, replay fails fast with a readiness error before publishing anything. Enable storage for the relevant entity types in get_step_storage_config and run a normal inventory first to populate.

🔎 Debug Modal Details

"View Details" buttons in the Logs and Steps tabs open a unified DetailsModal.

Logs Tab

Eye icon in action column opens DetailsModal with: datetime, message, logger_name, labels (JSON viewer).

Steps Tab

Eye icon opens DetailsModal with: datetime, event_type, message, labels, parameters, next_inventory_input. Both columns in visibility chooser.

Pattern

Uses reusable DetailsModal from common components (same as Analytics LogViewer). Declarative Modal controlled via state, not imperative Modal.error().

📚 Plugin Documentation UI

Documentation and Validation tabs for strategies and connectors.

Documentation Tab

• PluginDocumentation component displays markdown for each plugin class in the file
• Uses marked for markdown-to-HTML
• Fetches via listConnectorPlugins / listStrategyPlugins, filters by source + module_path

Validation Tab

• PluginValidation shows success or error alerts per class
• Red badge when any class has error
• Shows traceback for validation failures

Connector Doc Buttons in Universal Designer

Documentation buttons at: Settings Tab connector dropdown, Health Check card title, ConnectorCallDrawer Parameters card. Modal z-index 5100 for drawer overlay.

📖 Glossary

✔️ Testing Checklist

Verification steps from implementation docs.

Debugging Tab

• Universal strategy debugging tab works
• Non-Universal strategy (e.g., Sapias) has same Debugging tab UI
• System edit page has Debugging tab
• Auto-clear on save works for all
• Test/Stop/Clear buttons work
• Empty state shows BugOutlined + Test button
• Templates tab "Debug All" shows evaluations (not "Click Test...")
• Templates tab individual "Debug" buttons open ExpressionEditorDrawer

Expression Editor

• String literal "start_index: 50100" stays as string after save/reopen
• Invalid YAML shows validation error
• Typing in Literal Value doesn't transform on every keystroke
• Radio clicks change type and stay selected
• Can switch from object to jinja
• Sandbox OFF: editor readonly, Evaluate disabled
• Sandbox ON: editor editable, Evaluate enabled, changes don't persist
• Sandbox resets when selecting different row

Schema Debug

• Cell selection highlights row and column
• Reliability check shows ✓/✗ per cell
• Evaluate shows new values with strikethrough on originals
• Add Field creates new field; column appears after Evaluate
• Columns chooser Show All / Minimal works

Download

• Download as JSON exports with inventory_run_id
• Download as YAML exports correctly
• Button disabled when no events

⚡ Quick Reference

API Endpoints

Canonical Docs

File Locations

🩺 HealthCheckResultsModal

The HealthCheckResultsModal renders the result of two API calls — testSystem (run from a system or system-type edit page) and testCredentialPointer (run from a credential edit page or the Inventories table's per-row Test action). It's the standard surface for "is this thing healthy right now?", separate from the time-series HealthCheckMiniGraph on the Inventories page.

Per-result rows

The modal lists one row per health check the test produced. Each row shows:

• Status icon — green check on success, red cross on failure
• Connector badge — which connector ran the check
• Credential pointer (for credential tests) or system (for system tests)
• Duration — milliseconds, useful for spotting slow auth flows
• Error message — primary line surfaces immediately on failed rows; full traceback / response is one click away

Per-row detail modal

Clicking a row opens a nested modal with the full HealthCheckDetails result: the params the connector was called with, the response body, any error_data (status code, type, hints), and the raw response when it differs from the parsed view. This is the same shape that ends up in the Admin Events health_check category — the modal is a real-time view; Admin Events is the historical view.

Where it appears

• System / system-type edit pages — Test Health buttons in the header and Connection tab
• Credential edit pages — Test Credential button
• Inventories table — per-row Test action (calls testSystem for the row's system)

All entry points gate through withAuth using the appropriate domain (dc-ui:system manage / dc-ui:credential manage). Failed health checks also fan out to Admin Events, where the same per-row detail is reachable via the Failed Health Checks filter preset.

❓ Debugging FAQ

Why does the Connector Calls tab badge show 4 when I expected 1334?

The tab badge shows actual API calls (step_start count), not expression evaluations. 4 means 4 connector call executions; 1334 would be the total template_evaluation count if that was what was being shown.

Why is Entity Production badge 0 when I have produce actions?

Entity Production shows change_dispatch count scoped to the current call. If the call hasn't dispatched any changes (e.g., empty result, condition not met), it stays 0. Verify step_id→call correlation; check that produce actions ran.

When should I use Sandbox mode?

When viewing expressions from Templates tab—you can't save anyway. Enable Sandbox to try different expression syntax, switch types, and hit Evaluate to see new values. Changes don't persist. Resets when you select another row.

Why do columns in Schema Debug grid not update after Add Field?

Grid columns come from server data only. Add Field updates local editedExpressions. Click Evaluate to send modified mappings to server; server returns new data; grid then includes the new column.

What's the difference between Evaluate and Check Reliability?

Evaluate: Sends your edited expressions to server, re-applies transformation, returns new values. Shows strikethrough when different from original. Reliability Check: Re-applies original mappings to raw data, compares with original transformed. Identifies serialization/roundtrip issues.

How do I share a debug session with my team?

Use the Download dropdown (JSON or YAML). Filename includes inventory_run_id. Share the file; teammates can inspect events offline. No live replay—static export.

✨ Summary

Debugging DataCollector spans two layers: the UI layer (the Debugging tab on every system / system-type / schema designer, the "Follow the Rabbit" badge trail, the ExpressionEditorDrawer with sandbox mode, the SchemaDebugDrawer with cell selection and reliability checks, the DebugResultsPanel with all the per-event sub-tabs, JSON/YAML download) and the engine layer (the /debug/strategy protocol with four Universal-strategy operations and SSE streaming, the e2e test harness running --execute headless, the integration-test pattern with file-diverted destinations, get_step_statistics patterns, eager-dispatch limits, throttle-group debugging via the Performance Gantt, visibility-cascade decisions in the Diffs tab). Use this page as the debugging entry point; use day 26 for the underlying concepts and day 27 for the full UI feature catalog.