Skip to content

bug: TypeError in Replication Service (v4.4.5) #3519

@Mathis001

Description

@Mathis001

Provide environment information

PS C:\Users\Chas\Documents\Github\AiAutomations\automations> npx envinfo --system --binaries

System:
OS: Windows 10 10.0.19045
CPU: (12) x64 AMD Ryzen 7 5800X 8-Core Processor
Memory: 8.56 GB / 24.00 GB
Binaries:
Node: 24.11.1 - C:\Program Files\nodejs\node.EXE
npm: 11.6.2 - C:\Program Files\nodejs\npm.CMD

Describe the bug

[BUG] TypeError: Cannot read properties of undefined (reading 'getTime') in ClickHouse Replication

Description

This is a server-side bug within the self-hosted Docker infrastructure, specifically in the background replication service that moves data from PostgreSQL CDC (via logical replication) to ClickHouse. As such, it cannot be reproduced in a client-side environment like Stackblitz or CodeSandbox.

However, the issue is directly verifiable by inspecting the toSessionInsertArray and toTaskRunInsertArray utility functions in the apps/webapp source. The stack trace indicates the failure occurs here:

This appears to be a new bug in the data transformation logic, distinct from the permission issues discussed in #2726.

Environment Details

  • Docker Image: ghcr.io/triggerdotdev/trigger.dev:v4.4.5 (also observed in v4-beta)
  • Deployment: Docker Compose
  • PostgreSQL: 16.2
  • ClickHouse: 24.3 (LTS)
  • Relevant Env Vars:
    • RUN_REPLICATION_ENABLED=1
    • EVENT_REPOSITORY_DEFAULT_STORE=clickhouse

Error Logs

{"error":{"message":"Cannot read properties of undefined (reading 'getTime')","stack":"TypeError: Cannot read properties of undefined (reading 'getTime')\n    at toSessionInsertArray (/triggerdotdev/apps/webapp/build/index.js:103386:23)\n    at /triggerdotdev/apps/webapp/build/index.js:103261:48\n    at Array.map (<anonymous>)\n    at /triggerdotdev/apps/webapp/build/index.js:103261:34\n    at /triggerdotdev/apps/webapp/build/index.js:12461:20\n    ...

Reproduction repo

N/A

To reproduce

Steps to Reproduce

  1. Deploy Trigger.dev v4.4.5 with ClickHouse replication enabled.
  2. Trigger tasks.
  3. Observe trigger-webapp logs; the replication service will fail with the TypeError and stop processing the stream.

Reproduction

This is a server-side bug within the self-hosted Docker infrastructure, specifically in the background replication service that moves data from PostgreSQL CDC (via logical replication) to ClickHouse. As such, it cannot be reproduced in a client-side environment like Stackblitz or CodeSandbox.

However, the issue is directly verifiable by inspecting the toSessionInsertArray and toTaskRunInsertArray utility functions in the apps/webapp source. The stack trace indicates the failure occurs here:

at toSessionInsertArray (/triggerdotdev/apps/webapp/build/index.js:103386:23)
at /triggerdotdev/apps/webapp/build/index.js:103261:48

The bug is triggered when a task run or session is synced before its updatedAt or createdAt timestamps are fully populated in the CDC message, causing a TypeError when .getTime() is called on an undefined value.

Proposed Fix / Workaround

We successfully resolved this in our environment by injecting null-safety checks into the minified webapp bundle. The following sed commands patch the getTime() calls to handle undefined fields:

# Patch TaskRun sync
docker exec -it trigger-webapp sed -i "s/run.updatedAt.getTime()/(run.updatedAt?run.updatedAt.getTime():0)/g" /triggerdotdev/apps/webapp/build/index.js
docker exec -it trigger-webapp sed -i "s/run.createdAt.getTime()/(run.createdAt?run.createdAt.getTime():0)/g" /triggerdotdev/apps/webapp/build/index.js

# Patch Session sync
docker exec -it trigger-webapp sed -i "s/session.createdAt.getTime()/(session.createdAt?session.createdAt.getTime():0)/g" /triggerdotdev/apps/webapp/build/index.js
docker exec -it trigger-webapp sed -i "s/session.updatedAt.getTime()/(session.updatedAt?session.updatedAt.getTime():0)/g" /triggerdotdev/apps/webapp/build/index.js

docker restart trigger-webapp

Additional information

Additional Context

We also encountered the PostgreSQL role/permission issues described in #2726 during setup. For replication to function at all, we had to ensure the database user had REPLICATION and SUPERUSER privileges:

ALTER USER trigger_user WITH REPLICATION SUPERUSER;

While #2726 addresses the connectivity and permissions, this issue covers a data-level crash once replication is active.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions