View Issue Details

Related ChangesetsJump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0006872SymmetricDSNew Featurepublic2025-05-06 18:032025-06-08 01:07
Reporteremiller Assigned Toemiller  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.16.0 
Target Version3.16.3Fixed in Version3.16.3 
Summary0006872: Add a JSON transform type
DescriptionAdd a JSON transform type which converts the entire row to JSON format. Depending on the transform_expression, it should be in one of three formats. Below are examples of the three formats:

Row for table:
"id","name","state","time"
"customer001","John","inactive","2025-01-01 02:03:04"

1. No header:
{
    "id":"customer001",
    "name":"John",
    "state":"inactive",
    "time":"2025-01-01 02:03:04"
}

2. Customizable data header:
{
    "payload":{
        "id":"customer001",
        "name":"John",
        "state":"inactive",
        "time":"2025-01-01 02:03:04"
    }
}

3. Customizable ID and data header:
{
    "_id":"customer001",
    "data":{
        "id":"customer001",
        "name":"John",
        "state":"inactive",
        "time":"2025-01-01 02:03:04"
    }
}
Steps To ReproduceCreate two different tables with the same name:
(assuming PostgreSQL syntax)

1. On the source node:
CREATE TABLE demo.demo4json(
    id INTEGER NOT NULL,
    name varchar(20) NOT NULL,
    PRIMARY KEY (id)
);

2. On the target node:
CREATE TABLE demo.demo4json(
    id INTEGER NOT NULL,
    payload jsonb NOT NULL,
    PRIMARY KEY (id)
);

3. Set up JSON transform on EXCRACT & SPECIFIED:
INSERT INTO sym_transform_table
(transform_id, source_node_group_id, target_node_group_id, transform_point, source_catalog_name, source_schema_name, source_table_name, target_catalog_name, target_schema_name, target_table_name, update_first, update_action, delete_action, transform_order, column_policy, create_time, last_update_by, last_update_time, description)
VALUES('demo4json', 'lab27pg17source', 'lab27pg17target', 'EXTRACT', '', 'demo', 'demo4json', NULL, 'demo', 'demo4json', 0, 'UPD_ROW', 'DEL_ROW', 0, 'SPECIFIED', '2025-05-14 16:44:23.905', 'no_user', '2025-05-14 16:47:41.791', 'JSON test');


INSERT INTO sym_transform_column
(transform_id, include_on, target_column_name, source_column_name, pk, transform_type, transform_expression, transform_order, create_time, last_update_by, last_update_time, description)
VALUES('demo4json', '*', 'id', 'id', 1, 'copy', NULL, 0, '2025-05-14 16:42:50.024', 'no_user', '2025-05-14 16:42:50.023', NULL)
, VALUES('demo4json', '*', 'payload', NULL, 0, 'json', '', 1, '2025-05-14 16:42:50.022', 'no_user', '2025-05-14 16:42:50.022', NULL);

4. Insert test data at the source:
 INSERT INTO demo.demo4json (id, name) VALUES(1, 'name1');

5. Observe data on the target:
  select * from demo.demo4json;
id | payload
1 | {"id": "1", "name": "3"}
Tagstransformation

Activities

emiller

2025-05-13 15:38

developer   ~0002960

Open-source PR: https://github.com/JumpMind/symmetric-ds/pull/277

pbelov

2025-06-08 01:07

manager   ~0003030

Included in the 3.16.3 release

Related Changesets

SymmetricDS: 3.16 a2ecdcda

2025-05-14 21:07:23

evan-miller-jumpmind


Committer: GitHub Details Diff 		0006872: Added a JSON transform type (0000277)

0006872: Added a JSON transform type		 Affected Issues
0006872
mod - symmetric-assemble/src/asciidoc/configuration/transforms/types.ad Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/TransformService.java Diff File
add - symmetric-io/src/main/java/org/jumpmind/symmetric/io/data/transform/JsonColumnTransform.java Diff File
add - symmetric-io/src/test/java/org/jumpmind/symmetric/io/data/transform/JsonColumnTransformTest.java Diff File

Issue History

Date Modified Username Field Change
2025-05-06 18:03 emiller New Issue
2025-05-06 18:03 emiller Status new => assigned
2025-05-06 18:03 emiller Assigned To => emiller
2025-05-06 18:03 emiller Tag Attached: transformation
2025-05-07 12:16 emiller Description Updated View Revisions
2025-05-13 15:38 emiller Note Added: 0002960
2025-05-14 19:19 pbelov Relationship added related to 0006445
2025-05-14 19:19 pbelov Relationship deleted related to 0006445
2025-05-14 21:08 pbelov Status assigned => resolved
2025-05-14 21:08 pbelov Resolution open => fixed
2025-05-14 21:08 pbelov Fixed in Version => 3.16.3
2025-05-14 21:08 pbelov Steps to Reproduce Updated View Revisions
2025-05-14 22:00 Changeset attached => SymmetricDS 3.16 a2ecdcda
2025-06-08 01:07 pbelov Note Added: 0003030
2025-06-08 01:07 pbelov Status resolved => closed