View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0005175 | SymmetricDS | Bug | public | 2021-12-28 19:58 | 2022-08-09 13:29 |
| Reporter | pmarzullo | Assigned To | pmarzullo | ||
| Priority | normal | ||||
| Status | closed | Resolution | fixed | ||
| Product Version | 3.12.14 | ||||
| Target Version | 3.12.15 | Fixed in Version | 3.12.15 | ||
| Summary | 0005175: JdbcBatchBulkDatabaseWriter only uses one record in the batches for deletes, and fails on record 10000 | ||||
| Description | Batch commit siize is set to 10000, and the writer only writes one record at a time during delete event type. The issue is that the check in DefaultDatabaseWriter.requireNewStatement() checks the passed in applyChangesOnly against the lastApplyChangesOnly member variable to see if it changed, and if it did, returns an indicator that a new statement must be created. The delete() function that doesn't really use the applyChangesOnly variable, and passes in the passed in useConflictDetection variable, and never sets the lastApplyChangesOnly member variable, leaving it false. And the useConflictDetection passed in variable for bulk is usually set to true, and so ends up always requiring a new statement. The fix is to create a requireNewStatement() method in the JdbcBatchBulkDatabaseWriter class that passes in a false for applyChangesOnly variable when the eventType is a delete. Very surgical fix. This fixes the batching of deletes to use more than one delete at a time (actually uses the db.jdbc.bulk.execute.batch.size parameter value), and succeeds on all 10000 records. | ||||
| Steps To Reproduce | Set up a channel that uses jdbc bulk for change data capture. Insert 10000 records. Allow 10000 records go to the target. Delete 10000 records. The deletes will delete one at a time, and will fail on the 10000th record. | ||||
| Tags | No tags attached. | ||||
|
SymmetricDS: 3.12 a0a8db16 2021-12-28 20:32:50 Details Diff |
0005175: JdbcBatchBulkDatabaseWriter only uses one record in the batches for deletes, and fails on record 10000 |
Affected Issues 0005175 |
|
| mod - symmetric-client/src/main/java/org/jumpmind/symmetric/io/JdbcBatchBulkDatabaseWriter.java | Diff File | ||
|
SymmetricDS: 3.13 dc759cf4 2021-12-28 20:32:50 Details Diff |
0005175: JdbcBatchBulkDatabaseWriter only uses one record in the batches for deletes, and fails on record 10000 # Conflicts: # symmetric-client/src/main/java/org/jumpmind/symmetric/io/JdbcBatchBulkDatabaseWriter.java |
Affected Issues 0005175 |
|
| mod - symmetric-client/src/main/java/org/jumpmind/symmetric/io/JdbcBatchBulkDatabaseWriter.java | Diff File | ||
|
SymmetricDS: 3.14 97af6973 2021-12-28 20:32:50 Details Diff |
0005175: JdbcBatchBulkDatabaseWriter only uses one record in the batches for deletes, and fails on record 10000 # Conflicts: # symmetric-client/src/main/java/org/jumpmind/symmetric/io/JdbcBatchBulkDatabaseWriter.java |
Affected Issues 0005175 |
|
| mod - symmetric-client/src/main/java/org/jumpmind/symmetric/io/JdbcBatchBulkDatabaseWriter.java | Diff File | ||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2021-12-28 19:58 | pmarzullo | New Issue | |
| 2021-12-28 19:58 | pmarzullo | Status | new => assigned |
| 2021-12-28 19:58 | pmarzullo | Assigned To | => pmarzullo |
| 2021-12-28 19:58 | pmarzullo | Tag Attached: data writer | |
| 2021-12-28 20:31 | pmarzullo | Product Version | 3.12.13 => 3.12.14 |
| 2021-12-28 20:49 | pmarzullo | Status | assigned => resolved |
| 2021-12-28 20:49 | pmarzullo | Resolution | open => fixed |
| 2021-12-28 20:49 | pmarzullo | Fixed in Version | => 3.12.15 |
| 2021-12-28 20:49 | pmarzullo | Note Added: 0002042 | |
| 2021-12-28 21:00 | pmarzullo | Changeset attached | => SymmetricDS 3.12 a0a8db16 |
| 2021-12-28 21:00 | pmarzullo | Changeset attached | => SymmetricDS 3.13 dc759cf4 |
| 2021-12-28 21:00 | pmarzullo | Changeset attached | => SymmetricDS 3.14 97af6973 |
| 2022-01-17 13:39 | admin | Status | resolved => closed |
| 2022-08-09 13:29 | admin | Tag Detached: data writer |