View Issue Details

IDProjectCategoryView StatusLast Update
0005906SymmetricDSBugpublic2024-06-19 17:39
Reporterpmarzullo Assigned Topmarzullo  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.14.7 
Target Version3.14.8Fixed in Version3.14.8 
Summary0005906: Non transactional initial load setup can cause load events to be missed ini table_reload_status
DescriptionWhen the following parameter is set to false:
datareload.batch.insert.transactional=false

it is possible for the events that are being inserted into the database for the load are routed and synchronized to the target before the counts in table_reload_status are completed. If this occurs, then the state of those batches that are completed before the table_reload_status is completed will be lost.

For example, if there is a load of one table, and a setup batch is created to delete the records for that table before sending the load, the setup batch can be routed and synchronized to the target before the table_reload_status table gets the range of batches that are created for the data load. This will cause the update of the table_reload_status for that setup batch to not update the record because the batch ID for the setup batch does not fall before the start batch ID for the load batches (because both the start_batch_id and end_batch_id are set to 0 when the setup batch is acknowledged).

To fix this, we should set the setup, load, and finalize batches, as well as the extract requests, to a status of LS for Load Setup. Then when all information is completely set up in the table_reload_status table for the load, then the status values can be set to NE for the setup and finalize batches, and for the extract requests, and the load batches can be set to NE when not using the initial load extractor job or they can be set to RQ when the initial load extractor job is used.
Tagsinitial/partial load

Relationships

related to 0005651 closedpmarzullo Row in sym_table_reload_status can have incorrect values for completed, data_batch_loaded and finalize_batch_loaded 

Activities

pmarzullo

2023-06-30 12:07

developer   ~0002349

Patch has been provided, patch-0005906.jar, but it does not patch the user interface changes that were required. The functionality is fixed by the patch, the user interface may show some anomalies during setup of the initial load, but when the setup is complete, the user interface will smooth out.

https://downloads.jumpmind.com/symmetricds/patches/3.14/patch-0005906.jar

Related Changesets

SymmetricDS: 3.14 23744fbc

2023-06-29 20:50:49

pmarzullo

Details Diff
0005906: Non transactional initial load setup can cause load events to
be missed ini table_reload_status
Affected Issues
0005906
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/model/AbstractBatch.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/model/ExtractRequest.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/IDataExtractorService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/IOutgoingBatchService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataExtractorService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataExtractorServiceSqlMap.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataServiceSqlMap.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/OutgoingBatchService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/OutgoingBatchServiceSqlMap.java Diff File

Issue History

Date Modified Username Field Change
2023-06-29 20:47 pmarzullo New Issue
2023-06-29 20:47 pmarzullo Status new => assigned
2023-06-29 20:47 pmarzullo Assigned To => pmarzullo
2023-06-29 20:47 pmarzullo Tag Attached: initial/partial load
2023-06-29 20:49 pmarzullo Description Updated View Revisions
2023-06-29 21:00 pmarzullo Changeset attached => SymmetricDS 3.14 23744fbc
2023-06-30 12:07 pmarzullo Status assigned => resolved
2023-06-30 12:07 pmarzullo Resolution open => fixed
2023-06-30 12:07 pmarzullo Fixed in Version => 3.14.8
2023-06-30 12:07 pmarzullo Note Added: 0002349
2023-07-19 12:58 admin Status resolved => closed
2024-06-19 17:39 emiller Relationship added related to 0005651