View Issue Details

IDProjectCategoryView StatusLast Update
0003670SymmetricDSImprovementpublic2019-03-01 17:54
Reporterelong Assigned Toelong  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.8.40 
Target Version3.8.40Fixed in Version3.8.40 
Summary0003670: Improve purge performance of sym_data and sym_data_event
DescriptionThe delete statements used on sym_data and sym_data_event use joins to verify the rows are part of a batch in OK status, which can be slow on some systems. Use a first pass on the tables that looks at all rows before the smallest data_id (for sym_data) or batch_id (for sym_data_event) associated with outstanding (not OK) batches. Once the safe range for purging is determined, it can delete without any joins. Afterwards, the normal purge routines run, but on a healthy system (one without any or very few outstanding batches), there shouldn't any data left to purge.

This feature is disabled by default for now. Enable with job.purge.first.pass=true parameter.
TagsNo tags attached.

Relationships

related to 0003514 closedelong SymmetricDS Improve purge performance of sym_data and sym_data_event 
related to 0003828 closedelong SymmetricDS Pro Fast Purge Job 

Activities

There are no notes attached to this issue.

Related Changesets

SymmetricDS: 3.8 022d403e

2018-08-09 15:04:24

admin

Details Diff
0003670: Improve purge performance of sym_data and sym_data_event Affected Issues
0003670
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/common/ParameterConstants.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PurgeService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PurgeServiceSqlMap.java Diff File
mod - symmetric-core/src/main/resources/symmetric-default.properties Diff File

Issue History

Date Modified Username Field Change
2018-08-09 19:03 elong New Issue
2018-08-09 19:03 elong Status new => assigned
2018-08-09 19:03 elong Assigned To => elong
2018-08-09 19:03 elong Relationship added related to 0003514
2018-08-09 19:05 elong Status assigned => resolved
2018-08-09 19:05 elong Resolution open => fixed
2018-08-09 19:05 elong Fixed in Version => 3.8.40
2018-08-09 19:13 elong Status resolved => closed
2018-08-09 20:00 admin Changeset attached => SymmetricDS 3.8 022d403e
2019-03-01 17:54 elong Relationship added related to 0003828