View Issue Details

IDProjectCategoryView StatusLast Update
0004625SymmetricDSImprovementpublic2020-11-24 20:55
Reporterelong Assigned Toelong  
Prioritynormal 
Status closedResolutionfixed 
Product Version3.12.0 
Target Version3.12.5Fixed in Version3.12.5 
Summary0004625: Multi-primary out of sync with initial load on busy system
DescriptionFor a multi-primary configuration, a new node added to the group should automatically become consistent with the other nodes. The registration (auto.reload or registration.secret) and initial load (auto.reload) can be automated through parameters, so the user expects the automation to handle the timing of events. The two events that need coordination are:

1) Registration of the new node should reach all other nodes before allowing the initial load to begin. This ensures that existing nodes will begin change capture for the new node.

2) When the initial load begins for the new node, the pull/push for incoming changes from other nodes should be denied. This ensures the initial load will have exclusive access to load the database. Afterwards, changes will apply and any conflicts can be detected and resolved.

With these two rules, the initial load followed by catching up of changes results in a consistent database for the new node.

Additional Information- New HTTP code of 668 for InitialLoadPendingException that will be logged at info level

- Add initial_load_end_time to sym_node_security. This table is cached, making it fast to check for pull/push request. The initial_load_time represents the start of the load, while the initial_load_end_time is the time of the last batch acknowledgement for the load.

- Add full_load flag to sym_table_reload_status. This table is already updated with each batch acknowledgement, so the flag makes an easy check to see if the initial_load_end_time should be updated.
Tagsinitial/partial load

Relationships

related to 0004660 closedelong Multi-primary subsequent data loads blocked 

Activities

There are no notes attached to this issue.

Related Changesets

SymmetricDS: 3.12 b39c1b10

2020-11-06 17:40:48

admin

Details Diff
0004625: Multi-primary out of sync with initial load on busy system Affected Issues
0004625
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/AbstractSymmetricEngine.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/load/ConfigurationChangedDatabaseWriterFilter.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/model/NodeSecurity.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/model/RemoteNodeStatus.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/model/TableReloadStatus.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/route/ConfigurationChangedDataRouter.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/IDataService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/INodeService.java Diff File
add - symmetric-core/src/main/java/org/jumpmind/symmetric/service/InitialLoadPendingException.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/AbstractOfflineDetectorService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataExtractorService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataServiceSqlMap.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/InitialLoadService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/NodeService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/NodeServiceSqlMap.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PullService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/PushService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/RegistrationService.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/transport/http/HttpOutgoingTransport.java Diff File
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/web/WebConstants.java Diff File
mod - symmetric-core/src/main/resources/symmetric-schema.xml Diff File
mod - symmetric-core/src/test/java/org/jumpmind/symmetric/service/impl/MockNodeService.java Diff File
mod - symmetric-server/src/main/java/org/jumpmind/symmetric/web/NodeConcurrencyInterceptor.java Diff File

SymmetricDS: 3.12 29251c4a

2020-11-11 13:01:24

admin

Details Diff
0004625: Multi-primary out of sync with initial load on busy system Affected Issues
0004625
mod - symmetric-core/src/main/java/org/jumpmind/symmetric/db/DatabaseUpgradeListener.java Diff File

Issue History

Date Modified Username Field Change
2020-11-03 21:23 elong New Issue
2020-11-03 21:23 elong Status new => assigned
2020-11-03 21:23 elong Assigned To => elong
2020-11-03 21:23 elong Tag Attached: initial/partial load
2020-11-06 17:49 elong Status assigned => resolved
2020-11-06 17:49 elong Resolution open => fixed
2020-11-06 17:49 elong Fixed in Version => 3.12.5
2020-11-06 18:00 admin Changeset attached => SymmetricDS 3.12 b39c1b10
2020-11-09 15:15 admin Status resolved => closed
2020-11-11 14:00 admin Changeset attached => SymmetricDS 3.12 29251c4a
2020-11-24 20:55 elong Relationship added related to 0004660