0003828: Fast Purge Job - SymmetricDS

ID	Project	Category	View Status	Date Submitted	Last Update

0003828	SymmetricDS Pro	Improvement	public	2018-12-13 14:38	2019-03-01 17:57

Reporter	gilles.riand	Assigned To	elong
Priority	normal
Status	closed	Resolution	duplicate
Product Version	3.7.36

Summary	0003828: Fast Purge Job
Description	Hello i'm using SymDS in a multi asterenvironnement wit a lot of store and a Master Node who make the Route for evey nodes. I think that i can be possible to do the ougoing Purge job faster 1) calculate the older batch not OK... SELECT MIN(CREATE_TIME) AS OLDEST_BATCH FROM SYM_OUTGOING_BATCH WHERE STATUS!='OK' ORDER BY 1 DESC After delete everything (in SYM_DATA) before this time or 10 minutes before .... probably in multiple statements... I think it will be faster than this kind of queries delete from sym_data where data_id between 3394307 and 3394407 and create_time < {ts '2018-12-10 14:33:37.003'} and data_id in (select e.data_id from sym_data_event e where e.data_id between 3394307 and 3394407) and data_id not in (select e.data_id from sym_data_event e where e.data_id between 3394307 and 3394407 and (e.data_id is null or e.batch_id in (select batch_id from sym_outgoing_batch where status != 'OK'))) What do you think about it ?
Tags	No tags attached.

Date Modified	Username	Field	Change
2018-12-13 14:38	gilles.riand	New Issue
2019-03-01 17:54	elong	Relationship added	related to 0003670
2019-03-01 17:55	elong	Relationship added	related to 0003784
2019-03-01 17:57	elong	Assigned To	=> elong
2019-03-01 17:57	elong	Status	new => closed
2019-03-01 17:57	elong	Resolution	open => duplicate
2019-03-01 17:57	elong	Note Added: 0001397

related to	0003670	closed	elong	SymmetricDS	Improve purge performance of sym_data and sym_data_event
related to	0003784	closed	elong	SymmetricDS	More aggressive purge settings

elong 2019-03-01 17:57 developer ~0001397	It's a great idea. We implemented it in latest 3.8 and 3.9 and can be enabled with job.purge.first.pass=true parameter. It's been working so well, that it will be enabled by default in 3.10.