View Revisions: Issue #5134
| Summary | 0005134: Client node service often hang and no error logs to output | ||
|---|---|---|---|
| Revision | 2021-12-01 18:46 by elong | ||
| Description | My symmetric system use version 3.9.21, and have 4 nodes: 1 server node( called: corp-000 ) is as data source , 3 client nodes are only data destination(called : store-001, mydata-myiddp, mysql-sseipo ), and all nodes run in one machine . My system runs 3 JVM instances: JVM1: corp-000 node instance connects ORACLE: data source db JVM2: store-001 node instance connects ORACLE : destination db JVM3: mydata-myiddp and mysql-sseipo node instance connects MYSQL: destination db JVM3 often hang and no response: no heartbeat, no ERROR or INFO logs to output, Jetty no response with follwing test url : http://10.10.20.31:34415/sync/mysql-sseipo/ping *** I am SURE THAT NETWORK IS OK and ORACLE/MYSQL is running correctly. I logged on to LINUX and do as following and reboot symmetric , but system still hang frequently: 1: add some timeout parameters to engine file as following : ===================================================== engine.name=mysql-sseipo # The class name for the JDBC Driver db.driver=com.mysql.jdbc.Driver db.url=jdbc:mysql://10.10.11.9/symmetric?tinyInt1isBit=false&connectTimeout=10000&socketTimeout=10000 db.user=syncapp2 db.password=syncapp2 db.validation.query=select 1 # The HTTP URL of the root node to contact for registration registration.url=http://10.10.20.31:32415/sync/corp-000 sync.url=http://10.10.20.31:34415/sync/mysql-sseipo # Do not change these for running the demo group.id=mysql external.id=sseipo # This is how often the routing job will be run in milliseconds job.routing.period.time.ms=5000 # This is how often the push job will be run. job.push.period.time.ms=10000 # This is how often the pull job will be run. job.pull.period.time.ms=10000 #20210618.sxp.add: try for hang randomly db.pool.max.active=20 http.concurrent.workers.max=10 pull.thread.per.server.count=5 push.thread.per.server.count=5 http.timeout.ms=10000 ====================================== 2： After hanging，i try to dump JAVA stack to check status, but no idea. see attachment files. commad : jstack -l 2914 > js2914.server.txt jstack -l 20317 >js20317.client.txt After i done , hanging status canceled automatically, and system sync normally. 3: I checked tcp port with lsof command : server port: lsof -i :32415 ====> found 5 CLOSE_WAIT client port : lsof -i :34415 ====> found 1 CLOSE_WAIT PLEASE HELP ME ! I searched passed discussion topic : https://sourceforge.net/p/symmetricds/discussion/1354726/thread/1ad5f034/?limit=25#677c BUT I didnt found any solution to resolve this problem. |
||
| Revision | 2021-11-26 03:26 by jtsunxp | ||
| Description | My symmetric system use version 3.9.21, and have 4 nodes: 1 server node( called: corp-000 ) is as data source , 3 client nodes are only data destination(called : store-001, mydata-myiddp, mysql-sseipo ), and all nodes run in one machine . My system runs 3 JVM instances: JVM1: corp-000 node instance connects ORACLE: data source db JVM2: store-001 node instance connects ORACLE : destination db JVM3: mydata-myiddp and mysql-sseipo node instance connects MYSQL: destination db JVM3 often hang and no response: no heartbeat, no ERROR or INFO logs to output, Jetty no response with follwing test url : http://10.10.20.31:34415/sync/mysql-sseipo/ping *** I am SURE THAT NETWORK IS OK and ORACLE/MYSQL is running correctly. I logged on to LINUX and do as following and reboot symmetric , but system still hang frequently: 1: add some timeout parameters to engine file as following : ===================================================== engine.name=mysql-sseipo # The class name for the JDBC Driver db.driver=com.mysql.jdbc.Driver db.url=jdbc:mysql://10.10.11.9/symmetric?tinyInt1isBit=false&connectTimeout=10000&socketTimeout=10000 db.user=syncapp2 db.password=syncapp2 db.validation.query=select 1 # The HTTP URL of the root node to contact for registration registration.url=http://10.10.20.31:32415/sync/corp-000 sync.url=http://10.10.20.31:34415/sync/mysql-sseipo # Do not change these for running the demo group.id=mysql external.id=sseipo # This is how often the routing job will be run in milliseconds job.routing.period.time.ms=5000 # This is how often the push job will be run. job.push.period.time.ms=10000 # This is how often the pull job will be run. job.pull.period.time.ms=10000 #20210618.sxp.add: try for hang randomly db.pool.max.active=20 http.concurrent.workers.max=10 pull.thread.per.server.count=5 push.thread.per.server.count=5 http.timeout.ms=10000 ====================================== 2： After hanging，i try to dump JAVA stack to check status, but no idea. see attachment files. commad : jstack -l 2914 > js2914.server.txt jstack -l 20317 >js20317.client.txt After i done , hanging status canceled automatically, and system sync normally. 3: I checked tcp port with lsof command : server port: lsof -i :32415 ====> found 5 CLOSE_WAIT client port : lsof -i :34415 ====> found 1 CLOSE_WAIT PLEASE HELP ME ! I searched passed discussion topic : https://sourceforge.net/p/symmetricds/discussion/1354726/thread/1ad5f034/?limit=25#677c BUT I didnt found any solution to resolve this problem. |
||