View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0003974 | SymmetricDS | Bug | public | 2019-05-21 20:02 | 2019-06-25 18:06 |
| Reporter | diltonn | Assigned To | elong | ||
| Priority | urgent | ||||
| Status | closed | Resolution | fixed | ||
| Product Version | 3.10.2 | ||||
| Target Version | 3.10.3 | Fixed in Version | 3.10.3 | ||
| Summary | 0003974: Too many threads, crashing server | ||||
| Description | We have a multi-tenant system running around 20 replication engines. Replication happens both ways, 'cloud-to-client' and 'client-to-cloud'. We upgraded from 3.9.16 to 3.10.2 and our server crashed a couple times before we realized it was due to SymmetricDS. First we noticed a huge increase in memory usage, we upgraded the server memory which made it stay up for longer. Later we noticed the number of threads open by the Symmetric service was too high and it was continuously increasing. We went back to v3.9.16 which won't go over 400 threads, while v3.10.2 got to more than 11K threads at some point. No changes were made on the settings (at least not knowingly), so I guess the issue is related to the new version. AWS Ubuntu Server SymmetricDS v3.10.2 running with "sym_service" PostgreSQL 10 Java 1.8.0_212 | ||||
| Tags | data sync | ||||
|
|
I'm guessing 0003873 is the cause of the additional threads. It was an attempt to keep the connection alive when the acknowledgement comes in. In retrospect, using an additional thread for each ack was not a good idea. Let me see if I can preserve the feature and get rid of the extra threads. |
|
|
Attaching patch file for 3.10.2 that removes additional threads from acknowledgement. Feel free to test and give feedback. Sending fix to integration testing. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2019-05-21 20:02 | diltonn | New Issue | |
| 2019-05-21 20:39 | elong | Assigned To | => elong |
| 2019-05-21 20:39 | elong | Status | new => confirmed |
| 2019-05-21 20:39 | elong | Note Added: 0001500 | |
| 2019-05-22 17:38 | elong | File Added: patch-0003974.jar | |
| 2019-05-22 17:38 | elong | Note Added: 0001501 | |
| 2019-05-22 17:47 | elong | Relationship added | related to 0003873 |
| 2019-05-22 17:53 | elong | Priority | normal => urgent |
| 2019-05-22 17:53 | elong | Target Version | => 3.10.3 |
| 2019-05-22 18:00 | admin | Changeset attached | => SymmetricDS 3.10 29416197 |
| 2019-05-22 18:07 | elong | Tag Attached: data sync | |
| 2019-05-29 16:57 | elong | Status | confirmed => resolved |
| 2019-05-29 16:57 | elong | Resolution | open => fixed |
| 2019-05-29 16:57 | elong | Fixed in Version | => 3.10.3 |
| 2019-06-25 18:06 | admin | Status | resolved => closed |