View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0003282 | SymmetricDS | Bug | public | 2017-10-17 13:29 | 2017-11-15 21:00 |
| Reporter | mmichalek | Assigned To | mmichalek | ||
| Priority | normal | ||||
| Status | closed | Resolution | fixed | ||
| Product Version | 3.8.0 | ||||
| Target Version | 3.8.31 | Fixed in Version | 3.8.31 | ||
| Summary | 0003282: Common batch extracts may fail when a cluster is using a shared staging area | ||||
| Description | Typically batches are extracted on demand. And common batches are batches that share the same data, but need to be delivered to different nodes. So a common batch only should be extracted once to serve up to any number of clients. But when clients simultaneously pull from multiple servers in a cluster, and that cluster is using a shared staging area (ie, some shared storage like a SAN) there is no locking mechanism at the file system level to say who is in control of extracting the batch. In other words, there is a race condition when extracting common batches on a clustered server that uses a shared staging location. This can result in undefined behavior at the file writing level. A couple of the observed errors related to this include: Caused by: java.io.IOException: Stream closed OR Caused by: java.io.IOException: Stale file handle OR java.lang.IllegalStateException: There is no content to read. Memory buffer was empty and /Users/userid/tmp/shared-staging/staging/server/outgoing/common/0000000116.create was not found. OR java.lang.IllegalStateException: Could not find the staged resource for batch client3-143 Full stack trace examples: 2017-10-04 05:03:16,525 ERROR [SymmetricDS] [DataExtractorService] [http-bio-8080-exec-6] Failed to extract batch 08286-210478 StackTraceKey.init [RuntimeException:1766863430] java.lang.RuntimeException: org.jumpmind.exception.IoException: java.io.IOException: Stale file handle at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:664) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:537) at org.jumpmind.symmetric.web.PullUriHandler.pull(PullUriHandler.java:130) at org.jumpmind.symmetric.web.PullUriHandler.handleWithCompression(PullUriHandler.java:100) at org.jumpmind.symmetric.web.AbstractCompressionUriHandler.handle(AbstractCompressionUriHandler.java:84) at org.jumpmind.symmetric.web.SymmetricServlet.service(SymmetricServlet.java:114) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:442) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1083) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:640) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: org.jumpmind.exception.IoException: java.io.IOException: Stale file handle at org.jumpmind.symmetric.io.data.writer.StagingDataWriter.print(StagingDataWriter.java:122) at org.jumpmind.symmetric.io.data.writer.AbstractProtocolDataWriter.println(AbstractProtocolDataWriter.java:253) at org.jumpmind.symmetric.io.data.writer.AbstractProtocolDataWriter.write(AbstractProtocolDataWriter.java:161) at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.write(NestedDataWriter.java:64) at org.jumpmind.symmetric.model.ProcessInfoDataWriter.write(ProcessInfoDataWriter.java:65) at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.write(NestedDataWriter.java:64) at org.jumpmind.symmetric.io.data.writer.TransformWriter.write(TransformWriter.java:207) at org.jumpmind.symmetric.io.data.DataProcessor.forEachDataInTable(DataProcessor.java:200) at org.jumpmind.symmetric.io.data.DataProcessor.forEachTableInBatch(DataProcessor.java:170) at org.jumpmind.symmetric.io.data.DataProcessor.process(DataProcessor.java:116) at org.jumpmind.symmetric.service.impl.DataExtractorService.extractOutgoingBatch(DataExtractorService.java:897) at org.jumpmind.symmetric.service.impl.DataExtractorService.extractBatch(DataExtractorService.java:774) at org.jumpmind.symmetric.service.impl.DataExtractorService$1.call(DataExtractorService.java:604) at org.jumpmind.symmetric.service.impl.DataExtractorService$1.call(DataExtractorService.java:602) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: java.io.IOException: Stale file handle at java.io.FileOutputStream.writeBytes(Native Method) at java.io.FileOutputStream.write(FileOutputStream.java:326) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207) at java.io.BufferedWriter.write(BufferedWriter.java:188) at org.jumpmind.symmetric.io.stage.ThresholdFileWriter.write(ThresholdFileWriter.java:87) at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129) at java.io.BufferedWriter.write(BufferedWriter.java:230) at java.io.Writer.write(Writer.java:157) at java.io.Writer.append(Writer.java:269) at org.jumpmind.symmetric.io.data.writer.StagingDataWriter.print(StagingDataWriter.java:119) ... 17 more 2017-10-13 11:40:40,796 ERROR [server] [DataExtractorService] [qtp1381233299-14] Failed to extract batch client1-116 java.lang.IllegalStateException: There is no content to read. Memory buffer was empty and /Users/userid/tmp/shared-staging/staging/server/outgoing/common/0000000116.create was not found. at org.jumpmind.symmetric.io.stage.StagedResource.getReader(StagedResource.java:195) at org.jumpmind.symmetric.service.impl.DataExtractorService.transferFromStaging(DataExtractorService.java:1144) at org.jumpmind.symmetric.service.impl.DataExtractorService.sendOutgoingBatch(DataExtractorService.java:1105) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:652) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:539) at org.jumpmind.symmetric.web.PullUriHandler.pull(PullUriHandler.java:130) at org.jumpmind.symmetric.web.PullUriHandler.handleWithCompression(PullUriHandler.java:100) at org.jumpmind.symmetric.web.AbstractCompressionUriHandler.handle(AbstractCompressionUriHandler.java:77) at org.jumpmind.symmetric.web.SymmetricServlet.service(SymmetricServlet.java:114) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at org.jumpmind.symmetric.web.HttpMethodFilter.doFilter(HttpMethodFilter.java:62) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) 2017-10-13 11:57:02,758 ERROR [server] [DataExtractorService] [qtp89832223-15] Failed to extract batch client3-143 java.lang.IllegalStateException: Could not find the staged resource for batch client3-143 at org.jumpmind.symmetric.service.impl.DataExtractorService.sendOutgoingBatch(DataExtractorService.java:1128) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:652) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:539) at org.jumpmind.symmetric.web.PullUriHandler.pull(PullUriHandler.java:130) at org.jumpmind.symmetric.web.PullUriHandler.handleWithCompression(PullUriHandler.java:100) at org.jumpmind.symmetric.web.AbstractCompressionUriHandler.handle(AbstractCompressionUriHandler.java:77) at org.jumpmind.symmetric.web.SymmetricServlet.service(SymmetricServlet.java:114) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at org.jumpmind.symmetric.web.HttpMethodFilter.doFilter(HttpMethodFilter.java:62) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) [server] - DataExtractorService - Failed to extract batch client5-140 java.lang.RuntimeException: org.jumpmind.exception.IoException: java.io.IOException: Stream closed at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:669) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:539) at org.jumpmind.symmetric.web.PullUriHandler.pull(PullUriHandler.java:130) at org.jumpmind.symmetric.web.PullUriHandler.handleWithCompression(PullUriHandler.java:100) at org.jumpmind.symmetric.web.AbstractCompressionUriHandler.handle(AbstractCompressionUriHandler.java:77) at org.jumpmind.symmetric.web.SymmetricServlet.service(SymmetricServlet.java:114) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at org.jumpmind.symmetric.web.HttpMethodFilter.doFilter(HttpMethodFilter.java:62) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: org.jumpmind.exception.IoException: java.io.IOException: Stream closed at org.jumpmind.symmetric.io.data.writer.StagingDataWriter.print(StagingDataWriter.java:126) at org.jumpmind.symmetric.io.data.writer.AbstractProtocolDataWriter.println(AbstractProtocolDataWriter.java:253) at org.jumpmind.symmetric.io.data.writer.AbstractProtocolDataWriter.start(AbstractProtocolDataWriter.java:96) at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.start(NestedDataWriter.java:56) at org.jumpmind.symmetric.model.ProcessInfoDataWriter.start(ProcessInfoDataWriter.java:51) at org.jumpmind.symmetric.io.data.writer.NestedDataWriter.start(NestedDataWriter.java:56) at org.jumpmind.symmetric.io.data.writer.TransformWriter.start(TransformWriter.java:97) at org.jumpmind.symmetric.service.impl.DataExtractorService.cleanupIgnoredBatch(DataExtractorService.java:1019) at org.jumpmind.symmetric.service.impl.DataExtractorService.extractOutgoingBatch(DataExtractorService.java:866) at org.jumpmind.symmetric.service.impl.DataExtractorService.extractBatch(DataExtractorService.java:779) at org.jumpmind.symmetric.service.impl.DataExtractorService$1.call(DataExtractorService.java:609) at org.jumpmind.symmetric.service.impl.DataExtractorService$1.call(DataExtractorService.java:607) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more Caused by: java.io.IOException: Stream closed at java.io.BufferedWriter.ensureOpen(BufferedWriter.java:116) at java.io.BufferedWriter.write(BufferedWriter.java:221) at java.io.Writer.write(Writer.java:157) at java.io.Writer.append(Writer.java:269) at org.jumpmind.symmetric.io.data.writer.StagingDataWriter.print(StagingDataWriter.java:123) ... 15 more 2017-10-13 11:57:02,758 ERROR [server] [DataExtractorService] [qtp89832223-15] Failed to extract batch client3-143 java.lang.IllegalStateException: Could not find the staged resource for batch client3-143 at org.jumpmind.symmetric.service.impl.DataExtractorService.sendOutgoingBatch(DataExtractorService.java:1128) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:652) at org.jumpmind.symmetric.service.impl.DataExtractorService.extract(DataExtractorService.java:539) at org.jumpmind.symmetric.web.PullUriHandler.pull(PullUriHandler.java:130) at org.jumpmind.symmetric.web.PullUriHandler.handleWithCompression(PullUriHandler.java:100) at org.jumpmind.symmetric.web.AbstractCompressionUriHandler.handle(AbstractCompressionUriHandler.java:77) at org.jumpmind.symmetric.web.SymmetricServlet.service(SymmetricServlet.java:114) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at org.jumpmind.symmetric.web.HttpMethodFilter.doFilter(HttpMethodFilter.java:62) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) | ||||
| Steps To Reproduce | This was reproduced in dev by pointed 2 SymmetricDS servers at the same MySQL database and the same staging location on the HD (a valid, 2 server cluster). Then the following script was used and an error was observed the majority of the time when running this script: # start script rm -rf /Users/userid/tmp/shared-staging/staging/server/outgoing/common/* # 6 nodes hitting server1. curl 'http://localhost:31415/sync/server/pull?nodeId=client1&securityToken=secret' & curl 'http://localhost:31415/sync/server/pull?nodeId=client2&securityToken=secret' & curl 'http://localhost:31415/sync/server/pull?nodeId=client3&securityToken=secret' & curl 'http://localhost:31415/sync/server/pull?nodeId=client4&securityToken=secret' & curl 'http://localhost:31415/sync/server/pull?nodeId=client5&securityToken=secret' & curl 'http://localhost:31415/sync/server/pull?nodeId=client6&securityToken=secret' & # 6 nodes hitting server2. curl 'http://localhost:31416/sync/server/pull?nodeId=client1&securityToken=secret' & curl 'http://localhost:31416/sync/server/pull?nodeId=client2&securityToken=secret' & curl 'http://localhost:31416/sync/server/pull?nodeId=client3&securityToken=secret' & curl 'http://localhost:31416/sync/server/pull?nodeId=client4&securityToken=secret' & curl 'http://localhost:31416/sync/server/pull?nodeId=client5&securityToken=secret' & curl 'http://localhost:31416/sync/server/pull?nodeId=client6&securityToken=secret' & | ||||
| Tags | No tags attached. | ||||
|
SymmetricDS: 3.8 50da75ce 2017-10-17 11:04:13 Details Diff |
0003282: Common batch extracts may fail when a cluster is using a shared staging area |
Affected Issues 0003282 |
|
| mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataExtractorService.java | Diff File | ||
| mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/data/writer/StagingDataWriter.java | Diff File | ||
| mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/IStagingManager.java | Diff File | ||
| mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/StagedResource.java | Diff File | ||
| add - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/StagingFileLock.java | Diff File | ||
| mod - symmetric-io/src/main/java/org/jumpmind/symmetric/io/stage/StagingManager.java | Diff File | ||
|
SymmetricDS: 3.8 8d59af6c 2017-11-15 15:24:43 Details Diff |
0003282: Common batch extracts may fail when a cluster is using a shared staging area |
Affected Issues 0003282 |
|
| mod - symmetric-core/src/main/java/org/jumpmind/symmetric/service/impl/DataExtractorService.java | Diff File | ||
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2017-10-17 13:29 | mmichalek | New Issue | |
| 2017-10-17 13:29 | mmichalek | Status | new => assigned |
| 2017-10-17 13:29 | mmichalek | Assigned To | => mmichalek |
| 2017-10-17 13:29 | mmichalek | Project | SymmetricDS Pro => SymmetricDS |
| 2017-10-17 15:05 | mmichalek | Status | assigned => resolved |
| 2017-10-17 15:05 | mmichalek | Resolution | open => fixed |
| 2017-10-17 15:05 | mmichalek | Fixed in Version | => 3.8.31 |
| 2017-10-17 16:00 | mmichalek | Changeset attached | => SymmetricDS 3.8 50da75ce |
| 2017-11-06 13:13 | chenson | Status | resolved => closed |
| 2017-11-15 21:00 | chenson42 | Changeset attached | => SymmetricDS 3.8 8d59af6c |