I have a complex adobe air app using livecycleDS 2.6.1 on server side. App uses dataserviceas and messaging. Server is running smoothly without any problem, but after some time (few hours running - very random) is shutting down. No logs no errors etc, only hs_err_pidXXX.log in tomcat BIN directory.
during development in flex builder with same versions of jre/lcds/tomcat there were no problems, random shutting never happened..
I'm lost, I have no clue where the problem could be or what it could be causing. Is it problem of livecycle/channels or tomcat or jvm? Can you give me any hints, ideas or something clue how i could debug this?
here is hs_err_pid log:
A fatal error has been detected by the Java Runtime Environment:
EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d8a3795, pid=3500, tid=3568
JRE version: 6.0_16-b01 Java VM: Java HotSpot(TM) Client VM (14.2-b01 mixed mode, sharing windows-x86 ) Problematic frame: V [jvm.dll+0xa3795]
If you would like to submit a bug report, please visit: http://java.sun.com/webapps/bugreport/crash.jsp
--------------- T H R E A D ---------------
Current thread (0x02c8b800): VMThread [stack: 0x02d70000,0x02dc0000] [id=3568]
siginfo: ExceptionCode=0xc0000005, reading address 0x00000009
Registers: EAX=0x00000009, EBX=0x09682800, ECX=0x02dbfa60, EDX=0x09682800 ESP=0x02dbf81c, EBP=0x02dbf824, ESI=0x02dbfa60, EDI=0x09682648 EIP=0x6d8a3795, EFLAGS=0x00010287
Top of Stack: (sp=0x02dbf81c) 0x02dbf81c: 075826a4 09682648 02dbf83c 6d96a42a 0x02dbf82c: 09682648 09682628 26c8c0b8 02dbf880 0x02dbf83c: 02dbf860 6d9a2bc1 09682628 02dbf880 0x02dbf84c: 09682600 00000080 09682600 09150000 0x02dbf85c: 09682800 02dbf894 6d9a2119 09682600 0x02dbf86c: 00000080 800c0c02 09682800 02dbf880 0x02dbf87c: 03149698 6da24f0c 09682600 02c82090 0x02dbf88c: 09150000 02dbfa60 02dbf8bc 6d9a20ad
Instructions: (pc=0x6d8a3795) 0x6d8a3785: 8b 7d 08 8b 07 85 c0 8b f1 74 55 3b 46 20 73 50 0x6d8a3795: 8b 08 83 e1 03 80 f9 03 75 1f 8a 0d 9d 19 a4 6d
Stack: [0x02d70000,0x02dc0000], sp=0x02dbf81c, free space=318k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [jvm.dll+0xa3795] V [jvm.dll+0x16a42a] V [jvm.dll+0x1a2bc1] V [jvm.dll+0x1a2119] V [jvm.dll+0x1a20ad] V [jvm.dll+0x620d4] V [jvm.dll+0x61943] V [jvm.dll+0x61da6] V [jvm.dll+0x62227] V [jvm.dll+0xccbab] V [jvm.dll+0x62022] V [jvm.dll+0xc6aab] V [jvm.dll+0xa333c] V [jvm.dll+0xc79f5] V [jvm.dll+0x7d50b] V [jvm.dll+0x1e6b1b] V [jvm.dll+0x1e92a3] V [jvm.dll+0x1e853e] V [jvm.dll+0x1e888c] V [jvm.dll+0x1e8cb2] V [jvm.dll+0x173e5c] C [msvcr71.dll+0x9565] C [kernel32.dll+0xb713]
VM_Operation (0x03acf34c): GenCollectForAllocation, mode: safepoint, requested by thread 0x0346a800
--------------- P R O C E S S ---------------
Java Threads: ( => current thread ) 0x032bf000 JavaThread "my-rtmp-SocketServer-WorkerThread-33" [_thread_blocked, id=2672, stack(0x04070000,0x040c0000)]
0x0309ac00 JavaThread "my-rtmp-SocketServer-WorkerThread-30" [_thread_blocked, id=2688, stack(0x04110000,0x04160000)]
0x0309b000 JavaThread "my-rtmp-SocketServer-WorkerThread-22" [_thread_blocked, id=2816, stack(0x04160000,0x041b0000)]
0x02d2b400 JavaThread "my-rtmp-SocketServer-WorkerThread-19" [_thread_blocked, id=2344, stack(0x040c0000,0x04110000)]
0x02ce0800 JavaThread "Timer-1" daemon [_thread_blocked, id=2312, stack(0x04cd0000,0x04d20000)]
0x0457d000 JavaThread "JotmClock" daemon [_thread_blocked, id=3548, stack(0x04c80000,0x04cd0000)]
0x04562800 JavaThread "JotmBatch" daemon [_thread_blocked, id=4020, stack(0x04c30000,0x04c80000)]
0x032e6000 JavaThread "TP-Monitor" daemon [_thread_blocked, id=1328, stack(0x04020000,0x04070000)]
0x03506c00 JavaThread "TP-Processor4" daemon [_thread_in_native, id=3180, stack(0x03fd0000,0x04020000)]
0x03505800 JavaThread "TP-Processor3" daemon [_thread_blocked, id=996, stack(0x03f80000,0x03fd0000)]
0x03114000 JavaThread "TP-Processor2" daemon [_thread_blocked, id=396, stack(0x03f30000,0x03f80000)]
0x03112c00 JavaThread "TP-Processor1" daemon [_thread_blocked, id=4084, stack(0x03ee0000,0x03f30000)]
0x02d36400 JavaThread "http-8400-12" daemon [_thread_blocked, id=3820, stack(0x03e90000,0x03ee0000)]
0x02d35000 JavaThread "http-8400-11" daemon [_thread_blocked, id=4080, stack(0x03e40000,0x03e90000)]
0x032aa000 JavaThread "http-8400-10" daemon [_thread_blocked, id=3664, stack(0x03df0000,0x03e40000)]
0x032a8c00 JavaThread "http-8400-9" daemon [_thread_blocked, id=472, stack(0x03da0000,0x03df0000)]
0x0340f000 JavaThread "http-8400-8" daemon [_thread_blocked, id=3464, stack(0x03d50000,0x03da0000)]
0x03498c00 JavaThread "http-8400-7" daemon [_thread_blocked, id=3472, stack(0x03d00000,0x03d50000)]
0x030c8c00 JavaThread "http-8400-6" daemon [_thread_blocked, id=3080, stack(0x03cb0000,0x03d00000)]
0x030c6800 JavaThread "http-8400-5" daemon [_thread_blocked, id=4064, stack(0x03c60000,0x03cb0000)]
0x0343c400 JavaThread "http-8400-4" daemon [_thread_blocked, id=1348, stack(0x03c10000,0x03c60000)]
0x03311400 JavaThread "http-8400-3" daemon [_thread_blocked, id=1960, stack(0x03bc0000,0x03c10000)]
0x02d39800 JavaThread "http-8400-2" daemon [_thread_blocked, id=572, stack(0x03b70000,0x03bc0000)]
0x0346c400 JavaThread "http-8400-1" daemon [_thread_blocked, id=2388, stack(0x03b20000,0x03b70000)]
0x03123400 JavaThread "http-8400-Acceptor-0" daemon [_thread_in_native, id=1084, stack(0x03ad0000,0x03b20000)]
0x0346a800 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=4076, stack(0x03a80000,0x03ad0000)] 0x0349fc00 JavaThread "RedeployManager" daemon [_thread_blocked, id=2756, stack(0x03a30000,0x03a80000)]
0x0349e000 JavaThread "my-nio-server-Acceptor1-port-2880" [_thread_in_native, id=3952, stack(0x039e0000,0x03a30000)]
0x034bdc00 JavaThread "my-nio-server-Reactor1Writer" [_thread_blocked, id=2448, stack(0x03990000,0x039e0000)]
0x03470400 JavaThread "my-nio-server-Reactor1" [_thread_in_native, id=2464, stack(0x03940000,0x03990000)]
0x0340b800 JavaThread "my-rtmp-SocketServer-Acceptor1-port-2038" [_thread_in_native, id=2396, stack(0x038a0000,0x038f0000)]
0x03497400 JavaThread "my-rtmp-SocketServer-Reactor1Writer" [_thread_blocked, id=2972, stack(0x03850000,0x038a0000)]
0x0350e400 JavaThread "my-rtmp-SocketServer-Reactor1" [_thread_in_native, id=2592, stack(0x03800000,0x03850000)]
0x0339e400 JavaThread "Thread-5" [_thread_blocked, id=3584, stack(0x03760000,0x037b0000)] 0x034bb400 JavaThread "Timer-0" [_thread_blocked, id=2656, stack(0x03710000,0x03760000)] 0x032e0400 JavaThread "MySQL Statement Cancellation Timer" daemon [_thread_blocked, id=1316, stack(0x036c0000,0x03710000)]
0x03483c00 JavaThread "RedeployManager" daemon [_thread_blocked, id=2608, stack(0x03670000,0x036c0000)]
0x03381400 JavaThread "RedeployManager" daemon [_thread_blocked, id=3884, stack(0x03610000,0x03660000)]
0x02cd9800 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=2764, stack(0x02f50000,0x02fa0000)]
0x02cd3000 JavaThread "CompilerThread0" daemon [_thread_blocked, id=3656, stack(0x02f00000,0x02f50000)]
0x02cd1800 JavaThread "Attach Listener" daemon [_thread_blocked, id=3356, stack(0x02eb0000,0x02f00000)]
0x02cd0400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=476, stack(0x02e60000,0x02eb0000)]
0x02c91c00 JavaThread "Finalizer" daemon [_thread_blocked, id=3068, stack(0x02e10000,0x02e60000)]
0x02c8d000 JavaThread "Reference Handler" daemon [_thread_blocked, id=3940, stack(0x02dc0000,0x02e10000)]
0x00316c00 JavaThread "main" [_thread_in_native, id=1884, stack(0x00900000,0x00950000)]
Other Threads: =>0x02c8b800 VMThread [stack: 0x02d70000,0x02dc0000] [id=3568] 0x02cdb000 WatcherThread [stack: 0x02fa0000,0x02ff0000] [id=2748]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event]) [0x00315d78] Threads_lock - owner thread: 0x02c8b800 [0x00316188] Heap_lock - owner thread: 0x0346a800
Heap def new generation total 18176K, used 17292K [0x069f0000, 0x07da0000, 0x09150000) eden space 16192K, 100% used [0x069f0000, 0x079c0000, 0x079c0000) from space 1984K, 55% used [0x07bb0000, 0x07cc3218, 0x07da0000) to space 1984K, 0% used [0x079c0000, 0x079c1060, 0x07bb0000) tenured generation total 241984K, used 14900K [0x09150000, 0x17da0000, 0x269f0000) the space 241984K, 6% used [0x09150000, 0x09fdd3f0, 0x09fdd400, 0x17da0000) compacting perm gen total 15360K, used 15243K [0x269f0000, 0x278f0000, 0x2a9f0000) the space 15360K, 99% used [0x269f0000, 0x278d2d50, 0x278d2e00, 0x278f0000) ro space 8192K, 63% used [0x2a9f0000, 0x2af09920, 0x2af09a00, 0x2b1f0000) rw space 12288K, 53% used [0x2b1f0000, 0x2b864dd0, 0x2b864e00, 0x2bdf0000)
Dynamic libraries: 0x00400000 - 0x00424000 C:\Program Files\Java\jre6\bin\java.exe 0x7c900000 - 0x7c9af000 C:\WINDOWS\system32\ntdll.dll 0x7c800000 - 0x7c8f6000 C:\WINDOWS\system32\kernel32.dll 0x77dd0000 - 0x77e6b000 C:\WINDOWS\system32\ADVAPI32.dll 0x77e70000 - 0x77f02000 C:\WINDOWS\system32\RPCRT4.dll 0x77fe0000 - 0x77ff1000 C:\WINDOWS\system32\Secur32.dll 0x7c340000 - 0x7c396000 C:\Program Files\Java\jre6\bin\msvcr71.dll 0x6d800000 - 0x6da8b000 C:\Program Files\Java\jre6\bin\client\jvm.dll 0x7e410000 - 0x7e4a1000 C:\WINDOWS\system32\USER32.dll 0x77f10000 - 0x77f59000 C:\WINDOWS\system32\GDI32.dll 0x76b40000 - 0x76b6d000 C:\WINDOWS\system32\WINMM.dll 0x5dac0000 - 0x5dac8000 C:\WINDOWS\system32\rdpsnd.dll 0x76360000 - 0x76370000 C:\WINDOWS\system32\WINSTA.dll 0x5b860000 - 0x5b8b5000 C:\WINDOWS\system32\NETAPI32.dll 0x77c10000 - 0x77c68000 C:\WINDOWS\system32\msvcrt.dll 0x76bf0000 - 0x76bfb000 C:\WINDOWS\system32\PSAPI.DLL 0x6d7b0000 - 0x6d7bc000 C:\Program Files\Java\jre6\bin\verify.dll 0x6d330000 - 0x6d34f000 C:\Program Files\Java\jre6\bin\java.dll 0x6d290000 - 0x6d298000 C:\Program Files\Java\jre6\bin\hpi.dll 0x6d7f0000 - 0x6d7ff000 C:\Program Files\Java\jre6\bin\zip.dll 0x6d610000 - 0x6d623000 C:\Program Files\Java\jre6\bin\net.dll 0x71ab0000 - 0x71ac7000 C:\WINDOWS\system32\WS2_32.dll 0x71aa0000 - 0x71aa8000 C:\WINDOWS\system32\WS2HELP.dll 0x71a50000 - 0x71a8f000 C:\WINDOWS\system32\mswsock.dll 0x662b0000 - 0x66308000 C:\WINDOWS\system32\hnetcfg.dll 0x71a90000 - 0x71a98000 C:\WINDOWS\System32\wshtcpip.dll 0x68000000 - 0x68036000 C:\WINDOWS\system32\rsaenh.dll 0x769c0000 - 0x76a74000 C:\WINDOWS\system32\USERENV.dll 0x76f20000 - 0x76f47000 C:\WINDOWS\system32\DNSAPI.dll 0x76fb0000 - 0x76fb8000 C:\WINDOWS\System32\winrnr.dll 0x76f60000 - 0x76f8c000 C:\WINDOWS\system32\WLDAP32.dll 0x76fc0000 - 0x76fc6000 C:\WINDOWS\system32\rasadhlp.dll 0x6d560000 - 0x6d569000 C:\Program Files\Java\jre6\bin\management.dll 0x6d630000 - 0x6d639000 C:\Program Files\Java\jre6\bin\nio.dll 0x73080000 - 0x7309d000 C:\WINDOWS\system32\rsvpsp.dll
VM Arguments: jvm_args: -Xms256m -Xmx512m -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=C:_server\lcds\tomcat\conf\logging.properties -Djava.endorsed.dirs=C:_server\lcds\tomcat\endorsed -Dcatalina.base=C:_server\lcds\tomcat -Dcatalina.home=C:_server\lcds\tomcat -Djava.io.tmpdir=C:_server\lcds\tomcat\temp java_command: org.apache.catalina.startup.Bootstrap start Launcher Type: SUN_STANDARD
Environment Variables: JAVA_HOME=c:\Program Files\Java\jdk1.6.0_16 JRE_HOME=C:Program Files\Java\jre6 CLASSPATH=c:\Program Files\Java\jdk1.6.0_16\lib\tools.jar;C:_server\lcds\tomcat\bin\bootstrap.jar PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:_server\php\;C:_server\php\ext; JAVA_HOME\bin USERNAME=server OS=Windows_NT PROCESSOR_IDENTIFIER=x86 Family 6 Model 10 Stepping 0, AuthenticAMD
--------------- S Y S T E M ---------------
OS: Windows XP Build 2600 Service Pack 3
CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 10 stepping 0, cmov, cx8, fxsr, mmx, sse, mmxext, 3dnow, 3dnowext
Memory: 4k page, physical 1048052k(494324k free), swap 2520340k(1473844k free)
vm_info: Java HotSpot(TM) Client VM (14.2-b01) for windows-x86 JRE (1.6.0_16-b01), built on Jul 31 2009 11:26:58 by "java_re" with MS VC++ 7.1
time: Wed Dec 16 16:28:31 2009 elapsed time: 8669 seconds
