Project

General

Profile

tigase can't login and many close_wait connection

shen wr
Added over 4 years ago

today tigase can't login and cannot be automatically restored;

the following is the information of system,

please help me to look at what's the problem? thanks

/var/log/messages

Aug 23 23:38:09 lx24 kernel: [4679430.652832] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.652839] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.652845] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.652852] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.652859] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.656824] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.656829] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.656834] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.656841] Out of socket memory
Aug 23 23:38:09 lx24 kernel: [4679430.656846] Out of socket memory
Aug 23 23:38:15 lx24 kernel: [4679435.923750] net_ratelimit: 2926 callbacks suppressed
Aug 23 23:38:15 lx24 kernel: [4679435.923755] Out of socket memory
Aug 23 23:38:15 lx24 kernel: [4679436.035554] Out of socket memory
Aug 23 23:38:15 lx24 kernel: [4679436.339031] Out of socket memory
Aug 23 23:38:15 lx24 kernel: [4679436.610562] Out of socket memory
Aug 23 23:38:16 lx24 kernel: [4679437.057796] Out of socket memory
Aug 23 23:38:16 lx24 kernel: [4679437.121684] Out of socket memory
Aug 23 23:38:16 lx24 kernel: [4679437.536962] Out of socket memory
Aug 23 23:38:16 lx24 kernel: [4679437.552934] Out of socket memory
Aug 23 23:38:17 lx24 kernel: [4679437.984189] Out of socket memory
Aug 23 23:38:17 lx24 kernel: [4679438.291662] Out of socket memory
Aug 23 23:38:29 lx24 kernel: [4679450.682310] net_ratelimit: 3 callbacks suppressed
Aug 23 23:38:29 lx24 kernel: [4679450.682315] Out of socket memory

tigase-console.log

2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 140.207.22.51 - 392
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 219.233.212.82 - 438
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 115.231.71.187 - 387
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 114.217.57.44 - 315
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 118.249.223.68 - 390
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 183.4.80.171 - 420
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 223.68.189.142 - 348
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 14.16.29.93 - 402
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 211.138.199.185 - 368
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 117.136.8.56 - 451
2014-08-23 23:38:27.113 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 124.79.217.202 - 355
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 180.114.189.134 - 377
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 112.65.5.66 - 349
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 219.136.99.168 - 405
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 223.104.21.140 - 391
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 112.64.26.74 - 440
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 223.104.1.90 - 353
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 124.166.32.200 - 368
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 116.228.31.162 - 449
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 58.211.229.58 - 363
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 116.231.124.42 - 314
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 14.16.43.186 - 340
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 27.115.116.215 - 459
2014-08-23 23:38:27.114 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 116.236.238.170 - 341
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 180.168.109.114 - 315
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 180.164.109.230 - 368
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 111.173.34.206 - 381
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 117.136.40.43 - 377
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 58.35.172.191 - 379
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 36.63.149.202 - 430
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 58.246.241.206 - 446
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 114.232.97.119 - 392
2014-08-23 23:38:27.115 [IPMonitor Timer]  IPMonitor$1.run()                  WARNING:  Many disconnects for IP: 223.11.223.18 - 313
2014-08-23 23:38:27.245 [in_12-c2s]        ClientConnectionManager.processCommand()  WARNING: Cant find sevice for STARTTLS command: {0}
2014-08-23 23:38:28.378 [in_23-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:38:29.009 [in_33-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:10.327 [in_6-c2s]         TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:10.646 [in_46-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:10.650 [in_44-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:10.656 [in_33-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:10.673 [in_18-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:10.695 [in_18-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:10.704 [in_41-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:14.185 [in_54-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ
2014-08-23 23:40:14.345 [in_14-c2s]        TLSIO.writeBuff()                  WARNING:  Infinite loop detected in writeBuff(buff) TLS code, tlsWrapper.getStatus(): NEED_READ

top

top - 23:46:49 up 54 days,  6:13,  1 user,  load average: 1.33, 1.26, 0.94
Tasks: 156 total,   1 running, 155 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.8%us,  0.4%sy,  0.0%ni, 98.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65965112k total, 47389856k used, 18575256k free,   435068k buffers
Swap: 24742908k total,        0k used, 24742908k free, 14071724k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                         
 6791 greenlin  20   0 59.2g  29g  12m S  105 47.2   5502:01 java                                                                                                                                                                            
    1 root      20   0 24312 2256 1360 S    0  0.0   0:26.47 init     

netstat

LISTEN 11
SYN_RECV 26
CLOSE_WAIT 10816
TIME_WAIT 187
ESTABLISHED 6727


Replies (2)

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

This looks to me like the OS configuration problem. I suggest you search internet for "Out of socket memory" error. Looks like there is plenty of articles on this. I found this one very informative: http://blog.tsunanet.net/2011/03/out-of-socket-memory.html

Added by shen wr over 4 years ago

thank you very much , Artur Hefczyc;

my tcp_max_orphans is too small

    (1-2/2)