java - timeout errors from Tomcat talking to MongoDB in Azure -


my system tomcat 7 server running on ubuntu talking mongodb cluster running in centos. have on aws , working fine.

i brought exact same thing on azure , having constant, seemingly random timeouts when tomcat app tries query mongodb. typical error is:

jan 31 08:13:54 catalina.out:  jan 31, 2014 4:14:09 pm com.mongodb.dbportpool goterror jan 31 08:13:54 catalina.out:  warning: emptying dbportpool xxx.cloudapp.net/xxx.xxx.xxx.xxx:21191 b/c of error jan 31 08:13:54 catalina.out:  java.net.socketexception: connection timed out jan 31 08:13:54 catalina.out:   @ java.net.socketinputstream.socketread0(native method) jan 31 08:13:54 catalina.out:   @ java.net.socketinputstream.read(socketinputstream.java:146) jan 31 08:13:54 catalina.out:   @ java.io.bufferedinputstream.fill(bufferedinputstream.java:235) jan 31 08:13:54 catalina.out:   @ java.io.bufferedinputstream.read1(bufferedinputstream.java:275) jan 31 08:13:54 catalina.out:   @ java.io.bufferedinputstream.read(bufferedinputstream.java:334) jan 31 08:13:54 catalina.out:   @ org.bson.io.bits.readfully(bits.java:46) jan 31 08:13:54 catalina.out:   @ org.bson.io.bits.readfully(bits.java:33) jan 31 08:13:54 catalina.out:   @ org.bson.io.bits.readfully(bits.java:28) jan 31 08:13:54 catalina.out:   @ com.mongodb.response.<init>(response.java:40) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbport.go(dbport.java:142) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbport.call(dbport.java:92) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbtcpconnector.innercall(dbtcpconnector.java:244) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbtcpconnector.call(dbtcpconnector.java:216) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbapilayer$mycollection.__find(dbapilayer.java:288) jan 31 08:13:54 catalina.out:   @ com.mongodb.db.command(db.java:262) jan 31 08:13:54 catalina.out:   @ com.mongodb.db.command(db.java:244) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbcollection.getcount(dbcollection.java:985) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbcollection.getcount(dbcollection.java:956) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbcollection.getcount(dbcollection.java:931) jan 31 08:13:54 catalina.out:   @ com.mongodb.dbcollection.count(dbcollection.java:878) jan 31 08:13:54 catalina.out:   @ com.eweware.service.base.store.impl.mongo.dao.basedaoimpl._exists(basedaoimpl.java:788) jan 31 08:13:54 catalina.out:   @ com.eweware.service.base.store.impl.mongo.dao.groupdaoimpl._exists(groupdaoimpl.java:18)

i using java driver 2.11.4 , initializing follows:

        builder.autoconnectretry(true)                 .connectionsperhost(10)                 .writeconcern(writeconcern.fsynced)                 .connecttimeout(30000)                 .socketkeepalive(true);

in reading around interwebs saw materials suggesting there azure issue , c# suggestions have not seen on how correct java.

some more details:

  1. this happens when mongodb single node or replica set
  2. this happens whether or not server under load. in fact, server seems perform better under load cold start. under constant load timeout
  3. the timeouts appear random, in there no discernable pattern in call fail or when fail. go hour no issues, other times every call fail
  4. if retry calls on timeout error, work. takes >100 retries, other times single 1 work.

here retry code trying:

private dbobject findoneretry(dbobject criteria, dbobject fields, dbcollection collection) throws systemerrorexception {     dbobject obj = null;     (int attempt = 1; attempt < max_retries; attempt++) {         try {             obj = collection.findone(criteria, fields);  // getting socketexception inside here             return obj;         } catch (exception e) {             if (attempt > max_retries) {                 throw new systemerrorexception(makeerrormessage("findoneretry", "find", attempt, e, null), e, errorcodes.server_db_error);             } else {                 logger.warning(getclass().getname() + ": findoneretry failed , retry in attempt #" + attempt + " in collection " + _getcollection());             }         }     }     return obj; }

any suggestions on how correct?

thanks in advance!

this due high tcp_keepalive_time in centos server.

in mongos server: sudo nano /proc/sys/net/ipv4/tcp_keepalive_time change 7200 60

restart azure instance.

update: in order make sure vm has value of tcp_keepalive_time:

add line:

bash -c 'echo 60 > /proc/sys/net/ipv4/tcp_keepalive_time'

to:

/etc/rc.d/rc.local

update update: flavors linux, there /etc/sysctl.d/ directory. create file, e.g. mongo.conf containing:

net.ipv4.tcp_keepalive_time = 60

put file in directory , run:

sysctl -p /etc/sysctl.d/mongo.conf

verify change with:

sysctl net.ipv4.tcp_keepalive_time

this survive reboots in redhat-based , debian-based systems i've come across. you'll want sure , check /etc/sysctl.conf , other files in /etc/sysctl.d see if variable being set else , make appropriate changes.


-

Comments

Post a Comment

Popular posts from this blog

html - Sizing a high-res image (~8MB) to display entirely in a small div (circular, diameter 100px) -

first of all, thank being patient know simple question more couple of weeks' experience figure out. i've seen folks persevere , helpful on here , hope you're willing me. here fiddle: http://jsfiddle.net/65uzf/ here 1 of places searched answers on here: ( vertically center image thumbnails fix-sized div ) tried solutions on there , didn't work out me. i've searched ~4 or 5 others, , been unsuccessful those, too. what want take large image (what linked background-image #ysypic , #tabpic divs) , display thumbnail version of image in confines of divs have set it. is, using background properties have laid out in css, portion of image displays inside of circular divs. without these properties, display defaults smaller section of image [presumably pi*(50^2) area in top left] terrible resolution. i not care if entire image doesn't display inside of circles (it won't, given difference in shape), , overflow not being displayed fine. want able see figu
Read more

java - IntelliJ - No such instance method -

using intellij's 12 ultimate, i'm running following code in debugger: java import play.api.libs.json.jsvalue; public class foo { ... public jsvalue tojson() { ... } public class fooexample { ... foo foo = new foo(); system.out.println("...); //<-- breakpoint at breakpoint, right-clicked source code , picked "evaluate expression," typing in: foo.tojson() . but following error showed up: no such instance method: play.api.libs.json.jsvalue$class.com.foo.foo.tojson () am doing wrong? foo#tojson calls scala code, if matters. edit had breakpoint after instantiation of foo. downvoted it, deserved it. edit my answer no longer valid after op modified question (i.e. moved breakpoint actual is).. a breakpoint hit before line executed. in case foo has not yet been declared or instantiated (i.e. foo constructor has not yet been called). you'll need put breakpoint @ next line (or step on current line) if want evaluate foo .
Read more

identifier - Is it possible for an html5 document to have two ids? -

i wondering if html5 element can have 2 ids, this: <button id="thing1" id="thing2" name="button1" onclick="change()">click here</button> or this: <button id="thing1, thing2" name="button1" onclick="change()">click here</button> also, if 1 of ids said 1 thing, , other id said thing, 1 used? tl;dr : no. in versions of html, each element can have 1 id. see the spec : the id attribute specifies element's unique identifier (id). [dom] the value must unique amongst ids in element's home subtree , must contain @ least 1 character. the value must not contain space characters. note: there no other restrictions on form id can take; in particular, ids can consist of digits, start digit, start underscore, consist of punctuation , etc. note: element's unique identifier can used variety of purposes, notably way link specific parts of document usin
Read more