Operational Defect Database

BugZero found this defect 2496 days ago.

MongoDB | 378338

[SERVER-29006] Query failed with error code 6 with YCSB workload

Last update date:

10/27/2023

Affected products:

MongoDB Server

Affected releases:

No affected releases provided.

Fixed releases:

No fixed releases provided.

Description:

Info

Hi , I have the following setup 4 nodes, 1 client Client -> node1 - node2 - node3 - node4 node1 and node2 config server is running node[1-4] are also acting as shard server , Sharding is enabled for YCSB database and for collection usertable. Mongos is running on client. YCSB: https://github.com/brianfrankcooper/YCSB/tree/master/mongodb Workload: I insert 10M doc , it works fine, When i run workloade issue started to happen With Version 3.4 Enterprise it crashed. Issue looks similar SERVER-27701 With version 3.5.6 Lot of queries throwing following Error com.mongodb.MongoQueryException: Query failed with error code 6 and error message

Top User Comments

mark.agarunov commented on Thu, 22 Jun 2017 21:04:23 +0000: Hello sannandi@indiana.edu We still need additional information to diagnose the problem. If this is still an issue for you, would you please provide the complete log files from both mongod and mongos when this issue is present? Thanks, Mark mark.agarunov commented on Fri, 9 Jun 2017 21:03:11 +0000: Hello sannandi@indiana.edu, Thank you for providing the script. Looking into this further, the error code you are seeing, 6, is "HostUnreachable". As this can be caused by a few different paths in the code, I would like to request the complete log files from both mongod and mongos when this issue is present so that we can get a better idea of what the issue may be. I've generated a secure upload portal so that you can send us these files privately. Thanks, Mark talha commented on Fri, 2 Jun 2017 08:27:00 +0000: Hello i'm facing same issue on my mongodb cluster. I have a collection in which sharding is not enabled. When i run single find query it seems ok. But during concurrent query it started showing exception "com.mongodb.MongoQueryException: Query failed with error code 6 and error message 'Operation aborted.' on server" . And on "Exit Codes and Statuses" of mongodb documentation there are nothing about error code 6 sannandi@indiana.edu commented on Tue, 2 May 2017 23:25:25 +0000: The configuration is here: Client -> node1 - node2 - node3 - node4 node1 and node2 config server is running node[1-4] are also acting as shard server , Sharding is enabled for YCSB database and for collection usertable. Mongos is running on client. Only a Single instance of mongos running on client Commands used to run: Attached the script mongodb.py.txt What is the configuration you are using ? mark.agarunov commented on Tue, 2 May 2017 21:25:26 +0000: Hello sannandi@indiana.edu, I have not yet been able to reproduce the error that you've described, and was hoping to get some more information. If possible, please provide the following: The complete output of YCSB The complete logs from mongos The complete logs from all mongod nodes The configuration you are using for mongod and mongos The exact command you are using to execute YCSB The should help shed some light on what may be causing this behavior. Thanks, Mark jeff.yemin commented on Thu, 27 Apr 2017 22:20:47 +0000: I don't think this is a networking issue between client and mongos. Running against MongoDB >= 3.2, the driver will only throw a MongoQueryException in response to a non-ok response from the server to either the find or getMore command (in this case probably a find since all YCSB queries are for single documents). The error code of 6 and the error message of "'End of file' on server x.x.x.x:27017." is what's being returned from mongos in the command response, e.g.: { ok: 0, code : 6, errmsg : "'End of file' on server x.x.x.x:27017." } dan@10gen.com commented on Thu, 27 Apr 2017 22:05:53 +0000: that's the error message given by the java driver back to the client. It looks like a networking issue between the client and the mongos. sannandi@indiana.edu commented on Thu, 27 Apr 2017 21:58:33 +0000: Only error message i can see is com.mongodb.MongoQueryException: Query failed with error code 6 and error message 'End of file' on server x.x.x.x:27017. All the clusters are up while the test is on. Is there anything wrong with the setup ? As I have mentioned if i use only write query it works fine. dan@10gen.com commented on Thu, 27 Apr 2017 21:50:15 +0000: What is the error on the mongos? Are all the nodes in the cluster up and running at the time of the error message? sannandi@indiana.edu commented on Thu, 27 Apr 2017 20:38:42 +0000: The YCSB workload e is 95% scan and 5% insert. If I change the workload to 100% insert it works fine, When I do scan operation it is throwing error. The earlier version was crashing. With version 3.5.6 crash issue on mongos has fixed but exception starts to happen. I am using 4 instances of YCSB and each one doing 30 threads.

Additional Resources / Links

Share:

BugZero Risk Score

Coming soon

Status

Closed

Have you been affected by this bug?

cost-cta-background

Do you know how much operational outages are costing you?

Understand the cost to your business and how BugZero can help you reduce those costs.

Discussion

Login to read and write comments.

Have you ever...

had your data corrupted from a

VMware

bug?

Search:

...