Operational Defect Database

BugZero found this defect 2631 days ago.

MongoDB | 338459

[SERVER-27400] RocksDB insertion count failure in concurrency test

Last update date:

12/6/2022

Affected products:

MongoDB Server

Affected releases:

No affected releases provided.

Fixed releases:

No fixed releases provided.

Description:

Info

concurrency_simultaneous failed on ubuntu1404-rocksdb Project: mongodb-mongo-master fsm_all_simultaneous.js - Logs | History [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.913+0000 2016-12-02T19:55:12.913+0000 E QUERY [main] Error: 2 threads threw [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.913+0000 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.913+0000 Foreground jstests/concurrency/fsm_workloads/indexed_insert_unordered_bulk.js [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.913+0000 Error: [0] != [15] are not equal : undefined [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.913+0000 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.913+0000 quietlyDoAssert@jstests/concurrency/fsm_libs/assert.js:53:15 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 assert.eq@src/mongo/shell/assert.js:54:5 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 wrapAssertFn@jstests/concurrency/fsm_libs/assert.js:60:13 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 assertWithLevel/</assertWithLevel[fn]@jstests/concurrency/fsm_libs/assert.js:99:13 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 find@jstests/concurrency/fsm_workloads/indexed_insert_base.js:47:13 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 runFSM@jstests/concurrency/fsm_libs/fsm.js:37:13 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 @<unknown> line 6 > eval:10:9 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 main@jstests/concurrency/fsm_libs/worker_thread.js:104:17 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 @<unknown> line 6 > eval:7:1 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 @<unknown> line 6 > eval:5:24 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 _threadStartWrapper@:24:16 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.914+0000 : [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.915+0000 throwError@jstests/concurrency/fsm_libs/runner.js:339:23 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.915+0000 runWorkloads@jstests/concurrency/fsm_libs/runner.js:734:17 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.915+0000 parallel@jstests/concurrency/fsm_libs/runner.js:756:1 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.915+0000 @jstests/concurrency/fsm_all_simultaneous.js:24:1 [js_test:fsm_all_simultaneous] 2016-12-02T19:55:12.915+0000 failed to load: jstests/concurrency/fsm_all_simultaneous.js

Top User Comments

milkie commented on Mon, 27 Feb 2017 21:04:18 +0000: Hi igor The compilation of this project has broken because we decided to rename a function in the storage API – sorry for that! It should be easy to fix, or we can submit a pull request. Once this project starts compiling and running the full test suite again, I suspect we will still see failures in the concurrency tests. Do you have any leads on what may be going wrong? -Eric milkie commented on Wed, 15 Feb 2017 16:50:47 +0000: Here's another failure: concurrency_replication failed on Ubuntu 14.04 (RocksDB) Project: MongoDB (3.4) fsm_all_replication.js - Logs | History igor commented on Tue, 24 Jan 2017 02:15:09 +0000: Ah, I though I was lucky! I'll try reproing again I guess. Thanks Eric! milkie commented on Wed, 18 Jan 2017 19:15:51 +0000: It turns out I was looking at Jira incorrectly and was mistaken – this type of failure continues to happen in our test suite. Here's one of the latest failures: concurrency failed on Ubuntu 14.04 (RocksDB) December 2 is the earliest incidence of failure that I can find. igor commented on Wed, 18 Jan 2017 18:43:44 +0000: Hi Eric, I haven't been able to reproduce unfortunately (I ran the test in a loop for a day). I also looked deep through the code and haven't detected anything that might have caused this. It might be that there was an underlying bug in RocksDB, since the tests in Evergreen run on RocksDB's master branch. Would it make sense to close it for now and reopen if we see the issue happening again? milkie commented on Wed, 18 Jan 2017 18:39:46 +0000: Hi igor, Did you happen to commit a fix for this? I haven't seen any failures for a while. igor commented on Wed, 14 Dec 2016 14:47:42 +0000: Thanks Eric, taking a look. milkie commented on Tue, 13 Dec 2016 17:00:00 +0000: Finally, here is a similar failure but in a different way: concurrency failed on ubuntu1404-rocksdb Project: mongodb-mongo-master fsm_all.js - Logs | History [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 2016-12-02T19:54:33.534+0000 E QUERY [main] Error: 1 thread threw [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 Foreground jstests/concurrency/fsm_workloads/touch_base.js [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 Error: [0] != [100] are not equal : collection scan should return the number of documents this thread inserted [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 quietlyDoAssert@jstests/concurrency/fsm_libs/assert.js:53:15 [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 assert.eq@src/mongo/shell/assert.js:54:5 [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 wrapAssertFn@jstests/concurrency/fsm_libs/assert.js:60:13 [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 assertWithLevel/</assertWithLevel[fn]@jstests/concurrency/fsm_libs/assert.js:99:13 [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 query@jstests/concurrency/fsm_workloads/touch_base.js:36:1 [js_test:fsm_all] 2016-12-02T19:54:33.535+0000 runFSM@jstests/concurrency/fsm_libs/fsm.js:37:13 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 @<unknown> line 6 > eval:10:9 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 main@jstests/concurrency/fsm_libs/worker_thread.js:104:17 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 @<unknown> line 6 > eval:7:1 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 @<unknown> line 6 > eval:5:24 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 _threadStartWrapper@:24:16 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 : [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 throwError@jstests/concurrency/fsm_libs/runner.js:339:23 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 runWorkloads@jstests/concurrency/fsm_libs/runner.js:734:17 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 serial@jstests/concurrency/fsm_libs/runner.js:747:1 [js_test:fsm_all] 2016-12-02T19:54:33.536+0000 @jstests/concurrency/fsm_all.js:16:1 [js_test:fsm_all] 2016-12-02T19:54:33.537+0000 failed to load: jstests/concurrency/fsm_all.js milkie commented on Tue, 13 Dec 2016 15:50:35 +0000: Here is another instance of the failure: concurrency_sharded failed on ubuntu1404-rocksdb Project: mongodb-mongo-master fsm_all_sharded_replication.js - Logs | History [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 2016-12-07T21:27:21.450+0000 E QUERY [main] Error: 2 threads threw [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 Foreground jstests/concurrency/fsm_workloads/indexed_insert_ordered_bulk.js [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 Error: [0] != [15] are not equal : undefined [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 quietlyDoAssert@jstests/concurrency/fsm_libs/assert.js:53:15 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 assert.eq@src/mongo/shell/assert.js:54:5 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 wrapAssertFn@jstests/concurrency/fsm_libs/assert.js:60:13 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 assertWithLevel/</assertWithLevel[fn]@jstests/concurrency/fsm_libs/assert.js:99:13 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 find@jstests/concurrency/fsm_workloads/indexed_insert_base.js:47:13 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 runFSM@jstests/concurrency/fsm_libs/fsm.js:37:13 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.451+0000 @<unknown> line 6 > eval:10:9 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 main@jstests/concurrency/fsm_libs/worker_thread.js:104:17 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 @<unknown> line 6 > eval:7:1 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 @<unknown> line 6 > eval:5:24 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 _threadStartWrapper@:24:16 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 : [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 throwError@jstests/concurrency/fsm_libs/runner.js:339:23 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 runWorkloads@jstests/concurrency/fsm_libs/runner.js:734:17 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 serial@jstests/concurrency/fsm_libs/runner.js:747:1 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 @jstests/concurrency/fsm_all_sharded_replication.js:99:1 [js_test:fsm_all_sharded_replication] 2016-12-07T21:27:21.452+0000 failed to load: jstests/concurrency/fsm_all_sharded_replication.js milkie commented on Tue, 13 Dec 2016 15:49:38 +0000: Hi igor, One of our concurrency tests encountered a failure with our RocksDB builder. Can you help diagnose? -Eric

Additional Resources / Links

Share:

BugZero Risk Score

Coming soon

Status

Closed

Have you been affected by this bug?

cost-cta-background

Do you know how much operational outages are costing you?

Understand the cost to your business and how BugZero can help you reduce those costs.

Discussion

Login to read and write comments.

Have you ever...

had your data corrupted from a

VMware

bug?

Search:

...