MongoDB | 2419311

Info

When an ExclusionProjectionExecutor is used to apply a projection like: {a: 0, b: 0} The code walks the input document, and for each field, determines whether to "project" it. Since the code for applying the projection is generalized to work for both inclusion and exclusion projections, the value for the field is always provided. By reading the value field, we load it into the document cache. For exclusion projections, the value of the field doesn't matter, since we're going to exclude it. So the work of loading it into cache is completely wasted. Recently, two fast paths have been added for exclusion projection, which do a direct BSON -> BSON transform. These were added under SERVER-61284 (ProjectionSimple) and SERVER-70353 which adds a fast path in ExclusionProjectionExecutor. These both went into 6.2, and make it harder to hit this problem, since the fast paths are used more frequently than the generic path which has the bug. However, for older versions, it's still very easy to run into this problem. Anyone upgrading from 4.2 -> 4.4 is likely to see this issue. For example, here is a test script which creates one document with a field blocks which is a 40,000 element array. Then it runs a query to exclude just the blocks field. Since blocks is such a large field, the (wasted) time spent loading it into cache dominates the runtime for 4.4: (function() { let doc = {blocks: []}; for (let i = 0; i < 40 * 1000; ++i) { doc.blocks.push({hash: 123, size: 456, otherdata: "hello world"}); } // Add 100 other top level fields. for (let i = 0; i < 20; ++i) { doc["field" + i] = "test"; } assert.commandWorked(db.c.insert(doc)); let now = Date.now(); for (let i = 0; i < 1000; ++i) { db.c.find({}, {"blocks": 0}).toArray(); } let after = Date.now(); print("Total time: " + tojson(after - now)); })(); If you compare this with a run on 6.2 or later, it is much slower (around 30x on my machine). In short, on affected versions, an exclusion projection requires us to completely copy and shred the fields we don't want to keep. This ticket tracks the work of fixing the problem in the default ExclusionProjectionExecutor path. Whether we also want to backport the new fast paths to older versions is a separate question.

Top User Comments

xgen-internal-githook commented on Thu, 5 Oct 2023 00:21:10 +0000: Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'} Message: SERVER-80157 / BACKPORT-16898 Don't load excluded fields into the cache Branch: v6.0 https://github.com/mongodb/mongo/commit/9bc2da112c531d89911938d265072cc59bcd8e62 xgen-internal-githook commented on Thu, 5 Oct 2023 00:08:55 +0000: Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'} Message: SERVER-80157 / BACKPORT-16900 Don't load excluded fields into the cache Branch: v4.4 https://github.com/mongodb/mongo/commit/40c22ee0081c0a780d103db17e11b16339995847 xgen-internal-githook commented on Wed, 4 Oct 2023 21:35:52 +0000: Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'} Message: SERVER-80157 / BACKPORT-16899 Don't load excluded fields into the cache Branch: v5.0 https://github.com/mongodb/mongo/commit/54b5f063d937219e3102581f3c5413ce5b4d5d4a xgen-internal-githook commented on Wed, 4 Oct 2023 21:31:56 +0000: Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'} Message: SERVER-80157 / BACKPORT-16896 Don't load excluded fields into the cache Branch: v7.0 https://github.com/mongodb/mongo/commit/0f99389827f8602d9d50e83087a03082464ae5ae xgen-internal-githook commented on Wed, 4 Oct 2023 18:24:13 +0000: Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'} Message: SERVER-80157 Don't load excluded fields into the cache Branch: master https://github.com/mongodb/mongo/commit/6f7fefbbe4528517f77a5c17f8fcff2912239ab4

Steps to Reproduce