TransWikia.com

Mongodb index keeps growing in size eventhough the collection has a TTL index

Database Administrators Asked on November 30, 2021

I work on a mongodb sharded cluster (3.0.2 i know it’s very outdated, not my call), on which i have a sharded db and a sharded collection on which i have multiple indexes.

One of the indexes is a TTL index (24h only, 86400) on a date field named d, used to expire old data, this leads to almost fixing the number of entries in the collection within our use case. Given the fixed structure and model of the data, this means an almost fixed size of the collection and it’s indexes, it has always been the case for index and collection sizes throughout the whole cluster.

Our issue is, we have another index on this collection, on a field named k which is a large string.

ex : GigabitEthernet_XXXX-XXXXX-01_ABC_DEFG-HIJKLMNOP-5-QRS_TUV_7_0/0/1

This index keeps growing indefinitely in size, and causes memory outages and we don’t know why. Any help ?

Knowing that the TTL index always expires/removes data from the collection, isn’t the second index, on the k field supposed to also remove the references to that data, hence be fixed or at least not be evergrowing in size ?

We use this same technique (TTL index + other indexes) on all our collections and it always works in terms of fixing the sizes, not on this collection though. It is also important to note that other collections have a much smaller k field than this one, is that the issue ? Any explanations ? Any mongodb docs about this ?

Happy to provide any needed details to explain the issue.

Thanks in advance

One Answer

Compact the collection in your staging environment. If index size decreases, space is taken up by overhead.

Compare the size of compacted collection & index to the size of both pre compaction, identify the overhead amount/percentage.

Answered by D. SM on November 30, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP