How to solve the problem of opening a large number of files in mongodb
How to solve the problem of opening a large number of files in mongodb, aiming at this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.
Problem description:
The mac osx environment creates 3000 connections through the tcp test tool Report the specific error of too many open files: `2016-05-11T00:59:29.772+0800 E STORAGE [thread1] WiredTiger (23) [1462899569vis771906] [60793:0x70000020a000], log-server: / data/db/journal: opendir: Too many open files in system2016-05-11T00:59:29.772+0800 E STORAGE [thread1] WiredTiger (23) [1462899569STORAGE 772155] [60793:0x70000020a000] Log-server: dirlist journal prefix WiredTigerPreplog: Too many open files in system2016-05-11T00:59:29.772+0800 E STORAGE [thread1] WiredTiger (23) [1462899569 11T00:59:29.772+0800 772199] [60793:0x70000020a000], log-server: log pre-alloc server error: Too many open files in system2016-05-11T00:59:29.772+0800 E STORAGE [thread1] WiredTiger (23) [1462899569vis772239] [60793:0x70000020a000] Log-server: log server error: Too many open files in system2016-05-11T00:59:33.022+0800 W FTDC [ftdc] Uncaught exception in 'FileNotOpen: Failed to open interim file / data/db/diagnostic.data/metrics.interim.temp' in full-time diagnostic data capture subsystem. Shutting down the full-time diagnostic data capture subsystem.2016-05-11T00:59:33.566+0800 E STORAGE [thread2] WiredTiger (23) [1462899573 Too many open files in system2016 566795] [60793:0x700000393000], file:WiredTiger.wt, WT_SESSION.checkpoint: WiredTiger.turtle: fopen: Too many open files in system2016-05-11T00:59:33.568+0800 E STORAGE [thread2] WiredTiger (23) [1462899573 thread2 568812] [60793:0x700000393000] Checkpoint-server: checkpoint server error: Too many open files in system2016-05-11T00:59:33.568+0800 E STORAGE [thread2] WiredTiger (- 31804) [1462899573 WiredTiger library panic2016 568879] [60793:0x700000393000], checkpoint-server: the process must exit and restart: WT_PANIC: WiredTiger library panic2016-05-11T00:59:33.568+0800 I-[thread2] Fatal Assertion 285582016-05-11T00:59:33.568+0800 I-[thread2]
You can check how many open files are allowed first.
Yaoxings-MacBook-Pro:db yaoxing$ ulimit-acore file size (blocks,-c) 0data seg size (kbytes,-d) unlimitedfile size (blocks,-f) unlimitedmax locked memory (kbytes,-l) unlimitedmax memory size (kbytes,-m) unlimitedopen files (- n) 256pipe size (512 bytes,-p) 1stack size (kbytes -s) 8192cpu time (seconds,-t) unlimitedmax user processes (- u) 709virtual memory (kbytes,-v) unlimited
In fact, you will be reminded of this as soon as you enter mongoshell.
2016-05-11T17:45:17.536+0800 I CONTROL [initandlisten] * * WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
WiredTiger works differently from MMAPv1, collecting one file at a time and indexing one file at a time. So how many collections you have, at least x2 open files (calculated as 1 collection file + 1 index file) will be occupied. Taking into account the number of network connections, make sure that the open files value is large enough. Open files can be modified with the following command:
This is the answer to ulimit-n 1024's question on how to solve the problem of opening a large number of files in mongodb. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.
