How the local memstore allocates buffers

This article introduces the knowledge of "how to allocate buffers in local memstore". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1. Long-lived keyvalue instances are brushed to disk-"holes are generated in the old heap -" there are more and more holes that cannot be allocated contiguously large enough-"JRE uses the stop-the-world garbage collector -" to rewrite the entire heap space and compress the remaining available objects

2. Ways to solve the above problems:

Use local memstore to allocate buffer (memstore-local allocation buffer) MSLAB after version 0.92

That is, when objects of the same size are allocated from the heap, fixed-size holes will be left in the heap when they are finally recycled.

3. Configure in the configuration file:

1 configure via hbase.hregion.memstore.mslab.enabled

2 allocation buffer size: hbase.hregion.memstore.mslab.chunksize, the default is 2m, and it is reasonable.

Adjust according to the size of your keyvalue instance.

3 upper boundary of storage buffer: hbase.hregion.memstore.mslab.max.allocation attribute, default 256kb

Beyond this value, apply for space directly in the java heap.

Four advantages and disadvantages:

Advantage: postpone the occurrence of garbage collection pause

Disadvantages: 1 waste heap space and can not make full use of it

2 using buffers requires additional memory copy work, which is better than using keyvalues instances directly

This is the end of how the local memstore allocates buffers. Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!