Shulou(Shulou.com)06/01 Report--

This article is about how to solve the delay and expiration of RabbtiMQ message queues. I think it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it.

With regard to this matter, let's sort it out one by one. Let's assume a scenario where we have a consumer malfunction, and then a large number of messages are overstocked in mq. Now there is an accident and panic.

A lot of news has been backlog in mq for several hours and has not been solved.

The backlog of tens of millions of pieces of data in MQ lasted for seven or eight hours, from more than 4 p.m., to more than 11:00 in the evening. This is a real scenario we have encountered, which is indeed an online glitch, or fix the consumer problem, let it restore the consumption speed, and then foolishly wait for a few hours to finish the consumption. I'm sure you can't talk about this during the interview.

One consumer is 1000 per second, three consumers per second is 3000, and one minute is 180000. So if you have a backlog of millions to tens of millions of data, even if the consumer recovers, it will take about an hour to recover.

Generally speaking, at this time, we can only expand capacity temporarily and urgently. The specific operation steps and ideas are as follows:

First fix the problem with consumer, make sure it returns to consumption speed, and then stop all existing consumer.

Create a new topic,partition that is 10 times the original, and temporarily set up 10 times the number of queue.

Then write a temporary distribution data consumer program, this program is deployed to consume the backlog of data, after consumption does not do time-consuming processing, directly and evenly poll to write the temporarily established 10 times the number of queue.

Then 10 times as many machines are temporarily requisitioned to deploy consumer, and each batch of consumer consumes one temporary queue data. This is equivalent to temporarily expanding queue resources and consumer resources by 10 times, consuming data at a normal rate of 10 times.

After quickly consuming the backlog of data, you have to restore the previously deployed architecture and re-use the original consumer machine to consume messages.

The message in mq expires.

Suppose you are using RabbitMQ,RabbtiMQ and you can set the expiration time, that is, TTL. If the message is overstocked in queue for more than a certain period of time, it will be cleared by RabbitMQ, and the data will be gone. Then this is the second trap. This is not to say that a large amount of data will be overstocked in mq, but that a large amount of data will be lost directly.

In this case, it does not mean to increase the backlog of consumer consumption, because there is actually no backlog, but a lot of information has been lost. We can take a plan, that is, batch redirect, which we have done on the front line. That is, when we had a large backlog, we simply discarded the data, and then after the peak period, for example, people stayed up late drinking coffee until 12:00 at night, and all the users went to bed. At this time, we began to write programs, write a temporary program for the lost data, check it out bit by bit, and then re-inject it into mq to make up for the data lost during the day. It's the best we can do.

Suppose there is a backlog of 10,000 orders in mq, which is not processed, and 1000 of them are lost. You can only write a manual program to check out the 1000 orders and manually send them to mq to make up again.

Mq is almost full.

What if there is a backlog of messages in mq and you haven't disposed of it for a long time, resulting in almost full mq? Is there any other way to do this? No, who asked you to implement the first program too slowly? you write programs temporarily, access data to consume, consume one by one, discard one by one, and quickly consume all the messages. Then take the second plan and fill in the data in the evening.

The above is how to solve the delay and expiration of RabbtiMQ message queues. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.