Record the client request timeout analysis process

Description of request processing process

The server uses the framework to load the business so, and there is no business so to create the collaborative process, and the response is given after the processing is completed. The so entry function receives (const Type&in, Type& out,Type & extend). The prerequisite for giving a response is that there must be the processing result of the business so in the out, that is, the out cannot be empty. The reason for this is that some clients only care about whether the message is received by the server and do not care about the processing result. At this time, the server directly sends a response to tell the client to receive the message after receiving the request, while the server business so will not send the response to the same request again if the out is empty after processing the message.

The framework calls the service so pseudo code:

Try

{

Loadso

Proc_func=get_symbol: entry function

Proc_func (in,out,extend)

}

Catch (...)

{

Printf "unkow exception"

}

Problem points:

20% of the current network request message of the business times out, and there is "unkow exception" in the server framework log. However, because the framework has to handle a lot of business, the framework log is public, and it is not possible to confirm whether "unkow exception" is caused by the timeout request.

Adjust the framework log and the business so level to debug, and the framework calls the business so to output two interface logs (req,ans)

Analysis.

a. Suspect is a bad request, the framework failed to parse the request. According to the keywords in the request, check the API log and find that there is only req, but no ans. You can confirm that the framework has received the request, indicating that there is no problem with the network (packet capture can be carried out at this step, and there is an 80% success rate), and the request is fine.

b. The business so has the running log of the request. It is found that the request will not be processed further after running to a specific line, and no core file is generated. You can confirm that the problem does not lie in the framework. Combined with the exception log generated by the framework, it indicates that an exception has occurred in the business so.

c. From the last debug log point of the business so output, the exception must be generated later; line-by-line code analysis focuses on the function that may throw an exception (the old driver can see it at a glance, cow), and finally locked in string str.substr (pos). The function prototype is

String substr (size_t pos=0,size_t len=npos) const

If this is greater than the string length, it throws out_of_range.

Summary

a. If you find a problem in the current network, first report to the direct leader that you are dealing with it, and the leader can help coordinate resources and assess the impact.

b. Open the log level to debug, and all logs should be opened. (the interface log was not printed at first, so the framework did not output req. It is not sure whether the framework did not receive the message.)

c. Determine the last few lines of code for request processing. Analyze line by line. (experienced drivers can see the problem code lines at a glance.)

d. It is recommended that the framework output the details of the standard exception .e.what (), catch (exception e) {} catch (...) {} now that the framework has caught the exception.

e. Business so uses local objects of guard class to determine whether out is empty in the destructor to determine whether it is abnormal exit, and to ensure that the log can be output when business so exits due to business exception. At the same time, resource release and recovery can also be carried out in guard to prevent resource leakage.