Shulou(Shulou.com)06/03 Report--

Need to parse the log of httpd

107.106.8.237.145-[12/Feb/2020:15:09:52 + 0800] "GET / noindex/css/fonts/Light/OpenSans-Light.ttf HTTP/1.1" 404 240 "http://39.96.45.213/noindex/css/open-sans.css"" Mozilla/5.0 (Macintosh; Intel Mac OS X 10 / 12 / 6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36 "

Fluentd configuration

@ id httpd @ type tail path / data/logs/kim-todo-api-prod/log/statistics.log pos_file / home/web_server/ingress-nginx/statistics.log.pos tag * read_from_head true format / ^ (? [^] *) [^] * (? [^] *)\ [(? [^\]] *)\] "(?\ S+) (: + (? [^] *) +\ S*)? (? [ ^] *) (? [^] *) (?: "(? [^\"] *) "(? [^\"] *) ")? $/

The effect after analysis

1. Format is followed by a regular matching statement

2. () means to capture a packet. () will save the matching values in each packet. It is not difficult to see that there is a key in angle brackets, and the regular match in parentheses () matches the desired value.

3. (?:) represents a non-capture group. the only difference between a capture packet and a capture packet is that the matching value of the non-capture packet will not be saved. Here you will wonder that the characters in (?:) are still matched. There are several special uses here.

Non-capture arrays do not participate in number allocation but participate in matching

A: in the form of (?: (exp))

That is to say, grouping according to normal grouping logic in exp

B: shaped like (?: () | ())

Strictly speaking, writing like this is a way to solve the problem of choosing caching. (using parentheses directly has a side effect, which is that the relevant matches are cached, which is available at this time?: put before the first option to eliminate this side effect. )

C: like ((?: exp))

I think this writing is equivalent to (exp)

4. Parentheses [] outside parentheses remove some useless characters

5. The right slash\ is an escape character.

6.\ s means to match if it is not blank + means to match the previous subexpression one or more times

7. [^] * means to match all empty characters

8 、? Indicates a non-greedy match, that is, if a match reaches a match, ok will not continue to match.

9. The expression begins and ends with "indicates that what matches" is the content in "."

10, + means to match the previous subexpression one or more times

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.