How to solve the problems when using flume

This article mainly explains "how to solve the problems when using flume". The content of the explanation in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn how to solve the problems when using flume.

Question: java.lang.NullPointerException: Expected timestamp

Sink is the hdfs, and then uses the automatic directory generation feature. If there is an error such as the title, the official website document says that you need to have a timestamp at the beginning of each file record line, but the format of the timestamp may be difficult to adjust, so you can also set the parameter hdfs.useLocalTimeStamp, such as every hour as a folder, then the configuration should be like this:

A1.sinks.k1.hdfs.path = hdfs://ubuntu:9000/flume/events/%y-%m-%d/%Ha1.sinks.k1.hdfs.filePrefix = events-a1.sinks.k1.hdfs.round = truea1.sinks.k1.hdfs.roundValue = 1a1.sinks.k1.hdfs.roundUnit = houra1.sinks.k1.hdfs.useLocalTimeStamp = true

Or modify the parameter hdfs.timeZone so that it can be the same as the date format of the log file we uploaded. It should be fine. It has not been tested.

Thank you for your reading, the above is the content of "how to solve the problems when using flume". After the study of this article, I believe you have a deeper understanding of how to solve this problem when using flume, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!