Shulou(Shulou.com)06/03 Report--

Problems in the online environment

Version: kafka-0.10.2.1

Phenomenon:

2017-11-29/14:45:02.937/CST WARN [kafka.utils.Logging$class.warn (Logging.scala:89)]: Failed to send producer request with correlation id 5 to broker 5 with data for partitions [mm-recommend-dev,2]

Java.io.IOException: broken pipe

At

Sun.nio.ch.FileDispatcherImpl.writev0 (Native Method)

At

Sun.nio.ch.SocketDispatcher.writev (SocketDispatcher.java:51)

At

Sun.nio.ch.IOUtil.write (IOUtil.java:148)

At

Sun.nio.ch.SocketChannelImpl.write (SocketChannelImpl.java:524)

At java.nio.channels.SocketChannel.write (SocketChannel.java:493)

At kafka.network.BoundedByteBufferSend.writeTo (BoundedByteBufferSend.scala:56)

At kafka.network.Send$class.writeCompletely (Transmission.scala:75)

At kafka.network.BoundedByteBufferSend.writeCompletely (BoundedByteBufferSend.scala:26)

At kafka.network.BlockingChannel.send (BlockingChannel.scala:92)

At kafka.producer.SyncProducer.liftedTree1 $1 (SyncProducer.scala:72)

At kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend (SyncProducer.scala:71)

At kafka.producer.SyncProducer$$anonfun$send$1 $$anonfun$apply$mcV$sp$1.apply$mcV$sp (SyncProducer.scala:102)

At kafka.producer.SyncProducer$$anonfun$send$1 $$anonfun$apply$mcV$sp$1.apply (SyncProducer.scala:102)

At kafka.producer.SyncProducer$$anonfun$send$1 $$anonfun$apply$mcV$sp$1.apply (SyncProducer.scala:102)

At kafka.metrics.KafkaTimer.time (KafkaTimer.scala:33)

At kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$sp (SyncProducer.scala:101)

Solution:

The configuration file for modifying the server.property of kafka broker is as follows:

# increase the maximum value of each message

Message.max.bytes defaults to 1000000 byte

After adjustment:

Message.max.bytes=5252880

Replica.fetch.max.bytes=6252880

Request.timeout.ms=600000

[@ zw ~] # kafka-topics.sh-alter-topic mm-recommend-dev-config max.message.bytes=5252880-zookeeper ip:2181/kafka-0.10.2.1-test

WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases.

Going forward, please use kafka-configs.sh for this functionality

Updated config for topic "mm-recommend-dev".

A new way

Bin/kafka-configs.sh-zookeeper localhost:2181-entity-type-alter-topic mm-recommend-dev-config max.message.bytes=5252880

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.