快捷導(dǎo)航

解決kafka:org.apache.kafka.common.errors.TimeoutException問(wèn)題

更新時(shí)間：2024年01月15日 09:55:54 作者：小明同學(xué)YYDS

這篇文章主要介紹了解決kafka:org.apache.kafka.common.errors.TimeoutException問(wèn)題,具有很好的參考價(jià)值,希望對(duì)大家有所幫助,如有錯(cuò)誤或未考慮完全的地方,望不吝賜教

問(wèn)題重現(xiàn)

java 生產(chǎn)者向kafka集群生產(chǎn)消息，報(bào)錯(cuò)如下：

2018-06-03 00:10:02.071 INFO 80 --- [nio-8080-exec-1] o.a.kafka.common.utils.AppInfoParser : Kafka version : 0.10.2.0
2018-06-03 00:10:02.071 INFO 80 --- [nio-8080-exec-1] o.a.kafka.common.utils.AppInfoParser : Kafka commitId : 576d93a8dc0cf421
2018-06-03 00:10:32.253 ERROR 80 --- [ad | producer-1] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='test1' and payload='hello122' to topic t2:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for t2-0: 30042 ms has passed since batch creation plus linger time

明顯連接kafka cluster超時(shí)，但是實(shí)際上并不知道具體是什么原因。

排除找原因

確定kafka集群已經(jīng)啟動(dòng)，包括zookeeper、kafka集群。

可以通過(guò)命令

ps -ef | grep java

個(gè)人排查，kafka集群即zookeeper已經(jīng)成功啟動(dòng)

確定生產(chǎn)者程序一方的kafka配置是否有誤。

spring boot的配置如下

spring.kafka.bootstrap-servers=39.108.61.252:9092
spring.kafka.consumer.group-id=springboot-group1
spring.kafka.consumer.auto-offset-reset=earliest

自然也沒(méi)有問(wèn)題

確定kafka集群所在機(jī)器防火墻或者說(shuō)安全組是否已經(jīng)開(kāi)放相關(guān)端口。

若是在windows上，打開(kāi)telnet工具可以查看是否端口開(kāi)放被監(jiān)聽(tīng)

cmd進(jìn)入命令行

telnet 39.108.61.252 9092

執(zhí)行后成功連接，說(shuō)明問(wèn)題在程序所在的一方。

打開(kāi)debug日志級(jí)別看錯(cuò)誤詳細(xì)信息

spring boot 的設(shè)置方式是：

logging.level.root=debug

然后重啟應(yīng)用

發(fā)現(xiàn)后臺(tái)在不停的刷錯(cuò)誤

如下：

2018-06-03 00:22:37.703 DEBUG 5972 --- [ t1-0-C-1] org.apache.kafka.clients.NetworkClient : Initialize connection to node 0 for sending metadata request
2018-06-03 00:22:37.703 DEBUG 5972 --- [ t1-0-C-1] org.apache.kafka.clients.NetworkClient : Initiating connection to node 0 at izwz9c79fdwp9sb65vpyk3z:9092.
2018-06-03 00:22:37.703 DEBUG 5972 --- [ t1-0-C-1] org.apache.kafka.clients.NetworkClient : Error connecting to node 0 at izwz9c79fdwp9sb65vpyk3z:9092:

java.io.IOException: Can't resolve address: izwz9c79fdwp9sb65vpyk3z:9092
at org.apache.kafka.common.network.Selector.connect(Selector.java:182) ~[kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:629) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:57) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:768) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:684) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:347) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:226) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:203) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitMetadataUpdate(ConsumerNetworkClient.java:138) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:216) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:193) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:275) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1030) [kafka-clients-0.10.2.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995) [kafka-clients-0.10.2.0.jar:na]
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:558) [spring-kafka-1.2.2.RELEASE.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_144]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
Caused by: java.nio.channels.UnresolvedAddressException: null
at sun.nio.ch.Net.checkAddress(Net.java:101) ~[na:1.8.0_144]
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) ~[na:1.8.0_144]
at org.apache.kafka.common.network.Selector.connect(Selector.java:179) ~[kafka-clients-0.10.2.0.jar:na]
... 17 common frames omitted

可知建立socket時(shí)不能解析kafka所在服務(wù)器地址

查看日志可知，解析的地址是izwz9c79fdwp9sb65vpyk3z

這個(gè)地址是遠(yuǎn)程服務(wù)器的實(shí)例名稱（阿里云服務(wù)器）。

自己配置的明明是ip，程序內(nèi)部卻去獲取他的別名，那如果生產(chǎn)者所在機(jī)器上沒(méi)有配置這個(gè)ip的別名，就不能解析到對(duì)應(yīng)的ip，所以連接失敗報(bào)錯(cuò)。

解決

windows則去添加一條host映射

C:\Windows\System32\drivers\etc\hosts

39.108.61.252 izwz9c79fdwp9sb65vpyk3z
127.0.0.1 localhost

linux則

vi /etc/hosts 修改方式一致

此時(shí)重啟生產(chǎn)者應(yīng)用

日志如下部分，成功啟動(dòng)，后臺(tái)會(huì)一直在心跳檢測(cè)連接和更新offset，所以debug日志一直在刷，此時(shí)就可以把日志級(jí)別修改為info

2018-06-03 00:29:46.543 DEBUG 12772 --- [ t2-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : Group springboot-group1 committed offset 10 for partition t2-0
2018-06-03 00:29:46.543 DEBUG 12772 --- [ t2-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : Completed auto-commit of offsets {t2-0=OffsetAndMetadata{offset=10, metadata=''}} for group springboot-group1
2018-06-03 00:29:46.563 DEBUG 12772 --- [ t1-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : Group springboot-group1 committed offset 0 for partition t1-0
2018-06-03 00:29:46.563 DEBUG 12772 --- [ t1-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : Completed auto-commit of offsets {t1-0=OffsetAndMetadata{offset=0, metadata=''}} for group springboot-group1
2018-06-03 00:29:46.672 DEBUG 12772 --- [ t2-0-C-1] o.a.k.c.consumer.internals.Fetcher : Sending fetch for partitions [t2-0] to broker izwz9c79fdwp9sb65vpyk3z:9092 (id: 0 rack: null)
2018-06-03 00:29:46.867 DEBUG 12772 --- [ t1-0-C-1] essageListenerContainer$ListenerConsumer : Received: 0 records
2018-06-03 00:29:46.872 DEBUG 12772 --- [ t2-0-C-1] essageListenerContainer$ListenerConsumer : Received: 0 records