KafkaJS status

[jesuslopezxing]

Hi,

Just wondering what the current status of KafkaJS is? It’s an important and widely-used library in the javascript community, but the last release was over two years ago and there haven’t been many updates since.

Is the project still actively maintained or are there any plans for future updates?

Thanks!

[saimonmoore]

Would very much like to know as well...

[saimonmoore]

Nm...I saw it's in maintenance mode

[leppaott]

Migrate to https://github.com/confluentinc/confluent-kafka-javascript but it's still a bit "beta".

[Charly-xepelin]

https://github.com/confluentinc/confluent-kafka-javascript is in version 1.3.1, seems to be productive now.

[AlbertoCalvoRubio]

Also worth trying from the Platformic team and Matteo Collina: https://blog.platformatic.dev/why-we-created-another-kafka-client-for-nodejs

[Dhruv-Garg79]

https://github.com/confluentinc/confluent-kafka-javascript
Have too many issues. The biggest drawback, which makes it unusable in production, is the maximum batch size of 32. It's hardcoded and doesn't follow kafka batching configuration.

[Dhruv-Garg79]

https://blog.platformatic.dev/why-we-created-another-kafka-client-for-nodejs

This looks promising, did anyone try it in production?

[AlbertoCalvoRubio]

https://blog.platformatic.dev/why-we-created-another-kafka-client-for-nodejs

This looks promising, did anyone try it in production?

I'm still testing https://github.com/confluentinc/confluent-kafka-javascript and https://github.com/platformatic/kafka. I'll let you know about the migration.
For now, none of them are supported out of the box by Instana https://www.ibm.com/docs/en/instana-observability/1.0.297?topic=nodejs-supported-libraries-frameworks#messaging

[Dhruv-Garg79]

@AlbertoCalvoRubio I've tried both extensively, and as of now, I would go with https://github.com/confluentinc/confluent-kafka-javascript. However, I think @platformic/kafka will be better in the future. During my benchmarks, I saw that the Confluent library is better in terms of consumer performance, but slower on the producer side when compared to @platformic/kafka. Nevertheless, that doesn't really matter. It seems more battle-tested as of now, and that was more important.

[siblznbitz]

Hey @Dhruv-Garg79 ! Would you mind sharing a bit about how you conducted your test? I'm on the Platformatic team and we've been unable to reproduce those results. Thanks!

@AlbertoCalvoRubio , is compatibility with Instana a hard requirement for your Kafka client?

[Dhruv-Garg79]

@siblznbitz I used the platformic benchmark module itself, and shared my results in this PR as well - platformatic/kafka#59.

Faced some dependency collision issues as well at the start with v1.4.0, so had to make work with what was available at the moment. i.e. confluent-kafka-javascript. Granted, the issue was quickly fixed by maintainers within a few days.

@platformic/kafka is a great project, and I am sure that I will be trying it again after few more releases.

[shahinghasemi]

I've switched to @platformatic/kafka a while ago. So far everything seems smooth. I think in the long term it's the sane choice.

[joebowbeer]

A more recent blog from Platformatic that mentions Confluent's client

https://blog.platformatic.dev/how-we-made-platformatickafka-223-faster-and-what-we-learned-along-the-way

[joebowbeer]

@Dhruv-Garg79 wrote:

https://github.com/confluentinc/confluent-kafka-javascript Have too many issues. The biggest drawback, which makes it unusable in production, is the maximum batch size of 32. It's hardcoded and doesn't follow kafka batching configuration.

Configurable batch size will be released soon in 1.6.1 and 1.7.0

confluentinc/confluent-kafka-javascript#393