The Apache Kafka C/C++ library

Перейти к файлу

Magnus Edenhill 034187d9da Fix Cygwin compilation, again		2014-02-01 13:25:34 +01:00
examples	…
tests	…
.dir-locals.el	…
.gitignore	…
.travis.yml	…
CONFIGURATION.md	…
INTRODUCTION.md	…
LICENSE	…
LICENSE.pycrc	…
LICENSE.snappy	…
Makefile	…
README.md	…
endian_compat.h	…
librdkafka.lds	…
rd.c	…
rd.h	…
rdaddr.c	…
rdaddr.h	…
rdcrc32.c	…
rdcrc32.h	…
rdevent.h	…
rdgz.c	…
rdgz.h	…
rdkafka.c	…
rdkafka.h	…
rdkafka_broker.c	…
rdkafka_broker.h	…
rdkafka_defaultconf.c	…
rdkafka_int.h	…
rdkafka_msg.c	…
rdkafka_msg.h	…
rdkafka_offset.c	…
rdkafka_offset.h	…
rdkafka_proto.h	…
rdkafka_timer.c	…
rdkafka_timer.h	…
rdkafka_topic.c	…
rdkafka_topic.h	…
rdlog.c	…
rdlog.h	…
rdqueue.c	…
rdqueue.h	…
rdrand.c	…
rdrand.h	…
rdsignal.h	…
rdsysqueue.h	…
rdthread.c	…
rdthread.h	…
rdtime.h	…
rdtypes.h	…
snappy.c	…
snappy.h	…
snappy_compat.h	…

README.md

librdkafka - Apache Kafka C client library

https://github.com/edenhill/librdkafka

librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. It was designed with message delivery reliability and high performance in mind, current figures exceed 800000 msgs/second for the producer and 3 million msgs/second for the consumer.

librdkafka is licensed under the 2-clause BSD license.

For an introduction to the performance and usage of librdkafka, see INTRODUCTION.md

NOTE: A fully fledged C++ interface will be available end of January 2014.

Apache Kafka 0.8 support:

Branch: master
Producer: supported
Consumer: supported
Compression: snappy and gzip
Debian package: librdkafka1 and librdkafka-dev in Debian testing and unstable
ZooKeeper: not supported
API: Stable, not backwards compatible
Tests: Regression tests in tests/ directory.
Statistics: JSON formatted, see rd_kafka_conf_set_stats_cb in rdkafka.h.
Status: Stable

Apache Kafka 0.7 support:

Branch: 0.7
Producer: supported
Consumer: supported
Compression: not supported
ZooKeeper: not supported
API: backwards compatible with 0.6
Status: Stable

Apache Kafka 0.6 support:

Branch: 0.6
Producer: supported
Consumer: supported
Compression: not supported
ZooKeeper: not supported
Status: Testing

#Users of librdkafka#

Wikimedia's varnishkafka
redBorder
Headweb
node-kafka
Produban's log2kafka
phpkafka
Let me know if you are using librdkafka

Usage

Requirements

The GNU toolchain
pthreads
zlib

Instructions

Building

  make all
  sudo make install
  # or to install in another location than /usr/local, set DESTDIR env
  # to the filesystem root of your choice.
  sudo make DESTDIR=/usr make install

Usage in code

See examples/rdkafka_example.c for an example producer and consumer.

Link your program with -lrdkafka -lz -lpthread -lrt.

Documentation

The API is documented in rdkafka.h

Configuration properties are documented in CONFIGURATION.md

For a librdkafka introduction, see INTRODUCTION.md

Examples

See the examples/sub-directory.

Tests

See the tests/sub-directory.

Support

File bug reports, feature requests and questions using GitHub Issues

Questions and discussions are also welcome on irc.freenode.org, #apache-kafka, nickname Snaps.

Commercial support

Commercial support is available from Edenhill services