Functional Kafka

Juchli, Marc and Wolf, Lorenz (2015) Functional Kafka. Bachelor thesis, HSR Hochschule für Technik Rapperswil.

[thumbnail of Thesis_eprints_mjuchli_lwolf.pdf]
Thesis_eprints_mjuchli_lwolf.pdf - Supplemental Material

Download (2MB) | Preview


The aim of this thesis is to provide a summary of the current state of message oriented-middleware and eventually build a message broker in Haskell, adapted from the concepts of Apache Kafka which was originally built at LinkedIn. The implementation shall provide basic functionalities such as producing and consuming messages, with the aim to approximate performance of Apache Kafka in a non-clustered setup. The Apache Kafka Protocol is being used as the underlying wire-protocol and is implemented in a standalone library. On top of the procotol library, a separate client library is provided. Thus, the Haskell Message Broker (HMB) as well as its producer and consumer clients have been successfully proofed as compatible with Apache Kafka.
This thesis first examines the fundamental concepts behind messaging and discloses the needs for message brokers. In a second stage of this technology research, the purpose of event-streaming is described, containing a comparison of batch and stream processing by explaining the differences in their nature. Finally the concept and features of Apache Kafka is presented. Insights into the HMB implementation is provided in the technical report and is split into two stages. At first, the protocol and client library is introduced. Subsequently the broker implementation is explained including its capabilities as well as the provided set of features. After all, HMB is applied to a benchmark against Apache Kafka.
The results of this proof of concept show that Haskell is well suited to build messaging applications as well as implementing protocols based on context free grammars. The from HMB provided performance hit the one of Apache Kafka for transmission of larger message sizes during the benchmark. For the most tested scenarios the performance suffers as HMB is not sufficiently optimized yet. However, the Haskell Message Broker is a well established basis of a state-of-the-art message broker implementation. The authors recommend to apply further optimization techniques as well as extending the feature-set before any other use.

Item Type: Thesis (Bachelor)
Subjects: Topics > Software
Topics > Software > Performance
Topics > Software > Optimization
Topics > Communication Systems
Area of Application > Business oriented
Divisions: Bachelor of Science FHO in Informatik > Bachelor Thesis
Depositing User: HSR Deposit User
Thesis advisor
Joller, Josef M.
Date Deposited: 20 Oct 2015 07:50
Last Modified: 20 Oct 2015 07:50

Actions (login required)

View Item
View Item