{"id":8427,"date":"2021-11-24T08:16:21","date_gmt":"2021-11-24T08:16:21","guid":{"rendered":"https:\/\/berg-software.com\/?p=8427"},"modified":"2025-09-02T14:44:53","modified_gmt":"2025-09-02T14:44:53","slug":"apache-kafka-synchronized-communications","status":"publish","type":"post","link":"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/","title":{"rendered":"Apache Kafka hands-on: synchronized communications between platforms"},"content":{"rendered":"<p>[et_pb_section fb_built=&#8221;1&#8243; fullwidth=&#8221;on&#8221; _builder_version=&#8221;4.4.1&#8243;][et_pb_fullwidth_post_title meta=&#8221;off&#8221; featured_placement=&#8221;background&#8221; _builder_version=&#8221;4.7.7&#8243; title_font=&#8221;||||||||&#8221; title_text_color=&#8221;#ff6317&#8243; title_font_size=&#8221;3.5em&#8221; meta_font=&#8221;|300|||||||&#8221; meta_text_color=&#8221;#ffffff&#8221; meta_font_size=&#8221;1em&#8221; background_enable_color=&#8221;off&#8221; use_background_color_gradient=&#8221;on&#8221; background_color_gradient_start=&#8221;rgba(248,248,248,0.85)&#8221; background_color_gradient_end=&#8221;rgba(248,248,248,0.75)&#8221; background_color_gradient_overlays_image=&#8221;on&#8221; min_height=&#8221;20vh&#8221; height=&#8221;400px&#8221; custom_padding=&#8221;6vh||6vh||false|false&#8221; global_module=&#8221;403&#8243; locked=&#8221;off&#8221;][\/et_pb_fullwidth_post_title][\/et_pb_section][et_pb_section fb_built=&#8221;1&#8243; admin_label=&#8221;intro&#8221; _builder_version=&#8221;4.4.6&#8243; custom_padding=&#8221;|||0px||&#8221; locked=&#8221;off&#8221;][et_pb_row use_custom_gutter=&#8221;on&#8221; _builder_version=&#8221;4.4.6&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;6vh||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h2>Intro<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; hover_enabled=&#8221;0&#8243; border_color_left=&#8221;#ff6317&#8243; sticky_enabled=&#8221;0&#8243;]<\/p>\n<p>How can you improve your data flows and processes with Apache Kafka?<\/p>\n<p>If you\u2019re in a hybrid online\/offline business, possibly spread across multiple platforms, then probably one of your most significant challenges is the complexity of your software ecosystem. Managing multiple software solutions coming from different providers and with independent lifecycles can have a significant impact on how (and whether) your business functions.<\/p>\n<p>In this case, you might need solutions to simplify your ecosystem, either by reducing the number of its components or by better integrating\/synchronizing them.<\/p>\n<p>In this article, we discuss how we optimized online\/offline data flows and processes with <a href=\"https:\/\/kafka.apache.org\" target=\"_blank\" rel=\"noopener noreferrer\">Apache Kafka<\/a>.<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=&#8221;on&#8221; _builder_version=&#8221;4.4.6&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;6vh||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h2>Basic intro to Kafka<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>Kafka is a data store optimized for ingesting and processing streaming data in real-time. With data that\u2019s generated and sent at the same time by thousands of data sources, Kafka can handle the influx and process it sequentially and incrementally.<\/p>\n<p>It can be distributed and is highly scalable, able to support trillions of messages per day.<\/p>\n<p>Very important: Kafka is open-source and community-driven, therefore free to install.<\/p>\n<p>(Disclaimer: Kafka appliances are huge, and we hereby cover only a quick hands-on, suited for a Kafka introduction and for a quick practical solution to a relatively simple business case.)<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=&#8221;on&#8221; _builder_version=&#8221;4.4.6&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;6vh||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h2>The business situation<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>The client is a mixed business in the fashion industry, with both an online store and a vast network of physical shops. The online store was powered by its own software infrastructure (a customized e-shop platform), while the brick-and-mortar stores were running custom-made cashbox software.<\/p>\n<p>Of course, the two systems need to be synchronized. For example, the value vouchers have to be recognized\/accepted across the whole business, regardless of where they are purchased.<\/p>\n<p>Using direct communications between the two systems\/services is not the optimal solution since it needs both services to always be available at the same time \u2013 which cannot be guaranteed.<\/p>\n<p>Enter Apache Kafka: it helps us build a message-based communication system between the two platforms that run even if one of the platforms is temporarily unavailable\/inaccessible.<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=&#8221;on&#8221; _builder_version=&#8221;4.4.6&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;6vh||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h2>Installing Kafka, hands-on<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>For a quick start, Kafka is available in three ways:<\/p>\n<ul>\n<li>A local installation<\/li>\n<li>A local installation using Docker<\/li>\n<li>Kafka as a service, on cloud<\/li>\n<\/ul>\n<p>Since we went through all of them, here\u2019s our quick take.<\/p>\n<p>(Please note: should you need heavy-duty Kafka, and you have the technical expertise, there are other options, too \u2013 just check the managed Kafka instances commercially available on all major cloud providers.)<\/p>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Local installation of Kafka<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>The installation steps are quite simple:<\/p>\n<ul>\n<li>Make sure you have Java (at least version 8) on the system.<\/li>\n<li>Download two applications\/packages: Kafka is a bundle of two applications (the Broker and the Zookeeper).<\/li>\n<li>Start them individually via the provided scripts.<\/li>\n<li>Use other provided scripts to administer Kafka, but also to test the installation by creating a test topic and by producing and consuming test messages.<\/li>\n<\/ul>\n<p>Here is a detailed description of the process: <a href=\"https:\/\/kafka.apache.org\/quickstart\">https:\/\/kafka.apache.org\/quickstart<\/a>.<\/p>\n<p>Our client\u2019s infrastructure is based on a group of on-premises Windows-2012 servers, so we decided to install Kafka on one of them. We did get some early warnings about Kafka on Windows, but since the installation process looked fairly simple, we just went along with it and gave it a try.<\/p>\n<p>Everything went smoothly, and in a few minutes, we were able to see Kafka in action. Soon after, we started implementing our particular producers and consumers, and it was almost too good to be true.<\/p>\n<p>However, 24 hours later, the Kafka Broker crashed, complaining about some locked files. Restarting the Broker \u201cfixed\u201d the issue, but a few hours later, the same error appeared.<\/p>\n<p>After some research and attempts to find a proper solution (more about the error here: <a href=\"https:\/\/github.com\/apache\/kafka\/pull\/6329\">https:\/\/github.com\/apache\/kafka\/pull\/6329<\/a>), we decided to move on to a different option.<\/p>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Cloud Kafka<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>We then searched for a Kafka service on the cloud. We selected Cloudkarafka for two main reasons: it has a free tier for testing and development, and it has accessible commercial plans once you decide to continue with it.<\/p>\n<p>We just registered for a trial version and instantly got all the connection details we need to start using it.<\/p>\n<p>Migrating the code from one Kafka (locally installed) to another (on Cloud) was just a matter of changing configurations, considering we didn\u2019t yet have any actual events to migrate.<\/p>\n<p>Here are the pros of Kafka as a cloud service:<\/p>\n<ul>\n<li>No installation time and no hassle, so it\u2019s available right away<\/li>\n<li>No worries about maintenance and configuration<\/li>\n<li>Security is configured by default<\/li>\n<\/ul>\n<p>&#8230;And the cons:<\/p>\n<ul>\n<li>It does cost a monthly fee that depends on the data speed and volume processed through Kafka.<\/li>\n<li>For European companies, the data must be stored inside the EU, and not all Cloud providers can guarantee that. Cloudkarafka does, though.)<\/li>\n<\/ul>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Linux installation of Kafka using Docker<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||25px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>Although the Kafka cloud solution was good enough for our business purposes, we still wanted to have Kafka as a local installation.<\/p>\n<p>Since the Windows server was not an option anymore, we prepared a new Ubuntu server for that purpose. Here are the steps needed to have Kafka on it:<\/p>\n<ul>\n<li>Make sure that you have docker and docker-compose available on the system<\/li>\n<li>Prepare a docker-compose.yml file (there are plenty of default configurations for that).<\/li>\n<li>Fire up the docker images using the common docker-compose commands.<\/li>\n<li>You will still have the admin scripts to test your installation, but since all is running inside a docker, you will need to fire them:<\/li>\n<\/ul>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;|300|||||||&#8221; text_font_size=&#8221;1em&#8221; text_line_height=&#8221;1.3em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;1em&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; background_color=&#8221;#f8f8f8&#8243; custom_margin=&#8221;||25px|20px|false|false&#8221; custom_padding=&#8221;||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221; border_style_left=&#8221;none&#8221; locked=&#8221;off&#8221;]<\/p>\n<pre class=\"text\" style=\"font-family: monospace;\">docker exec -it &lt;container-name&gt; &lt;command&gt;<\/pre>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>(Please note: for a detailed description of such a process, you can check the \u201cSingle Node setup\u201d section from <a href=\"https:\/\/www.baeldung.com\/ops\/kafka-docker-setup)\">https:\/\/www.baeldung.com\/ops\/kafka-docker-setup<\/a>).<\/p>\n<p>This time, the installation was as stable as it can be, and we eventually used it as our final solution that went into production.<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=&#8221;on&#8221; _builder_version=&#8221;4.4.6&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;6vh||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h2>Using Kafka \u2013 bare minimum<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>When it comes to actually use Kafka, the three most important terms are:<\/p>\n<ul>\n<li>Topic: a category used to organize messages; each topic will have its own queue of messages<\/li>\n<li>Producer: a client application that produces messages on a certain topic<\/li>\n<li>Consumer: a client application that subscribes to messages coming from a certain topic<\/li>\n<\/ul>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Topics<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||25px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>Creating topics can be done via the admin scripts mentioned above. For our first particular business case, we created two topics named test.voucher and prod.voucher.<\/p>\n<p>We have a convention to name the topics by the particular environment (prefix) followed by the business event (suffix). Here is the command to create a topic:<\/p>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;|300|||||||&#8221; text_font_size=&#8221;1em&#8221; text_line_height=&#8221;1.3em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;1em&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; background_color=&#8221;#f8f8f8&#8243; custom_margin=&#8221;||35px|20px|false|false&#8221; custom_padding=&#8221;||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221; border_style_left=&#8221;none&#8221; locked=&#8221;off&#8221;]<\/p>\n<pre class=\"text\" style=\"font-family: monospace;\">kafka-topics.sh --create --topic test.voucher --bootstrap-server localhost:9092<\/pre>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Producer and Consumer<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>Kafka\u2019s community offers support to write producers and consumers in all major languages. The exact indication and clear code snippets can be found here: <a href=\"https:\/\/docs.confluent.io\/clients-confluent-kafka-dotnet\/current\/overview.html\">https:\/\/docs.confluent.io\/clients-confluent-kafka-dotnet\/current\/overview.html<\/a><\/p>\n<p>To map this to our simple business case, a workflow will look like this:<\/p>\n<ol>\n<li>A voucher is bought at the physical store<\/li>\n<li>A dedicated Producer is activated to produce a message with all the details about the voucher purchase, then pushed to a certain Topic.<\/li>\n<li>A dedicated Consumer, acting on behalf of the Web-shop and listening to that Topic, will \u201ccatch\u201d that message and notify the Web-shop to register the voucher into its own system.<\/li>\n<\/ol>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=&#8221;on&#8221; _builder_version=&#8221;4.4.6&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;6vh||||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h2>Kafka best practices, the Berg Software edition<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Kafka REST proxy<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>The code that implements the producers and consumers is better kept as a separate microservice, acting like a Kafka REST proxy. This proxy will stay between your business services and the Kafka instance.<\/p>\n<ul>\n<li>When the business decides to broadcast a message, it will simply fire a REST call to the Kafka proxy, which will then use the dedicated Producer to actually push that message to the right topic.<\/li>\n<li>When the business decides that it\u2019s interested in a particular message, it will register for it with a REST endpoint at the Kafka proxy, which will then instruct a Consumer to call that endpoint when a message arrives at the right topic.<\/li>\n<\/ul>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Retry and fallback strategy<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||35px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<ul>\n<li>Use the Commit mechanism to ensure that messages are properly processed. A failure during processing will also fail to commit that message, and so it will stay in the queue.<\/li>\n<li>An alternative to the above technique is to use a dedicated topic for messages that failed to be processed. They will be inserted into this topic and processed by a different consumer.<\/li>\n<\/ul>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||15px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h3>Monitoring<\/h3>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||25px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<ul>\n<li>Kafka comes without any graphical user interface. You will need some viewing and administration tools beyond the barebones command interface.<\/li>\n<li>Use monitoring tools like Confluent Control Center, Prometheus &amp; Grafana, or Conduktor, to have a clear, instant view of your Kafka instance.<\/li>\n<\/ul>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row use_custom_gutter=&#8221;on&#8221; _builder_version=&#8221;4.7.7&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;6vh||0px||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;2em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||10px||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<h2>Conclusions<\/h2>\n<p>[\/et_pb_text][et_pb_text _builder_version=&#8221;4.7.7&#8243; text_font=&#8221;||||||||&#8221; text_font_size=&#8221;1.1em&#8221; text_line_height=&#8221;1.6em&#8221; quote_font=&#8221;|700|||||||&#8221; quote_text_align=&#8221;left&#8221; quote_font_size=&#8221;16px&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;1.5em&#8221; header_2_line_height=&#8221;0.9em&#8221; header_3_font_size=&#8221;23px&#8221; header_4_font=&#8221;||||||||&#8221; header_4_font_size=&#8221;16px&#8221; header_4_line_height=&#8221;1.5em&#8221; header_5_font_size=&#8221;14px&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;||||false|false&#8221; border_color_left=&#8221;#ff6317&#8243;]<\/p>\n<p>With the use of Apache Kafka, we were able to solve the main issue on hand \u2013 i.e., the synchronization of various, not-directly-connected parts of the client\u2019s software ecosystem.<\/p>\n<p>But then, we reached other, equally important benefits, such as:<\/p>\n<ul>\n<li>ecosystem resilience,<\/li>\n<li>flexibility,<\/li>\n<li>preservation of all important data;<\/li>\n<li>even when one part of the system is down for some time, the communication is still functional eventually.<\/li>\n<\/ul>\n<p>At Berg Software, <a href=\"https:\/\/www.berg-software.com\/en\/contact-berg-software\/\">we turn business ideas into software<\/a> \u2013 and this Apache Kafka case is a straightforward example of how we deploy software solutions to support and streamline clients\u2019 businesses.<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=&#8221;1&#8243; _builder_version=&#8221;4.4.1&#8243; custom_padding=&#8221;|||0px||&#8221; locked=&#8221;off&#8221;][et_pb_row use_custom_gutter=&#8221;on&#8221; gutter_width=&#8221;3&#8243; admin_label=&#8221;\uff3f&#8221; _builder_version=&#8221;4.4.6&#8243; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;4vh||8vh||false|false&#8221; border_color_left=&#8221;rgba(0,0,0,0)&#8221;][et_pb_column type=&#8221;4_4&#8243; _builder_version=&#8221;4.4.6&#8243;][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=&#8221;1&#8243; _builder_version=&#8221;3.22&#8243; background_color=&#8221;#eeeeee&#8221; custom_padding=&#8221;50px||50px||false|false&#8221; border_color_top=&#8221;#ff6317&#8243; global_module=&#8221;1642&#8243;][et_pb_row column_structure=&#8221;1_3,1_3,1_3&#8243; _builder_version=&#8221;4.4.1&#8243; custom_padding=&#8221;0px|||||&#8221; locked=&#8221;off&#8221;][et_pb_column type=&#8221;1_3&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_social_media_follow _builder_version=&#8221;4.4.4&#8243; text_orientation=&#8221;left&#8221;][et_pb_social_media_follow_network social_network=&#8221;linkedin&#8221; url=&#8221;https:\/\/www.linkedin.com\/company\/berg-computers-srl\/&#8221; _builder_version=&#8221;4.4.4&#8243; background_color=&#8221;#007bb6&#8243; follow_button=&#8221;off&#8221; url_new_window=&#8221;on&#8221;]linkedin[\/et_pb_social_media_follow_network][et_pb_social_media_follow_network social_network=&#8221;twitter&#8221; url=&#8221;https:\/\/twitter.com\/berg_software&#8221; _builder_version=&#8221;4.4.4&#8243; background_color=&#8221;#00aced&#8221; follow_button=&#8221;off&#8221; url_new_window=&#8221;on&#8221;]twitter[\/et_pb_social_media_follow_network][et_pb_social_media_follow_network social_network=&#8221;facebook&#8221; url=&#8221;https:\/\/www.facebook.com\/bergCOMPUTERS&#8221; _builder_version=&#8221;4.4.4&#8243; background_color=&#8221;#3b5998&#8243; follow_button=&#8221;off&#8221; url_new_window=&#8221;on&#8221;]facebook[\/et_pb_social_media_follow_network][et_pb_social_media_follow_network social_network=&#8221;instagram&#8221; url=&#8221;https:\/\/www.instagram.com\/berg_software\/&#8221; _builder_version=&#8221;4.4.4&#8243; background_color=&#8221;#ea2c59&#8243; follow_button=&#8221;off&#8221; url_new_window=&#8221;on&#8221;]instagram[\/et_pb_social_media_follow_network][\/et_pb_social_media_follow][\/et_pb_column][et_pb_column type=&#8221;1_3&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_post_nav in_same_term=&#8221;on&#8221; show_next=&#8221;off&#8221; _builder_version=&#8221;4.4.1&#8243; title_text_color=&#8221;#ff6317&#8243; custom_padding=&#8221;|25px|||false|false&#8221;][\/et_pb_post_nav][\/et_pb_column][et_pb_column type=&#8221;1_3&#8243; _builder_version=&#8221;4.4.1&#8243;][et_pb_post_nav in_same_term=&#8221;on&#8221; show_prev=&#8221;off&#8221; _builder_version=&#8221;4.4.1&#8243; title_text_color=&#8221;#ff6317&#8243; custom_padding=&#8221;|||25px|false|false&#8221;][\/et_pb_post_nav][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=&#8221;1&#8243; admin_label=&#8221;CONTACT&#8221; _builder_version=&#8221;4.4.1&#8243; background_color=&#8221;#d2d2d2&#8243; custom_padding=&#8221;75px||75px||false|false&#8221; global_module=&#8221;1544&#8243;][et_pb_row column_structure=&#8221;1_4,3_4&#8243; admin_label=&#8221;Service Section Title&#8221; _builder_version=&#8221;4.4.1&#8243; custom_padding=&#8221;||25px||false|false&#8221; animation_direction=&#8221;top&#8221; locked=&#8221;off&#8221;][et_pb_column type=&#8221;1_4&#8243; _builder_version=&#8221;3.25&#8243; custom_padding=&#8221;|||&#8221; custom_padding__hover=&#8221;|||&#8221;][\/et_pb_column][et_pb_column type=&#8221;3_4&#8243; _builder_version=&#8221;3.25&#8243; custom_padding=&#8221;|||&#8221; custom_padding__hover=&#8221;|||&#8221;][et_pb_text _builder_version=&#8221;4.4.1&#8243; text_font=&#8221;|300|||||||&#8221; text_text_color=&#8221;#ffffff&#8221; text_line_height=&#8221;1.1em&#8221; header_5_font=&#8221;|600|||||||&#8221; header_5_text_color=&#8221;#ffffff&#8221; header_5_font_size=&#8221;14px&#8221; header_5_line_height=&#8221;1.5em&#8221; custom_margin=&#8221;||||false|false&#8221;]29 years in business | 2700 software projects | 760 clients | 24 countries<\/p>\n<h5>We turn ideas into software. What is yours?<\/h5>\n<p>[\/et_pb_text][et_pb_text admin_label=&#8221;Title&#8221; _builder_version=&#8221;4.4.6&#8243; header_text_align=&#8221;center&#8221; header_2_font=&#8221;|300|||||||&#8221; header_2_text_align=&#8221;left&#8221; header_2_text_color=&#8221;#ff6317&#8243; header_2_font_size=&#8221;50px&#8221; header_2_line_height=&#8221;0.9em&#8221; custom_margin=&#8221;||||false|false&#8221; custom_padding=&#8221;25px||25px||false|false&#8221;]<\/p>\n<h2 id=\"Getintouch\">Get in touch<\/h2>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=&#8221;1_4,3_4&#8243; _builder_version=&#8221;4.4.4&#8243;][et_pb_column type=&#8221;1_4&#8243; _builder_version=&#8221;4.4.4&#8243;][\/et_pb_column][et_pb_column type=&#8221;3_4&#8243; _builder_version=&#8221;4.4.4&#8243;][et_pb_contact_form email=&#8221;contact@berg-software.com&#8221; custom_message=&#8221;WEBFORM MESSAGE||et_pb_line_break_holder||\uff3f||et_pb_line_break_holder||FROM: %%Name%%||et_pb_line_break_holder||EMAIL: %%Email%%||et_pb_line_break_holder||PHONE NUMBER: %%Phone_number%%||et_pb_line_break_holder||COMPANY: %%Company_name%%||et_pb_line_break_holder||TERMS &amp; CONDITIONS: %%Terms_and_Conditions%%||et_pb_line_break_holder||DATA PRIVACY POLICY: %%Data_Privacy_Policy%%||et_pb_line_break_holder||\uff3f||et_pb_line_break_holder||MESSAGE:||et_pb_line_break_holder||%%Message%%&#8221; success_message=&#8221;Thank you for reaching out! Your message was sent. We will get back to you right away.&#8221; _builder_version=&#8221;4.7.7&#8243; form_field_background_color=&#8221;#d2d2d2&#8243; form_field_text_color=&#8221;#ffffff&#8221; form_field_focus_background_color=&#8221;#ffffff&#8221; form_field_focus_text_color=&#8221;#000000&#8243; title_level=&#8221;h2&#8243; title_font=&#8221;|300|||||||&#8221; title_text_color=&#8221;#ff6317&#8243; title_font_size=&#8221;30px&#8221; form_field_line_height=&#8221;1.5em&#8221; custom_button=&#8221;on&#8221; button_text_size=&#8221;14px&#8221; button_text_color=&#8221;#ffffff&#8221; button_bg_color=&#8221;#ff6317&#8243; button_border_width=&#8221;0px&#8221; button_border_radius=&#8221;0px&#8221; button_icon=&#8221;%%3%%&#8221; button_on_hover=&#8221;off&#8221; border_color_all=&#8221;#ffffff&#8221; border_width_bottom=&#8221;1px&#8221;][et_pb_contact_field field_id=&#8221;Name&#8221; field_title=&#8221;Name *&#8221; fullwidth_field=&#8221;on&#8221; _builder_version=&#8221;4.4.4&#8243; form_field_background_color=&#8221;#d2d2d2&#8243; form_field_text_color=&#8221;#ffffff&#8221; form_field_focus_background_color=&#8221;#ffffff&#8221; form_field_focus_text_color=&#8221;#000000&#8243; border_width_bottom=&#8221;1px&#8221; button_text_size__hover_enabled=&#8221;off&#8221; button_one_text_size__hover_enabled=&#8221;off&#8221; button_two_text_size__hover_enabled=&#8221;off&#8221; button_text_color__hover_enabled=&#8221;off&#8221; button_one_text_color__hover_enabled=&#8221;off&#8221; button_two_text_color__hover_enabled=&#8221;off&#8221; button_border_width__hover_enabled=&#8221;off&#8221; button_one_border_width__hover_enabled=&#8221;off&#8221; button_two_border_width__hover_enabled=&#8221;off&#8221; button_border_color__hover_enabled=&#8221;off&#8221; button_one_border_color__hover_enabled=&#8221;off&#8221; button_two_border_color__hover_enabled=&#8221;off&#8221; button_border_radius__hover_enabled=&#8221;off&#8221; button_one_border_radius__hover_enabled=&#8221;off&#8221; button_two_border_radius__hover_enabled=&#8221;off&#8221; button_letter_spacing__hover_enabled=&#8221;off&#8221; button_one_letter_spacing__hover_enabled=&#8221;off&#8221; button_two_letter_spacing__hover_enabled=&#8221;off&#8221; button_bg_color__hover_enabled=&#8221;off&#8221; button_one_bg_color__hover_enabled=&#8221;off&#8221; button_two_bg_color__hover_enabled=&#8221;off&#8221;][\/et_pb_contact_field][et_pb_contact_field field_id=&#8221;Email&#8221; field_title=&#8221;Email address *&#8221; field_type=&#8221;email&#8221; fullwidth_field=&#8221;on&#8221; _builder_version=&#8221;4.4.4&#8243; button_text_size__hover_enabled=&#8221;off&#8221; button_one_text_size__hover_enabled=&#8221;off&#8221; button_two_text_size__hover_enabled=&#8221;off&#8221; button_text_color__hover_enabled=&#8221;off&#8221; button_one_text_color__hover_enabled=&#8221;off&#8221; button_two_text_color__hover_enabled=&#8221;off&#8221; button_border_width__hover_enabled=&#8221;off&#8221; button_one_border_width__hover_enabled=&#8221;off&#8221; button_two_border_width__hover_enabled=&#8221;off&#8221; button_border_color__hover_enabled=&#8221;off&#8221; button_one_border_color__hover_enabled=&#8221;off&#8221; button_two_border_color__hover_enabled=&#8221;off&#8221; button_border_radius__hover_enabled=&#8221;off&#8221; button_one_border_radius__hover_enabled=&#8221;off&#8221; button_two_border_radius__hover_enabled=&#8221;off&#8221; button_letter_spacing__hover_enabled=&#8221;off&#8221; button_one_letter_spacing__hover_enabled=&#8221;off&#8221; button_two_letter_spacing__hover_enabled=&#8221;off&#8221; button_bg_color__hover_enabled=&#8221;off&#8221; button_one_bg_color__hover_enabled=&#8221;off&#8221; button_two_bg_color__hover_enabled=&#8221;off&#8221;][\/et_pb_contact_field][et_pb_contact_field field_id=&#8221;Phone_number&#8221; field_title=&#8221;Phone number&#8221; required_mark=&#8221;off&#8221; fullwidth_field=&#8221;on&#8221; _builder_version=&#8221;4.4.4&#8243;][\/et_pb_contact_field][et_pb_contact_field field_id=&#8221;Company_name&#8221; field_title=&#8221;Company name&#8221; fullwidth_field=&#8221;on&#8221; _builder_version=&#8221;4.4.4&#8243;][\/et_pb_contact_field][et_pb_contact_field field_id=&#8221;Message&#8221; field_title=&#8221;Message *&#8221; field_type=&#8221;text&#8221; fullwidth_field=&#8221;on&#8221; _builder_version=&#8221;4.4.4&#8243; form_field_background_color=&#8221;#d2d2d2&#8243; form_field_focus_background_color=&#8221;#ffffff&#8221; form_field_focus_text_color=&#8221;#000000&#8243; border_width_bottom=&#8221;1px&#8221; button_text_size__hover_enabled=&#8221;off&#8221; button_one_text_size__hover_enabled=&#8221;off&#8221; button_two_text_size__hover_enabled=&#8221;off&#8221; button_text_color__hover_enabled=&#8221;off&#8221; button_one_text_color__hover_enabled=&#8221;off&#8221; button_two_text_color__hover_enabled=&#8221;off&#8221; button_border_width__hover_enabled=&#8221;off&#8221; button_one_border_width__hover_enabled=&#8221;off&#8221; button_two_border_width__hover_enabled=&#8221;off&#8221; button_border_color__hover_enabled=&#8221;off&#8221; button_one_border_color__hover_enabled=&#8221;off&#8221; button_two_border_color__hover_enabled=&#8221;off&#8221; button_border_radius__hover_enabled=&#8221;off&#8221; button_one_border_radius__hover_enabled=&#8221;off&#8221; button_two_border_radius__hover_enabled=&#8221;off&#8221; button_letter_spacing__hover_enabled=&#8221;off&#8221; button_one_letter_spacing__hover_enabled=&#8221;off&#8221; button_two_letter_spacing__hover_enabled=&#8221;off&#8221; button_bg_color__hover_enabled=&#8221;off&#8221; button_one_bg_color__hover_enabled=&#8221;off&#8221; button_two_bg_color__hover_enabled=&#8221;off&#8221;][\/et_pb_contact_field][et_pb_contact_field field_id=&#8221;Terms_and_Conditions&#8221; field_title=&#8221; &#8221; field_type=&#8221;checkbox&#8221; checkbox_options=&#8221;%91{%22value%22:%22I have read and accepted the Terms and Conditions%22,%22checked%22:0,%22dragID%22:-1}%93&#8243; fullwidth_field=&#8221;on&#8221; _builder_version=&#8221;4.4.4&#8243;][\/et_pb_contact_field][et_pb_contact_field field_id=&#8221;Data_Privacy_Policy&#8221; field_title=&#8221; &#8221; field_type=&#8221;checkbox&#8221; checkbox_options=&#8221;%91{%22value%22:%22I have read and accepted the Data Privacy Policy%22,%22checked%22:0,%22dragID%22:-1}%93&#8243; fullwidth_field=&#8221;on&#8221; _builder_version=&#8221;4.4.4&#8243;][\/et_pb_contact_field][\/et_pb_contact_form][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We discuss the optimization of online\/offline data flows and processes with Apache Kafka.<\/p>\n","protected":false},"author":5,"featured_media":8428,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"2880","footnotes":""},"categories":[138],"tags":[],"class_list":["post-8427","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-how-to"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v16.1.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Apache Kafka hands-on: synchronized communications | Berg Software<\/title>\n<meta name=\"description\" content=\"How can you improve your data flows and processes with Apache Kafka? If you\u2019re in a hybrid online\/offline business, possibly spread across multiple\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Apache Kafka hands-on: synchronized communications | Berg Software\" \/>\n<meta property=\"og:description\" content=\"How can you improve your data flows and processes with Apache Kafka? If you\u2019re in a hybrid online\/offline business, possibly spread across multiple\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/\" \/>\n<meta property=\"og:site_name\" content=\"Berg Software\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/bergCOMPUTERS\/\" \/>\n<meta property=\"article:published_time\" content=\"2021-11-24T08:16:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-09-02T14:44:53+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.berg-software.com\/wp-content\/uploads\/Berg-Software-Kafka-hands-on-Adi-Nitu.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@berg_software\" \/>\n<meta name=\"twitter:site\" content=\"@berg_software\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data1\" content=\"11 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.berg-software.com\/de\/#organization\",\"name\":\"Berg Software\",\"url\":\"https:\/\/www.berg-software.com\/de\/\",\"sameAs\":[\"https:\/\/www.facebook.com\/bergCOMPUTERS\/\",\"https:\/\/www.instagram.com\/berg_software\/\",\"https:\/\/www.linkedin.com\/company\/berg-computers-srl\/\",\"https:\/\/www.youtube.com\/channel\/UCw1FfcRJnC-CoKPwlcM10Iw\",\"https:\/\/twitter.com\/berg_software\"],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/www.berg-software.com\/de\/#logo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/berg-software.com\/wp-content\/uploads\/berg-software-logo.png\",\"contentUrl\":\"https:\/\/berg-software.com\/wp-content\/uploads\/berg-software-logo.png\",\"width\":512,\"height\":512,\"caption\":\"Berg Software\"},\"image\":{\"@id\":\"https:\/\/www.berg-software.com\/de\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.berg-software.com\/de\/#website\",\"url\":\"https:\/\/www.berg-software.com\/de\/\",\"name\":\"Berg Software\",\"description\":\"We turn ideas into software.\",\"publisher\":{\"@id\":\"https:\/\/www.berg-software.com\/de\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/www.berg-software.com\/de\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/www.berg-software.com\/wp-content\/uploads\/Berg-Software-Kafka-hands-on-Adi-Nitu.jpg\",\"contentUrl\":\"https:\/\/www.berg-software.com\/wp-content\/uploads\/Berg-Software-Kafka-hands-on-Adi-Nitu.jpg\",\"width\":1200,\"height\":600,\"caption\":\"Berg Software - Kafka hands on\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#webpage\",\"url\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/\",\"name\":\"Apache Kafka hands-on: synchronized communications | Berg Software\",\"isPartOf\":{\"@id\":\"https:\/\/www.berg-software.com\/de\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#primaryimage\"},\"datePublished\":\"2021-11-24T08:16:21+00:00\",\"dateModified\":\"2025-09-02T14:44:53+00:00\",\"description\":\"How can you improve your data flows and processes with Apache Kafka? If you\\u2019re in a hybrid online\/offline business, possibly spread across multiple\",\"breadcrumb\":{\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.berg-software.com\/en\/\",\"url\":\"https:\/\/www.berg-software.com\/en\/\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"position\":2,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.berg-software.com\/en\/category\/how-to\/\",\"url\":\"https:\/\/www.berg-software.com\/en\/category\/how-to\/\",\"name\":\"How to\"}},{\"@type\":\"ListItem\",\"position\":3,\"item\":{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/\",\"url\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/\",\"name\":\"Apache Kafka hands-on: synchronized communications between platforms\"}}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#webpage\"},\"author\":{\"@id\":\"https:\/\/www.berg-software.com\/de\/#\/schema\/person\/348c86c1bd957c7a27a124f4189a0491\"},\"headline\":\"Apache Kafka hands-on: synchronized communications between platforms\",\"datePublished\":\"2021-11-24T08:16:21+00:00\",\"dateModified\":\"2025-09-02T14:44:53+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#webpage\"},\"publisher\":{\"@id\":\"https:\/\/www.berg-software.com\/de\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.berg-software.com\/en\/apache-kafka-synchronized-communications\/#primaryimage\"},\"articleSection\":\"How to\",\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.berg-software.com\/de\/#\/schema\/person\/348c86c1bd957c7a27a124f4189a0491\",\"name\":\"Adrian Nitu\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","_links":{"self":[{"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/posts\/8427","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/comments?post=8427"}],"version-history":[{"count":0,"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/posts\/8427\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/media\/8428"}],"wp:attachment":[{"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/media?parent=8427"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/categories?post=8427"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.berg-software.com\/en\/wp-json\/wp\/v2\/tags?post=8427"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}