top of page
Search

Kakfa part 1: Confluent VS Strimzi open source

Updated: Mar 30




You already know Confluent provides an advanced Kafka platform—complete with robust governance tools (Schema Registry, Enterprise Security), streaming and query functionality (ksqlDB), and a user-friendly Control Center UI. It’s a powerful, enterprise-grade lamborghini solution.


However, if you don’t need the full Confluent suite or you’re looking for ways to reduce costs while running Kafka on Kubernetes, Strimzi might be a great alternative. Below, we’ll explore why Strimzi could be a better fit for certain environments and how Stacktic can help automate many of the manual tasks that come with open-source Kafka. I won’t go deep into every feature—Confluent/ Strimzi’s docs have you covered. Instead, I’ll zoom out and share a few personal insights on topicing, sinking, and automation this great complexity. I’ve also created a Stacktic-generated sample repo to highlight these efforts—deliberately avoiding Confluent plugins and sticking to pure Apache Camel connectors for fairness. https://github.com/stackticio/strimzi_basic_setup.git


Why Consider Strimzi Over Confluent?


Why Confluent?

  • Small Teams with Minimal Traffic: If you’re a small team paying around $2–$3K per month for light Kafka usage, Confluent’s simplicity and support can be worth it—even if you’re not leveraging every premium feature.

  • Mission-Critical Kafka: When Kafka is at the core of your application and advanced capabilities (e.g., Schema Registry, RBAC, ksqlDB, Control Center) are essential, Confluent’s integrated ecosystem is tough to beat.


Why Strimzi?

  • Cost Savings: If you only need one or two connectors on a single cluster but still find yourself paying thousands each month, switching to an open-source solution like Strimzi could cut costs significantly—provided you have the in-house skills to manage it.

  • Enterprise Scale with In-House Expertise: For organizations spending hundreds of thousands on Kafka, Strimzi can be very attractive if you don’t need every single Confluent feature and your team is comfortable running open-source software.


From the official Strimzi web site
From the official Strimzi web site


 Strimzi Configuration Overhead

Strimzi is powerful, but it can require a substantial amount of YAML configuration:


  • Defining Kafka clusters, Connect workers, topics, and connectors

  • Managing secrets, environment variables, and storage

  • Integrating monitoring tools like Prometheus and Grafana

  • Handling more complex integrations (for example, with Keycloak)

  • Managing versioning for Kafka and KafkaConnect images ,migrations from zookeper to kraft, etc...



Is it really possible to spin up Strimzi in just minutes and then operate and version it with only a few clicks?

Stacktic employs a relationship-based automation framework to streamline operating Kafka on Kubernetes. With Stacktic, creating a single link between components triggers an entire automation process  for example, linking a topic to a connector:




  • KafkaConnect Creation: Sets up the KafkaConnect resource and includes any required JARs or plugins

  • Secrets & Environment Variables: Automates the creation and management of authentication, secrets, and necessary environment variables:

  • Topic Definitions: Ensures that all required topics exist or are referenced correctly for each connector.

  • Metrics & Dashboards: If Prometheus and Grafana are in place, Stacktic automatically generates PodMonitors, ConfigMaps, and dashboard JSON files for connector metrics.

  • mTLS & User Configuration: Applies the relevant certificates and user credentials, securing your setup from the outset.

Examples:


In essence, you define what you want (e.g., “Connect this topic to that connector”), and Stacktic takes care of how it’s done, requiring only minimal configuration on your end and leaving for you to Kustomize.


You’ll notice that this repository has two branches: stacktic and main.

  • stacktic: Contains the skeleton for apps or setups (like this one) for Kafka.

  • main: This is where you apply your customizations.


We’ve developed a merge strategy so you can easily pull in updates from the stacktic branch (the skeleton) without disrupting any of the custom changes you’ve made on main. This approach strikes a balance between staying up-to-date with the latest setup improvements and retaining full freedom to tailor your configuration.





What Else we can Do?

Simplified Ops:  Stacktic sets up a structured repository with distinct operational layers. We start with the Strimzi Helm chart to create a base YAML, then apply Kustomize for automated management. Our approach also converts ConfigMaps into secrets as needed and produces dynamic documentation that references your actual namespaces, topics, and domains—eliminating guesswork and stale references.


Version Management:  Versioning can be a significant challenge in Kafka environments, especially when transitioning from Zookeeper to Kraft or performing major version upgrades. Stacktic integrates versioning directly into its framework. You simply select the new Kafka version, and Stacktic creates the necessary resources under a fresh version behind the scenes, making upgrades smoother and less error-prone. We’ll go into more detail on our versioning mechanism in a future post.


MOVE FROM V1.1 (Zookeeper) to v1.2(Kraft)
MOVE FROM V1.1 (Zookeeper) to v1.2(Kraft)

Short Demo:




The beauty of running open-source software on Kubernetes is that you gain access to a rich ecosystem of complementary services, each capable of enhancing and extending your platform. We’re exploring additional integrations—let us know what you’d like to see!OPA (Open Policy Agent) + Strimzi


  • Policy-Driven Kafka: Use OPA’s Rego policies to govern Strimzi resources (clusters, topics, connectors). For instance, OPA can enforce naming conventions, TLS requirements, and resource quotas per namespace—automatically rejecting anything that doesn’t comply.


  • Multi-Tenant Governance: Streamline ownership by letting each team operate in their own namespace, confident that OPA will uphold organizational rules.


KEDA (Kubernetes Event-Driven Autoscaling) + Strimzi

  • Lag-Aware Scaling: KEDA monitors Kafka consumer lag to autoscale Connect workers or MirrorMaker 2 pods. When message volume spikes, more replicas spin up; once the backlog clears, the cluster scales down—saving costs.

  • Brokers-on-Demand: In advanced use cases, KEDA can also help (with careful partition balancing) to scale broker pods, aligning resources with actual load.


Use your imagination—when it comes to open-source on Kubernetes, everything is on the table!


Summary


Our overarching goal is to democratize open-source complexity by making outstanding projects like Strimzi easier to adopt, ultimately strengthening the community that supports them. 


We welcome your input—tell us what needs improvement, or suggest features you’d like to see automated. community@stacktic.io

 feel free to explore our sample repository for a ready-to-use baseline that can help jump-start your Strimzi journey. its just a basic example but good start point.

https://github.com/stackticio/strimzi_basic_setup.git By simplifying Strimzi and integrating it with the many innovative services in the Kubernetes ecosystem, you can build a strong offering that truly resonates—one that gets the popularity it deserves. last thought. could Confluent’s deeper involvement with Flink shines a brighter spotlight on advanced stream processing in the Kafka ecosystem— could this spark a new wave of Strimzi-Flink synergy on Kubernetes? Coming Soon:

  • Part 2 – Show Me the Money: A deep dive into cost savings and ROI with open-source Kafka VS managed.

  • Part 3 – Production Power: How to run Strimzi on Kubernetes without breaking a sweat.



 
 
 

Comentarios

bottom of page