Kafka use cases

Kafka use cases

In short, Kafka gets used for stream processing, website activity tracking, metrics collection and monitoring, log aggregation, real-time analytics, CEP, ingesting data into Spark, ingesting data into Hadoop, CQRS, replay messages, error recovery, and guaranteed distributed commit log for in-memory computing (microservices).

Comments

Post a Comment

Popular posts from this blog

Kafka: Streaming Architecture

Image
Kafka: Streaming Architecture Kafka gets used most often for real-time streaming of data into other systems. Kafka is a middle layer to decouple your real-time data pipelines. Kafka core is not good for direct computations such as data aggregations, or CEP.  Kafka Streaming  which is part of the Kafka ecosystem does provide the ability to do real-time analytics. Kafka can be used to feed fast lane systems (real-time, and operational data systems) like Storm, Flink, Spark Streaming and your services and CEP systems. Kafka is also used to stream data for batch data analysis. Kafka feeds Hadoop. It streams data into your BigData platform or into RDBMS, Cassandra, Spark, or even S3 for some future data analysis. These data stores often support data analysis, reporting, data science crunching, compliance auditing, and backups. Kafka Streaming Architecture Diagram
Read more

Who uses Kafka?

Who uses Kafka? A lot of large companies who handle a lot of data use Kafka. LinkedIn, where it originated, uses it to track activity data and operational metrics. Twitter uses it as part of Storm to provide a stream processing infrastructure. Square uses Kafka as a bus to move all system events to various Square data centers (logs, custom events, metrics, and so on), outputs to Splunk, Graphite (dashboards), and to implement an Esper-like/CEPalerting systems. It gets used by other companies too like Spotify, Uber, Tumbler, Goldman Sachs, PayPal, Box, Cisco, CloudFlare, NetFlix, and much more.
Read more

A systematic approach makes a programmatic reuse works

A systematic approach makes a programmatic reuse works: obstacles in Reuse process:: 1.Engineering 2.Process 3.Organizational 4.Business Oriented 1.Engineering:: Identify basic elements of model For Ex: Bank application::requirement tools identification reduces programmer to work on application area rather than design area. 2.Process:: continuation still there
Read more