China
Africa
Explore chapters and articles related to this topic
Middleware
Published in Chandrasekar Vuppalapati, Building Enterprise IoT Applications, 2019
Apache Kafka is a distributed streaming platform. A Streaming platform has the following characteristics (see Figure 4) [1, 2]: Publish and Subscribe to stream of records. For instance, a central message topic publishes restful services to be consumed by clients. In our case, Hanumayamma Dairy IoT Sensor uses Kafka to publish real-time events.Persistent of Stream records in a fault-tolerant durable way.Process stream of records as they occur.
Building Permissioned Blockchain Networks Using Hyperledger Fabric
Published in Sandeep Kumar Panda, Ahmed A. Elngar, Valentina Emilia Balas, Mohammed Kayed, Bitcoin and Blockchain, 2020
K. Varaprasada Rao, Mutyala Sree Teja, P. Praneeth Reddy, S. Saikrishna
Most of the distributed blockchains, which are permissionless, permit anyone to participate in the network consensus, where the transactions are bundled and ordered into block. Hyperledger Fabric works differently as explained in the previous sections, that it introduced a new architecture called order-execute-validate. It contains a module known as the ordering node also known as orderer, which performs ordering of transactions. A Fabric ordering service accepts transactions that have been endorsed, orders them into a block, and delivers the blocks to committing peers. It does not maintain a ledger or chaincode with the sole purpose of ordering transactions into a block and distributing the block to all peers in the network. This is considered an ordering service. Because Fabric relies on deterministic algorithms of consensus, it is difficult to fork the ledger. Hyperledger Fabric offers three ordering mechanisms: SOLO: It is most widely used by programmers working with Hyperledger Fabric networks that are the Hyperledger Fabric ordering process. SOLO requires a single command node.Kafka: Recommended for production use is the Hyperledger Fabric ordering system. This mechanism uses Apache Kafka, an open-source stream processing platform that provides a unified low-latency, high-throughput platform for handling data feeds in real time. The data consists of approved transactions and RW sets in this case. A crash-tolerant solution for ordering is given by the Kafka method.SBFT: It stands for Simplified Byzantine Fault Tolerance. The ordering system is both crash-tolerant and BFT, which ensures that even in the presence of malicious or faulty nodes it can reach agreement. This method has not yet been adopted by the Hyperledger Fabric team, as it is under development.
Role of Open Source, Standards, and Public Clouds in Autonomous Networks
Published in Mazin Gilbert, Artificial Intelligence for Autonomous Networks, 2018
According to its homepage: “Apache Kafka is used for building real-time data pipelines and streaming applications. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies…” [41].
An open source approach to the design and implementation of Digital Twins for Smart Manufacturing
Published in International Journal of Computer Integrated Manufacturing, 2019
Violeta Damjanovic-Behrendt, Wernher Behrendt
In situations when large amounts of incoming data are expected, Apache Kafka is often used as an event store engine that maintains ordered sequences of entries, allowing multiple consumers to pull in the data and process it. Dobbelaere and Esmaili (2017) perform a qualitative and quantitative comparison of the common features of Apache Kafka vs. RabbitMQ, two popular open-source and commercially supported pub/sub messaging systems. The best suited use cases for Apache Kafka are those implementing pub/sub-messaging with simple routing logic; scalable ingestion systems enabling high throughput processing of stored data; capturing change feeds and stream processing (with Kafka Streams). RabbitMQ is often used as pub/sub-messaging with complex routing logic, or for operational metrics tracking for real-time processing.
Applying big data and stream processing to the real estate domain
Published in Behaviour & Information Technology, 2019
Herminio García-González, Daniel Fernández-Álvarez, José Emilio Labra-Gayo, Patricia Ordóñez de Pablos
Python Transformer: This is responsible for bringing the data from Twitter real-time API to the event queue middleware. Transformations are made here and then transferred to the event queue.Apache Kafka: Kafka is a distributed event queue that aims to connect different sources and consumers of information through a distributed and fault tolerant system.Scala API: As we mentioned with the batch layer, Scala API will collect real-time events and expose different aggregated views to clients.