China 
Africa 
Explore chapters and articles related to this topic
Big Data Stream Processing
Published in Vivek Kale, Parallel Computing Architectures and APIs, 2019
Historically, stream processing systems have been relegated to a somewhat niche market providing low-latency, inaccurate, or speculative results, often in conjunction with a more capable batch system to provide eventually correct results, i.e., Lambda architecture. The essential idea of Lambda architecture is that it runs a stream processing system alongside a batch system—both performing essentially the same calculation. The stream processing system gives a low-latency, inaccurate result (either because of the use of an approximation algorithm, or because the stream processing system itself does not provide correctness), and subsequently a separate batch system provides the correct output. Originally proposed by Twitter’s Nathan Marz (creator of Storm), Lambda architecture successfully addressed the need of reconciling: Stream processing engines challenged on the accuracy dimension.Batch engines which were inherently unwieldy.
Transition from Relational Database to Big Data and Analytics
Published in Mohiuddin Ahmed, Al-Sakib Khan Pathan, Data Analytics, 2018
Santoshi Kumari, C. Narendra Babu
The key features of the emerging database systems for advanced analytics have been a boundless motivation to take a look into big data processing frameworks, like Hadoop [11,12] and Spark. A thoughtful review on the development of new systems to overcome the drawbacks of traditional analytical methods for large data is taken into account. To develop a generalized system for processing large dataset, the Lambda Architecture [6] provides three layers of architecture structure, which in turn helps to understand the basic requirement for processing and analyzing large data in batch, stream, and real time.
Big data text mining in the financial sector
Published in Noura Metawa, Mohamed Elhoseny, Aboul Ella Hassanien, M. Kabir Hassan, Expert Systems in Finance, 2019
Mirjana Pejić Bach, Živko Krstić, Sanja Seljan
The Lambda architecture (Marz and Warren, 2015) is architecture that consists of three layers: batch, speed and serving. The batch layer is part of architecture where raw data or processed data can be stored. Arbitrary views are stored in the serving layer. The batch layer contains all data to recent hours. Because the batch layer is time-consuming, views stored in the serving layer for additional use do not contain fresh and new data (from the last hours depending on time of computation of the batch layer). New and fresh data is ingested with the support of the speed layer.
Applying big data and stream processing to the real estate domain
Published in Behaviour & Information Technology, 2019
Herminio García-González, Daniel Fernández-Álvarez, José Emilio Labra-Gayo, Patricia Ordóñez de Pablos
Our proposed architecture is a specialisation of Lambda architecture in the terms of: it is intended for big data, it has a batch layer and a streaming layer, and it combines real-time data with non-real-time data. However, Lambda architecture is meant, mainly, for getting information from the same source. While our one is intended, from the very beginning, to mix and integrate heterogeneous data sources. This is present not only in the existence of various data sources (see Figure 2) but also in the existence of an integrator component.