Results 11 to 20 of about 48,436 (195)
In-transit molecular dynamics analysis with Apache flink [PDF]
Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, 2018 In this paper, an on-line parallel analytics framework is proposed to process and store in transit all the data being generated by a Molecular Dynamics (MD) simulation run using staging nodes in the same cluster executing the simulation. The implementation and deployment of such a parallel workflow with standard HPC tools, managing problems such as ...
Henrique C. Zanúz +3 more
openaire +5 more sources
Predictive topology refinements in distributed stream processing system. [PDF]
PLoS ONE, 2020 Cloud computing has evolved the big data technologies to a consolidated paradigm with SPaaS (Streaming processing-as-a-service). With a number of enterprises offering cloud-based solutions to end-users and other small enterprises, there has been a boom ...
Muhammad Hanif, Choonhwa Lee, Sumi Helal
doaj +2 more sources
Comparative Study of Record Linkage Approaches for Big Data
Walailak Journal of Science and Technology, 2021 Record linkage is a challenging task for Big Data. This paper, hence, attempts to shed light on record linkage approaches for Big Data by comparing three dimensions involving record linkage phases, dataset properties, and parallel processing approach ...
Randa MOHAMED +3 more
doaj +3 more sources
Optimization for Large-Scale Dimension Table Connection Technology in Distributed Environment [PDF]
Jisuanji kexue yu tansuo, 2022 The large-scale dimension table connection technology in the distributed environment is one of the key technologies in online big data analysis, which is widely used in real-time recommendation, real-time analysis and other fields.
ZHAO Hengtai, ZHAO Yuhai, YUAN Ye, JI Hangxu, QIAO Baiyou, WANG Guoren
doaj +1 more source
GeoFlink: An Efficient and Scalable Spatial Data Stream Management System
IEEE Access, 2022 This era is witnessing an exponential growth in spatial data due to the increase in GPS-enabled devices. Spatial data can be of extreme use to commercial businesses, governments and NGOs if processed timely.
Salman Ahmed Shaikh +4 more
doaj +1 more source
VeilGraph: incremental graph stream processing
Journal of Big Data, 2022 Graphs are found in a plethora of domains, including online social networks, the World Wide Web and the study of epidemics, to name a few. With the advent of greater volumes of information and the need for continuously updated results under temporal ...
Miguel E. Coimbra +3 more
doaj +1 more source
Towards autoscaling of Apache Flink jobs [PDF]
Acta Universitatis Sapientiae, Informatica, 2021 Abstract Data stream processing has been gaining attention in the past decade. Apache Flink is an open-source distributed stream processing engine that is able to process a large amount of data in real time with low latency. Computations are distributed among a cluster of nodes.
Varga Balázs +2 more
openaire +2 more sources
s2p: Provenance Research for Stream Processing System
Applied Sciences, 2021 The main purpose of our provenance research for DSP (distributed stream processing) systems is to analyze abnormal results. Provenance for these systems is not nontrivial because of the ephemerality of stream data and instant data processing mode in ...
Qian Ye, Minyan Lu
doaj +1 more source
Influencing Factors in the Scalability of Distributed Stream Processing Jobs
IEEE Access, 2021 More and more use cases require fast, accurate, and reliable processing of large volumes of data. To do this, a distributed stream processing framework is needed which can distribute the load over several machines.
Giselle Van Dongen, Dirk Van Den Poel
doaj +1 more source
A Performance Analysis of Fault Recovery in Stream Processing Frameworks
IEEE Access, 2021 Distributed stream processing frameworks have gained widespread adoption in the last decade because they abstract away the complexity of parallel processing. One of their key features is built-in fault tolerance.
Giselle van Dongen, Dirk Van Den Poel
doaj +1 more source