Results 41 to 50 of about 22,068 (223)

Diaspore: Diagnosing Performance Interference in Apache Spark

IEEE Access, 2021
Apache Spark is being increasingly used to execute big data applications on cluster computing platforms. To increase system utilization, cluster operators often configure their clusters such that multiple co-located applications can simultaneously share ...
Sarah Shah, Yasaman Amannejad, Diwakar Krishnamurthy   +2 more
doaj   +1 more source

Scientific Computing Meets Big Data Technology: An Astronomy Use Case

, 2015
Scientific analyses commonly compose multiple single-process programs into a dataflow. An end-to-end dataflow of single-process programs is known as a many-task application.
Barbary, Kyle, Franklin, Michael J., Nothaft, Frank Austin, Patterson, David A., Perlmutter, Saul, Sparks, Evan, Zahn, Oliver, Zhang, Zhao   +7 more
core   +2 more sources

Accelerating Large-Scale Data Analysis by Offloading to High-Performance Computing Libraries using Alchemist

, 2018
Apache Spark is a popular system aimed at the analysis of large data sets, but recent studies have shown that certain computations---in particular, many linear algebra computations that are the basis for solving common machine learning problems---are ...
Gerhardt, Lisa, Gittens, Alex, Kottalam, Jey, Mahoney, Michael W., Maschhoff, Kristyn, Prabhat, Ringenburg, Michael, Rothauge, Kai, Wang, Shusen   +8 more
core   +1 more source

Real-time Analysis of NetFlow Data for Generating Network Traffic Statistics using Apache Spark [PDF]

, 2016
—In this paper, we present a framework for the realtime generation of network traffic statistics on Apache Spark Streaming, a modern distributed stream processing system.
Jirsík Tomáš, Laštovička Martin, Čermák Milan   +2 more
core   +1 more source

Matrix Computations and Optimization in Apache Spark

, 2016
We describe matrix computations available in the cluster programming framework, Apache Spark. Out of the box, Spark provides abstractions and implementations for distributed matrices and optimization routines using these matrices. When translating single-
Meng, Xiangrui, Pu, Li, Sparks, Evan, Staple, Aaron, Ulanov, Alexander, Venkataraman, Shivaram, Yavuz, Burak, Zadeh, Reza Bosagh, Zaharia, Matei   +8 more
core   +1 more source

Model of Point Cloud Data Management System in Big Data Paradigm

ISPRS International Journal of Geo-Information, 2018
Modern geoinformation technologies for collecting and processing data, such as laser scanning or photogrammetry, can generate point clouds with billions of points. They provide abundant information that can be used for different types of analysis. Due to
Vladimir Pajić, Miro Govedarica, Mladen Amović   +2 more
doaj   +1 more source

ReForeSt: Random Forests in Apache Spark [PDF]

, 2017
Random Forests (RF) of tree classifiers are a popular ensemble method for classification. RF are usually preferred with respect to other classification techniques because of their limited hyperparameter sensitivity, high numerical robustness, native capacity of dealing with numerical and categorical features, and effectiveness in many real world ...
Lulli A., Oneto L., Anguita D.
openaire   +1 more source

Real-time high-throughput cotton phenotyping using distributed computing and deep learning

Smart Agricultural Technology
In this paper, we present an approach for real-time high-throughput cotton phenotyping using distributed computing and deep learning. The objective of this study is to develop a big data pipeline to efficiently ingest and process large amounts of image ...
Vaishnavi Thesma, Glen C. Rains, Javad Mohammadpour Velni   +2 more
doaj   +1 more source

Apache Spark Tabanlı Duygu Analizi

Osmaniye Korkut Ata Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 2021
Bu çalışmada, büyük verileri bellek içi hesaplama yöntemi ile hızlı bir şekilde işleyebilen Apache Spark açık kaynak kodlu çerçeve kullanılarak duygu analizi gerçekleştirilmiştir. Duygu analizi işleminde Spark içerisinde bulunan MLlib makine öğrenimi kütüphanesi kullanılmıştır.
Emre YILDIRIM, Ali ÇALHAN
openaire   +2 more sources

Major Cybersecurity Breaches: Shaping Corporate Cybersecurity Policies and Closing the Gaps

Journal of Corporate Accounting &Finance, EarlyView.
ABSTRACT As digitalization accelerates, cybercrime has intensified in both scale and impact over the past two decades. This study aims to critically examine major cybersecurity events, assess them through the lens of routine activity theory, examine insight from three other established criminological and organizational theories, and address central ...
Laura K. Rickett, Deborah Smith
wiley   +1 more source