Explore chapters and articles related to this topic
Knowledge Discovery and Information Analysis from Remote Sensing Big Data
Published in Lizhe Wang, Jining Yan, Yan Ma, Cloud Computing in Remote Sensing, 2019
Lizhe Wang, Jining Yan, Yan Ma
As an increasing number of RS applications need to process or analyze the massive volume of RS data collections, the stand-alone mode processing can not satisfy the computation requirement. To process the large-scale RS data efficiently, we built a distributed executed engine using the dask – a distributed computing framework focusing on scientific data analysisdata analysis!dask. Compared with the popular distributed computing tools such as Apache Spark, dask supports the multidimensional data model natively and has a similar API with pandas and numpydata analysis!python. Therefore, it is more suitable for computing an N-Dimensional array. Similar to Spark, dask is also a master-slave system framework that consists of one schedule node and several work nodes. The schedule node is responsible for scheduling the tasks, while the work nodes are responsible for executing tasks. If all the tasks have being performed, these workers’ computation results would be reduced to the scheduler and the final result would be obtained.
p †q : a tool for prototyping many-body methods for quantum chemistry
Published in Molecular Physics, 2021
Nicholas C. Rubin, A. Eugene DePrince
An important part of computer algebra systems is the potential to translate equations into usable code. pq comes with a front-end parser that produces code for residual equations in a NumPy einsum convention. Appropriate summation limits, occupied or virtual indices, are enforced using NumPy array slicing. Using the NumPy einsum format allows a user to implement the generated code with any of a variety of einsum backends. For example, tensor contraction engines in Tensorflow [97], Jax [98], or Dask [99] can be used to implement coupled cluster iterations on a variety of different hardware platforms.
Urban path travel time estimation using GPS trajectories from high-sampling-rate ridesourcing services
Published in Journal of Intelligent Transportation Systems, 2022
To solve the Spatial Join problem, the Python Dask package was used. A Dask dataframe is a large parallel dataframe composed of many smaller dataframes split along the index. The route and OD data files contains around 54.2 million points, using Dask, the dataframe was divided up into small partitions for efficient calculations. The algorithm takes advantage of Dask’s map_partitions function to do a spatial join with the TAZ zones on every partition. The function was called to apply the spatial join to each of the routing points locations in the dataframes that make up the Dask DataFrame. At this point, the trips imported in Dask dataframe has a valid TAZ zone_id information.
Genetic Folding (GF) Algorithm with Minimal Kernel Operators to Predict Stroke Patients
Published in Applied Artificial Intelligence, 2022
The SVM kernels and proposed mGF model were developed as Python scripts integrated with the PGFPyLib toolbox and compared with (Mezher 2021). Dask technology (Rocklin 2015) and other Python frameworks such as NumPy and pandas developed in the toolbox. The open-source Python code PGFPyLib was developed to integrate the Stroke dataset in the toolkit source codes for prediction. The performance table for accuracy values and corresponding statistics (complexity, time, and AUC) were generated from the calculations on each generic kernel in the test data sets.