China
Africa
Explore chapters and articles related to this topic
Grid Computing
Published in Praveen Kumar, Jay Alameda, Peter Bajcsy, Mike Folk, Momcilo Markus, Hydroinformatics: Data Integrative Approaches in Computation, Analysis, and Modeling, 2005
GridFTPrefers to a standardized protocol [11] for data transfer on the grid. Servers supporting gridFTPdata transfer have been developed by the Globus Project and NCSA; tools such as globus-url-copy (part of the default globus toolset) and uberftp [12] have been developed to interact with the gridftp servers to allow users to move their data as needed on the computational grid. Gridftp has supported performance controls, such as TCPbuffer size and the provision for parallel data streams, to allow for the movement of large volumes of scientific data. New versions of the Gridftp server will provide for striping of data transfers among a number of gridftp hosts that are connected in parallel to a parallel filesystem. This is necessary to take advantage of some of the emerging high bandwidth transcontinental networks that were deployed to support scientific computing, such as the network dedicated to TeraGrid [13]. Finally, it is possible to build advanced interactive clients such as in Figure 12.2 on top of toolkits such as the Java Commodity Grid to provide for easy data management by end-users.
FedIDS: a federated cloud storage architecture and satellite image delivery service for building dependable geospatial platforms
Published in International Journal of Digital Earth, 2018
J. L. Gonzalez-Compean, Victor J. Sosa-Sosa, Arturo Diaz-Perez, Jesus Carretero, Ricardo Marcelin-Jimenez
Transport protocols, inspired mainly by FTP, are a key component in the life cycle of satellite imagery. For instance, GridFTP (2017) (in union with Globus 2017 ) is a traditional large file transfer solution for GRID environments, which increases the throughput per I/O operation through the deployment of multiple concurrent data streams. FDT and mdtmFTP (Zhang et al. 2017) are protocols that also deploy multiple concurrent streams with acceptable performance in dedicated network scenarios (Zhang et al. 2017). In FedIDS, the connection drivers for the transportation of ensured images are exchangeable. This means that different connectors can be added to the IDS service, so administrators can be chosen from those best suitable to meet a given requirement of a specific partner. By now, IDS service only includes connection drivers for HTTP and HTTPS (bit-torrent drivers are under developing) to deliver images to the heterogeneous and multiple cloud resources in the federation. These protocols are quite suitable for computers of users to establish connections with minimal requirements avoiding dedicated networks and the management of certificates required by other protocols, which avoids being intercepted by firewalls. Nevertheless, in scenarios where a set of partners requiring increased throughput, having the resource availability (dedicated networks and an agreement for managing certificates for transportation) and the agreement to use a single file management as globus online (including reliability based on S3), the use of GridFTP/mdtmFTP would be an option for IDS service. In FedIDS, the management of files is relegated to local file systems to reduce requirements to be met by partners in the federation.