China
Africa
Explore chapters and articles related to this topic
Trauma Outcome Prediction in the Era of Big Data: From Data Collection to Analytics
Published in Ervin Sejdić, Tiago H. Falk, Signal Processing and Machine Learning for Biomedical Big Data, 2018
Shiming Yang, Peter F. Hu, Colin F. Mackenzie
Storing, processing, and learning from massive medical data require intensive calculation. First, having a high-performance file format to store and organize large amounts of data is critical for the input and output (I/O) of data processing. For a typical TBI patient staying 7 days in a trauma center, five 240 Hz waveforms are monitored and up to 700 million data points (equivalent to 1-gigabyte disk size, if data are stored in 12-bit format) would be collected. Traditional spreadsheet-based data management becomes less efficient within such a Big Data scenario. Hierarchical Data Format (HDF) is a high-performance data format that offers on-the-fly data compression and high I/O performance. It also supports reading from or writing to a subset of a data set, without loading the entire data file into memory.
Coastal and Estuarine Waters: Optical Sensors and Remote Sensing
Published in Yeqiao Wang, Coastal and Marine Environments, 2020
Post-flight data processing is an extremely important process for removing atmospheric and seabed effects to convert the digital radiances into reflectance data. There are several commercially available software packages to process aircraft and satellite remote-sensing data such as ERDAS IMAGINE (Leica Geosystems GIS & Mapping) and ENVI (Research Systems International). Data are provided to a data user in a standard image processing format (e.g., ERDAS, ENVI, and GEOTIFF) or in a generic scientific data format such as HDF (Hierarchal Data Format). The HDF is an efficient structure for storing multiple sets of scientific, image, and ancillary data in a single data file. The data may be sent to a user as raw radiance files with no processing, image files with radiometric calibrations applied, or as radiometrically calibrated and atmospherically corrected digital image files geoferenced to a map projection. Complex atmospheric correction procedures and models [such as MODTRAN 4.0 (MODerate resolution TRANSsmittance)] are employed to compute the ocean color signal by determining the magnitude of and removing atmospheric scattering and absorption effects between the water surface and the sensor[13,15,16] However, a commonly used and simple approach is the “clear water pixel” or “dark pixel” subtraction technique, which assumes that the sensor has a spectral band for which clear water is essentially a black body (i.e., no reflectance). Therefore, any radiance measured by the instrument in this band is due to atmospheric backscatter and can be subtracted from all pixels in the image.[17,18] In the shallow waters of estuarine and coastal systems, the seabed reflects part of the incident light in a way that is highly dependent on the bottom material and roughness. The reflected light is spectrally different than that of deep water, which allows scientists to obtain useful information about the nature of the seabed. The maximum depth at which a sensor receives any significant signal varies as a function of spectral wavelength and the clarity of the water. In some coastal waters, the bottom is detected to less than 10 m. In highly turbid waters, the bottom would not be visible as the depth of light penetration is less than a meter.[19] Once the data have been corrected for atmospheric and seabed effects, the standard method is to then georeference the imagery that links specific pixel locations in the image to their corresponding location on a mapped surface for which the mapped coordinates are well known.
Management of local multi-sensors applied to SHM and long-term infrared monitoring: Cloud2IR implementation
Published in Quantitative InfraRed Thermography Journal, 2019
Antoine Crinière, Jean Dumoulin, Laurent Mevel
Firstly proposed by the National Center for Supercomputing Applications and now developed by the HDF Group [5], the Hierarchical Data Format is a generic data container able to structure and describe a huge amount of data. Available in BSD (Berkeley Software Distribution License) open source, the HDF library is common to multiple systems and programming languages (Matlab, C/C++, Java, Python). Various versions of the format have been proposed, the Hdf5 (the last one) simplifies the structure, as well as the access strategies to the data contained in the file. The Hdf5 file structure includes only two major types of objects (Datasets and Groups) for which metadata can be specified: