China
Africa
Explore chapters and articles related to this topic
Data mining
Published in Catherine Dawson, A–Z of Digital Research Methods, 2019
Data mining is the process that is used to turn raw data into useful information that will help to uncover hidden patterns, relationships and trends. It is a data-driven technique in which data are examined to determine which variables and their values are important and understand how variables are related to each other. It involves applying algorithms to the extraction of hidden information with the aim of building an effective predictive or descriptive model of data for explanation and/or generalisation. The focus is on data sourcing, pre-processing, data warehousing, data transformation, aggregation and statistical modelling. The availability of big data (extremely large and complex datasets, some of which are free to use, re-use, build on and redistribute, subject to stated conditions and licence: see Chapter 3) and technological advancement in software and tools (see below), has led to rapid growth in the use of data mining as a research method.
Big data for construction cost management
Published in Weisheng Lu, Chi Cheung Lai, Tung Tse, BIM and Big Data for Construction Cost Management, 2018
Weisheng Lu, Chi Cheung Lai, Tung Tse
What Ace did was to hire a big data analyst, who worked with the directors to seek their support, with the company’s in-house IT team to have the data interoperability, and equally importantly with the frontline surveyors to understand their needs. The database of the in-house system was opened for analysis. Some of the paper documents need to be digitalised and made machine-readable using optical character recognition (OCR) and other technologies. Data cleansing helped detect and correct incomplete, incorrect, inaccurate, or irrelevant parts of the raw data. The big data was analysed using regular statistical software such as SPSS or R without necessarily involving fascinating terms such as ‘pattern finding algorithms’, ‘unattended machine learning’, ‘deep learning’, or ‘artificial intelligence’. Data visualisation is also important, as it can attract employees’ attention and further seek their true engagement in the big data movement.
Digital twin of tunnel construction for safety and efficiency
Published in Daniele Peila, Giulia Viggiani, Tarcisio Celestino, Tunnels and Underground Cities: Engineering and Innovation meet Archaeology, Architecture and Art, 2020
R. Tomar, J. Piesk, H. Sprengel, E. Isleyen, S. Duzgun, J. Rostami
The remainder of the section describes the construction of a virtual tunnel based on LiDAR scanning. Point cloud of Colorado School of Mines’ Edgar Experimental Mine is collected with LiDAR (Figure 2). The raw data requires data cleaning and noise filtering. After pre-processing of the data, the point cloud is transformed into a polygon mesh (Figure 3).
Big Earth Data science: an information framework for a sustainable planet
Published in International Journal of Digital Earth, 2020
Huadong Guo, Stefano Nativi, Dong Liang, Max Craglia, Lizhe Wang, Sven Schade, Christina Corban, Guojin He, Martino Pesaresi, Jianhui Li, Zeeshan Shirazi, Jie Liu, Alessandro Annoni
Data cleaning is used to transform raw data into an understandable format, as real world data can be incomplete (lacking attribute values, lacking certain attributes of interest, or containing only aggregated data), noisy (containing errors or outliers) and inconsistent (containing discrepancies in codes or names).