China
Africa
Explore chapters and articles related to this topic
Knowledge-enhanced Deep Learning for Efficient Response Estimation of Nonlinear Structures
Published in M.Z. Naser, Leveraging Artificial Intelligence in Engineering, Management, and Safety of Infrastructure, 2023
Compared with conventional feedforward neural network (Fig. 12.1), the neurons in RNNs have a hidden state to memorize the past data and recurrently pass the hidden state to the next-step input. Figure 12.2a shows the data flow of a single-neuron RNN unfolded in time steps i – 1 and i. At each time-step, the neurons receive data from both the input (feedforward edge) and the hidden state of the last time-step (recurrent edge). With the hidden state, the output of at time i is related to both the input at time i and the previous outputs. Thus, the RNN architecture is well suited for ‘remembering’ the previous outputs (LeCun et al., 2015). With the ability of remembering the previous outputs, the RNN has been extensively applied to sequence data-related tasks, including natural language processing, speech recognition, and audio classification (Graves et al., 2013; Mogren, 2016; Sezer and Ozbayoglu, 2018).
Deep Learning in Smart Agriculture Applications
Published in Rashmi Priyadarshini, R M Mehra, Amit Sehgal, Prabhu Jyot Singh, Artificial Intelligence, 2023
Recurrent neural networks (RNNs) area type of neural network where output of the previous loop is considered as input for the current loop. General applications of generative neural networks are speech recognition, hand written recognition and analysis of sequence of data. Also generative neural networks automatically generate programming codes that give a predefined objective. The working process of RNN consists of providing input to the model. Representation of the data in the input layer is computed and sent to the hidden layer, where it conducts sequence modeling and training in a forward or backward directions. Multiple hidden layers can also be used, final hidden layer sends the processed result to the output layer. Long-short-term memory RNN is currently a popular RNN model. It is effective on data sequences that require memory or details of the last events. Some of the applications of RNN are language modeling and prediction, speech recognition, machine translation, image recognition and translation. Long-short-term memory is the latest improvement of RNN networks; these networks are known as cells [6–10]. These cells consider the input from previous state and present input, and also decide which information needs to be considered, and which should be disregarded. The previous condition, present memory and the current input combine together to predict the next output. Figure 8.3 shows the RNN in crop classification.
Neural Networks Machines Mimic Human Intelligence
Published in Chong Ho Alex Yu, Data Mining and Exploration, 2022
Recurrent Neural Network, which uses sequential data or time series data, originates from the work of Rumelhart (1986). Based on the notion that intelligence relies on memories and experiences, RNN utilizes feedback from the output of the previous temporal sequences to process the data at the current time point. As such, a typical RNN comprises three layers: input, recurrent, and output (Rezk et al. 2020). Although it is a highly dynamic system, training a RNN could be challenging because its backpropagation grows or shrinks substantially in each step, and as a result the steps either explode or vanish (Liang et al. 2017). If the gradient is near or larger than 1, it is called the exploding gradient. If it is near zero, it is called the vanishing gradient. RNN is commonly applied in speech recognition, text generation, sentiment analysis, language translation, and image captioning (IBM Cloud Education 2020; Rezk et al. 2020).
Knowledge reasoning with multiple relational paths
Published in Connection Science, 2023
To model multi-step relational paths in knowledge graphs, a Recurrent Neural Network RNN (Medsker & Jain, 2001), can be used. RNN is the expansion of a sequence in time, because its network structure can handle the correlation between time series data. The model structure is shown in Figure 2, where x represents the input layer, H represents the hidden layer, O represents the output layer, and there is a loop operation on the hidden layer H, U, V, and W are linear relationship parameters, RNN passes the weight matrix W to realise the dependencies between hidden layers. With the research and application of RNN, some issues have been found in many practical problems that RNN has shortcomings such as gradient disappearance, gradient explosion and poor ability to model long-range dependency information. Therefore, the long short-term memory network LSTM is introduced. LSTMs are able to memorise long-range dependencies in data sequences, thereby modeling long-term dependencies. In knowledge reasoning techniques, such long-term sequential patterns are crucial for predictions between entities and relations in knowledge graphs using multi-step relation paths.
Discrete wavelet transform application for bike sharing system check-in/out demand prediction
Published in Transportation Letters, 2023
Yu Chen, Wei Wang, Xuedong Hua, Weijie Yu, Jialiang Xiao
For a long time, the recurrent neural network (RNN) has been one of the most widely used models for analyzing time series (Xiao et al. 2015; P. C. Chen et al. 2017; Lee and Ku 2022). However, when dealing with long-term memory problems, RNN has also difficulties such as exponential increase in computation, resulting in excessive model training time and vanishing gradients (X. Y. Li et al. 2023). As one of the best solutions to the aforementioned problems, the Long-Short Term Memory (LSTM) is an extension of the traditional RNN network. Its core unit is an LSTM memory cell composed of three gates: an input gate, output gate, and forget gate. These gates can be used to adjust the information flow, where the input gate determines the amplitude of an input (new memory), the forget gate controls the input amplitude of the previous memory state and the output gate adjusts the final output amplitude. Figure 4 shows the information flow process in LSTM.
Evaluation of neural network models for landslide susceptibility assessment
Published in International Journal of Digital Earth, 2022
Yaning Yi, Wanchang Zhang, Xiwei Xu, Zhijie Zhang, Xuan Wu
RNN has been widely used in serialized data processing, such as text language and audio processing. For serialized data, the current data are usually related to the previous data. The implicit relationship is not considered in other neural networks, because the units in the layer are independent and not connected, except RNN. As shown in Figure 6, in the hidden layer of RNN, each unit is connected to other units at different time intervals. In other words, the current output (yt+1) considers not only the present input (xt+1), but also the history information of previous elements (such as xt and xt-1) of a sequence. Thus, RNN can learn valuable information in sequence data of various lengths (Fang et al. 2020).