China
Africa
Explore chapters and articles related to this topic
Deep Learning
Published in Subasish Das, Artificial Intelligence in Highway Safety, 2023
Models that use attention show greater prediction accuracy by locating global dependencies between data points and disregarding their distance in output and input sequences, and attention mechanisms also make calculations by the neural network more parallelizable. They are usually utilized along with convolution and recurrence, but, in a minimal portion of neural network architectures, attention may take the place of convolution and recurrence schemes. This architecture uses an attention scheme called intra-attention or self-attention, wherein multiple relationships are extrapolated between a data sequence’s different positions. Thus, the Transformer’s attention mechanism produces a more robust model creation by finding more patterns from input data.
Survey of fabric defect detection using UNet architecture
Published in Sangeeta Jadhav, Rahul Desai, Ashwini Sapkal, Application of Communication Computational Intelligence and Learning, 2022
Saiqa Khan, Ansari Almas Eram, Ansari Saadan, Ansari Raheen Bano, Narjis Fatema Suterwala
The cognitive process of selectively focusing on one or a few things while dismissing others is known as attention. The attention mechanism enables the classification algorithm to pay closer attention to some of the images more discriminative local regions, resulting in improved model classification accuracy in complicated scenarios. The attention method is being used in skip-connection because some fabric images contain minor faults that are easy to overlook during the segmentation process. The attention technique is precisely used to model the dependencies between feature channels. It’s implemented in the CU-skip- connect Net’s feature to decrease the number of redundant functions by suppressing the typical response of disconnected regions.
Introduction to Deep Learning
Published in Lia Morra, Silvia Delsanto, Loredana Correale, Artificial Intelligence in Medical Imaging, 2019
Lia Morra, Silvia Delsanto, Loredana Correale
In recent years, attention modules are also gaining widespread importance. In their more general formulation, attention modules are simply a gating function that regulates which values in an array should be passed through, and which are instead irrelevant. To this aim, a linear layer can be used to implement a weighting function. Attention modules are being increasingly used for sequence modelling, in conjunction with or substitution of RNNs [44], and are also being integrated with CNNs, to identify the most relevant parts of an image for a given task [45].
A spatiotemporal grammar network model with wide attention for short-term traffic flow prediction
Published in Transportmetrica B: Transport Dynamics, 2023
The deep temporal correlation feature maps and the deep spatial correlation feature maps can be extracted through the dual-branch grammar model. However, different points in the same channel or different channels in these two feature maps have different effects on the prediction task. Therefore, the features attention mechanism is usually used to make the prediction model controllable, enhancing useful features and suppressing redundant features. The attention mechanism consists of two essential parts: the feature maps and the attention weights. In most studies, the weights of attention are obtained from the feature maps, which increases the number of layers of the prediction model, thus increasing the computational cost of model training. According to the feature maps and the weights of attention in the self-attention (Vaswani et al. 2017) are derived from the same input, the weights of attention for map points and channels are generated from the initial input of the model in this paper, so that only the width of the model is increased, and the computational cost is much smaller than the structure that the weights of attention are obtained from the feature maps while ensuring the attention effect.
An identification and positioning method for coal gangue based on lightweight mixed domain attention
Published in International Journal of Coal Preparation and Utilization, 2023
De-Yong Li, Guo-Fa Wang, Yong-Cun Guo, Yong Zhang, Shuang Wang
In recent years, attention mechanism has been widely used in various fields of deep learning, such as image processing, speech recognition, and natural language processing. The visual attention mechanism can obtain the key target areas by rapidly scanning the global image, invest more attention resources, acquire the target details needing attention, and suppress other useless information. Cheng et al. (2020) applied the mixed domain attention mechanism to the detection of ground fissures in mined-out areas, strengthening the contribution of specific channels and spatial positions of the feature images to the detection of ground fissures. Yan, Fang, and Gao (2020) used the mixed domain attention mechanism to obtain the features of the key focus areas and channels of the feature images, so as to improve the detection ability of the network for 3D targets.
Prediction of the total solar irradiance based on the CEEMDAN-BiGRU-Attention model
Published in Energy Sources, Part A: Recovery, Utilization, and Environmental Effects, 2023
Xuchu Jiang, Nisang Chen, Jinghong Huang, Ying Li, Xiaobing Luo
The attention mechanism (Vaswani et al. 2017) mimics the attention function of humans. When dealing with specific information, people’s attention is focused on key areas to obtain more important information. The attention mechanism identifies the information that is more critical to the target by assigning different weights to the target data and determining the information that is more critical to the target based on the weight score. If RNN models are given the ability to focus on important features, it helps to extract the hidden key features in the information. It improves the efficiency of information processing. The attention mechanism is shown in Figure 2, and the calculation formulas are shown in Eqs. (15)-(17).