China
Africa
Explore chapters and articles related to this topic
Natural Language Processing
Published in Vishal Jain, Akash Tayal, Jaspreet Singh, Arun Solanki, Cognitive Computing Systems, 2021
V. Vishnuprabha, Lino Murali, Daleesha M. Viswanathan
The skip-gram model with negative subsampling has a better accuracy over all other methods. In addition, this can identify the semantics of a word quickly. That is, the word “bank” can be a financial institution or riverbank. Google’s word2vec uses CBOW and skip-gram model.Facebook’s Fast Text is an extension of the word2vec model, but it feeds N-grams of words instead of feeding N-grams of sentences. It helps in identifying the context of the given word even if that word is not present in the dictionary. Consider a legal term; it can be present in the standard dictionary, but some of the legal terms may not be present in the standard dictionary that can be found out with the help of this method.Example: Trigram of the word ‘crane’ >>> {cra,ran,ane}.
Supervised Learning for Aggression Identification and Author Profiling over Twitter Dataset
Published in Sk Md Obaidullah, KC Santosh, Teresa Gonçalves, Nibaran Das, Kaushik Roy, Document Processing Using Machine Learning, 2019
One paper that outlines various features is given by Stamatatos [18]. He outlines lexical, character, syntactic, semantic and application-specific features. These feature types were evident in submissions to the author profiling task of PAN, an ongoing project of the Conference and Labs of the Evaluation Forum (CLEF). There are various editions, but the ones pertinent to our task are PAN 2015 [19], whose task is to classify tweets for four languages, and PAN 2016 [20], whose task is to do a cross-genre evaluation. The main types of features extracted include style-based, content-based, n-gram–based, information retrieval–based and collocation-based. In PAN 2015, for instance, character n-gram models were used in [21,22]. Word n-grams were also used in works such as those of [23–25]. TFIDF n-grams were used in [26,27]. POS n-grams were also used in [26,28]. Another set of features stem from word vectors. There are various word vector implementations proposed. One that is widely used is Word2Vec by Mikolov [29,30]. The Word2Vec algorithm essentially has two modes: continuous bag of words and skip-grams. To obtain the word vectors, the program must initialize the vectors from random numbers. Then it reads from a large corpus such as Wikipedia. Using the sequence of words in the text given in the corpus, the program uses the initial random vectors that represent words to predict other words. In continuous bag of words, for instance, a target word is predicted using the surrounding context words. Skip-grams, on the other hand, use a word to predict its contextual words.
Machine Learning
Published in Ravi Das, Practical AI for Cybersecurity, 2021
From this point onwards, there are actually two types of models that are used for the word2vec architecture, and they are known as the “CBOW” and the “Skip-Gram.” The common features between these two are the following: The d-dimensional input;The d-dimensional output;If H < d has an “X” number of Hidden Units and an “X” number of Hidden Layers, then this will closely resemble an Autoencoder (as reviewed in the last subsection).
An optimal context-aware content-based movie recommender system using genetic algorithm: a case study on MovieLens dataset
Published in Journal of Experimental & Theoretical Artificial Intelligence, 2022
Alireza Abdolmaleki, Mohammad Hossein Rezvani
Predictive methods generally outperform counting methods in terms of efficiency. The most popular predictive word embedding algorithm is Word-to-Vector (Word2Vec). Two essential Word2Vec models are Skip-Gram and Continuous Bag of Words (CBOW) (Zhou, 2019). The Skip-Gram model predicts a target word using its nearby appearing words. In contrast, the CBOW model predicts the target word according to the context. In this method, the context is a window of a specified size around the word. Figure 2 shows the difference between Skip-Gram and CBOW model with an illustrative example. Suppose our goal is to break down the phrase ‘I am selling these fine leather jackets,’ and the target word is ‘fine.’ The CBOW model predicts the word using the information of all the surrounding terms, including ‘selling,’ ‘these,’ ‘leather,’ and ‘jackets.’ The sum of these word vectors is used to predict the target word in the CBOW model. The Skip-Gram model uses a nearby random word to predict the target word instead of all words. Previous research has shown that, in general, the Skip-Gram model performs better than the CBOW model.
Geographic context-aware text mining: enhance social media message classification for situational awareness by integrating spatial and temporal features
Published in International Journal of Digital Earth, 2021
Christopher Scheele, Manzhu Yu, Qunying Huang
Figure 3 presents the CNN architecture, which is the configuration for tweet message classification. Preprocessing converts tweets into lists of 50 integers and represents each word of the tweet by an integer. The preprocessed tweet then passes through the first layer, word embedding, which expands the word integers to a larger matrix and represents them in a more meaningful way. The word embedding layer uses Word2Vec (Mikolov et al. 2013) to embed semantic similarity information in the representation of words and expands each word into a vector of 300. The convolution layer extracts features from the word embedding and transforms them through global max pooling. The convolution layer uses neurons with filter size of 3, stride of 1, zero padding, and depth of 250 (Table 6). The extracted features are then concatenated with spatial and temporal features, which represent the tweet’s distance from hurricane center and the surrounding geographic and meteorological environment. Then two fully connected layers predict the themes of each tweet. Dropout layers are utilized before the convolution layer and the last fully connected layer, while activation functions are used after the convolution layer and the fully connected layers.
Optimal Deep Neural Network-Based Model for Answering Visual Medical Question
Published in Cybernetics and Systems, 2022
Karim Gasmi, Ibtihel Ben Ltaifa, Gaël Lejeune, Hamoud Alshammari, Lassaad Ben Ammar, Mahmood A. Mahmood
For question representation, we used a Bi-LSTM variant of RNNs, which captures long-range dependencies with their hidden states. Word embeddings are learned from word2vec and given as input to the Bi-LSTM. Word2vec is a prediction-based model designed to predict words based on their surrounding linguistic context by using one of two distinct neural network language models—skip-gram and Continuous Bag of Words—where, in each step, the neural network is trained with a set of words in the window (Altszyler, Sigman, and Slezak 2018). Our word2vec model used the skip-gram model with the default parameters. To learn the output vectors of skip-gram, we used the hierarchical softmax algorithm for better optimization.