China
Africa
Explore chapters and articles related to this topic
Information Retrieval Methods for Big Data Analytics on Text
Published in Mohiuddin Ahmed, Al-Sakib Khan Pathan, Data Analytics, 2018
Abhay Kumar Bhadani, Ankur Narang
fastText is one of the recent advances in the area of NLP, which helps to generate word embeddings, very similar to what word2vec does [9]. It is an open-source library created by Facebook for carrying out their NLP activities, such as word embeddings or text classification. However, the beauty of fastText lies in the fact that it takes several orders of magnitude less time to train a model. fastText takes less than ten minutes to train a model using a standard multicore CPU and can classify 312K classes in less than a minute for half a million sentences.
BERT- and FastText-Based Research Paper Recommender System
Published in Pallavi Vijay Chavan, Parikshit N Mahalle, Ramchandra Mangrulkar, Idongesit Williams, Data Science, 2022
Nemil Shah, Yash Goda, Naitik Rathod, Vatsal Khandor, Pankaj Kulkarni, Ramchandra Mangrulkar
FastText is a library created by Facebook to perform word embeddings to perform sentence classification and word representations. FastText supports continuous bag of words (CBOW) or skip-gram models using two different methods to address classification and training word representations in text: hierarchical softmax and word n-grams.
Takeover Transition in Autonomous Vehicles: A YouTube Study
Published in International Journal of Human–Computer Interaction, 2020
Feng Zhou, X. Jessie Yang, Xin Zhang
Third, in order to further automate this process, we conducted a topic mining analysis using fastText (Joulin, Grave, Bojanowski, & Mikolov, 2017) based on the human factors issues identified in step 2. fastText is a library created by Facebook and it is used to learn word representation and sentence classification. It was reported that its performance was on par with deep learning methods, but with extreme efficiency (Joulin et al., 2017). Figure 1 shows the model architecture of fastText. It first looks up the N word vectors (L2 short for Layer 2) in a preprocessed comment (L1), which are then averaged (O2 short for Operation 2) into a hidden comment representation (L3). The comment representation, shared among features and classes, is then fed into a linear classifier with rank constraint and a fast loss approximation (O3). Finally, the output is a softmax layer (L4) producing a probability distribution over labeled classes (O4), which are topic (L5) in this research. In the lookup step, fastText applies a hashing trick (Weinberger, Dasgupta, Langford, Smola, & Attenberg, 2009) (O1) that is a fast and space efficient way of vectorizing features by using the hash values of the features as direct indices of the vector. For ngram features, it makes use of sub-word information (i.e., character n-grams) so that wrongly spelled words (e.g., ‘goood’) can have a similar word vector to the correct one (‘good’), which are often seen in social media. It also gives an option to use hierarchical softmax at the output layer when the number of classes is large, reducing the computational complexity from linear time to log time.
Learning Bilingual Word Embedding Mappings with Similar Words in Related Languages Using GAN
Published in Applied Artificial Intelligence, 2022
Ghafour Alipour, Jamshid Bagherzadeh Mohasefi, Mohammad-Reza Feizi-Derakhshi
A method, proposed by Google researchers, for learning word embeddings is based on either the skip-gram or the continuous bag-of-words (CBOW) architectures, which are implemented in Word2vec (Mikolov et al. 2013b) and Fast-Text (Joulin et al. 2016) libraries. FastText is an extension of Word2vec, representing sentences with a bag of words, a bag of n-grams, sub-word information, and sharing information across classes through a hidden representation. Another approach, proposed by Stanford university researchers, is Glove, which is achieved by mapping words into a latent space where the distance between words is related to semantic similarity (Pennington, Socher, and Manning 2014).
A text classification method based on LSTM and graph attention network
Published in Connection Science, 2022
We compare our model with the following 10 baseline models. CNN (Kim, 2014): The first use of convolutional neural networks for text classification tasks, using pre-trained word vectors and maximum pooling operations to obtain text representations.LSTM (Liu et al., 2016): The model uses the last hidden state as a representation of the whole text. Pre-trained word vectors are used in our experiments.Bi-LSTM: is a bi-directional LSTM that uses pre-trained word embeddings.fastText (Joulin et al., 2016): A simple and efficient text classification method that does not require initialisation with pre-trained word vectors and is faster to train than a typical neural network.Bert (Devlin et al., 2018): Bidirectional Encoder Representations from Transformer. It is a very advanced natural language processing framework.Graph-CNN (Defferrard et al., 2016): a graph CNN model that operates convolutions over word embedding similarity graphs, in which Chebyshev filter is used.TextGCN (Yao et al., 2019): A model for text classification using GCN, which constructs a big picture for the entire corpus.InducT-GCN (Wang et al., 2022): An inductive text classification model based on GCN, which uses less parameters and space than TextGCN.Syntax-AT-Capsule (Jia & Wang, 2022): An enhanced capsule network text classification model, which uses GCN as a submodule to encode the syntactic dependency tree and extract the syntactic information in the text.Text-level-GNN (Huang et al., 2019): uses GNN for message passing between nodes, which builds text graphs for a text individually.