China
Africa
Explore chapters and articles related to this topic
Bidirectional Encoder Representations from Transformers (BERT)
Published in Uday Kamath, Kenneth L. Graham, Wael Emara, Transformers for Machine Learning, 2022
Uday Kamath, Kenneth L. Graham, Wael Emara
TaBERT [289] is the first model to have been pre-trained on both natural language sentences and tabular data formats. These representations are advantageous for problems involving cooperative reasoning over natural language sentences and tables. Semantic parsing over databases is a sample example, in which a natural language inquiry (e.g., “Which nation has the largest GDP?”) is translated to a program executable over database (DB) tables. This is the first technique of pre-training that spans structured and unstructured domains, and it opens up new possibilities for semantic parsing, where one of the primary issues has been comprehending the structure of a database table and how it fits with a query. TaBERT was trained on a corpus of 26 million tables and the English phrases that accompany them. Historically, language models have been trained only on free-form natural language text. While these models are excellent for jobs that need reasoning only in free-form natural language, they are insufficient for activities such as database-based question answering, which involves reasoning in both free-form language and database tables.
Survey on frontiers of language and robotics
Published in Advanced Robotics, 2019
T. Taniguchi, D. Mochihashi, T. Nagai, S. Uchida, N. Inoue, I. Kobayashi, T. Nakamura, Y. Hagiwara, N. Iwahashi, T. Inamura
Note that we are not insisting on ‘grounding’ any word or phrase on the sensory–motor information provided by robots, i.e. external information. Sensory–motor information provides the cognitive system of a robot with observations, e.g. visual, auditory, and haptic information. However, many words representing abstract concepts cannot be directly ‘grounded’ on such sensory–motor information. For example, we cannot determine a proper probabilistic distribution of sensory–motor information for ‘in', ‘freedom’, or ‘the'. Even a verb can be considered as an abstract concept. ‘Through’ can represent different trajectories or a controller depending on target objects. Even though ‘in’ is abstract, ‘in front of the door’ seems concrete and more conducive to an association with sensory–motor information. Semantic parsing with real-world information and finding a way to handle abstract concepts is an important challenge.