Licor's Space

First introduced in computer vision, visual attention explained the perception towards the world of creatures in some way. I remember the first time I met attention was reading the paper Show and tell: A neural image caption generator during doing work related to image captioning. And in the area of NLP, tasks like machine translation, speech recognition, attention really makes a difference, it was started by combining with Seq2Seq model, then new model like Transformer tried to getting out of this. »

Task Specification Topic models are type of statistical models for discovering the abstract “topics” that occur in a collection of documents. Topic modeling is a frequently used text-mining tool for discovery of hidden semantic structures in a text body. About LDA Latent Dirichlet Allocation (LDA) is a generative probabilistic model for collections of discrete data such as text corpora and explain why some parts of the data are similar. It’s widely being used in tasks like topic modeling, text classification and collaborative filtering. »

Brief Definition Latent Dirichlet Allocation(LDA) is a generative probabilistic model, a three-level hierarchical Bayesian model. In the model, each item of a collection is modeled as a finite mixture over an underlying set of topics. And each topic is modeled as an infinite mixture over an underlying set of topic probabilities. Process Overview LDA assumes the following generative process for each document w in a corpus D: Choose N ∼ Poisson(ξ). »

When it comes to text generation, there’re always so many questions about it: Where’s the word come from? Does the model learn languages as a child do? Characteristics of the text Text data has a time dimension. The order of words is highly important. Text or language is rule-based, which we call grammar. Text data is discrete, characters or words are not continuous, which makes it hard to represent the data compared with image pixels. »

Table of content Chapter 1 | What Is Language Chapter 2 | Morphology: The Words of Language Chapter 3 | Syntax: Infinite Use of Finite Means Chapter 4 | The Meaning of Language Chapter 5 | Phonetics: The Sounds of Language Chapter 6 | Phonology: The Sound Patterns of Language Chapter 7 | Language in Society Chapter 8 | Language Change: The Syllables of Time Chapter 9 | Language Acquisition »