L61 Text Representation 2I3 - Vector Space Models4

Apuntes

L6: Text Representation (I) - Vector Space Models: How can we represent words as vectors? Basic representations: ▪ Manually hand-crafted features: POS tagging, word shape, lemma, prefix, sufix, etc. ▪ Word atoms or one-hot vectors ( one 1 corresponding to the index of the word, the rest 0’s ) Very sparse representations. Vector dimension is equal to the number of words in the vocabulary These representations not capture word semantics (the meaning of words) so that words that have similar meanings have similar representations. To understand the context and information Applications:Information retrieval, Text classification, Question answering, Machine translation, Natural language generation, Language modeling, Plagiarism detection, Document clustering ▪ The Distributional Hypothesis The contexts in which a word appears tells us a lot about what it means. Words that appear in similar contexts have similar meanings. Example: What is tejuino? A bottle of tejuino, tejuino makes you drunk, make tejuino out of corn, … Basic assumption: Use the set of contexts in which words (= word types) appear to define a vector representation of each word Words with similar contexts (similar...