Tags

Type your tag names separated by a space and hit enter

Estimating the average need of semantic knowledge from distributional semantic models.
Mem Cognit 2017; 45(8):1350-1370MC

Abstract

Continuous bag of words (CBOW) and skip-gram are two recently developed models of lexical semantics (Mikolov, Chen, Corrado, & Dean, Advances in Neural Information Processing Systems, 26, 3111-3119, 2013). Each has been demonstrated to perform markedly better at capturing human judgments about semantic relatedness than competing models (e.g., latent semantic analysis; Landauer & Dumais, Psychological Review, 104(2), 1997 211; hyperspace analogue to language; Lund & Burgess, Behavior Research Methods, Instruments, & Computers, 28(2), 203-208, 1996). The new models were largely developed to address practical problems of meaning representation in natural language processing. Consequently, very little attention has been paid to the psychological implications of the performance of these models. We describe the relationship between the learning algorithms employed by these models and Anderson's rational theory of memory (J. R. Anderson & Milson, Psychological Review, 96(4), 703, 1989) and argue that CBOW is learning word meanings according to Anderson's concept of needs probability. We also demonstrate that CBOW can account for nearly all of the variation in lexical access measures typically attributable to word frequency and contextual diversity-two measures that are conceptually related to needs probability. These results suggest two conclusions: One, CBOW is a psychologically plausible model of lexical semantics. Two, word frequency and contextual diversity do not capture learning effects but rather memory retrieval effects.

Authors+Show Affiliations

Department of Psychology, University of Alberta, P217 Biological Sciences Building, Edmonton, AB, T6G 2E9, Canada. hollis@ualberta.ca.

Pub Type(s)

Journal Article

Language

eng

PubMed ID

28707176

Citation

Hollis, Geoff. "Estimating the Average Need of Semantic Knowledge From Distributional Semantic Models." Memory & Cognition, vol. 45, no. 8, 2017, pp. 1350-1370.
Hollis G. Estimating the average need of semantic knowledge from distributional semantic models. Mem Cognit. 2017;45(8):1350-1370.
Hollis, G. (2017). Estimating the average need of semantic knowledge from distributional semantic models. Memory & Cognition, 45(8), pp. 1350-1370. doi:10.3758/s13421-017-0732-1.
Hollis G. Estimating the Average Need of Semantic Knowledge From Distributional Semantic Models. Mem Cognit. 2017;45(8):1350-1370. PubMed PMID: 28707176.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - Estimating the average need of semantic knowledge from distributional semantic models. A1 - Hollis,Geoff, PY - 2017/7/15/pubmed PY - 2018/6/23/medline PY - 2017/7/15/entrez KW - CBOW KW - Contextual diversity KW - Likely need KW - Needs probability KW - Rational analysis KW - Skip-gram KW - Word frequency KW - average need SP - 1350 EP - 1370 JF - Memory & cognition JO - Mem Cognit VL - 45 IS - 8 N2 - Continuous bag of words (CBOW) and skip-gram are two recently developed models of lexical semantics (Mikolov, Chen, Corrado, & Dean, Advances in Neural Information Processing Systems, 26, 3111-3119, 2013). Each has been demonstrated to perform markedly better at capturing human judgments about semantic relatedness than competing models (e.g., latent semantic analysis; Landauer & Dumais, Psychological Review, 104(2), 1997 211; hyperspace analogue to language; Lund & Burgess, Behavior Research Methods, Instruments, & Computers, 28(2), 203-208, 1996). The new models were largely developed to address practical problems of meaning representation in natural language processing. Consequently, very little attention has been paid to the psychological implications of the performance of these models. We describe the relationship between the learning algorithms employed by these models and Anderson's rational theory of memory (J. R. Anderson & Milson, Psychological Review, 96(4), 703, 1989) and argue that CBOW is learning word meanings according to Anderson's concept of needs probability. We also demonstrate that CBOW can account for nearly all of the variation in lexical access measures typically attributable to word frequency and contextual diversity-two measures that are conceptually related to needs probability. These results suggest two conclusions: One, CBOW is a psychologically plausible model of lexical semantics. Two, word frequency and contextual diversity do not capture learning effects but rather memory retrieval effects. SN - 1532-5946 UR - https://www.unboundmedicine.com/medline/citation/28707176/Estimating_the_average_need_of_semantic_knowledge_from_distributional_semantic_models_ L2 - https://dx.doi.org/10.3758/s13421-017-0732-1 DB - PRIME DP - Unbound Medicine ER -