Psycholinguistics James Myers April 30, 2004 Language production OVERVIEW: 1. Evidence for language production models 2. Speech errors 3. Stages in language production 4. Parallel models of language production ============================================================= 1. Evidence for language production models Why does the book have four chapters on perception and comprehension, but only one chapter on production? Not because production is less important or interesting, but because it's harder to test production models: It's easy to control what a person will perceive, but hard to control what a person is going to produce. Two basic sources of data on language production: (a) experiments, and (b) natural speech; (b) has been more commonly used until very recently (the last 10 years or so). 1.1 Experiments 1.1.1 Experiments to induce (cause) speech errors. Plus: Increases the number of speech errors to study. Minus: Doesn't always give better insight into what causes speech errors; artificially produced speech errors may show different properties from those of natural speech errors. 1.1.2 Experiments that control what subjects produce: Reading aloud Minus: Orthographic influences on production. (e.g. American subjects may use spelling to change their phonological productions; Chinese subjects may be influenced by characters when producing two-syllable words) Picture-naming Minus: Must make sure that all subjects name the pictures the same way; cannot provide clear pictures for all interesting words, etc. Implicit priming (e.g. Meyer 1990) Subjects must learn to associate a prompt word with a target word that they must produce when they see the prompt; target words in training set are either phonologically similar or not; if there's a difference in production times in these two conditions, this shows "priming", but it's "implicit". Plus: Can study time course of phonological production Minus: Can't study other aspects of production, can't be sure that training task doesn't tap into off-line processing An example of a language production experiment: Schriefers, Meyer and Levelt (1990) [Max Planck Institute for Psycholinguistics in Holland] Issue: During the production of lexical items, do people first retrieve semantics, and then only later the phonology? That is, is the serial model for language production accurate? Task: Picture-naming. (First do a pretest to make sure that people consistently name pictures the same way.) RT measured by "voice key" (microphone connected to a computer) During the picture-naming task, get auditory interfering stimuli (IS) of different kinds: > semantically related to picture-word > phonologically related to picture-word > unrelated IS presented at different times: > 150 msecs before the picture (EARLY) > same time as picture (MIDDLE) > 150 msecs after the picture (LATE) Results: EARLY: Semantic IS slows down RT, Phonological IS has no effect. MIDDLE: Semantic IS has no effect; Phonological IS speeds up RT. LATE: Semantic IS still has no effect; Phonological IS speeds up RT even more. Interpretation: During lexical access in production, semantics is retrieved before phonology. For a comprehensive review of a highly detailed production model and the experiments that have been done to support it, see Levelt, Roelofs, & Meyer (1999). 1.2 Natural speech Of course speech errors are the most widely used source of evidence, but natural speech can be used to study language processing in many other ways as well. Much phonetic research is really psycholinguistics; e.g. Tsay, Charles-Luce, and Guo (1999): Issue: When Taiwanese speakers apply the tone-changing rule, do they modify their articulation of tone, or do they substitute one lexical tone for another? (That is, at what stage does the tone rule apply, lexicon or phonetics?) Example: Mid tone becomes Low tone: 地 地 球 te[M] te[L] kiu[LH] Note: There is also a lexical Low tone: 帝 te[L] Test: speakers produce (read aloud) sentences that are matched for everything except the environment for the tone rule (i.e. within a word or across words): 彼 粒 紅 地 球 真 值 錢。 hit-liap ang te-kiu tsin tat tsi~ [M] [L] [LH] 彼 個 皇 帝 球 真 多。 hit-le hong-te kiu tsin tse [M] [L] [LH] Results: the only difference between the "changed" tone and the lexical tone is the general drop in pitch at the end of a phrase. Interpretation: The Taiwanese tone "rule" involves lexical tone substitution rather than modification of tone articulation (with intonation coming in later). 2. Speech errors (slips of the tongue) 2.1 Things to keep in mind "Errors", NOT "ignorance" (or different grammar)! "Freudian slips"? Freud thought that slips revealed one's hidden thoughts, and this could be true (i.e. a speaker has more than one semantic representation in mind). But it isn't the only kind of error! Why study speech errors? Because if you want to know how something is built, one way is to break the thing and see what the parts are and how they interrelate (like atom-smashers). 2.2 How to collect speech errors: Carry around a notebook (or better, a tape-recorder). Pay attention to spontaneous speech only: TV news, for example, usually involves reading (reading errors are not like other kinds of natural speech errors). Record date, setting, error, target, and something about the speaker. Ideally, collect ALL errors of the type you are interested in, because otherwise you may be biased to only "hear" the errors that support your theory. Danger: "Slips of the ear". People tend to hear things the way they expect to hear them. 2.3 Categories of speech errors: (Unless otherwise noted, these errors come from a small corpus of Jane Tsay's English speech errors that I collected in the mid-1990's) NOTE: the "units" affected in these ways can be anything: features, tones, segments, syllables, morphemes (including affixes), words, syntactic phrases, etc. Intended utterance is given in square brackets. Shift: a unit moves ...in case she decide to hits it {from Carroll} {/s/ morpheme has shifted from "decide" to "hit"} Exchange: two units switch places sibbalus [syllabus] {/b/ and /l/ phonemes have exchanged} bruuk [group] {features [labial] and [velar] have exchanged!} Perseveration: a unit is copied later ("preserved") his final fish [wish] {feature [voiceless], or phoneme /f/ has perseverated} a stiff sneck [neck] {phoneme /s/ has perseverated} Anticipation: a unit is copied earlier ("comes too early") Prissmas present [Christmas present] {/p/ phoneme is anticipated} We have to wait for the woit -- water to start boiling. {vowels /oi/ are anticipated} Addition: a unit is added ...explain this clarefully enough {from Carroll} {phoneme /l/ is added, coming "out of nowhere"} Deletion: a unit is deleted plattic [plastic] {phoneme /s/ is deleted} skinner [skinnier] {phoneme or morpheme /i/ is deleted} Substitution: a unit is changed into a different unit Don't remember tomorrow night [forget] {word "remember" is substituted for "forget"} Blend: two units are combined garliflower [garlic + cauliflower] In real life, of course, many speech errors are difficult to classify, for example: I won't ask any flavors [favors] {phoneme addition or lexical substitution?} I'm always very amused [amazed] {phoneme substitution /e/->/ju/, or lexical substitution?} Why is natural glass... [gas] {addition of /l/, or perseveration from /l/ in "natural"?} 2.4 Implications of speech errors for models of language production: some examples from Wan & Jaeger (1998). Issue: How are Mandarin tones processed in production? Some results (i.e. observations about their natural speech errors): In lexical blends, the tone that remains is the one found on the vowel that remains: 談／說 --> ㄕㄢˊ This implies that during lexical access, the tone is "part" of the vowel. However, if phonological units are substituted or exchanged, the tone acts as if it's independent of the vowel: Anticipated falling tone: 同學多半。。。 --> 同學【ㄉㄨㄥˋ】半。。。 Correct tone remains even though vowel is perseverated: 哥，你肚子。。。 --> 哥，你【ㄍㄜˋ】子。。。 This suggests that during production of phrases (after lexical access), tones are represented separately from vowels. The Mandarin tone rule is sensitive to the surface tone environment, even if it's created by a speech error: 他很傲。【ㄊㄚ】【ㄏㄣˇ】【ㄠˋ】。 --> 【ㄊㄚ】【ㄏㄣˊ】【ㄠˇ】。 First the lexical tone is erroneously changed on the last syllable; then the tone rule applies. 我早回。 【ㄨㄛˊ】【ㄗㄠˇ】【ㄏㄨㄟˊ】 -->【ㄨㄛˇ】 【ㄏㄨㄟˊ】 First the second morpheme is erroneously deleted; then when the tone rule comes along, it can't apply! 3. Stages in language production 3.1 The mother of all models: Fromkin (1971) The following representations are created in order during language production: Propositional representations Syntactic frame Intonation (1) Content words are inserted into syntactic frame (2) Function words and affixes are inserted (3) Phonemic representations are added (4) Phonological rules etc (5) Sample evidence: (1): intonation will remain even when words or sounds are switched around (i.e. intonation is built BEFORE the words are inserted): hammer and SICKLE --> sickle and HAMMER a computer in our own LABoratory --> a laboratory in our own comPUter (3): affixes and function words can shift (as in some earlier examples); thus there is a function word inserter that can "miss" the proper location (5): Phonological accommodations: phonological rules seem to "accommodate" errors, that is, apply AFTER them to produce outputs that are consistent with the phonological rules of the language (the Mandarin tone rule pattern above is an example). (2) before (4): see error cited in the "review questions" in 2.e in Carroll, p. 220. 3.2 Further refinements have been made in this model, and new evidence has been collected, as summarized in Levelt (1989) and Levelt, et al. (1999). One key issue has been whether it's really true that semantics is processed before phonology and writing in the production of words. Evidence for this comes from a variety of sources: 3.2.1 Experimental evidence like Schriefers, et al. (1990) 3.2.2 Speech error evidence like Garrett (1980): Word exchanges (semantic stage) involve words of same class (e.g. all nouns, or all verbs) and they always occur across phrases: "I left [the briefcase] [in my cigar]" {exchanged items are both nouns, moving between NP and PP} But sound exchanges (phonological stage) can involve words of different class and they always occur within phrases: "...a choice of stummer sipends" [summer stipends] {shifted phoneme /t/ goes from N to modifier within a single NP} 3.2.3 Tip-of-the-tongue (TOT) states, first studied by Brown & McNeill (1966): When a speaker knows the syntax and semantics of a word, but only the tiniest bit about the phonology (usually just the first phoneme and/or the prosodic pattern). This implies that speakers can have semantic knowledge about words before phonological information becomes available. 3.2.3 Aphasia evidence (Caramazza, 1997): there is a double dissociation between word production disorders caused by semantic processing, and those caused by problems with the access of lexical phonological forms. 3.3 What are the very last things to happen in speech production? Articulatory plans: abstract instructions for your articulators, but all in the brain, not the mouth Covert self-monitoring (?): unconsciously "hearing" these articulatory plans and then correcting mistakes before you actually talk {this is controversial} Actual articulation Overt self-monitoring: correcting yourself after you make an actual mistake 3.3.1 Evidence for articulatory plans: Anticipations, shifts and exchanges show that some speech processing is "backwards," so there must be a stage in production that is "timeless", where earlier words and later words are all available in the same representation Planning and production cycles: plan for a while (speech is hesitant), then articulate for a while (speech is fluent), and then back again (see Carroll, p. 207) 3.3.2 Overt self-monitoring: "self-repairs" after we've made a mistake. In natural speech errors, about 64% are corrected, usually after the first word boundary after the error. In experimental task, Levelt (1983) found the same thing: 51% of corrections occur immediately after the error word. 3.3.3 Is there also COVERT self-monitoring? That is, do speakers correct the silent "plan" BEFORE commands are sent to articulators? Some relevant evidence: Speech-error-inducing experiment (Baars, Motley and MacKay 1975): Issue: Are people more likely to make sound-based errors if they generate a real word? Task: Subjects read pairs of words silently until they are signalled to respond (speak aloud); context is designed to induce an exchange error involving the syllable-initial consonants. Lexical condition (exchange errors produce real words): ball doze bash door bean deck bell dark darn bore RESPOND --> "barn door" 30% of the time Nonlexical condition (errors produce nonwords): big dutch bang doll bill deal bark dog dart board RESPOND --> "bart doard" 10% of the time Results: Yes, there is a lexical bias effect: subjects are more likely to make the error if it creates real words. Interpretation I (serial model version): Speakers generate the same number of sound errors in both conditions, but only in the silent articulatory plan; then they perform covert self- monitoring and filter out the nonwords. Interpretation II (parallel model version): The lexical bias effect is actually due to top-down processing: the word level is influencing the sound level. (More on this below) 4. Parallel models of language production 4.1 Claim: Lexical bias effect is caused by the "word" level being active IN PARALLEL with the "phoneme" level (e.g. Stemberger, 1985; Dell, 1986) 4.2 But is this the correct way to interpret this effect? 4.2.1 Garrett (1988); Garrett (personal communication ca. 1991): Lexical bias effect is probably partly an artifact of experimentally induced speech errors, since he didn't find it in his naturally collected corpus. 4.2.2 Dell and Reich (1981): Yeah, but they found that real words appeared in their natural corpus more often than by chance. 4.2.3 Garrett (personal communication ca. 1991): But Dell's collection was made by inexperienced undergraduate students, so his lexical bias was probably caused by how they collected the errors -- errors that create real words are easier to remember than errors that don't create real words ("slips of the ear").... 4.3 A closer look at lexical bias effect in error-inducing experiment: an examination of a prediction involving the effect of speaking rate on the lexical bias effect 4.3.1 Parallel model says: Lexical bias effect is caused by activation spreading "backwards", from phoneme level back into word level. Because speech production normally involves activation spreading only from word level down to sound level, this backwards spreading takes extra time. Therefore the lexical bias effect should only be found if subjects are allowed to respond more slowly. This is true! (e.g. Dell, 1985; see also Carroll, p. 204) 4.3.2 BUT -- Serial model says: Lexical bias effect is due to self-monitoring. Covert self-monitoring takes extra time. Therefore the lexical bias effect should only be found if subjects are allowed to respond more slowly. This is true! (e.g. above again!) 4.4 Conclusion? According to Levelt, et al. (1999), the lexical bias effect is real (they cite Martin, Gagnon, Schwartz, Dell, & Saffran, 1996). 4.4.1 But they say it isn't well explained by feedback from the phonological level back to the lexical level, since it isn't automatic: it only happens in induced-error experiments when there are real words mixed with the materials, which supports the covert self-monitoring hypothesis. 4.4.2 Moreover, Levelt et al.'s own computer model, which doesn't have feedback from phonological level back up to the lexical level (or any self-monitoring), ALSO produces a lexical bias effect! This happens because the model makes phonology-based word substitutions, e.g. "cat" > "rat" is a more common error type than "cat" > "dog". Therefor, "cat" > "rat" is also more common than "cat" > "lat". REFERENCES Baars, B. J., Motley, M. T., & MacKay, D. G. (1975). Output editing for lexical status in artificially elicited slips of the tongue. Journal of Verbal Learning and Verbal Behavior 14:382-391. Brown, R., McNeill, D. (1966). The 'tip of the tongue' phenomenon. Journal of Verbal Learning and Verbal Behavior, 5, 325-337. Caramazza, A. (1997). How many levels of processing are there in lexical access? Cognitive Neuropsychology, 14(1), 177-208. Dell, G. S. (1985). Positive feedback in hierarchical connectionist models: Applications to language production. Cognitive Science, 9, 3-23. Dell, G. S. (1986). A spreading-activation theory of retrieval in sentence production. Psychological Review, 93, 283-321. Dell, G. S., & Reich, P. (1981). Stages in sentence production: An analysis of speech error data. Journal o Verbal Learning and Verbal Behavior, 20, 611-629. Fromkin, V. A. (1971). The non-anomalous nature of anomalous utterances," Language, 47, 27-52. Garrett, M. F. (1980). The limits of accommodation. In V. A. Fromkin (Ed.) Errors in linguistic performance (pp. 263-271). Academic Press. Garrett, M. F. (1988). Processes in language production. In F. J. Newmeyer (Ed.) Linguistics: The Cambridge Survey, Vol III: Language: Psychological and biological aspects (pp. 69-96). Cambridge, UK: Cambridge University Press. Levelt, W. J. M. (1983). Monitoring and self-repair in speech. Cognition, 14, 41-104. Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press. Levelt, Willem J. M., Roelofs, A., & Meyer, A. S. (1999). A theory of lexical access in speech production. Behavioral and Brain Sciences, 22, 1-75. Martin, N., Gagnon, D. A., Schwarz, M. F., Dell, G. S., & Saffran, E. M. (1996). Phonological facilitation of semantic errors in normal and aphasic speakers. Langauge and Cognitive Processes, 11, 257-282. Meyer, A. S. (1990). The time course of phonological encoding in language production: The encoding of successive syllables of a word. Journal of Memory and Language, 29, 524-545. Schriefers, H., Meyer, A. S., & Levelt, W. J. M. (1990). Exploring the time course of lexical access in language production: Picture-word interference studies. Journal of Memory and Language, 29, 86-102. Stemberger, J. P. (1985). An interactive activation model of language production. In A. W. Ellis (Ed.) Progress in the psychology of language, Vol. 1 (pp. 143-186). Erlbaum. Tsay, J., Charles-Luce, J., & Guo, Y-S. (1999). The syntax-phonology interface in Taiwanese: acoustic evidence. Proceedings of the XIVth International Congress of Phonetic Sciences (pp. 2407-2410). Berkeley: University of California. Wan I-P., & Jaeger, J. J. (1998). Speech errors and the representation of tone in Mandarin Chinese. Phonology, 15, 417-461.