Sharon Goldwater's publications

Forthcoming

Full PDFs may be available on request.

A role for the developing lexicon in phonetic category acquisition. Naomi H. Feldman, Thomas L. Griffiths, Sharon Goldwater, James L. Morgan. . In submission.

[ bib | abstract ]

Infants learn to segment words from fluent speech during the same period as they learn native language phonetic categories, yet accounts of phonetic category acquisition typically ignore information about the words in which sounds appear. We use a Bayesian model to illustrate how feedback from segmented words might constrain phonetic category learning by providing information about which sounds occur together in words. Simulations demonstrate that word-level information can successfully disambiguate overlapping English vowel categories. Learning patterns in the model are shown to parallel human behavior from artificial language learning tasks. These findings point to a central role for the developing lexicon in phonetic category acquisition and provide a framework for incorporating top-down constraints into models of category learning.
```
@Article{	feldman.griffiths.ea:role,
	author	= {Naomi H. Feldman and Thomas L. Griffiths and Sharon Goldwater and James L. Morgan},
	year	= {In submission},
	title	= {A role for the developing lexicon in phonetic category acquisition}
}
```
Adding sentence types to a model of syntactic category acquisition. Stella Frank, Sharon Goldwater, Frank Keller. TopiCS in Cognitive Science. In press.

[ bib | abstract ]

The acquisition of syntactic categories is a crucial step in the process of acquiring syntax. At this stage, before a full grammar is available, only surface cues are available to the learner. Previous computational models have demonstrated that local contexts are informative for syntactic categorization. However, local contexts are affected by sentence-level structure. In this paper, we add sentence type as an observed feature to a model of syntactic category acquisition, based on experimental evidence showing that pre-syntactic children are able to distinguish sentence type using prosody and other cues. The model, a Bayesian Hidden Markov Model, allows for adding sentence type in a few different ways; we find that sentence type can aid syntactic category acquisition if it is used to characterize the differences in word order between sentence types. In these models, knowledge of sentence type permits similar gains to those found by extending the local context.
```
@Article{	frank.goldwater.ea:adding,
	author	= {Stella Frank and Sharon Goldwater and Frank Keller},
	year	= {In press},
	journal	= {TopiCS in Cognitive Science},
	title	= {Adding sentence types to a model of syntactic category acquisition}
}
```

Statistical Learning, Inductive Bias, and Bayesian Inference in Language Acquisition. Lisa Pearl, Sharon Goldwater. In Jeffrey Lidz, editors. Oxford Handbook of Developmental Linguistics. Oxford University Press. In press.

[ bib ]

@InCollection{	pearl.goldwater:statistical,
	publisher	= {Oxford University Press},
	editor	= {Jeffrey Lidz},
	booktitle	= {Oxford Handbook of Developmental Linguistics},
	author	= {Lisa Pearl and Sharon Goldwater},
	year	= {In press},
	title	= {Statistical Learning, Inductive Bias, and Bayesian Inference in Language Acquisition}
}

Existing

2013

A summary of the 2012 JHU CLSP Workshop on Zero Resource Speech Technologies and Models of Early Language Acquisition. Aren Jansen, Emmanuel Dupoux, Sharon Goldwater, Mark Johnson, Sanjeev Khudanpur, Kenneth Church, Naomi Feldman, Hynek Hermansky, Florian Metze, Richard Rose, Mike Seltzer, Pascal Clark, Ian McGraw, Balakrishnan Varadarajan, Erin Bennett, Benjamin Borschinger, Justin Chiu, Ewan Dunbar, Abdallah Fourtassi, David Harwath, Chia-ying Lee, Keith Levin, Atta Norouzian, Vijay Peddinti, Rachel Richardson, Thomas Schatz, Samuel Thomas. In Proceedings of ICASSP. 2013.
[ pdf | bib | abstract ]

We summarize the accomplishments of a multi-disciplinary workshop exploring the computational and scientific issues surrounding zero resource (unsupervised) speech technologies and related models of early language acquisition. Centered around the tasks of phonetic and lexical discovery, we consider unified evaluation metrics, present two new approaches for improving speaker independence in the absence of supervision, and evaluate the application of Bayesian word segmentation algorithms to automatic subword unit tokenizations. Finally, we present two strategies for integrating zero resource techniques into supervised settings, demonstrating the potential of unsupervised methods to improve mainstream technologies.
```
@InProceedings{	jansen.dupoux.ea:summary,
	booktitle	= {Proceedings of ICASSP},
	author	= {Aren Jansen and Emmanuel Dupoux and Sharon Goldwater and Mark Johnson and Sanjeev Khudanpur and Kenneth Church and Naomi Feldman and Hynek Hermansky and Florian Metze and Richard Rose and Mike Seltzer and Pascal Clark and Ian McGraw and Balakrishnan Varadarajan and Erin Bennett and Benjamin Borschinger and Justin Chiu and Ewan Dunbar and Abdallah Fourtassi and David Harwath and Chia-ying Lee and Keith Levin and Atta Norouzian and Vijay Peddinti and Rachel Richardson and Thomas Schatz and Samuel Thomas},
	year	= {2013},
	title	= {A summary of the 2012 JHU CLSP Workshop on Zero Resource Speech Technologies and Models of Early Language Acquisition}
}
```
Unsupervised dependency parsing with acoustic cues. John K. Pate, Sharon Goldwater. Transactions of the Association for Computational Linguistics 1, pp. 63--74. 2013.
[ pdf | bib | abstract ]

Unsupervised parsing is a difficult task that infants readily perform. Progress has been made on this task using text-based models, but few computational approaches have considered how infants might benefit from acoustic cues. This paper explores the hypothesis that word duration can help with learning syntax. We describe how duration information can be incorporated into an unsupervised Bayesian dependency parser whose only other source of information is the words themselves (without punctuation or parts of speech). Our results, evaluated on both adult-directed and child-directed utterances, show that using word duration can improve parse quality relative to words-only baselines. These results support the idea that acoustic cues provide useful evidence about syntactic structure for language-learning infants, and motivate the use of word duration cues in NLP tasks with speech.
```
@Article{	pate.goldwater:unsupervised*1,
	volume	= {1},
	author	= {John K. Pate and Sharon Goldwater},
	year	= {2013},
	pages	= {63--74},
	journal	= {Transactions of the Association for Computational Linguistics},
	title	= {Unsupervised dependency parsing with acoustic cues}
}
```
Minimally-Supervised Morphological Segmentation using Adaptor Grammars. Kairit Sirts, Sharon Goldwater. Transactions of the Association for Computational Linguistics. 2013. (Volume/page numbers not yet assigned)
[ pdf | bib | abstract ]

This paper explores the use of Adaptor Grammars, a nonparametric Bayesian modelling framework, for minimally supervised morphological segmentation. We compare three training methods: unsupervised training, semisupervised training, and a novel model selection method. In the model selection method, we train unsupervised Adaptor Grammars using an over-articulated metagrammar, then use a small labelled data set to select which potential morph boundaries identified by the metagrammar should be returned in the final output. We evaluate on five languages and show that semi-supervised training provides a boost over unsupervised training, while the model selection method yields the best average results over all languages and is competitive with state-of-the-art semi-supervised systems. Moreover, this method provides the potential to tune performance according to different evaluation metrics or downstream tasks.
```
@Article{	sirts.goldwater:minimally-supervised,
	author	= {Kairit Sirts and Sharon Goldwater},
	year	= {2013},
	journal	= {Transactions of the Association for Computational Linguistics},
	title	= {Minimally-Supervised Morphological Segmentation using Adaptor Grammars}
}
```

2012

Turning the pipeline into a loop: Iterated unsupervised dependency parsing and PoS induction. Christos Christodoulopoulos, Sharon Goldwater, Mark Steedman. In Proceedings of the NAACL-HLT Workshop on the Induction of Linguistic Structure. 2012.
[ pdf | bib ]

@InProceedings{	christodoulopoulos.goldwater.ea:turning,
	booktitle	= {Proceedings of the NAACL-HLT Workshop on the Induction of Linguistic Structure},
	author	= {Christos Christodoulopoulos and Sharon Goldwater and Mark Steedman},
	year	= {2012},
	title	= {Turning the pipeline into a loop: Iterated unsupervised dependency parsing and PoS induction}
}

Bootstrapping a Unified Model of Lexical and Phonetic Acquisition. Micha Elsner, Sharon Goldwater, Jacob Eisenstein. In Proceedings of the 50th Annual Meeting of the Association of Computational Linguistics. 2012.
[ pdf | bib | abstract ]

During early language acquisition, infants must learn both a lexicon and a model of phonetics that explains how lexical items can vary in pronunciation—for instance "the" might be realized as [Di] or [D@]. Previous models of acquisition have generally tackled these problems in isolation, yet behavioral evidence suggests infants acquire lexical and phonetic knowledge simultaneously. We present a Bayesian model that clusters together phonetic variants of the same lexical item while learning both a language model over lexical items and a log-linear model of pronunciation variability based on articulatory features. The model is trained on transcribed surface pronunciations, and learns by bootstrapping, without access to the true lexicon. We test the model using a corpus of child-directed speech with realistic phonetic variation and either gold standard or automatically induced word boundaries. In both cases modeling variability improves the accuracy of the learned lexicon over a system that assumes each lexical item has a unique pronunciation.
```
@InProceedings{	elsner.goldwater.ea:bootstrapping,
	booktitle	= {Proceedings of the 50th Annual Meeting of the Association of Computational Linguistics},
	author	= {Micha Elsner and Sharon Goldwater and Jacob Eisenstein},
	year	= {2012},
	title	= {Bootstrapping a Unified Model of Lexical and Phonetic Acquisition}
}
```
Semantic Parsing with Bayesian Tree Transducers. Bevan K. Jones, Mark Johnson, Sharon Goldwater. In Proceedings of the 50th Annual Meeting of the Association of Computational Linguistics. 2012.
[ pdf | bib | abstract ]

Many semantic parsing models use tree transformations to map between natural language and meaning representation. However, while tree transformations are central to several state-of-the-art approaches, little use has been made of the rich literature on tree automata. This paper makes the connection concrete with a tree transducer based semantic parsing model and suggests that other models can be interpreted in a similar framework, increasing the generality of their contributions. In particular, this paper further introduces a variational Bayesian inference algorithm that is applicable to a wide class of tree transducers, producing state-of-the-art semantic parsing results while remaining applicable to any domain employing probabilistic tree transducers.
```
@InProceedings{	jones.johnson.ea:semantic,
	booktitle	= {Proceedings of the 50th Annual Meeting of the Association of Computational Linguistics},
	author	= {Bevan K. Jones and Mark Johnson and Sharon Goldwater},
	year	= {2012},
	title	= {Semantic Parsing with Bayesian Tree Transducers}
}
```
A Probabilistic Model of Syntactic and Semantic Acquisition from Child-Directed Utterances and their Meanings. Tom Kwiatkowski, Sharon Goldwater, Luke Zettelmoyer, Mark Steedman. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. 2012.
[ pdf | bib | abstract ]

This paper presents an incremental probabilistic learner that models the acquistion of syntax and semantics from a corpus of child-directed utterances paired with possible representations of their meanings. These meaning representations approximate the contextual input available to the child; they do not specify the meanings of individual words or syntactic derivations. The learner then has to infer the meanings and syntactic properties of the words in the input along with a parsing model. We use the CCG grammatical framework and train a non-parametric Bayesian model of parse structure with online variational Bayesian expectation maximization. When tested on utterances from the CHILDES corpus, our learner outperforms a state-of-the-art semantic parser. In addition, it models such aspects of child acquisition as "fast mapping," while also countering previous criticisms of statistical syntactic learners.
```
@InProceedings{	kwiatkowski.goldwater.ea:probabilistic,
	booktitle	= {Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics},
	author	= {Tom Kwiatkowski and Sharon Goldwater and Luke Zettelmoyer and Mark Steedman},
	year	= {2012},
	title	= {A Probabilistic Model of Syntactic and Semantic Acquisition from Child-Directed Utterances and their Meanings}
}
```

2011

A Bayesian mixture model for part-of-speech induction using multiple features. Christos Christodoulopoulos, Sharon Goldwater, Mark Steedman. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2011.
[ pdf | bib | abstract ]

In this paper we present a fully unsupervised syntactic class induction system formulated as a Bayesian multinomial mixture model, where each word type is constrained to belong to a single class. By using a mixture model rather than a sequence model (e.g., HMM), we are able to easily add multiple kinds of features, including those at both the type level (morphology features) and token level (context and alignment features, the latter fromparallel corpora). Using only context features, our system yields results comparable to state-of-the art, far better than a similar model without the one-class-per-type constraint. Using the additional features provides added benefit, and our final system outperforms the best published results on most of the 25 corpora tested.
```
@InProceedings{	christodoulopoulos.goldwater.ea:bayesian,
	booktitle	= {Proceedings of the Conference on Empirical Methods in Natural Language Processing},
	author	= {Christos Christodoulopoulos and Sharon Goldwater and Mark Steedman},
	year	= {2011},
	title	= {A Bayesian mixture model for part-of-speech induction using multiple features}
}
```
Producing power-law distributions and damping word frequencies with two-stage language models. Sharon Goldwater, Thomas L. Griffiths, Mark Johnson. Journal of Machine Learning Research 12(Jul), pp. 2335--2382. 2011.
[ pdf | bib | abstract ]

Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statisticalmodels that can generically produce power laws, breaking generativemodels into two stages. The first stage, the generator, can be any standard probabilistic model, while the second stage, the adaptor, transforms the word frequencies of this model to provide a closer match to natural language. We show that two commonly used Bayesian models, the Dirichlet-multinomial model and the Dirichlet process, can be viewed as special cases of our framework. We discuss two stochastic processes—the Chinese restaurant process and its two-parameter generalization based on the Pitman-Yor process—that can be used as adaptors in our framework to produce power-law distributions over word frequencies. We show that these adaptors justify common estimation procedures based on logarithmic or inverse-power transformations of empirical frequencies. In addition, taking the Pitman-Yor Chinese restaurant process as an adaptor justifies the appearance of type frequencies in formal analyses of natural language and improves the performance of a model for unsupervised learning of morphology.
```
@Article{	goldwater.griffiths.ea:producing*1,
	volume	= {12(Jul)},
	author	= {Sharon Goldwater and Thomas L. Griffiths and Mark Johnson},
	year	= {2011},
	pages	= {2335--2382},
	journal	= {Journal of Machine Learning Research},
	title	= {Producing power-law distributions and damping word frequencies with two-stage language models}
}
```
Formalizing Semantic Parsing with Tree Transducers. Bevan K. Jones, Mark Johnson, Sharon Goldwater. In Proceedings of the Australasian Language Technology Workshop. 2011.
[ pdf | bib | abstract ]

This paper introduces tree transducers as a unifying theory for semantic parsing models based on tree transformations. Many existing models use tree transformations, but implement specialized training and smoothing methods, which makes it difficult to modify or extend the models. By connecting to the rich literature on tree automata, we show how semantic parsing models can be developed using completely general estimation methods. We demonstrate the approach by reframing and extending one state-of-the-art model as a tree automaton. Using a variant of the inside-outside algorithm with variational Bayesian estimation, our generative model achieves higher raw accuracy than existing generative and discriminative approaches on a standard data set.
```
@InProceedings{	jones.johnson.ea:formalizing,
	booktitle	= {Proceedings of the Australasian Language Technology Workshop},
	author	= {Bevan K. Jones and Mark Johnson and Sharon Goldwater},
	year	= {2011},
	title	= {Formalizing Semantic Parsing with Tree Transducers}
}
```
Lexical generalization in CCG grammar induction for semantic parsing. Tom Kwiatkowski, Luke Zettelmoyer, Sharon Goldwater, Mark Steedman. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2011.
[ pdf | bib | abstract ]

We consider the problem of learning factored probabilistic CCG grammars for semantic parsing from data containing sentences paired with logical-form meaning representations. Traditional CCG lexicons list lexical items that pair words and phrases with syntactic and semantic content. Such lexicons can be inefficient when words appear repeatedly with closely related lexical content. In this paper, we introduce factored lexicons, which include both lexemes to model word meaning and templates to model systematic variation in word usage. We also present an algorithm for learning factored CCG lexicons, along with a probabilistic parse-selection model. Evaluations on benchmark datasets demonstrate that the approach learns highly accurate parsers, whose generalization performance benefits greatly from the lexical factoring.
```
@InProceedings{	kwiatkowski.zettelmoyer.ea:lexical,
	booktitle	= {Proceedings of the Conference on Empirical Methods in Natural Language Processing},
	author	= {Tom Kwiatkowski and Luke Zettelmoyer and Sharon Goldwater and Mark Steedman},
	year	= {2011},
	title	= {Lexical generalization in CCG grammar induction for semantic parsing}
}
```
Unsupervised extraction of recurring words from infant-directed speech. Fergus R. McInnes, Sharon Goldwater. In Proceedings of the 33rd Annual Conference of the Cognitive Science Society. 2011.
[ pdf | bib | abstract ]

To date, most computational models of infant word segmentation have worked from phonemic or phonetic input, or have used toy datasets. In this paper, we present an algorithm for word extraction that works directly from naturalistic acoustic input: infant-directed speech from the CHILDES corpus. The algorithm identifies recurring acoustic patterns that are candidates for identification as words or phrases, and then clusters together the most similar patterns. The recurring patterns are found in a single pass through the corpus using an incremental method, where only a small number of utterances are considered at once. Despite this limitation, we show that the algorithm is able to extract a number of recurring words, including some that infants learn earliest, such as Mommy and the child's name. We also introduce a novel information-theoretic evaluation measure.
```
@InProceedings{	mcinnes.goldwater:unsupervised,
	booktitle	= {Proceedings of the 33rd Annual Conference of the Cognitive Science Society},
	author	= {Fergus R. McInnes and Sharon Goldwater},
	year	= {2011},
	title	= {Unsupervised extraction of recurring words from infant-directed speech}
}
```
Predictability effects in adult-directed and infant-directed speech: Does the listener matter?. John K. Pate, Sharon Goldwater. In Proceedings of the 33rd Annual Conference of the Cognitive Science Society. 2011.
[ pdf | bib | abstract ]

A well-known effect in speech production is that more predictable words tend to be phonetically reduced. Recent work has suggested that predictability effects result from hardwired properties of the language production system, rather than active modulation by the talker to accommodate the listener. However, these studies investigated only minor manipulations of listener characteristics. Here, we examine predictability effects with two very different listener populations: adults and preverbal infants. Using mixed effects regressions on spontaneous speech corpora, we compare the effect of word frequency, probability in context, and previous mention on word duration in adult-directed and infant-directed speech. We find that the effects of preceding context and word frequency differ according to listener. Contrary to previous work, these results suggest that talkers do modulate the phonetic effects of predictability based on listener characteristics. To our knowledge, this study is also the first published analysis of predictability effects in infant-directed speech.
```
@InProceedings{	pate.goldwater:predictability,
	booktitle	= {Proceedings of the 33rd Annual Conference of the Cognitive Science Society},
	author	= {John K. Pate and Sharon Goldwater},
	year	= {2011},
	title	= {Predictability effects in adult-directed and infant-directed speech: Does the listener matter?}
}
```
Unsupervised syntactic chunking with acoustic cues: Computational models for prosodic bootstrapping. John K. Pate, Sharon Goldwater. In Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics. 2011. (Received Best Student Paper award.)
[ pdf | bib | abstract ]

Learning to group words into phrases without supervision is a hard task for NLP systems, but infants routinely accomplish it. We hypothesize that infants use acoustic cues to prosody, which NLP systems typically ignore. To evaluate the utility of prosodic information for phrase discovery, we present an HMM-based unsupervised chunker that learns from only transcribed words and raw acoustic correlates to prosody. Unlike previous work on unsupervised parsing and chunking, we use neither gold standard part-of-speech tags nor punctuation in the input. Evaluated on the Switchboard corpus, our model outperforms several baselines that exploit either lexical or prosodic information alone, and, despite producing a flat structure, performs competitively with a state-of-the-art unsupervised lexicalized parser, with a substantial advantage in precision. Our results support the hypothesis that acoustic-prosodic cues provide useful evidence about syntactic phrases for languagelearning infants.
```
@InProceedings{	pate.goldwater:unsupervised,
	booktitle	= {Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics},
	author	= {John K. Pate and Sharon Goldwater},
	year	= {2011},
	title	= {Unsupervised syntactic chunking with acoustic cues: Computational models for prosodic bootstrapping}
}
```

2010

Two decades of unsupervised POS induction: How far have we come?. Christos Christodoulopoulos, Sharon Goldwater, Mark Steedman. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2010. (This is an updated version that corrects a minor bug in the computation of the vmb measure in Figure 1 and Table 1. Thanks to Andreas Zollmann for pointing this out.)
[ pdf | bib | abstract ]

Part-of-speech (POS) induction is one of the most popular tasks in research on unsupervised NLP. Many different methods have been proposed, yet comparisons are difficult to make since there is little consensus on evaluation framework, and many papers evaluate against only one or two competitor systems. Here we evaluate seven different POS induction systems spanning nearly 20 years of work, using a variety of measures. We show that some of the oldest (and simplest) systems stand up surprisingly well against more recent approaches. Since most of these systems were developed and tested using data from the WSJ corpus, we compare their generalization abil- ities by testing on both WSJ and the multilingual Multext-East corpus. Finally, we introduce the idea of evaluating systems based on their ability to produce cluster prototypes that are useful as input to a prototype-driven learner. In most cases, the prototype-driven learner outperforms the unsupervised system used to initialize it, yielding state-of-the-art results on WSJ and improvements on non-English corpora.
```
@InProceedings{	christodoulopoulos.goldwater.ea:two,
	booktitle	= {Proceedings of the Conference on Empirical Methods in Natural Language Processing},
	author	= {Christos Christodoulopoulos and Sharon Goldwater and Mark Steedman},
	year	= {2010},
	title	= {Two decades of unsupervised POS induction: How far have we come?}
}
```
Inducing tree substitution grammars. Trevor Cohn, Phil Blunsom, Sharon Goldwater. Journal of Machine Learning Research 11(Nov), pp. 3053--3096. 2010.
[ preprint pdf | bib | abstract | online journal ]

Inducing a grammar from text has proven to be a notoriously challenging learning task despite decades of research. The primary reason for its difficulty is that in order to induce plausible grammars, the underlying model must be capable of representing the intricacies of language while also ensuring that it can be readily learned from data. The majority of existing work on grammar induction has favoured model simplicity (and thus learnability) over representational capacity by using context free grammars and first order dependency grammars, which are not sufficiently expressive to model many common linguistic constructions. We propose a novel compromise by inferring a probabilistic tree substitution grammar, a formalism which allows for arbitrarily large tree fragments and thereby better represent complex linguistic structures. To limit the model's complexity we employ a Bayesian non-parametric prior which biases the model towards a sparse grammar with shallow productions. We demonstrate the model's efficacy on supervised phrase-structure parsing, where we induce a latent segmentation of the training treebank, and on unsupervised dependency grammar induction. In both cases the model uncovers interesting latent linguistic structures while producing competitive results.
```
@Article{	cohn.blunsom.ea:inducing,
	volume	= {11(Nov)},
	author	= {Trevor Cohn and Phil Blunsom and Sharon Goldwater},
	year	= {2010},
	pages	= {3053--3096},
	journal	= {Journal of Machine Learning Research},
	title	= {Inducing tree substitution grammars}
}
```
Beyond transitional probabilities: Human learners impose a parsimony bias in statistical word segmentation. Michael C. Frank, Inbal Arnon, Harry Tily, Sharon Goldwater. In Proceedings of the 32nd Annual Conference of the Cognitive Science Society. 2010.
[ pdf | bib | abstract ]

Human infants and adults are able to segment coherent sequences from unsegmented strings of auditory stimuli after only a short exposure, an ability thought to be linked to early language acquisition. Although some research has hypothesized that learners succeed in these tasks by computing transitional probabilities between syllables, current experimental results do not differentiate between a range of models of different computations that learners could perform. We created a set of stimuli that was consistent with two different lexicons---one consisting of two-syllable words and one of three-syllable words---but where transition probabilities would not lead learners to segment sentences consistently according to either lexicon. Participants' responses formed a distribution over possible segmentations that included consistent segmentations into both two- and three-syllable words, suggesting that learners do not use pure transitional probabilities to segment but instead impose a bias towards parsimony on the lexicons they learn.
```
@InProceedings{	frank.arnon.ea:beyond,
	booktitle	= {Proceedings of the 32nd Annual Conference of the Cognitive Science Society},
	author	= {Michael C. Frank and Inbal Arnon and Harry Tily and Sharon Goldwater},
	year	= {2010},
	title	= {Beyond transitional probabilities: Human learners impose a parsimony bias in statistical word segmentation}
}
```
Modeling human performance in statistical word segmentation. Michael C. Frank, Sharon Goldwater, Thomas L. Griffiths, Joshua B. Tenenbaum. Cognition 117 (2), pp. 107--125. 2010.
[ preprint pdf | bib | abstract | online journal ]

The ability to discover groupings in continuous stimuli on the basis of distributional information is present across species and across perceptual modalities. We investigate the nature of the computations underlying this ability using statistical word segmentation experiments in which we vary the length of sentences, the amount of exposure, and the number of words in the languages being learned. Although the results are intuitive from the perspective of a language learner (longer sentences, less training, and a larger language all make learning more difficult), standard computational proposals fail to capture several of these results. We describe how probabilistic models of segmentation can be modified to take into account some notion of memory or resource limitations in order to provide a closer match to human performance.
```
@Article{	frank.goldwater.ea:modeling*1,
	volume	= {117},
	number	= {2},
	author	= {Michael C. Frank and Sharon Goldwater and Thomas L. Griffiths and Joshua B. Tenenbaum},
	year	= {2010},
	pages	= {107--125},
	journal	= {Cognition},
	title	= {Modeling human performance in statistical word segmentation}
}
```
Using sentence type information for syntactic category acquisition. Stella Frank, Sharon Goldwater, Frank Keller. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics at ACL. 2010.
[ pdf | bib | abstract ]

In this paper we investigate a new source of information for syntactic category acquisition: sentence type (question, declarative, imperative). Sentence type correlates strongly with intonation patterns in most languages; we hypothesize that these intonation patterns are a valuable signal to a language learner, indicating different syntactic patterns. To test this hypothesis, we train a Bayesian Hidden Markov Model (and variants) on child-directed speech. We first show that simply training a separate model for each sentence type decreases performance due to sparse data. As an alternative, we propose two new models based on the BHMM in which sentence type is an observed variable which influences either emission or transition probabilities. Both models outperform a standard BHMM on data from English, Cantonese, and Dutch. This suggests that sentence type information available from intonational cues may be helpful for syntactic acquisition crosslinguistically.
```
@InProceedings{	frank.goldwater.ea:using,
	booktitle	= {Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics at ACL},
	author	= {Stella Frank and Sharon Goldwater and Frank Keller},
	year	= {2010},
	title	= {Using sentence type information for syntactic category acquisition}
}
```
Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. Sharon Goldwater, Daniel Jurafsky, Christopher D. Manning. Speech Communication 52 (3), pp. 181--200. 2010.
[ preprint pdf | bib | abstract | online journal ]

Despite years of speech recognition research, little is known about which words tend to be misrecognized and why. Previous work has shown that errors increase for infrequent words, short words, and very loud or fast speech, but many other presumed causes of error (e.g., nearby disfluencies, turn-initial words, phonetic neighborhood density) have never been carefully tested. The reasons for the huge differences found in error rates between speakers also remain largely mysterious.
Using a mixed-effects regression model, we investigate these and other factors by analyzing the errors of two state-of-the-art recognizers on conversational speech. Words with higher error rates include those with extreme prosodic characteristics, those occurring turn-initially or as discourse markers, and doubly confusable pairs: acoustically similar words that also have similar language model probabilities. Words preceding disfluent interruption points (first repetition tokens and words before fragments) also have higher error rates. Finally, even after accounting for other factors, speaker differences cause enormous variance in error rates, suggesting that speaker error rate variance is not fully explained by differences in word choice, fluency, or prosodic characteristics. We also propose that doubly confusable pairs, rather than high neighborhood density, may better explain phonetic neighborhood errors in human speech processing.
```
@Article{	goldwater.jurafsky.ea:which*1,
	volume	= {52},
	number	= {3},
	author	= {Sharon Goldwater and Daniel Jurafsky and Christopher D. Manning},
	year	= {2010},
	pages	= {181--200},
	journal	= {Speech Communication},
	title	= {Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates}
}
```
Inducing probabilistic CCG grammars from logical form with higher-order unification. Tom Kwiatkowski, Luke Zettelmoyer, Sharon Goldwater, Mark Steedman. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2010.
[ pdf | bib | abstract ]

This paper addresses the problem of learning to map sentences to logical form, given training data consisting of natural language sentences paired with logical representations of their meaning. Previous approaches have been designed for particular natural languages or specific meaning representations; here we present a more general method. The approach induces a probabilistic CCG grammar that represents the meaning of individual words and defines how these meanings can be combined to analyze complete sentences. We use higher-order unification to define a hypothesis space containing all grammars consistent with the training data, and develop an online learning algorithm that efficiently searches this space while simultaneously estimating the parameters of a log-linear parsing model. Experiments demonstrate high accuracy on benchmark data sets in four languages with two different meaning representations.
```
@InProceedings{	kwiatkowski.zettelmoyer.ea:inducing,
	booktitle	= {Proceedings of the Conference on Empirical Methods in Natural Language Processing},
	author	= {Tom Kwiatkowski and Luke Zettelmoyer and Sharon Goldwater and Mark Steedman},
	year	= {2010},
	title	= {Inducing probabilistic CCG grammars from logical form with higher-order unification}
}
```

How ideal are we? Incorporating human limitations into Bayesian models of word segmentation. Lisa Pearl, Sharon Goldwater, Mark Steyvers. In Proceedings of the 34th annual Boston University Conference on Child Language Development, pp. 315--326. 2010.
[ pdf | bib ]

@InProceedings{	pearl.goldwater.ea:how,
	publisher	= {Cascadilla Press},
	booktitle	= {Proceedings of the 34th annual Boston University Conference on Child Language Development},
	author	= {Lisa Pearl and Sharon Goldwater and Mark Steyvers},
	address	= {Somerville, MA},
	year	= {2010},
	pages	= {315--326},
	title	= {How ideal are we? Incorporating human limitations into Bayesian models of word segmentation}
}

Online Learning Mechanisms for Bayesian Models of Word Segmentation. Lisa Pearl, Sharon Goldwater, Mark Steyvers. Research on Language and Computation 8 (2), pp. 107-132. 2010.
[ preprint pdf | bib | abstract | online journal ]

In recent years, Bayesian models have become increasingly popular as a way of understanding human cognition. Ideal learner Bayesian models assume that cognition can be usefully understood as optimal behavior under uncertainty, a hypothesis that has been supported by a number of modeling studies across various domains (e.g., Griffiths & Tenenbaum, 2005; Xu & Tenenbaum, 2007). The models in these studies aim to explain why humans behave as they do given the task and data they encounter, but typically avoid some questions addressed by more traditional psychological models, such as how the observed behavior is produced given constraints on memory and processing. Here, we use the task of word segmentation as a case study for investigating these questions within a Bayesian framework. We consider some limitations of the infant learner, and develop several online learning algorithms that take these limitations into account. Each algorithm can be viewed as a different method of approximating the same ideal learner. When tested on corpora of English child-directed speech, we find that the constrained learner’s behavior depends non-trivially on how the learner's limitations are implemented. Interestingly, sometimes biases that are helpful to an ideal learner hinder a constrained learner, and in a few cases, constrained learners perform equivalently or better than the ideal learner. This suggests that the transition from a computational-level solution for acquisition to an algorithmic-level one is not straightforward.
```
@Article{	pearl.goldwater.ea:online,
	publisher	= {Springer Netherlands},
	volume	= {8},
	number	= {2},
	author	= {Lisa Pearl and Sharon Goldwater and Mark Steyvers},
	year	= {2010},
	pages	= {107-132},
	journal	= {Research on Language and Computation},
	title	= {Online Learning Mechanisms for Bayesian Models of Word Segmentation}
}
```

2009

A note on the implementation of Hierarchical Dirichlet Processes. Phil Blunsom, Trevor Cohn, Sharon Goldwater, Mark Johnson. In Proceedings of the 47th Annual Meeting of the Association of Computational Linguistics. 2009. (This is an updated version with corrected pseudocode for Algorithm 1: line 16 was previously missing. Thanks to Weng Wei for pointing this out.)
[ pdf | bib | abstract ]

The implementation of collapsed Gibbs samplers for non-parametric Bayesian models is non-trivial, requiring considerable book-keeping. Goldwater et al. (2006a) presented an approximation which significantly reduces the storage and computation overhead, but we show here that their formulation was incorrect and, even after correction, is grossly inaccurate. We present an alternative formulation which is exact and can be computed easily. However this approach does not work for hierarchical models, for which case we present an efficient data structure which has a better space complexity than the naive approach.
```
@InProceedings{	blunsom.cohn.ea:note,
	booktitle	= {Proceedings of the 47th Annual Meeting of the Association of Computational Linguistics},
	author	= {Phil Blunsom and Trevor Cohn and Sharon Goldwater and Mark Johnson},
	year	= {2009},
	title	= {A note on the implementation of Hierarchical Dirichlet Processes}
}
```
Inducing compact but accurate tree-substitution grammars. Trevor Cohn, Sharon Goldwater, Phil Blunsom. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 548--556. 2009.
[ pdf | bib | abstract ]

Tree substitution grammars (TSGs) are a compelling alternative to context-free grammars for modelling syntax. However, many popular techniques for estimating weighted TSGs (under the moniker of Data Oriented Parsing) suffer from the problems of inconsistency and overfitting. We present a theoretically principled model which solves these problems using a Bayesian non-parametric formulation. Our model learns compact and simple grammars, uncovering latent linguistic structures (e.g., verb subcategorisation), and in doing so far out-performs a standard PCFG.
```
@InProceedings{	cohn.goldwater.ea:inducing,
	booktitle	= {Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics},
	author	= {Trevor Cohn and Sharon Goldwater and Phil Blunsom},
	address	= {Boulder, Colorado},
	year	= {2009},
	pages	= {548--556},
	title	= {Inducing compact but accurate tree-substitution grammars}
}
```
Evaluating models of syntactic category acquisition without using a gold standard. Stella Frank, Sharon Goldwater, Frank Keller. In Proceedings of the 31st Annual Conference of the Cognitive Science Society. 2009.
[ pdf | bib | abstract ]

A number of different measures have been proposed for evaluating computational models of human syntactic category acquisition. They all rely on a gold standard set of manually determined categories. However, children’s syntactic categories change during language development, so evaluating against a fixed and final set of adult categories is not appropriate. In this paper, we propose a new measure, substitutable precision and recall, based on the idea that words which occur in similar syntactic environments share the same category. We use this measure to evaluate three standard category acquisition models (hierarchical clustering, frequent frames, Bayesian HMM) and show that the results correlate well with those obtained using two gold-standard-based measures.
```
@InProceedings{	frank.goldwater.ea:evaluating,
	booktitle	= {Proceedings of the 31st Annual Conference of the Cognitive Science Society},
	author	= {Stella Frank and Sharon Goldwater and Frank Keller},
	year	= {2009},
	title	= {Evaluating models of syntactic category acquisition without using a gold standard}
}
```
A Bayesian framework for word segmentation: Exploring the effects of context. Sharon Goldwater, Thomas L. Griffiths, Mark Johnson. Cognition 112 (1), pp. 21--54. 2009. (Results in this paper are based on a newer version of the code used in the ACL06 and BUCLD07 word segmentation papers and chapter 5 of my thesis. The new version corrects a small bug in the implementation of the bigram (HDP) model. Please cite results from this paper in future publications.)
[ preprint pdf | bib | abstract | online journal ]

Since the experiments of Saffran et al. [Saffran, J., Aslin, R., & Newport, E. (1996). Statistical learning in 8-month-old infants. Science, 274, 1926–1928], there has been a great deal of interest in the question of how statistical regularities in the speech stream might be used by infants to begin to identify individual words. In this work, we use computational modeling to explore the effects of different assumptions the learner might make regarding the nature of words – in particular, how these assumptions affect the kinds of words that are segmented from a corpus of transcribed child-directed speech. We develop several models within a Bayesian ideal observer framework, and use them to examine the consequences of assuming either that words are independent units, or units that help to predict other units. We show through empirical and theoretical results that the assumption of independence causes the learner to undersegment the corpus, with many two- and three-word sequences (e.g. what's that, do you, in the house) misidentified as individual words. In contrast, when the learner assumes that words are predictive, the resulting segmentation is far more accurate. These results indicate that taking context into account is important for a statistical word segmentation strategy to be successful, and raise the possibility that even young infants may be able to exploit more subtle statistical patterns than have usually been considered.
```
@Article{	goldwater.griffiths.ea:bayesian,
	volume	= {112},
	number	= {1},
	author	= {Sharon Goldwater and Thomas L. Griffiths and Mark Johnson},
	year	= {2009},
	pages	= {21--54},
	journal	= {Cognition},
	title	= {A Bayesian framework for word segmentation: Exploring the effects of context}
}
```
Improving nonparametric Bayesian inference: Experiments on unsupervised word segmentation with adaptor grammars. Mark Johnson, Sharon Goldwater. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2009.
[ pdf | bib | abstract ]

One of the reasons nonparametric Bayesian inference is attracting attention in computational linguistics is because it provides a principled way of learning the units of generalization together with their probabilities. Adaptor grammars are a framework for defining a variety of hierarchical nonparametric Bayesian models. This paper investigates some of the choices that arise in formulating adaptor grammars and associated inference procedures, and shows that they can have a dramatic impact on performance in an unsupervised word segmentation task. With appropriate adaptor grammars and inference procedures we achieve an 87% word token f-score on the standard Brent version of the Bernstein- Ratner corpus, which is an error reduction of over 35% over the best previously reported results for this corpus.
```
@InProceedings{	johnson.goldwater:improving,
	booktitle	= {Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics},
	author	= {Mark Johnson and Sharon Goldwater},
	address	= {Boulder, Colorado},
	year	= {2009},
	title	= {Improving nonparametric Bayesian inference: Experiments on unsupervised word segmentation with adaptor grammars}
}
```
Improving morphology induction by learning spelling rules. Jason Naradowsky, Sharon Goldwater. In Proceedings of IJCAI. 2009.
[ pdf | bib | abstract ]

Unsupervised learning of morphology is an important task for human learners and in natural language processing systems. Previous systems focus on segmenting words into substrings (taking) tak.ing), but sometimes a segmentation-only analysis is insufficient (e.g., taking may be more appropriately analyzed as take+ing, with a spelling rule accounting for the deletion of the stem-final e). In this paper, we develop a Bayesian model for simultaneously inducing both morphology and spelling rules. We show that the addition of spelling rules improves performance over the baseline morphology-only model.
```
@InProceedings{	naradowsky.goldwater:improving,
	booktitle	= {Proceedings of IJCAI},
	author	= {Jason Naradowsky and Sharon Goldwater},
	year	= {2009},
	title	= {Improving morphology induction by learning spelling rules}
}
```

2008

Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase ASR error rates. Sharon Goldwater, Daniel Jurafsky, Christopher D. Manning. In Proceedings of ACL-08: HLT. 2008.
[ pdf | bib | abstract ]

Many factors are thought to increase the chances of misrecognizing a word in ASR, including low frequency, nearby disfluencies, short duration, and being at the start of a turn. However, few of these factors have been formally examined. This paper analyzes a variety of lexical, prosodic, and disfluency factors to determine which are likely to increase ASR error rates. Findings include the following. (1) For disfluencies, effects depend on the type of disfluency: errors increase by up to 15% (absolute) for words near fragments, but decrease by up to 7.2% (absolute) for words near repetitions. This decrease seems to be due to longer word duration. (2) For prosodic features, there are more errors for words with extreme values than words with typical values. (3) Although our results are based on output from a system with speaker adaptation, speaker differences are a major factor influencing error rates, and the effects of features such as frequency, pitch, and intensity may vary between speakers.
```
@InProceedings{	goldwater.jurafsky.ea:which,
	booktitle	= {Proceedings of ACL-08: HLT},
	author	= {Sharon Goldwater and Daniel Jurafsky and Christopher D. Manning},
	address	= {Columbus, Ohio},
	year	= {2008},
	title	= {Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase ASR error rates}
}
```

2007

Modeling Human Performance in Statistical Word Segmentation. Michael C. Frank, Sharon Goldwater, Vikash Mansinghka, Thomas L. Griffiths, Joshua B. Tenenbaum. In Proceedings of the 29th Annual Conference of the Cognitive Science Society. 2007.
[ pdf | bib | abstract ]

What mechanisms support the ability of human infants, adults, and other primates to identify words from fluent speech using distributional regularities? In order to better characterize this ability, we collected data from adults in an artificial language segmentation task similar to Saffran, Newport, and Aslin (1996) in which the length of sentences was systematically varied between groups of participants. We then compared the fit of a variety of computational models— including simple statistical models of transitional probability and mutual information, a clustering model based on mutual information by Swingley (2005), PARSER (Perruchet & Vintner, 1998), and a Bayesian model. We found that while all models were able to successfully complete the task, fit to the human data varied considerably, with the Bayesian model achieving the highest correlation with our results.
```
@InProceedings{	frank.goldwater.ea:modeling,
	booktitle	= {Proceedings of the 29th Annual Conference of the Cognitive Science Society},
	author	= {Michael C. Frank and Sharon Goldwater and Vikash Mansinghka and Thomas L. Griffiths and Joshua B. Tenenbaum},
	year	= {2007},
	title	= {Modeling Human Performance in Statistical Word Segmentation}
}
```

Distributional cues to word segmentation: Context is important. Sharon Goldwater, Thomas L. Griffiths, Mark Johnson. In Proceedings of the 31st Boston University Conference on Language Development. 2007. (If you plan to cite results from this paper, see this note.)
[ pdf | bib ]

@InProceedings{	goldwater.griffiths.ea:distributional,
	booktitle	= {Proceedings of the 31st Boston University Conference on Language Development},
	author	= {Sharon Goldwater and Thomas L. Griffiths and Mark Johnson},
	year	= {2007},
	title	= {Distributional cues to word segmentation: Context is important}
}

A Fully Bayesian Approach to Unsupervised Part-of-Speech Tagging. Sharon Goldwater, Thomas L. Griffiths. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. 2007.
[ pdf | bib | abstract ]

Unsupervised learning of linguistic structure is a difficult problem. A common approach is to define a generative model and maximize the probability of the hidden structure given the observed data. Typically, this is done using maximum-likelihood estimation (MLE) of the model parameters. We show using part-of-speech tagging that a fully Bayesian approach can greatly improve performance. Rather than estimating a single set of parameters, the Bayesian approach integrates over all possible parameter values. This difference ensures that the learned structure will have high probability over a range of possible parameters, and permits the use of priors favoring the sparse distributions that are typical of natural language. Our model has the structure of a standard trigram HMM, yet its accuracy is closer to that of a state-of-the-art discriminative model (Smith and Eisner, 2005), up to 14 percentage points better than MLE. We find improvements both when training from data alone, and using a tagging dictionary.
```
@InProceedings{	goldwater.griffiths:fully,
	booktitle	= {Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics},
	author	= {Sharon Goldwater and Thomas L. Griffiths},
	address	= {Prague, Czech Republic},
	year	= {2007},
	title	= {A Fully Bayesian Approach to Unsupervised Part-of-Speech Tagging}
}
```
Adaptor Grammars: a Framework for Specifying Compositional Nonparametric Bayesian Models. Mark Johnson, Thomas L. Griffiths, Sharon Goldwater. In Advances in Neural Information Processing Systems 19, pp. 641--648. 2007. (This is an updated version that fixes a typo in equation 4. Thanks to Julia Hockenmaier for pointing this out.)
[ pdf | bib | abstract ]

This paper introduces adaptor grammars, a class of probabilistic models of language that generalize probabilistic context-free grammars (PCFGs). Adaptor grammars augment the probabilistic rules of PCFGs with "adaptors" that can induce dependencies among successive uses. With a particular choice of adaptor, based on the Pitman-Yor process, nonparametric Bayesian models of language using Dirichlet processes and hierarchical Dirichlet processes can be written as simple grammars. We present a general-purpose inference algorithm for adaptor grammars, making it easy to define and use such models, and illustrate how several existing nonparametric Bayesian models can be expressed within this framework.
```
@InProceedings{	johnson.griffiths.ea:adaptor,
	editor	= {B. Sch"olkopf and J. Platt and T. Hoffman},
	publisher	= {MIT Press},
	booktitle	= {Advances in Neural Information Processing Systems 19},
	author	= {Mark Johnson and Thomas L. Griffiths and Sharon Goldwater},
	address	= {Cambridge, MA},
	year	= {2007},
	pages	= {641--648},
	title	= {Adaptor Grammars: a Framework for Specifying Compositional Nonparametric Bayesian Models}
}
```
Bayesian Inference for PCFGs via Markov chain Monte Carlo. Mark Johnson, Thomas L. Griffiths, Sharon Goldwater. In Proceedings of Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics. 2007.
[ pdf | bib | abstract ]

This paper presents two Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference of probabilistic context free grammars (PCFGs) from terminal strings, providing an alternative to maximum-likelihood estimation using the Inside-Outside algorithm. We illustrate these methods by estimating a sparse grammar describing the morphology of the Bantu language Sesotho, demonstrating that with suitable priors Bayesian techniques can infer linguistic structure in situations where maximum likelihood methods such as the Inside-Outside algorithm only produce a trivial grammar.
```
@InProceedings{	johnson.griffiths.ea:bayesian,
	booktitle	= {Proceedings of Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics},
	author	= {Mark Johnson and Thomas L. Griffiths and Sharon Goldwater},
	year	= {2007},
	title	= {Bayesian Inference for PCFGs via Markov chain Monte Carlo}
}
```

2006

Contextual Dependencies in Unsupervised Word Segmentation. Sharon Goldwater, Thomas L. Griffiths, Mark Johnson. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics. 2006.
[ pdf | bib | abstract ]

Developing better methods for segmenting continuous text into words is important for improving the processing of Asian languages, and may shed light on how humans learn to segment speech. We propose two new Bayesian word segmentation methods that assume unigram and bigram models of word dependencies respectively. The bigram model greatly outperforms the unigram model (and previous probabilistic models), demonstrating the importance of such dependencies for word segmentation. We also show that previous probabilistic models rely crucially on suboptimal search procedures.
```
@InProceedings{	goldwater.griffiths.ea:contextual,
	booktitle	= {Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics},
	author	= {Sharon Goldwater and Thomas L. Griffiths and Mark Johnson},
	address	= {Syndney, Australia},
	year	= {2006},
	title	= {Contextual Dependencies in Unsupervised Word Segmentation}
}
```
Interpolating Between Types and Tokens by Estimating Power-Law Generators. Sharon Goldwater, Thomas L. Griffiths, Mark Johnson. In Advances in Neural Information Processing Systems 18, pp. 459--466. 2006. (This is a corrected version.)
[ pdf | bib | abstract ]

Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process---the Pitman-Yor process---as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.
```
@InProceedings{	goldwater.griffiths.ea:interpolating,
	publisher	= {MIT Press},
	booktitle	= {Advances in Neural Information Processing Systems 18},
	author	= {Sharon Goldwater and Thomas L. Griffiths and Mark Johnson},
	address	= {Cambridge, MA},
	year	= {2006},
	pages	= {459--466},
	title	= {Interpolating Between Types and Tokens by Estimating Power-Law Generators}
}
```
Nonparametric Bayesian Models of Lexical Acquisition. Sharon Goldwater. Ph.D. Dissertation, Brown University, 2006. (This is the tree-saving version of my thesis, single-spaced with minimal front matter. The official version is double-spaced and contains more front matter. If you plan to cite results on word segmentation, see this note.)
[ pdf | bib ]
```
@PhDThesis{	goldwater:nonparametric,
	school	= {Brown University},
	author	= {Sharon Goldwater},
	year	= {2006},
	title	= {Nonparametric Bayesian Models of Lexical Acquisition}
}
```
A Non-Parametric Bayesian Approach to Spike Sorting. Frank Wood, Sharon Goldwater, Michael Black. In Proceedings of the 28th IEEE Conference on Engineering in Medicine and Biologicial Systems. 2006.
[ pdf | bib | abstract ]

In this work we present and apply infinite Gaussian mixture modeling, a non-parametric Bayesian method, to the problem of spike sorting. As this approach is Bayesian, it allows us to integrate prior knowledge about the problem in a principled way. Because it is non-parametric we are able to avoid model selection, a difficult problem that most current spike sorting methods do not address. We compare this approach to using penalized log likelihood to select the best from multiple finite mixture models trained by expectation maximization. We show favorable offline sorting results on real data and discuss ways to extend our model to online applications.
```
@InProceedings{	wood.goldwater.ea:non-parametric,
	booktitle	= {Proceedings of the 28th IEEE Conference on Engineering in Medicine and Biologicial Systems},
	author	= {Frank Wood and Sharon Goldwater and Michael Black},
	year	= {2006},
	title	= {A Non-Parametric Bayesian Approach to Spike Sorting}
}
```

2005

Representational Bias in Unsupervised Learning of Syllable Structure. Sharon Goldwater, Mark Johnson. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CONLL '05). 2005.
[ pdf | bib | abstract ]

Unsupervised learning algorithms based on Expectation Maximization (EM) are often straightforward to implement and provably converge on a local likelihood maximum. However, these algorithms often do not perform well in practice. Common wisdom holds that they yield poor results because they are overly sensitive to initial parameter values and easily get stuck in local (but not global) maxima. We present a series of experiments indicating that for the task of learning syllable structure, the initial parameter weights are not crucial. Rather, it is the choice of model class itself that makes the difference between successful and unsuccessful learning. We use a language-universal rule-based algorithm to find a good set of parameters, and then train the parameter weights using EM. We achieve word accuracy of 95.9% on German and 97.1% on English, as compared to 97.4% and 98.1% respectively for supervised training.
```
@InProceedings{	goldwater.johnson:representational,
	booktitle	= {Proceedings of the Ninth Conference on Computational Natural Language Learning (CONLL '05)},
	author	= {Sharon Goldwater and Mark Johnson},
	address	= {Ann Arbor},
	year	= {2005},
	title	= {Representational Bias in Unsupervised Learning of Syllable Structure}
}
```
Improving Statistical MT Through Morphological Analysis. Sharon Goldwater, David McClosky. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2005.
[ pdf | bib | abstract ]

In statistical machine translation, estimating word-to-word alignment probabilities for the translation model can be difficult due to the problem of sparse data: most words in a given corpus occur at most a handful of times. With a highly inflected language such as Czech, this problem can be particularly severe. In addition, much of the morphological variation seen in Czech words is not reflected in either the morphology or syntax of a language like English. In this work, we show that using morphological analysis to modify the Czech input can improve a Czech-English machine translation system. We investigate several different methods of incorporating morphological information, and show that a system that combines these methods yields the best results. Our final system achieves a BLEU score of .333, as compared to .270 for the baseline word-to-word system.
```
@InProceedings{	goldwater.mcclosky:improving,
	booktitle	= {Proceedings of the Conference on Empirical Methods in Natural Language Processing},
	author	= {Sharon Goldwater and David McClosky},
	address	= {Vancouver},
	year	= {2005},
	title	= {Improving Statistical MT Through Morphological Analysis}
}
```

2004

Priors in Bayesian Learning of Phonological Rules. Sharon Goldwater, Mark Johnson. In Proceedings of the Seventh Meeting of the ACL Special Interest Group in Computational Phonology (SIGPHON '04). 2004.
[ pdf | bib | abstract ]

This paper describes a Bayesian procedure for unsupervised learning of phonological rules from an unlabeled corpus of training data. Like Goldsmith’s Linguistica program (Goldsmith, 2004b), whose output is taken as the starting point of this procedure, our learner returns a grammar that consists of a set of signatures, each of which consists of a set of stems and a set of suffixes. Our grammars differ from Linguistica’s in that they also contain a set of phonological rules, specifically insertion, deletion and substitution rules, which permit our grammars to collapse far more words into a signature than Linguistica can. Interestingly, the choice of Bayesian prior turns out to be crucial for obtaining a learner that makes linguistically appropriate generalizations through a range of different sized training corpora.
```
@InProceedings{	goldwater.johnson:priors,
	booktitle	= {Proceedings of the Seventh Meeting of the ACL Special Interest Group in Computational Phonology (SIGPHON '04)},
	author	= {Sharon Goldwater and Mark Johnson},
	address	= {Barcelona, Spain},
	year	= {2004},
	title	= {Priors in Bayesian Learning of Phonological Rules}
}
```

2003

Statically finding errors in spreadsheets. Yanif Ahmad, Tudor Antoniu, Sharon Goldwater, Shriram Krishnamurthi. In Proceedings of the IEEE International Conference on Software Engineering. 2003.
[ pdf | bib | abstract ]

We describe a methodology for detecting user errors in spreadsheets, using the notion of units as our basic elements of checking. We define the concept of a header and discuss two types of relationships between headers, namely is-a and has-a relationships. With these, we develop a set of rules to assign units to cells in the spreadsheet. We check for errors by ensuring that every cell has a well-formed unit. We describe an implementation of the system that allows the user to check Microsoft Excel spreadsheets. We have run our system on practical examples, and even found errors in published spreadsheets.
```
@InProceedings{	ahmad.antoniu.ea:statically,
	booktitle	= {Proceedings of the IEEE International Conference on Software Engineering},
	author	= {Yanif Ahmad and Tudor Antoniu and Sharon Goldwater and Shriram Krishnamurthi},
	year	= {2003},
	title	= {Statically finding errors in spreadsheets}
}
```
Learning OT Constraint Rankings Using a Maximum Entropy Model. Sharon Goldwater, Mark Johnson. In Proceedings of the Workshop on Variation within Optimality Theory, pp. 113--122. 2003.
[ pdf | bib | abstract ]

A weakness of standard Optimality Theory is its inability to account for grammars with free variation. We describe here the Maximum Entropy model, a general statistical model, and show how it can be applied in a constraint-based linguistic framework to model and learn grammars with free variation, as well as categorical grammars. We report the results of using the MaxEnt model for learning two different grammars: one with variation, and one without. Our results are as good as those of a previous probabilistic version of OT, the Gradual Learning Algorithm (Boersma, 1997), and we argue that our model is more general and mathematically well-motivated.
```
@InProceedings{	goldwater.johnson:learning,
	booktitle	= {Proceedings of the Workshop on Variation within Optimality Theory},
	author	= {Sharon Goldwater and Mark Johnson},
	address	= {Stockholm University},
	year	= {2003},
	pages	= {113--122},
	title	= {Learning OT Constraint Rankings Using a Maximum Entropy Model}
}
```

2000

Building a Robust Dialogue System with Limited Data. Sharon Goldwater, Elizabeth Owen Bratt, Jean-Mark Gawron, John Dowding. In Proceedings of the NAACL Workshop on Conversational Systems. 2000.
[ pdf | bib ]

@InProceedings{	goldwater.bratt.ea:building,
	booktitle	= {Proceedings of the NAACL Workshop on Conversational Systems},
	author	= {Sharon Goldwater and Elizabeth Owen Bratt and Jean-Mark Gawron and John Dowding},
	address	= {Seattle, WA},
	year	= {2000},
	title	= {Building a Robust Dialogue System with Limited Data}
}

Compiling language models from a linguistically motivated unification grammar. Rayner, Manny, Hockey, Beth Ann, James, Frankie, Bratt, Elizabeth Owen, Goldwater, Sharon, Gawron, Jean Mark. In Proceedings of the 18th Conference on Computational linguistics, Volume 2, pp. 670--676. 2000.
[ pdf | bib | abstract ]

Systems now exist which are able to compile unification grammars into language models that can be included in a speech recognizer, but it is so far unclear whether non-trivial linguistically principled grammars can be used for this purpose. We describe a series of experiments which investigate the question empirically, by incrementally constructing a grammar and discovering what problems emerge when successively larger versions are compiled into nite state graph representations and used as language models for a medium-vocabulary recognition task.
```
@InProceedings{	rayner.hockey.ea:compiling,
	booktitle	= {Proceedings of the 18th Conference on Computational linguistics, Volume 2},
	author	= {Rayner, Manny and Hockey, Beth Ann and James, Frankie and Bratt, Elizabeth Owen and Goldwater, Sharon and Gawron, Jean Mark},
	year	= {2000},
	pages	= {670--676},
	title	= {Compiling language models from a linguistically motivated unification grammar}
}
```

1999

Interpreting language in context in CommandTalk. John Dowding, Elizabeth Owen Bratt, Sharon Goldwater. In Proceedings of Communicative Agents: The Use of Natural Language in Embodied Systems. 1999.
[ pdf | bib ]

@InProceedings{	dowding.bratt.ea:interpreting,
	booktitle	= {Proceedings of Communicative Agents: The Use of Natural Language in Embodied Systems},
	author	= {John Dowding and Elizabeth Owen Bratt and Sharon Goldwater},
	year	= {1999},
	title	= {Interpreting language in context in CommandTalk}
}

1998

Edge-based best-first chart parsing. Eugene Charniak, Sharon Goldwater, Mark Johnson. In Proceedings of the Sixth Workshop on Very Large Corpora at COLING-ACL. 1998.
[ pdf | bib ]

@InProceedings{	charniak.goldwater.ea:edge-based,
	booktitle	= {Proceedings of the Sixth Workshop on Very Large Corpora at COLING-ACL},
	author	= {Eugene Charniak and Sharon Goldwater and Mark Johnson},
	year	= {1998},
	title	= {Edge-based best-first chart parsing}
}

Thanks to Charles Sutton for the scripts used to generate this page automatically from my BibTeX file. You too can download the scripts here.