对于词汇增长数学模型的实证性研究

论文摘要

基于BNC笔语语料库,本文研究了八千万英语笔语的篇际词汇增长曲线,并对现有数学模型进行了拟合度检验。本文主要解决了关于篇际词汇增长的四个问题,即单独语篇的词汇量分布,八千万篇际词汇增长模型,篇际新增词汇的递减模式,以及新增词汇的词性分析。研究表明,长度约为2000单词的语篇词汇量呈正态分布,由此可以计算出语篇词汇量的95%双向置信区间值。在对八千万单词的语料样本进行了Brunet模型,Herdan-Heaps模型,Guiraud模型,Tuldava模型,以及K（?）hler,Martináková模型检验后,发现当语料样本大于五万单词而小于八千万词单词时,Herdan-Heaps模型能很精确的描述词汇增长曲线;而对于小于五万单词的语料样本,Tuldava模型则能很好的描述词汇增长曲线。篇际新增词汇随语料样本增大而呈现递减曲线,而这些新词在词性分布上有明显的特征。研究表明,除专有名词,阿拉伯数字,连字符构成的合成词,以及非单词字符会大大增加英语词汇量外,大量的普通名词,形容词,动词以及副词也是英语新增词汇的主要来源。这些单词都是造成词汇分布及词汇增长中LNRE现象的主要原因。

论文目录

• 摘要
• Abstract
• Chapter One Introduction
• 1.1 Background of Vocabulary Study
• 1.2 Objective of This Paper
• 1.3 Significance of This Paper
• 1.4 Research Method
• 1.5 Related Concepts
• 1.5.1 Token and Type
• 1.5.2 Word Family and Lemma
• 1.5.3 Tokenization and Lemmatization
• 1.6 Layout of the Thesis
• Chapter Two Theoretical Background
• 2.1 Zipfs Law
• 2.2 Characteristic Constants
• 2.3 Lexical Growth Models
• 2.3.1 Guiraud's Model
• 2.3.2 Brunet's Model
• 2.3.3 Kohler-Martináková's Model
• 2.3.4 Tuldava's Model
• 2.3.5 Herdan-Heaps'Model
• 2.3.6 Fan's Model Test
• 2.4 LNRE Zone
• Chapter Three Data Collection and Processing
• 3.1 Corpus Description
• 3.2 Tools for Data Processing
• 3.3 Procedures for Data Processing
• Chapter Four Results and Analysis
• 4.1 Vocabulary Size Distributions of Individual Texts
• 4.2 Inter-textual Vocabulary Growth Pattern
• 4.2.1 Test of Vocabulary Growth Models for 50000≤N≤80000000
• 4.2.2 Analysis of Residual Errors for Existing Models
• 4.2.3 Calculation of Parameters
• 4.2.4 Vocabulary Growth Pattern for N≤50000
• 4.3 Decreasing Pattern of Newly Occurring Words
• 4.4 Analysis of Newly Occurring Words
• 4.5 Discussion
• Chapter Five Conclusion
• 5.1 Major Findings of the Study
• 5.2 Suggestions for Further Study
• Reference
• Appendix A
• Program for Random Sampling
• Program for Tokenization
• Program for Calculating V（N） and Vj against N
• Program for Obtaining Part of Speech Data on the Newly Occurring Words
• Appendix B
• Part of the Newly Occurring Words Brought by 2 Million Words Input into the 80 Million Words（Initial Letter A）
• Acknowledgements
• Resume