论文题目: 分析性口试评分表的效度研究
论文类型: 博士论文
论文专业: 英语语言文学
作者: 刘芹
导师: 邹申
关键词: 分析性口试评分表,效度研究,信度,构想效度,外部效度
文献来源: 上海外国语大学
发表年度: 2005
论文摘要: 中国大学师生历来十分重视英语口语能力的培养。他们在课内外教学上作出了很大努力。然而,如何全面、详细地评估这一能力却鲜有涉及。因此,本篇论文旨在提供一套分析性口试评分表(OARS)以试图解决这一问题。该评分表以Bachman的交际语言能力(CLA)模式,Cohen的语用口试等级评分表,Nunn的小组讨论等级评分表,英语专业及非英语专业英语教学大纲,以及对182位教师和1139位学生的问卷调查结果为基础设计而成。它主要分为语言能力和策略能力两大方面。语言能力包括组织能力(即语音、语法、语篇)和语用能力(言外语用能力和社会语用能力)。策略能力考察灵活交际能力和非言语交际能力(即体势语和副语言)。OARS共有八个小项,每项分五个等级(5分为最高,1分为最低),每个小项配备详细的等级描述语。语言测试是与实用性紧密相连的一门学科。理论上设计完好的测试体系在实际应用时难免发生问题。因此为检验OARS的信度、效度和实用性,笔者对其进行了如下两阶段的效度研究。第一阶段是小范围效度研究。它主要涉及该评分表在应用上可能存在的问题以及信度和效度。上海理工大学(USST)的三十名英语专业二年级学生参加了本次研究的试测工作。他们被随机分成三位一组,每组进行五分钟的讨论。试测全程录像后交由两位评分员按照OARS的等级描述语打细评分(即分析分),另由六位教师根据OARS的框架打粗评分(即印象分)。求出细评分的平均分作为整项研究的核心。考生在试测同期参加的英语专业四级考试(TEM4)和口语课程大考成绩亦记录在案作为外部效标。经过信度检验、平均分检验及相关检验得出如下结论。首先,OARS的信度系数为0.7902,符合口语测试标准。第二,绝大多数部分-整体相关系数超过0.7而部分-部分相关系数低于0.6,显示合理的构想效度。第三,细评分平均分与粗评分平均分及口语课程大考成绩的相关系数分别达到0.831和0.653,表明较高外部效度。另外,参加该项研究的师生对OARS持肯定态度,他们普遍认
论文目录:
Acknowledgements
Abstract
摘要
List of Acronyms
List of Figures
List of Tables
Table of Contents
Chapter 1 Introduction
1.1 Need to the study
1.1.1 Students’perspective
1.1.2 Teachers’perspective
1.1.3 Purpose of the study
1.2 Configuration of the study
1.2.1 Study fields
1.2.2 Study questions
1.3 Significance of the study
1.4 Layout of the dissertation
1.5 Summary
Chapter 2 Literature Review
2.1 Fundamentals of oral proficiency
2.1.1 Speech communication and its characteristics
2.1.2 Essence of speech communication
2.1.2.1 Language use versus usage
2.1.2.2 Communicative competence versus linguistic competence
2.1.2.3 Accuracy versus fluency
2.1.2.4 Correctness versus appropriateness
2.1.2.5 Proficiency versus achievement
2.1.2.6 Process versus product
2.1.2.7 Synthetic approach versus analytic approach
2.1.2.8 The learner versus the teacher
2.1.3 Components of oral proficiency
2.2 Classification of oral rating scales
2.2.1 Holistic rating scales
2.2.2 Analytic rating scales
2.2.3 Combination of holistic & analytic rating scales
2.2.4 Integration of holistic & analytic rating scales
2.2.5 Using analytic rating scales for the project
2.3 Summary
Chapter 3 Study Bases
3.1 Empirical basis
3.1.1 Syllabus description
3.1.2 Oral tests of CET & TEM
3.2 Theoretical basis
3.2.1 CLA model
3.2.1.1 Language competence
3.2.1.2 Strategic competence
3.2.1.3 Psychophysiological mechanisms
3.2.1.4 Significance of CLA to OARS
3.2.1.5 Trait factors of CLA
3.2.2 Cohen’s rating scale for pragmatic speaking
3.2.3 Nunn’s rating scales for small group interaction
3.2.3.1 Nunn’s rating scales of interactive skills
3.2.3.2 Nunn’s rating scales of intelligibility
3.2.3.3 Significance of Nunn’s rating scales to OARS
3.2.4 Reconsiderations on strategic competence
3.2.4.1 Classification of strategic competence
3.2.4.2 Evaluation of nonverbal communication
3.3 Summary
Chapter 4 Pre-validation Research
4.1 Needs analysis
4.1.1 Basic considerations on band descriptors
4.1.2 Questionnaire construction
4.1.2.1 The trial questionnaire
4.1.2.2 Population analysis
4.1.2.3 Data analysis
4.1.3 Survey results
4.1.3.1 Population analysis
4.1.3.2 The OARS questionnaire
4.1.3.3 Data analysis
4.2 Description of OARS
4.2.1 Framework of OARS
4.2.2 Band descriptors of OARS
4.2.2.1 Band descriptors of organizational competence
4.2.2.2 Band descriptors of pragmatic competence
4.2.2.3 Band descriptors of flexible interaction
4.2.2.4 Band descriptors of nonverbal communication
4.3 Sample rating
4.3.1 Sample rating I –role-play
4.3.1.1 Performance analysis
4.3.1.2 Suggested rating
4.3.2 Sample rating II –discussion
4.3.2.1 Performance analysis
4.3.2.2 Suggested rating
4.4 Summary
Chapter 5 The Pilot Validation Study
5.1 Research purpose
5.2 Research subjects and task
5.3 Research operation
5.4 Data analysis
5.4.1 Prior consideration
5.4.1.1 Candidates’perspective
5.4.1.2 Teachers’perspective
5.4.2 The profound study
5.4.2.1 Analyses of basic statistics
5.4.2.2 Analyses of construct validity
5.4.2.3 Analyses of external validity
5.5 Discussion
5.6 Summary
Chapter 6 The Field Validation Study
6.1 Research purpose
6.2 Research subjects and task
6.3 Research operation
6.3.1 Process of experimental tests
6.3.2 Rater training
6.3.3 Collection of data
6.4 Data analysis
6.4.1 Study on 6 types of candidates
6.4.1.1 Analyses of basic statistics
6.4.1.2 Analyses of construct validity
6.4.1.3 Analyses of external validity
6.4.2 Study on 180 candidates
6.4.2.1 Analyses of basic statistics
6.4.2.2 Analyses of construct validity
6.4.2.3 Analyses of external validity
6.4.3 Comparability study
6.4.3.1 English majors versus non English majors
6.4.3.2 Arts majors versus Science majors
6.4.3.3 English majors of key universities versus non English majors of key umiersities
6.4.3.4 English majors of local universities versus non English majors of local universities
6.4.3.5 English majors of key universities versus English majors of local universities
6.4.3.6 Non English majors of key universities versus non English majors of local universities
6.4.3.7 Arts majors of key universities versus Arts majors of local universities
6.4.3.8 Science majors of key universities versus Science majors of local universities
6.5 Discussion
6.6 Summary
Chapter 7 Conclusions and Recommendations
7.1 Validation results
7.2 Washback effects
7.2.1 Design of OARS as a model
7.2.2 Precautions in application of OARS
7.2.3 Contribution of OARS to oral instruction
7.3 Existing problems
7.3.1 Problems with OARS itself
7.3.2 Problems with the validation study
7.4 Further research
7.5 Summary
Bibliography
Appendix 1 Empirical Basis
Appendix 1.1 Requirements of oral proficiency of English majors (Chinese version)
Appendix 1.2 Requirements of oral proficiency of non English majors (Chinese version)
Appendix 1.3 Requirements of pronunciation of English majors at Band 2 & Band 4 (Chinese version)
Appendix 1.4 Rating scale & range descriptions of CET-SET (Chinese version)
Appendix 1.5 Range descriptions of spoken test of TEM4 (Chinese version)
Appendix 1.6 Range descriptions of spoken test of TEM8 (Chinese version)
Appendix 2 Questionnaires
Appendix 2.1 OARS trial questionnaire
Appendix 2.2 OARS teachers’questionnaire
Appendix 2.3 OARS students’questionnaire
Appendix 2.4 Candidate’s questionnaire –the Pilot Study
Appendix 3 Experiments
Appendix 3.1 Experimental test of sample rating I
Appendix 3.2 Experimental test of sample rating II
Appendix 3.3 Experimental test of the validation study –role cards
Appendix 4 Data of the Validation Study
Appendix 4.1 Candidates’raw scores of ORAL & TEM4 –the Pilot Study
Appendix 4.2 Raw scores of candidates’ratings –the Pilot Study
Appendix 4.3 Raw scores of raters’detailed ratings –the Pilot Study
Appendix 4.4 Raw scores of teachers’rough ratings –the Pilot Study
Appendix 4.5 Sub-total correlation matrix –the Pilot Study
Appendix 4.6 Sub-total correlation matrix for Type 1 candidates –the Field Study
Appendix 4.7 Sub-total correlation matrix for Type 2 candidates –the Field Study
Appendix 4.8 Sub-total correlation matrix for Type 3 candidates –the Field Study
Appendix 4.9 Sub-total correlation matrix for Type 5 candidates –the Field Study
Appendix 4.10 Sub-total correlation matrix for Type 6 candidates –the Field Study
Appendix 4.11 Sub-total correlation matrix of R1T for 180 candidate –the Field Study
Appendix 4.12 Sub-total correlation matrix of RAT for 180 candidate –the Field Study
Appendix 4.13 T-test (English majors versus non English majors) –the Field Study
Appendix 4.14 T-test (Arts majors versus Science majors) –the Field Study
Appendix 4.15 T-test (English majors of key universities versus non English majors of key universities) –the Field Study
Appendix 4.16 T-test (English majors of local universities versus non English majors of local universities) –the Field Study
Appendix 4.17 T-test (English majors of key universities versus English majors of local universities) –the Field Study
Appendix 4.18 T-test (non English majors of key universities versus non English majors of local universities) –the Field Study
Appendix 4.19 T-test (Arts majors of key universities versus Arts majors of local universities) –the Field Study
Appendix 4.20 T-test (Science majors of key universities versus Science majors of local universities) –the Field Study
发布时间: 2006-12-30
参考文献
- [1].考试效度研究的互动性[D]. 邹申.上海外国语大学2005
标签：分析性口试评分表论文; 效度研究论文; 信度论文; 构想效度论文; 外部效度论文;