Help

Learn About Scoring

Select Grade  


Introduction
Critical Information
Assessment Information
Understanding Scoring
Glossary
Rubrics
Introduction
+
 Welcome

Welcome to Understand Scoring-powered by Perspective

This program is designed to help you
  • gain a better understanding of the Virginia Standards of Learning Direct Writing Assessments for grade 5, grade 8, and high school
  • understand the use of rubrics in scoring student writing
  • develop consistency in scoring actual student work

Specifically, this program will
  • provide information about the Virginia Standards of Learning English Assessment
  • provide an opportunity to study the rubrics used for scoring the direct writing assessments
  • provide an opportunity to study anchor papers that illustrate each score point of each domain
  • provide an opportunity to practice scoring several sets of papers written by Virginia students
  • verify your progress by evaluating the accuracy of the scores you assign to a final set of papers (verification set)
The Critical Information sub-section below includes important information you need in order to understand the rubrics and the scoring process. It is extremely important for you to read the Critical Information sub-section before proceeding. You can review the other sub-sections at any time.
+
 Definitions of Sections
Learn About Scoring
   This section provides important information that covers many aspects of Assessment, Scoring, Virginia SOL, and a glossary
    Critical Information
    This section provides critical information about the Virginia English SOL Assessment and scoring student papers.
    AssessmentInformation
    This section provides information about Virginia SOL test development, prompts and committees.
    Understanding Scoring
    This section provides a deeper understanding of scoring. It discusses the people and process used to score Virginal SOL     writing assessments. This section also covers information about rubrics, anchor papers, alert papers, and nonscorable     papers.
    Rubric
    This section describes the criteria for each score point of the rubric for composing, written expression, and usage and     mechanics.
Anchor Papers
   This section allows you to review papers that represent each score point, with annotations and color highlights that explain the assigned scores. The anchor papers will help you understand differences between score points.
Practice Scoring
   This section allows you to practice scoring and check your scores. After scoring practice sets, you can test your progress by attempting a verification set.
Critical Information
+
 What Is Focused Holistic Scoring?
Focused holistic scoring is the evaluation of writing proficiency based on several specific elements or domains of writing. The writer is judged in each domain independent of the others. In Virginia's Direct Writing Assessment each paper receives three scores, one for each of the three writing domains identified: composing, written expression, and usage/mechanics. Each domain is scored holistically. The score reflects the reader's overall impression of each of the specified domains. The scorers weigh the student's relative strengths or weaknesses in each of the three domains and then assign a score point for each domain that most accurately describes the attributes of the paper. Therefore, each paper receives a separate score for composing, written expression, and usage/mechanics.

In focused holistic scoring, it is possible for a paper to receive a higher score in one area such as usage/mechanics and a lower score in another area such as written expression. Assigning separate scores assists a teacher in diagnosing a student's relative strengths and weaknesses so that instructional activities can be planned for those specific skills identified as features within the rubric-scoring guide. A focused holistic scale defines the requirements for each domain at each level of the rubric. Thus, a particular score represents not only an overall impression, but also a specific level of proficiency in the complete set of composition domains being observed.

Virginia's Direct Writing Assessment model uses a 4-point scale. Each Domain is scored independently, using the following scale:

4 =The writer demonstrates consistent, though not necessarily perfect, control* of almost all of the domain's features.

3 =The writer demonstrates reasonable, but not consistent, control* of most of the domain's features indicating some weakness in the domain.

2 =The writer demonstrates enough inconsistent control* of several features to indicate significant weakness in the domain.

1 =The writer demonstrates little or no control* of most of the domain's features.

*Control: The ability to use a given feature of written language effectively at appropriate grade level. A paper receives a higher score to the extent that it demonstrates control of the features in each domain.

As in any scoring, a range of papers is evident within each score point. In a 4-point scale there will be high 4's and low 4's, high 3's and low 3's, etc. Although trainers talk about writing assessments as having discrete score points, the assigned scores actually represent a continuum within the score points. At every score point level, there is a range of papers representing the lower, middle, and upper end of the score point. A paper assigned a score of 3, for example, might have some characteristics of a 2-level paper and some characteristics of a 4-level paper, but overall, its 3-level characteristics predominate. Because of this phenomenon, papers receiving the same score point may look very different from each other.The 3-level paper described above, for example, might be at the top end of the 3's, and look very different than a paper that has only 2- and 3-level characteristics. The second paper is still assigned a 3, but it represents the lower end of the 3's. It is important to understand the criteria used in assigning a score and not to expect papers at a certain score point always to look the same.
+
 How Are Rubrics Used to Score Papers?
Generally, rubrics for writing assessments are fairly generic. It is only in conjunction with anchor (exemplar) papers that the criteria at each score point begin to be clarified.The rubric provides the general description of a score point; the anchor papers are the exemplars of how those general descriptions look in practice. A good set of anchor papers attempts to represent the different ways a student might achieve a score by including papers which attain the same score point in different ways.

Because rubrics are not meant to be lists of mandatory criteria a student must meet in order to achieve a score point, the word "may" is often used in a score point description.This use of the word may encourages scorers to remember that a paper may demonstrate some characteristics of one score point and some of another. A paper ultimately should be assigned a "best fit" or most appropriate score. For example, in the 4-point focused holistic rubric used in Virginia's Direct Writing Assessment, one of the sentences used to describe a 2 in Composing is "Even if a single idea dominates, the writing may be a list of general, underdeveloped statements or a skeletal plot." While this is often the case, it is also possible for a paper that doesn't have a skeletal plot or isn't a list of underdeveloped statements to receive a score of 2 because other 2-level qualities predominate and keep the paper from achieving a higher score.

One of the most important points in learning how to apply a rubric consistently is to let go of existing prejudices. When scoring writing assessments, one of the most difficult tasks is to accept the boundaries of a rubric when those boundaries differ from a reader's own teaching practices or beliefs. For example, a teacher might value a nearly error-free paper in the area of usage/mechanics while a rubric might make allowances for certain errors. Assigning scores to papers in an assessment based on a rubric the teacher did not create is not to suggest teachers must adopt that rubric's criteria to classroom assignments. Large-scale assessments and classroom-based assessments may very well measure somewhat different skills in a different manner.
+
 What Are the General Considerations for the Writing Tests?
No provisions are made for examinees to recopy their final drafts. They are encouraged, however, to revise their work and to make whatever changes they believe will improve their papers. Readers, therefore, should expect and must be tolerant of revisions and interlinear editing in these papers. To gather as much instructionally useful information as possible in its writing assessment, Virginia has adopted a scoring model that calls for evaluations in three domains - composing, style, and usage-mechanics. In this type of scoring, the observation of writing is divided into several domains (categories), each of which is comprised of various features. Each domain is evaluated holistically, with the domain score indicating the extent to which the features in that domain appear to be under the control of the writer. Thus, an awareness of the features and their use contributes to the score, but the score is a judgment of the whole domain and not simply a counting of demonstrated features. Some skilled scorers can score for all domains after one complete reading, although scorers may re-read a portion in order to make a decision about one or more domains.

Application of the Scoring Scale
  1. Look as much at what the writer does well as at what he/she does poorly.
  2. Give consideration to all features within domain.
  3. Do not contaminate the scoring of one domain by awareness of either accomplishments or errors belonging to another domain.
  4. Do not create double jeopardy, e.g., a run-on sentence in Usage-Mechanics is not also a sentence error in Written Expression, and "brite" is one misspelling no matter how often the word appears.
  5. Compare papers to the requirements of each domain, not to each other, or to student writing that you may have previously read, or to some general standard of desired literacy.
  6. Score only what is on the paper. It is impossible to evaluate what the writer might have intended to say.
  7. Note that a word that seems to have been omitted due to haste in revising, editing, or copying need not demonstrate a total lack of control over that feature.
  8. Do not score a paper on only length. Length, in and of itself, is not a feature of any domain.
  9. Read the entire paper before assigning any scores.
Nonscorable Papers
All papers, no matter how brief, must be scored, unless they are off-topic, illegible, incoherent, refusals to respond, or written in a language other than English. Only the Scoring Director is permitted to assign a nonscorable code.
+
 What Is the Grade 5 Writing Prompt Used in Understand Scoring for Virginia?

Write about a happy memory you have.Tell what that memory is and why it is important to you.

CHECKLIST FOR WRITERS

______I planned my paper before writing it.

______I revised my paper to be sure that

______the subject of my paper was clear;

______everything in my paper told about my subject;

______my paper was logically organized so readers would understand my message;

______my words and information made my paper interesting to readers; and

______my sentences made sense, sounded like me, and read smoothly.

______I edited my paper to be sure that

______I used good grammar;

______I used capital letters and punctuation marks correctly;

______I let my readers know were I started new paragraphs; and

______I made my spelling correct.

______I proofread my paper to make sure that my paper was the way I wanted readers to read it.

+
 What Are the Allowable Scoring Interpretations?
When scoring writing samples, decisions must be made regarding how papers with unexpected content, organization, or structure will be scored. While some guidelines are general, other decisions must be made because a significant number of writers have chosen to respond to a specific prompt in an appropriate but unanticipated way. These "rules" generally cannot be planned in advance but come out of issues that arise during the scoring process. Teachers should consider that the following list does not necessarily represent rules that would apply to classroom assignments or even classroom assessments. They are offered here as examples of considerations of how the papers written in response to the prompts included in these training materials were scored.

Grade 5 Prompt: Write about a happy memory you have. Tell what that memory is and why it is important to you.

Grade 5 Prompt-Specific Scoring Guidelines
  • In general, if it is apparent that the student is responding to the prompt, the paper should be considered on topic and should be scored according to the rubric. (Pay special attention to those papers that look as if they are refusals at first glance but are actually responses to the prompt, e.g., "Asking us to write about a happy memory is silly. It would be better to ask us to write about...").
  • Once a paper is determined to be "on topic," that issue should no longer influence the score assigned. Papers are not scored lower for being slightly on topic or higher for being exactly on topic. The assigned score should reflect only the student's writing ability as described in the rubric.
  • A student may choose to write about several happy memories instead of just one. Such a response should be considered on topic. Depending on how skillfully the student handles the inclusion of multiple memories, the unity of the piece may be negatively affected.
  • It is not necessary for the student to tell what the memory is and tell why it is important. The student may do either.
  • The student may write about a happy memory without ever directly stating that this was his or her happy memory.
  • The reader should assume any memory the writer chooses to share is his/her happy memory. Some writers interpreted "happy" to mean funny or most memorable.
  • The writer may make up a happy memory that includes fantasy or details that are most likely not true, or the memory may come from a book or a movie with which the student is familiar.
  • The student may write about someone else's happy memory.
  • The memory may start out as a happy memory and then become sad or unpleasant in some way or visa versa.
  • The student may state that s/he has no happy memories and write about an unhappy memory instead
  • The reader should not judge the appropriateness of the experience the writer chooses to describe.
  • The reader should not judge the reasons the writer gives for the importance of the memory.
  • The reader should not judge the length of time the memory involves. Some writers may choose to tell about an event that happened over several weeks, others last only a moment.
  • Readers should be forgiving of naive or illogical explanations.
  • The student's score should not be affected by the mode used: narrative, expository, descriptive, imaginative.
+
 How Does Reader Bias Affect Scoring?
Reader bias refers to personal factors that may affect a reader's perception of a student's response, but have no basis in a rubric. Below are factors that affect some readers but must not be allowed to have an impact on scoring.

Appearance of Response.The quality of the handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria.

Length of Response.The length of a student paper is not part of the scoring criteria. Readers should take into consideration only whether the finished piece seems complete and has the components required. The size of a student's handwriting can make a paper look longer or shorter on the page than it actually is.

Repetition of Response.Although readers may tire of reading several essays on the same topic, it is important to remember that for each student the response represents a unique attempt.

Offensive or Disturbing Content.If a student uses vulgar language, adopts a sexist or racist point of view, or perhaps takes a naive or narrow approach to a topic, readers should not let the student's point of view affect the score. Likewise, readers should not let a student's lifestyle or maturity level influence them either positively or negatively regarding the writing.

Alert Papers.Because the prompts are designed to have meaning to students so they can create a personal response, sometimes the paper relates situations and information that are disturbing. It is Virginia's policy to bring disturbing papers to the attention of an adult close to the student as soon as possible so that positive action can be taken if necessary. Readers are instructed to indicate to the Scoring Director any paper that is found to be disturbing and then score the paper according to the criteria.

Response to Prompt.In the classroom, there may be a "correct" response to a writing assignment. For Virginia's assessment, students are free to respond in any way they choose. There is no right or wrong "answer" as long as it is clear that the student is attempting to reply to the prompt.

Reactions to Style.A reader's own grammatical biases should not play a part in assigning a score if the student has not violated standard writing conventions. In other words, beginning a sentence with "and," the absence of a formal thesis sentence, the use of first or second person, or an informal tone are not wrong in this type of assessment. Classroom assessments often have different requirements.

Writer's Personality.Writers may come across as brash, sassy, cute, self-aware, shy, surly, flat, honest, or naive. Readers are scoring the written passage, not the writer's personality.

Reactions to Direct Writing Assessments.Some readers may approach writing assessments with their own biases in favor of one type of assessment over another. Or, they may believe it is impossible to score writing fairly using any of these techniques. The prompts and rubrics used in Understand Scoring for Virginia have been analyzed and validated by experts in the field and adhere to standard practices in state assessments. Teachers should tailor their own classroom writing assessments around local curricular expectations.
Assessment Information
+
 How Are Standards of Learning Tests Developed?
The development of each SOL test begins with the blueprint for that test. The test blueprint serves as a guide to test developers as they write test questions and construct the SOL tests. The test blueprints show (a) the Standards covered by the test and which, if any, have been excluded; (b) which Standards are assigned to each reporting category; (c) the number of test items in each reporting category and on the total test; (d) general information about how the test questions were constructed; and (e) the materials that students are allowed to use while taking the test.

Using the test blueprint as a guide, the test development contractor writes test items designed to measure student achievement of the particular Standards of Learning covered by that test. Each test item is coded to a particular SOL.

The content review committee for that test reviews each proposed test item, answering the following questions for each item:

SOL Item Match:Does this item measure the SOL it was designed to measure?

Appropriateness:Does this item appropriately measure content or skills that students in Virginia should be required to have by spring of the designated grade level or near the end of the course?

Difficulty:Is the difficulty of this item appropriate?

Fairness:Is the item free from any content that would offend or unfairly penalize students on the basis of personal characteristics such as gender, ethnicity, religion, or socioeconomic status?

If the answer to each of these questions is "yes," the item is accepted for field testing and tried out with Virginia students. Items are tried out by placing them as field test items on an SOL test. Field test items do not count toward the student's score on the test; however, data are collected on how students respond to the items. The number of field test items on a test varies depending on the test: the multiple choice section of the English Writing Test for grade 5 has 12 field test items, while the multiple choice section of English Writing Test for grade 8 and the multiple choice section of Secondary English Writing Test both have 14 field test items.

Once the field test data are collected, the content review committee reviews the items again using the statistical information collected during the field test and decides whether each item should be accepted into the item bank for potential use on a future test form. In the case of the high school tests, a separate Bias Committee reviews each item a second time for fairness.

Using items that have been approved by the content review committee for inclusion in the item bank, the test development contractor follows the test blueprint to develop test forms. These test forms are reviewed by the content review committee prior to their administration. Once the test form is administered, each test item either returns to the bank or is released as a sample test item.

+
 How Are Prompts Selected?
The process of prompt development begins with the test development contractor's developing potential writing prompts for the direct writing component of the SOL English Writing Test. According to the test blueprints, prompts are to present to the student a context for writing in the form of a question, an issue, or a hypothetical situation. The content review committee then reviews each proposed writing prompt for potential field testing.

If the prompt is accepted for field testing, it is tried out with Virginia students. Papers written to the prompt during the field test are scored by the scoring contractor. The readers involved in the field test scoring evaluate the prompt in terms of whether it would be appropriate for use in live testing. This analysis in addition to information about the number of papers assigned each score point in each domain is provided to the content review committee.

Using this information from the field test, the content review committee reviews the prompts again and decides whether each prompt should be accepted into the prompt bank for potential use on a future test form. Prior to each administration of the SOL English Writing Test, the content review committee selects two prompts to be used: one for the main prompt and one to be used as the make-up prompt for students who are not tested on the statewide writing day.

+
 What Committees Are Involved in the Development of the Standards of Learning Tests?

Content Review Committee

Each SOL test is constructed according to the specifications of the SOL test blueprint, and with the cooperation of a Content Review Committee composed of Virginia educators who have experience and expertise in the content area and grade level of the test. Working with the Department and the testing contractor, the Content Review Committees review each test question before it is field-tested. In order to make it to field test status in Virginia, the potential question must, in the eyes of the Content Review Committee, meet the following four criteria:

  1. Does the question measure the SOL it was designed to measure?
  2. Does the question appropriately measure content or skills that students in Virginia should be expected to learn by the spring of the designated grade level or near the end of the course?
  3. Is the difficulty of the question appropriate?
  4. Is the question free from content that stereotypes, offends, or unfairly penalizes students on the basis of personal characteristics such as gender, ethnicity, religion, or socioeconomic status?

If the question meets these criteria, it is used as a field test question with Virginia students during a regular administration of the SOL tests. Field test questions appear among the operational questions on each SOL test. In this way a steady stream of new test questions is available for use.

After test questions have been field tested, the Content Review Committee is reconvened to review the questions and make the same four judgments again, but this time with statistics about the field-tested questions. If the answers to the four questions are yes, then the test question is placed in the item bank for potential use in an operational SOL test form.

When preparing a test form for operational use, the testing contractor selects questions from the item bank in accordance with the blueprint specifications. Once draft test forms are constructed, the Content Review Committees are again convened. Committee members assume the task of approving or editing two forms of each grade level or end-of-course test to determine the content validity and equivalency of the test forms as a whole. While the previous committee reviews were concerned with individual questions, the focus of this review is the full operational test forms. At this stage there may be additional minor edits or revisions.

In addition to the previously discussed duties, the content review committees for the SOL Writing Tests select the prompt to be used for each administration of the tests and review and approve the materials used to train readers to score the prompt.

Bias Committee
Because passing the high school SOL assessments will be a graduation requirement, it is especially important that the tests be free of factors that have an unfair impact upon a group of students. Therefore, an additional bias review is conducted by a separate Bias Review Committee representing each content area to be tested. Bias Review Committee members are asked to scrutinize items for potential stereotyping or other forms of bias. The purpose of the bias review is to identify any items that appear to have the potential to treat any group, regardless of ethnic background, gender, or region, differently from other groups. Committee members examine the response distribution for each of the demographic groups identified for the review. The intent of the review is to determine if members of a certain group were drawn to one or more of the answer choices for the item. If a large percent of one group selected a particular response or did not select a particular response, the item is carefully examined. Training and procedures are similar to those carried out during the Item Review meetings. The Bias Review Committee's task focuses solely on reviewing test items for potential bias after the items have been reviewed by the Content Review Committees. It is the Bias Review Committee's responsibility to ensure that items are fair to all students and that all students would have an equal opportunity to demonstrate achievement regardless of gender, ethnic background, religion, socioeconomic status, or geographic region.

Assessment Policy Advisory Committee
The Assessment Policy Advisory Committee composed of Virginia educators advises the Virginia Department of Education and the Board of Education on policy issues related to the state-mandated assessment programs. This committee was instrumental in developing the guidelines for testing students with disabilities and the guidelines for testing limited-English-proficient students in the SOL program.
Understanding Scoring
+
 What Is Focused Holistic Scoring?

Grade 5 Focused Holistic Scoring Guidelines

Focused holistic scoring is the evaluation of writing proficiency based on several specific elements or domains of writing. The writer is judged in each domain independent of the others. In Virginia's Direct Writing Assessment each paper receives three scores, one for each of the three writing domains identified: composing, written expression, and usage/mechanics. Each domain is scored holistically. The score reflects the reader's overall impression of each of the specified domains. The scorers weigh the student's relative strengths or weaknesses in each of the three domains and then assign a score point for each domain that most accurately describes the attributes of the paper. Therefore, each paper receives a separate score for composing, written expression, and usage/mechanics.

In focused holistic scoring, it is possible for a paper to receive a higher score in one area such as usage/mechanics and a lower score in another area such as written expression. Assigning separate scores assists a teacher in diagnosing a student's relative strengths and weaknesses so that instructional activities can be planned for those specific skills identified as features within the rubric-scoring guide. A focused holistic scale defines the requirements for each domain at each level of the rubric. Thus, a particular score represents not only an overall impression, but also a specific level of proficiency in the complete set of composition domains being observed.

Virginia's Direct Writing Assessment model uses a 4-point scale. Each Domain is scored independently, using the following scale:

4 =The writer demonstrates consistent, though not necessarily perfect, control* of almost all of the domain's features.

3 =The writer demonstrates reasonable, but not consistent, control* of most of the domain's features indicating some weakness in the domain.

2 =The writer demonstrates enough inconsistent control* of several features to indicate significant weakness in the domain.

1 =The writer demonstrates little or no control* of most of the domain's features.

*Control: The ability to use a given feature of written language effectively at appropriate grade level. A paper receives a higher score to the extent that it demonstrates control of the features in each domain.

As in any scoring, a range of papers is evident within each score point. In a 4-point scale there will be high 4's and low 4's, high 3's and low 3's, etc. Although trainers talk about writing assessments as having discrete score points, the assigned scores actually represent a continuum within the score points. At every score point level, there is a range of papers representing the lower, middle, and upper end of the score point. A paper assigned a score of 3, for example, might have some characteristics of a 2-level paper and some characteristics of a 4-level paper, but overall, its 3-level characteristics predominate. Because of this phenomenon, papers receiving the same score point may look very different from each other.The 3-level paper described above, for example, might be at the top end of the 3's, and look very different than a paper that has only 2- and 3-level characteristics. The second paper is still assigned a 3, but it represents the lower end of the 3's. It is important to understand the criteria used in assigning a score and not to expect papers at a certain score point always to look the same.
+
 How Should the Scale Be Applied?
  1. Look as much at what the writer does well as at what he/she does poorly.
  2. Give consideration to all features within domain.
  3. Do not contaminate the scoring of one domain by awareness of either accomplishments or errors belonging to another domain.
  4. Do not create double jeopardy, e.g., a run-on sentence in Usage-Mechanics is not also a sentence error in Written Expression, and "brite" is one misspelling no matter how often the word appears.
  5. Compare papers to the requirements of each domain, not to each other, or to student writing that you may have previously read, or to some general standard of desired literacy.
  6. Score only what is on the paper. It is impossible to evaluate what the writer might have intended to say.
  7. Note that a word that seems to have been omitted due to haste in revising, editing, or copying need not demonstrate a total lack of control over that feature.
  8. Do not score a paper on length alone. Length, in and of itself, is not a feature of any domain.
  9. Read the entire paper before assigning any scores.
+
 How Should the Rubrics Be Applied?
In learning how to apply a rubric consistently, teachers must let go of existing prejudices. One of the most difficult tasks in writing assessment is to accept the boundaries of a scoring guide when those boundaries differ from a teacher's own teaching practices or beliefs. For example, a teacher might insist upon an error-free paper in the area of usage and mechanics for classroom writing assignments while a rubric might emphasize consistent control of usage and mechanics. Assigning scores to papers in an assessment based on a rubric the teacher did not create is not to suggest teachers must apply that rubric's criteria to classroom assignments. Large scale assessments and classroom-based assessments may very well measure somewhat different skills in a different manner.

Rubrics for writing assessments are generally fairly generic. It is only in conjunction with anchor papers that the criteria at each score point begin to be clarified. The rubric provides the general description of a score point; the anchor papers are the exemplars of how those general descriptions look in practice.
+
 What Are the Procedures for Training Scorers and Scoring?

Introduction to the SOL Writing Assessment

Training of readers begins with a brief introduction to the Virginia SOL Writing Assessment in which readers are informed of the goals and purposes of the assessment program. In addition, readers are told that the assessment is an on-demand writing task in which students are given a prompt and have up to an entire school day to respond to it. Students have space in the answer booklet for notes and are given scratch paper for prewriting and drafting if they need more space. A checklist included with the writing prompt encourages students to follow the writing process. Students have four sheets of paper in the answer booklet to use for their response and are told that only what they write on these four sheets will be scored. They have access to dictionaries and may write their responses in cursive, or they may print. Following the description of the writing task given to students, readers are told that they are to consider the papers written for this assessment to be first draft writing and are to be tolerant of interlinear editing.

Training and Qualifying
A discussion of the three domains used in the scoring model: composing, written expression, and usage/mechanics follows the introduction to the assessment. Readers are introduced to the prompt and then training begins with the composing domain. Trainers first discuss the features of the composing domain as well as the general scale: 4 = consistent control, 3 = reasonable control, 2 = inconsistent control and 1 = little or no control. Score point 1 in the composing rubric is then read, and the score point 1 anchor papers are read aloud and discussed. The discussion emphasizes each paper's written annotation which uses the language of the rubric to explain the score point as it applies to that paper. The score point 2 rubric is then read followed by discussion of the anchor papers for this score point. A similar process is used for score points 3 and 4 in composing. Following the discussion of the rubric and anchor papers prospective readers independently score the papers in the Composing Training Set for the composing domain only. A similar process is used for the written expression domain and the usage/mechanics domain.

Once all domains have been discussed and all domain specific training sets scored, readers begin scoring the mixed domain sets. The purpose of the mixed domain sets is to familiarize readers with assigning scores in each of the three domains to a paper. There are three such sets of 10 papers each (sets A, B, and C), and readers are asked to assign a score in each domain to these papers. Again, scores are discussed in detail at the completion of each set.

After all of the domain training sets have been scored and discussed, prospective readers begin scoring qualifying sets. There are 4 qualifying sets of 10 papers each. Readers are required to have 70% or better exact agreement on each domain in at least one qualifying set with 60 % or better agreement in a second set. Readers who do not qualify are dismissed from the Virginia project.

Scoring
Once readers have qualified, they are assembled into teams and begin scoring papers. Each team is led by a team leader who has been selected based on demonstrated expertise in all phases of the scoring process including skill in training readers and in maintaining the organizational procedures necessary for scoring to move smoothly. During scoring, team leaders respond to readers' questions About Scoring and work with readers who are having difficulty applying the rubric. A primary duty of the table leader is to "read behind" readers to ensure that they are not drifting from the scoring criteria. Team leaders also review the scoring patterns of readers throughout the project and conduct retraining as necessary.

Two readers from different teams read each paper. If the two readers disagree by more than one point in any domain, a third reader reads the paper to resolve the score.

In addition to the team leaders, the scoring contractor's scoring directors and the Virginia Department of Education review daily reader reports that show how well readers are agreeing with each other. Furthermore, each day readers are asked to score packets of papers pre-scored by the content review committee. The reader reports resulting from these calibration packets serve to identify readers who are drifting off the scale, while the discussion of the score points assigned to each paper helps to re-calibrate all readers.

Readers are monitored throughout the scoring process. Those who fail to maintain acceptable agreement with other readers or who fail to accurately apply the scoring criteria despite retraining are dismissed.
+
 What Is Range Finding and How Are Scores Assigned?
For each administration of the SOL writing test, the prompt to be used for the direct writing component is selected by the content review committee for the English Writing Test at that grade level. After the prompt is selected, range finding begins with scoring contractor staff experienced with the Virginia SOL Writing Assessment. They score several hundred papers written to the prompt by Virginia students during the field test. Following this initial scoring, contractor staff discuss the papers and make preliminary selections of papers to be used in anchor sets, training sets, and qualifying sets. Then these preliminary sets are brought to the content review committee for discussion. The content review committee may replace papers or move them to different sets. Once the committee is satisfied with the papers and the scores assigned to them,the papers are ready for final preparation before being used for training readers.
+
 What Are the Domains and Definitions?

Composing

The Composing domain includes the focusing, structuring, and elaborating that a writer does to construct an effective message for a reader. It is the creation of a product, the building of a writing intended to be read. The writer crafts his/her message for the reader by focusing on a central idea, providing elaboration of the central idea, and delivering the central idea and its elaboration in an organized text.

Features:
  • Central idea
  • Elaboration
  • Unity
  • Organization
Written Expression
The Written Expression domain comprises those features that show the writer purposefully shaping and controlling language to affect readers. This domain focuses on the vividness, specificity, and rhythm of the piece and the writer's attitude and presence.

Features:
  • Deliberate word choice
  • Selected, precise information
  • Sentence variety
  • Tone
  • Voice
Usage/Mechanics
The Usage/Mechanics domain comprises the writer's ability to form competent, appropriately mature sentences and the use of word level features that cause written language to be acceptable and effective for standard discourse. This domain includes the system of symbols and cueing devices a writer uses to help readers make meaning.

Features:
  • Sentence formation
  • Usage
    • Standard inflections
    • Comparison of adjectives/adverbs
    • Agreement
  • Mechanics
    • Capitalization
    • Spelling
    • Punctuation
    • Formatting
+
 What Are the Writing Prompts and Checklists?

Grade 5 Writing Prompt and Checklist


Write about a happy memory you have. Tell what that memory is and why it is important to you.

CHECKLIST FOR WRITERS

______I planned my paper before writing it.

______I revised my paper to be sure that

______the subject of my paper was clear;

______everything in my paper told about my subject;

______my paper was logically organized so readers would understand my message;

______my words and information made my paper interesting to readers; and

______my sentences made sense, sounded like me, and read smoothly.

______I edited my paper to be sure that

______I used good grammar;

______I used capital letters and punctuation marks correctly;

______I let my readers know were I started new paragraphs; and

______I made my spelling correct.

______I proofread my paper to make sure that my paper was the way I wanted readers to read it.

+
 What Are the Rubrics?

Grade 5 Rubrics


Composing Rubric


Score Point 4
The writer demonstrates consistent,though not necessarily perfect, control of the composing domain's features. Full elaboration focuses the central idea both at the sentence level and throughout the entire piece by providing purposeful examples, anecdotes, illustrations, or details. Narrative organization is intact; in other modes, minor organizational lapses may occur. However, any organizational lapses that occur do not significantly detract from the presentation. Several elements in the writing provide evidence of unity: a consistent point of view (e.g., not switching from "I" to "you"), a lack of digressions, and the presence of a lead and closure.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of the composing domain's features; the writer may control some features more than others. Purposeful elaboration focuses the central idea both at the sentence level and throughout the entire piece of writing. However, some thinness or unevenness in elaboration may occur. Narrative organization is generally intact; in other modes, organizational lapses may occur, but an overall plan is apparent. Unity is evidenced by the fact that few, if any, minor digressions or shifts in point of view occur; further, an opening and closing, though not sophisticated, are present.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the composing domain. At this score point, major digressions may be present, or ideas compete and no one idea emerges as central. Even if a single idea dominates, the writing may be a list of general, underdeveloped statements or a skeletal plot. Often, little elaboration or organization is apparent, although some attempt at a lead or closure may be present. The lack of a logically elaborated central idea prevents unity from emerging.

Score Point 1
The writer demonstrates little or no control of most of the composing domain's features. Typically, the writing jumps from point to point without a unifying central idea. No organizational plan is apparent. No purposeful elaboration of any kind is present.

Written Expression Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the written expression domain's features. The result is a purposefully crafted message that the reader remembers, primarily because its precise information and vocabulary resonate as images in the reader's mind. Specific word choice and information also create tone in the writing and enhance the writer's voice. Sentences are often varied in length and beginnings, resulting in a rhythmic flow throughout the piece.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of the written expression domain's features; the writer may control some features more than others. On the whole, some specific word choice and information cause the message to be clear, although there may be some inclusion of unnecessary but related information. Along with instances of successful control, general statements or vague words might be present; when they are, the tone and voice of the piece will flatten somewhat. Typically, sentences are varied in length and structure. However, at times the rhythm of the paper may be diminished by a lack of sentence variety or by awkward constructions.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the written expression domain. While some specificity of word choice might exist, mostly the message consists of general information written in imprecise, bland language. As a result, the writer's voice emerges only on occasion, if at all. A relative lack of sentence variety may make the reading monotonous, and awkward constructions may be distracting enough to make the writer's meaning unclear on occasion. While a few brief rhythmic clusters of sentences may occur, an overall sense of rhythmic flow is not present.

Score Point 1
The writer demonstrates little or no control of most of the written expression domain's features. Both word choice and information are general, vague, and/or repetitive. A lack of sentence variety makes the presentation monotonous. The existence of several extremely awkward constructions may further reduce the paper's stylistic effect. The writer's lack of control of vocabulary and information prevents both tone and voice from emerging.

Usage/Mechanics Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the domain's features of usage/mechanics. The writing demonstrates a thorough understanding of usage and mechanics as specified in the Virginia K-5 SOL. The author uses capitalization, punctuation, usage, and sentence formation and applies the structural principles of spelling. A few errors in usage and mechanics may be present. However, the writer's control of the domain's many features is too strong for these mistakes to detract from the performance.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of most of the domain's features of usage/mechanics. The writing demonstrates a basic understanding of usage and mechanics as specified in the Virginia K-5 SOL. For the most part, the author appropriately applies both the rules of capitalization, punctuation, usage, and sentence formation and the structural principles of spelling expected of 5th graders.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the domain of usage/mechanics. Evidence of the author's knowledge of features of this domain appears alongside frequent errors. In terms of both usage and mechanics, the writer inconsistently applies the rules of capitalization, punctuation, usage, spelling, and sentence formation as specified in the Virginia K-5 SOL. The density of errors across features outweighs the feature control present in the paper.

Score Point 1
The writer demonstrates little or no control of most of the domain's features of usage/mechanics. Frequent and severe errors in the Virginia K-5 usage and mechanics SOL distract the reader and make the writing very hard to understand. Even when meaning is not significantly affected, the density and variety of errors overwhelm the performance and keep it from meeting minimum standards of competence.

Sentence Formation, Usage, and Mechanics Skills

Listed below are skills in the areas of sentence formation, usage, and mechanics (capitalization, punctuation, formatting, and spelling) which may be addressed on both the multiple choice and the direct writing components of the Grade 5 English writing test. This list is not intended to be exhaustive but to provide examples of the skills which may be addressed on the writing test at grade 5.

Standard Sentence Formation

Avoid fragments (use of clausal fragments such as "Although he was not supposed to go out of the house." is not penalized in direct writing)

Avoid run-ons (e.g., I opened the door the dog went out.). Commas splices such as "I opened the door, the dog went out." are not penalized in direct writing.

Avoid excessive coordination (e.g., I opened the door and the dog went out and he chased the cat and then he came back inside.)

Use standard word order

A few word omissions are considered miscues. A pattern of word omissions would indicate sentence formation problems.

Usage

Correct use of:

subject-verb agreement

"I" in compound subject situations

a, an

them, those

good, well

adjective comparisons

adverb comparisons

adverbs instead of adjectives where appropriate (e.g. "He played really well." instead of "He played real well.")

verb tenses

plurals (-s, -es, changes in spellings)

possessives (singular and plural)

negatives (includes avoidance of double negatives)

Use of correct words (e.g., "I could have ridden the bike." instead of "I could of ridden the bike.")

Mechanics

Punctuation

Correct use of:

periods, question marks, and exclamation points at the end of sentences

apostrophe in contractions

punctuation of letter parts

apostrophes in singular and plural possessives

periods after abbreviations

commas around interrupters (includes nouns of address and interrupting clauses)

commas in dates, series, addresses

quotation marks around dialogue ( Students are not responsible for commas before initial quotation marks or for placing end punctuation inside closing quotation marks in direct writing.)

 

Capitalization

Capitalize

the first word of a sentence

proper nouns

"I"

Unnecessary capitalization of the following is forgiven in direct writing:

directions ( South, West)

subjects ("Math")

my Mom and Dad

activities ("Football")

 

Format

indent paragraphs or double space between them

divide words between syllables with a hyphen at the end of the line

 

Spelling

spell frequently used and common words correctly(includes common homophones)

+
 What Are Writing Assessment Issues and Answers?

How are the anchor sets compiled in Virginia?

Typically, anchor sets are compiled from low score point to high score point or vice versa. Virginia arranges anchors sets from low score point to high score point. By starting with the lowest score point level, a person who trains scorers can talk about what a student paper does rather than what it doesn't do. While this may seem insignificant, it becomes very important in helping people internalize the rubric. For example, it's very tempting for readers to say, "This paper doesn't have a good opening, so it must not be a 4." Instead, when papers are ordered from low score point to high, scorers are encouraged to consider first what a paper does successfully and to what level it rises, before considering its weaknesses.

What role does the length of a response play in writing assessments?
The length in and of itself is not a factor in scoring student responses. Rather, the concern is how complete the piece of writing is based on the criteria the prompt and rubrics set forth. Sometimes very concise pieces of writing score higher than responses that are long and rambling. In addition, a student's handwriting can make a response look artificially longer or shorter than it actually is.

Does handwriting count in writing assessments?
No. The ability of a student to communicate in writing may be affected by handwriting that is difficult to read, but professional scorers do not let that influence the score that is assigned. Writing assessments focus on how well a student communicates important ideas on paper in a logical order and a readable fashion. Penmanship is a separate skill. The only exception is a piece of writing that is completely illegible and cannot be scored.

What role does spelling play in writing assessments?
The weight an assessment places on spelling depends entirely on who developed the rubrics. It may be based on a district's expectations, or on local or state standards. In Virginia's direct writing assessments spelling is one of the features of the usage/mechanics domain. Therefore, spelling is assessed with the other features of the usage and mechanics expected at a particular grade level. Students are allowed and encouraged to use dictionaries on Virginia's direct writing assessments.

Should students practice before taking a formal writing assessment?
Students should be given guided practice for writing as often as possible. However, not every piece of writing in a classroom should be a fully polished piece of writing. A variety of writing techniques and situations helps beginning writers understand that purpose and audience come under consideration each time they sit down to write.

If a writing assessment relies on a piece of writing on demand, doesn't that violate what teachers are trying to teach students about writing as a process?
It's important for students to understand that each time they sit down to write, they must consider audience, purpose, content, and deadlines. Sometimes writers have weeks, even months, to produce a piece of writing. Other times, responses must take shape during one class period or within a few hours. This time factor holds true in the "real world" as well, as represented by project reports, research papers, and legal briefs on the one hand, and memos, essay tests, and business letters on the other. One type of writing doesn't negate another. Teachers should expose students to all types of writing and situations.

Can I apply holistic, focused holistic, or analytic scoring to student writing in my classroom?
Yes, with some cautions. The purpose of most classroom writing assignments is instructional or diagnostic.If the writing is a very first draft, it is not instructive to assign a score, even with a well-written rubric at hand. If students are writing in a workshop setting and/or writing several drafts, there generally needs to be significant teacher and/or peer input in the form of margin notes and suggestions on at least one early draft to assist them in revising and polishing their writing. Once a piece has been turned in ready to publish, using a rubric to assign a score is a quick and effective way to evaluate the writer's ultimate success.

Not every piece of writing a student produces needs to be critically evaluated. Some quick writes or journal entries, for example, may be considered exercises similar to practice routines for ice skaters or scrimmages for basketball players.

And finally, using this type of assessment in the classroom, or for a high-stakes assessment, works only if the students know the criteria beforehand. The rubric can be a valuable teaching tool. Rubrics should be shared with students so that they understand how their writing will be evaluated. The ultimate goal is student self-awareness.
Glossary
+
 General Terms Used In Writing Assessments
Refer to the Annotation Terminology Section for terms used in annotations.

adjacent scores:when two scores assigned to a writing sample are within one score point of each other; for example, a paper given a score of "4" by one scorer and a "3" by another

analytic scoring:the method by which trained readers evaluate a piece of writing in various areas and assign a separate score for each area, e.g. conventions, style, word choice, organization, topic development

anchor papers:student-produced writings that exemplify the score standards for each level on the rubric

annotation:the explanation accompanying training papers which supports the assigned score

domain scoring:the method by which trained readers evaluate a piece of writing for its overall quality in previously designated areas or domains. In Virginia, this method is called "focused holistic scoring."

draft:a preliminary version of a piece of writing

exact or perfect agreement:when two scores assigned to a writing sample by different readers are the same

field test (tryout or pilot):testing a representative sample of a student population to determine the effectiveness of assessment items

final copy:the polished copy of a student's writing which is ready for publication

focused holistic scoring:the method by which trained readers evaluate a piece of writing for its overall quality in previously designated areas. In Virginia this method was formerly called "domain scoring." The designated areas are composing, written expression, and sentence usage/mechanics

holistic scoring:the method by which trained readers evaluate a piece of writing for its overall quality

non-adjacent:when two scores assigned to a writing sample are not exact or within one score point of each other; for example, a paper given a score of "4" by one scorer and a "2" by another

nonscorable:the category into which papers fall that cannot be scored because they are illegible, incoherent, in a language other than English, off-topic or blank; the nonscorable categories differ from assessment to assessment

off-prompt or off-topic:a paper that has not responded in any way to the given prompt

overlay:highlighting on anchor papers that illustrates points made in the annotations

practice set:training set where the user must continue to select score points until the exact score is chosen

primary trait scoring:the method by which trained scorers evaluate writing based on how well the writer has fulfilled the assigned task

prompt:writing task that states the topic and purpose of the piece of writing

range finding:a process in which a group of experts score and discuss student papers in order to come to consensus about how to apply the rubric; the scored papers are usually then used to make training sets

reader:also called a scorer, a person trained to apply the holistic scoring process to student writing

reader bias:personal factors, not included in the scoring criteria or the rubric, that may affect the score assigned to a student paper

rubric:scoring criteria at each score point of the scale

scoring guide:scoring criteria used to evaluate performance assessment and to assign score points to student work; sometimes called the rubric; sometimes includes exemplars of student work

surface feature errors:errors the student made in the area of conventions (except usage) that are not immediately apparent if the paper is read aloud; for example, errors in spelling, punctuation, indenting, and capitalization

tone:a writer's tone might be described as somber, tongue-in-cheek, formal, casual, ironic, melodramatic, etc.; the tone is a result of the writer's word choice and sentence structure

umbrella statement:similar to a thesis statement although less formal, an umbrella statement consists of one or two sentences that indicate what the writer intends to do in the rest of the paper. For example, asked to write about an invention that has been important in his/her life, a writer might begin by saying, "I can't write about just one invention since so many have affected my life." This would indicate to a reader that the writer intends to discuss several inventions.

verification set:training set in which the user is immediately informed of the correct score

voice:a writer's voice comes from the elements that contribute to the overall impression of the writer's personality as it comes through in one particular piece of writing; word choice, sentence structure, organizational strategy, details, insight, tone, and approach to the topic all work together to create a writer's voice

writing process:the typical steps that are used in the development of a piece of writing, including prewriting, drafting, revising, editing, and publishing

+
 Annotation Terminology
The following terms and phrases are used in the annotations and may need further explanation for teachers who are not familiar with them. In some cases, the terms may be used in a slightly different way than other sources. Reading the explanation for these terms will help further clarify the reason a paper received the score point it did, as well as provide a sense of the many elements scorers consider as they read a paper.

audience:This term refers to the intended reader or readers for whom a piece is written. A writer can show "audience awareness" in a variety of ways. Some of these include identifying characters and places as they appear in a narrative, giving readers clear time markers so they can easily follow a sequence of events, defining technical terms or jargon with which readers might be unfamiliar, tying thoughts or events together for a reader with insightful comments, and generally providing readers with sufficient information to make the writing clear. Audience awareness is also sometimes referred to as a writer "bringing the readers along."

best fit:This term is used most often in focused holistic scoring in which a paper may have characteristics of several different score points. For example, a paper may have some features that are at a 2 level while other features may seem to be at the 3 level. The paper would receive a score of 2 or 3 depending on the score point of the rubric that most accurately describes the attributes of the paper. A scorer would also consider whether the 2- and 3- level elements were in the high or low range of those score points. Ultimately, the paper would be given the "best fit" score. It had elements of both score points but, overall, was it more like a 2 or more like a 3?

borderline papers:Within each score point there will be a range of papers that represent the lower, middle, and upper levels of that score. Papers that represent the middle level are often referred to as "solid" examples. Papers representing the extreme lower or upper levels are sometimes called borderline papers because they fall between two score points. In other words, a 2/3 borderline paper has some 2-level characteristics and some 3-level characteristics. Ultimately, of course, a scorer must assign the paper a "high 2" or a "low 3." Scorers are cautioned, however, to remember that a "high 2" is still reported back to the student as a 2 and differs significantly from a score of "low 3" which will be reported back as a 3.

central idea:This might also be called the paper's overall theme or purpose. It is governed by the message the writer would like to see the reader take away from the piece of writing. The topic is not a central idea. The central idea is the message or idea about the topic.

clustering:Clustering is an organizational strategy in which related details are placed together in "clusters" or "clumps." Clustering is generally a weak organizational strategy because little progression of ideas is evident in the paper.

cohesiveness (see unity):A paper that gives the sense of completeness with no extra details and no details missing might be said to have cohesiveness. A paper that lacks cohesiveness may have an abrupt or no ending, an introduction that doesn't connect to the body of the paper, serious gaps in the story leaving a reader with unanswered questions, or a switch in topics mid-way to a completely different topic.

control:Score points in the SOL rubrics use the term "control" to describe the writer's ability to use a given feature of written language effectively.

composing domain:This domain of the Virginia SOL Writing test includes the focusing, structuring, and elaborating that a writer does to construct an effective message for a reader.

density of errors:The shorter a paper is, the more the density of errors in conventions affects a reader's overall impression of the paper. In other words, a paper that consists of 5 sentences with a total of 12 errors might be said to have a greater density of errors than a paper three pages long with a total of 12 errors.

elaboration:A paper is well elaborated if the details, examples, and illustrations focus the central idea in a purposeful way. Elaboration occurs at the sentence, paragraph, and paper levels.

fluency:This refers to the overall rhythm and flow of a paper and takes into consideration features such as word choice, sentence structure and variation, transitions, organizational strategy, and focus. A test of fluency might be reading the paper aloud and listening to how easily and smoothly it reads. Well-controlled fluency generally means there are few or no awkward moments where the reader stumbles.

mechanics:Mechanics is a feature in the usage/ mechanics domain that includes formatting, spelling, capitalization, and punctuation.

organization:Organization is the logic structure that facilitates presentation of the central idea. This characteristic takes into account the way a piece of writing is arranged and ordered.

precise vocabulary:Words or phrases are carefully placed and purposefully chosen by the writer to resonate as images in the reader's mind.

rhythm:Rhythm is the quality of fluency in a writing. A writer's rhythm can either be pleasant and unobtrusive, or it can be choppy and abrupt; a pleasingly rhythmic paper will include sentences which are varied in length and beginnings.

selected information:Specific, purposefully chosen information helps to propel the reader through a writing. A paper lacking information, giving too much information, or repeating ideas will affect the reader negatively.

sentence formation:Sentence formation is a feature in the usage/mechanics domain that includes standard word order, completeness, avoidance of sentence faults (run-ons, comma splices) and standard coordination.

sentence-to-sentence organization:Sometimes each sentence is tied to the sentence that comes before, but there is no overall organizational strategy. These papers might seem to have an internal organization at first glance, but on closer examination, the paper often ends up in a very different place than where it began. For example: "My favorite weather is sunny because I can play baseball. Baseball is a great warm weather outdoor sport. I'm a good baseball player. I play first base, which is the same position Joe Smith plays for the Rhinos. The Rhinos had a great season last year. This year my dad got us tickets to see 3 of the Rhinos home games. The home games are great because I can take my friends and eat hot dogs and popcorn." The initial point of "my favorite weather" has been lost. Do not mistake additional sentences as always being elaboration of the topic. Sentences that can be rearranged without affecting the message demonstrate a flaw in organization.

sentence variety:A paper with sentences of various lengths, constructions, and beginnings creates a pleasant, unobtrusive rhythm for readers.

surface feature errors:These are errors the student made in the area of conventions (except usage) that are not immediately apparent if the paper is read aloud; for example, errors in spelling, punctuation, indenting, and capitalization. (See Virginia Skills Lists for items addressed at each tested grade level.)

tone:A writer's tone might be described as somber, tongue-in-cheek, formal, casual, ironic, melodramatic, etc. The tone is a result of the writer's word choice and sentence structure. For example, a piece of writing using slang or colloquialisms might be considered casual. It could be an appropriate tone for a narrative, a personal letter, or an informal essay. Sometimes, students attempt a lofty, academic tone in their writing that generally produces stilted prose since the word choices and sentence constructions may be awkward or incorrect. For example, "One's choice of a college could well depend on where one's parents attended or that of which one's own siblings," An uneven tone occurs when a student mixes casual and formal terminology. For example, "I threw myself forward down the precipice. Every nerve was shrieking, "Stop, stop. Dang it all, Stop." A flat tone is characterized by ordinary word choices, repetitive and often simple sentence structure, and an overall lack of vitality in the writer's voice. It is also sometimes called "generic" since a flat tone gives no indication of a writer's unique personality. A tone closest to the student's own generally produces the most genuine type of writing.

unity:Unity is completeness in the elaboration of a central idea in an organized way. A unified paper has no extraneous details and no details missing.

usage/mechanics:This domain reflects the writer's ability to compose competent, appropriately mature sentences (observed independently of purpose and style), to construct these sentences grammatically at the word level, and to spell, capitalize, and punctuate writing mechanically at its surface level.

usage:Usage is a feature of the usage/mechanics domain that includes inflections, agreement, conventions, and word meaning.

voice:A writer's voice comes from the elements that contribute to the overall impression of the writer's presence as it comes through in one particular piece of writing. Word choice, sentence structure, organizational strategy, details, insight, tone, and approach to the topic all work together to create a writer's voice. A writer's voice can and often does change depending on the piece's audience and purpose, although more experienced writers tend to have a similar voice in all of their writing.

written expression domain:This domain of the Virginia SOL Writing test includes those features that show the writer can purposefully shape and control language to affect readers. The features of written expression are vivid and precise vocabulary, selected information, tone, voice, and sentence variety.

Rubrics
Introduction
Critical Information
Assessment Information
Understanding Scoring
Glossary
Rubrics
Introduction
+
 Welcome

Welcome to Understand Scoring-powered by Perspective

This program is designed to help you
  • gain a better understanding of the Virginia Standards of Learning Direct Writing Assessments for grade 5, grade 8, and high school
  • understand the use of rubrics in scoring student writing
  • develop consistency in scoring actual student work

Specifically, this program will
  • provide information about the Virginia Standards of Learning English Assessment
  • provide an opportunity to study the rubrics used for scoring the direct writing assessments
  • provide an opportunity to study anchor papers that illustrate each score point of each domain
  • provide an opportunity to practice scoring several sets of papers written by Virginia students
  • verify your progress by evaluating the accuracy of the scores you assign to a final set of papers (verification set)
The Critical Information sub-section below includes important information you need in order to understand the rubrics and the scoring process. It is extremely important for you to read the Critical Information sub-section before proceeding. You can review the other sub-sections at any time.
+
 Definitions of Sections

Learn About Scoring
   This section provides important information that covers many aspects of Assessment, Scoring, Virginia SOL, and a glossary
    Critical Information
    This section provides critical information about the Virginia English SOL Assessment and scoring student papers.
    AssessmentInformation
    This section provides information about Virginia SOL test development, prompts and committees.
    Understanding Scoring
    This section provides a deeper understanding of scoring. It discusses the people and process used to score Virginal SOL     writing assessments. This section also covers information about rubrics, anchor papers, alert papers, and nonscorable     papers.
    Rubric
    This section describes the criteria for each score point of the rubric for composing, written expression, and usage and     mechanics.
Anchor Papers
   This section allows you to review papers that represent each score point, with annotations and color highlights that explain the assigned scores. The anchor papers will help you understand differences between score points.
Practice Scoring
   This section allows you to practice scoring and check your scores. After scoring practice sets, you can test your progress by attempting a verification set.
Critical Information
+
 What Is Focused Holistic Scoring?
Focused holistic scoring is the evaluation of writing proficiency based on several specific elements or domains of writing. The writer is judged in each domain independent of the others. In Virginia's Direct Writing Assessment each paper receives three scores, one for each of the three writing domains identified: composing, written expression, and usage/mechanics. Each domain is scored holistically. The score reflects the reader's overall impression of each of the specified domains. The scorers weigh the student's relative strengths or weaknesses in each of the three domains and then assign a score point for each domain that most accurately describes the attributes of the paper. Therefore, each paper receives a separate score for composing, written expression, and usage/mechanics.

In focused holistic scoring, it is possible for a paper to receive a higher score in one area such as usage/mechanics and a lower score in another area such as written expression. Assigning separate scores assists a teacher in diagnosing a student's relative strengths and weaknesses so that instructional activities can be planned for those specific skills identified as features within the rubric-scoring guide. A focused holistic scale defines the requirements for each domain at each level of the rubric. Thus, a particular score represents not only an overall impression, but also a specific level of proficiency in the complete set of composition domains being observed.

Virginia's Direct Writing Assessment model uses a 4-point scale. Each Domain is scored independently, using the following scale:

4 =The writer demonstrates consistent, though not necessarily perfect, control* of almost all of the domain's features.

3 =The writer demonstrates reasonable, but not consistent, control* of most of the domain's features indicating some weakness in the domain.

2 =The writer demonstrates enough inconsistent control* of several features to indicate significant weakness in the domain.

1 =The writer demonstrates little or no control* of most of the domain's features.

*Control: The ability to use a given feature of written language effectively at appropriate grade level. A paper receives a higher score to the extent that it demonstrates control of the features in each domain.

As in any scoring, a range of papers is evident within each score point. In a 4-point scale there will be high 4's and low 4's, high 3's and low 3's, etc. Although trainers talk about writing assessments as having discrete score points, the assigned scores actually represent a continuum within the score points. At every score point level, there is a range of papers representing the lower, middle, and upper end of the score point. A paper assigned a score of 3, for example, might have some characteristics of a 2-level paper and some characteristics of a 4-level paper, but overall, its 3-level characteristics predominate. Because of this phenomenon, papers receiving the same score point may look very different from each other.The 3-level paper described above, for example, might be at the top end of the 3's, and look very different than a paper that has only 2- and 3-level characteristics. The second paper is still assigned a 3, but it represents the lower end of the 3's. It is important to understand the criteria used in assigning a score and not to expect papers at a certain score point always to look the same.
+
 How Are Rubrics Used to Score Papers?
Generally, rubrics for writing assessments are fairly generic.It is only in conjunction with anchor (exemplar) papers that the criteria at each score point begin to be clarified.The rubric provides the general description of a score point; the anchor papers are the exemplars of how those general descriptions look in practice. A good set of anchor papers attempts to represent the different ways a student might achieve a score by including papers which attain the same score point in different ways.

Because rubrics are not meant to be lists of mandatory criteria a student must meet in order to achieve a score point, the word "may" is often used in a score point description.This use of the word may encourages scorers to remember that a paper may demonstrate some characteristics of one score point and some of another. A paper ultimately should be assigned a "best fit" or most appropriate score. For example, in the 4-point focused holistic rubric used in Virginia's Direct Writing Assessment, one of the sentences used to describe a 2 in Composing is "Even if a single idea dominates, the writing may be a list of general, underdeveloped statements or a skeletal plot." While this is often the case, it is also possible for a paper that doesn't have a skeletal plot or isn't a list of underdeveloped statements to receive a score of 2 because other 2-level qualities predominate and keep the paper from achieving a higher score.One of the most important points in learning how to apply a rubric consistently is to let go of existing prejudices. When scoring writing assessments, one of the most difficult tasks is to accept the boundaries of a rubric when those boundaries differ from a reader's own teaching practices or beliefs. For example, a teacher might value a nearly error-free paper in the area of usage/mechanics while a rubric might make allowances for certain errors. Assigning scores to papers in an assessment based on a rubric the teacher did not create is not to suggest teachers must adopt that rubric's criteria to classroom assignments. Large-scale assessments and classroom-based assessments may very well measure somewhat different skills in a different manner.
+
 What Are the General Considerations for the Writing Tests?
No provisions are made for examinees to recopy their final drafts. They are encouraged, however, to revise their work and to make whatever changes they believe will improve their papers. Readers, therefore, should expect and must be tolerant of revisions and interlinear editing in these papers. To gather as much instructionally useful information as possible in its writing assessment, Virginia has adopted a scoring model that calls for evaluations in three domains - composing, style, and usage-mechanics. In this type of scoring, the observation of writing is divided into several domains (categories), each of which is comprised of various features. Each domain is evaluated holistically, with the domain score indicating the extent to which the features in that domain appear to be under the control of the writer. Thus, an awareness of the features and their use contributes to the score, but the score is a judgment of the whole domain and not simply a counting of demonstrated features. Some skilled scorers can score for all domains after one complete reading, although scorers may re-read a portion in order to make a decision about one or more domains.

Application of the Scoring Scale
  1. Look as much at what the writer does well as at what he/she does poorly.
  2. Give consideration to all features within domain.
  3. Do not contaminate the scoring of one domain by awareness of either accomplishments or errors belonging to another domain.
  4. Do not create double jeopardy, e.g., a run-on sentence in Usage-Mechanics is not also a sentence error in Written Expression, and "brite" is one misspelling no matter how often the word appears.
  5. Compare papers to the requirements of each domain, not to each other, or to student writing that you may have previously read, or to some general standard of desired literacy.
  6. Score only what is on the paper. It is impossible to evaluate what the writer might have intended to say.
  7. Note that a word that seems to have been omitted due to haste in revising, editing, or copying need not demonstrate a total lack of control over that feature.
  8. Do not score a paper on only length. Length, in and of itself, is not a feature of any domain.
  9. Read the entire paper before assigning any scores.
Nonscorable PapersAll papers, no matter how brief, must be scored, unless they are off-topic, illegible, incoherent, refusals to respond, or written in a language other than English. Only the Scoring Director is permitted to assign a nonscorable code.
+
 What Is the Grade 8 Writing Prompt Used in Understand Scoring for Virginia?

Think about a time when something funny happened to you or someone you know. Write about what happened.

CHECKLIST FOR WRITERS

______I planned my paper before writing it.

______I revised my paper to be sure that

______the introduction captures the reader's attention;

______the central idea is supported with specific information and examples that will be interesting to the reader;

______the content relates to my central idea;

______ideas are organized in a logical manner;

______my sentences are varied in length;

______my sentences are varied in the way that they begin; and

______the conclusion brings my ideas together.

______I edited my paper to be sure that

______correct grammar is used;

______words are capitalized when appropriate;

______sentences are punctuated correctly;

______words are spelled correctly; and

______paragraphs are clearly indicated.

+
 What Are the Allowable Scoring Interpretations?
When scoring writing samples, decisions must be made regarding how papers with unexpected content, organization, or structure will be scored. While some guidelines are general, other decisions must be made because a significant number of writers have chosen to respond to a specific prompt in an appropriate but unanticipated way. These "rules" generally cannot be planned in advance but come out of issues that arise during the scoring process. Teachers should consider that the following list does not necessarily represent rules that would apply to classroom assignments or even classroom assessments. They are offered here as examples of considerations of how the papers written in response to the prompts included in these training materials were scored.

Think about a time when something funny happened to you or someone you know. Write about what happened.

Grade 8 Prompt-Specific Scoring Guidelines
  • In general, if it is apparent that the student is responding to the prompt, the paper should be considered on topic and should be scored according to the rubric. (Pay special attention to those papers that look as if they are refusals at first glance but are actually responses to the prompt, e.g., "Asking us to write about something funny that happened to us is silly. It would be better to ask us to write about...").
  • Once a paper is determined to be "on topic," that issue should no longer influence the score assigned. Papers are not scored lower for being slightly on topic or higher for being exactly on topic. The assigned score should reflect only the student's writing ability as described in the rubric.
  • A student may choose to write about several funny occurrences instead of just one. Such a response should be considered on topic. Depending on how skillfully the student handles the inclusion of multiple funny occurrences, the unity of the piece may be negatively affected.
  • The student may write about a funny occurrence without ever directly mentioning that the event was funny.
  • The reader should assume that any incident the writer chooses to share is a time when something funny happened. Some writers interpreted "funny" as odd or unusual.
  • The writer may make up an incident that includes details that are obviously too fantastic to be true, e.g., describing a time s/he got to play basketball with Michael Jordan, or the funny occurrence may come from a book or a movie with which the student is familiar.
  • The student may state that s/he knows of no funny occurrence and write about something else that happened.
  • Readers should not judge the incident the writer chooses or whether or not they agree that it's funny. The reader should not judge the appropriateness of the occurrence the writer chooses to describe.
  • The reader should not judge the reasons the writer gives for believing that this event was funny.
  • Readers should be forgiving of naive or illogical explanations.
  • The writer need not explain why s/he thought the incident was funny.
  • The student's score should not be affected by the mode used: narrative, expository, descriptive, persuasive, imaginative.
+
 How Does Reader Bias Affect Scoring?
Reader bias refers to personal factors that may affect a reader's perception of a student's response, but have no basis in a rubric. Below are factors that affect some readers but must not be allowed to have an impact on scoring.

Appearance of Response.The quality of the handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria.

Length of Response.The length of a student paper is not part of the scoring criteria. Readers should take into consideration only whether the finished piece seems complete and has the components required. The size of a student's handwriting can make a paper look longer or shorter on the page than it actually is.

Repetition of Response.Although readers may tire of reading several essays on the same topic, it is important to remember that for each student the response represents a unique attempt.

Offensive or Disturbing Content.If a student uses vulgar language, adopts a sexist or racist point of view, or perhaps takes a naive or narrow approach to a topic, readers should not let the student's point of view affect the score. Likewise, readers should not let a student's lifestyle or maturity level influence them either positively or negatively regarding the writing.

Alert Papers.Because the prompts are designed to have meaning to students so they can create a personal response, sometimes the paper relates situations and information that are disturbing. It is Virginia's policy to bring disturbing papers to the attention of an adult close to the student as soon as possible so that positive action can be taken if necessary. Readers are instructed to indicate to the Scoring Director any paper that is found to be disturbing and then score the paper according to the criteria.

Response to Prompt.In the classroom, there may be a "correct" response to a writing assignment. For Virginia's assessment, students are free to respond in any way they choose. There is no right or wrong "answer" as long as it is clear that the student is attempting to reply to the prompt.

Reactions to Style.A reader's own grammatical biases should not play a part in assigning a score if the student has not violated standard writing conventions. In other words, beginning a sentence with "and," the absence of a formal thesis sentence, the use of first or second person, or an informal tone are not wrong in this type of assessment. Classroom assessments often have different requirements.

Writer's Personality.Writers may come across as brash, sassy, cute, self-aware, shy, surly, flat, honest, or naive. Readers are scoring the written passage, not the writer's personality.

Reactions to Direct Writing Assessments.Some readers may approach writing assessments with their own biases in favor of one type of assessment over another. Or, they may believe it is impossible to score writing fairly using any of these techniques. The prompts and rubrics used in Understand Scoring for Virginia have been analyzed and validated by experts in the field and adhere to standard practices in state assessments. Teachers should tailor their own classroom writing assessments around local curricular expectations.
Assessment Information
+
 How Are Standards of Learning Tests Developed?
The development of each SOL test begins with the blueprint for that test. The test blueprint serves as a guide to test developers as they write test questions and construct the SOL tests. The test blueprints show (a) the Standards covered by the test and which, if any, have been excluded; (b) which Standards are assigned to each reporting category; (c) the number of test items in each reporting category and on the total test; (d) general information about how the test questions were constructed; and (e) the materials that students are allowed to use while taking the test.

Using the test blueprint as a guide, the test development contractor writes test items designed to measure student achievement of the particular Standards of Learning covered by that test. Each test item is coded to a particular SOL.

The content review committee for that test reviews each proposed test item, answering the following questions for each item:

SOL Item Match:Does this item measure the SOL it was designed to measure?

Appropriateness:Does this item appropriately measure content or skills that students in Virginia should be required to have by spring of the designated grade level or near the end of the course?

Difficulty:Is the difficulty of this item appropriate?

Fairness:Is the item free from any content that would offend or unfairly penalize students on the basis of personal characteristics such as gender, ethnicity, religion, or socioeconomic status?

If the answer to each of these questions is "yes," the item is accepted for field testing and tried out with Virginia students. Items are tried out by placing them as field test items on an SOL test. Field test items do not count toward the student's score on the test; however, data are collected on how students respond to the items. The number of field test items on a test varies depending on the test: the multiple choice section of the English Writing Test for grade 5 has 12 field test items, while the multiple choice section of English Writing Test for grade 8 and the multiple choice section of Secondary English Writing Test both have 14 field test items.

Once the field test data are collected, the content review committee reviews the items again using the statistical information collected during the field test and decides whether each item should be accepted into the item bank for potential use on a future test form. In the case of the high school tests, a separate Bias Committee reviews each item a second time for fairness.

Using items that have been approved by the content review committee for inclusion in the item bank, the test development contractor follows the test blueprint to develop test forms. These test forms are reviewed by the content review committee prior to their administration. Once the test form is administered, each test item either returns to the bank or is released as a sample test item.

+
 How Are Prompts Selected?
The process of prompt development begins with the test development contractor's developing potential writing prompts for the direct writing component of the SOL English Writing Test. According to the test blueprints, prompts are to present to the student a context for writing in the form of a question, an issue, or a hypothetical situation. The content review committee then reviews each proposed writing prompt for potential field testing.

If the prompt is accepted for field testing, it is tried out with Virginia students. Papers written to the prompt during the field test are scored by the scoring contractor. The readers involved in the field test scoring evaluate the prompt in terms of whether it would be appropriate for use in live testing. This analysis in addition to information about the number of papers assigned each score point in each domain is provided to the content review committee.

Using this information from the field test, the content review committee reviews the prompts again and decides whether each prompt should be accepted into the prompt bank for potential use on a future test form. Prior to each administration of the SOL English Writing Test, the content review committee selects two prompts to be used: one for the main prompt and one to be used as the make-up prompt for students who are not tested on the statewide writing day.

+
 What Committees Are Involved in the Development of the Standards of Learning Tests?
Content Review Committee

Each SOL test is constructed according to the specifications of the SOL test blueprint, and with the cooperation of a Content Review Committee composed of Virginia educators who have experience and expertise in the content area and grade level of the test. Working with the Department and the testing contractor, the Content Review Committees review each test question before it is field-tested. In order to make it to field test status in Virginia, the potential question must, in the eyes of the Content Review Committee, meet the following four criteria:
  1. Does the question measure the SOL it was designed to measure?
  2. Does the question appropriately measure content or skills that students in Virginia should be expected to learn by the spring of the designated grade level or near the end of the course?
  3. Is the difficulty of the question appropriate?
  4. Is the question free from content that stereotypes, offends, or unfairly penalizes students on the basis of personal characteristics such as gender, ethnicity, religion, or socioeconomic status?

If the question meets these criteria, it is used as a field test question with Virginia students during a regular administration of the SOL tests. Field test questions appear among the operational questions on each SOL test. In this way a steady stream of new test questions is available for use.

After test questions have been field tested, the Content Review Committee is reconvened to review the questions and make the same four judgments again, but this time with statistics about the field-tested questions. If the answers to the four questions are yes, then the test question is placed in the item bank for potential use in an operational SOL test form.

When preparing a test form for operational use, the testing contractor selects questions from the item bank in accordance with the blueprint specifications. Once draft test forms are constructed, the Content Review Committees are again convened. Committee members assume the task of approving or editing two forms of each grade level or end-of-course test to determine the content validity and equivalency of the test forms as a whole. While the previous committee reviews were concerned with individual questions, the focus of this review is the full operational test forms. At this stage there may be additional minor edits or revisions.

In addition to the previously discussed duties, the content review committees for the SOL Writing Tests select the prompt to be used for each administration of the tests and review and approve the materials used to train readers to score the prompt.

Bias Committee
Because passing the high school SOL assessments will be a graduation requirement, it is especially important that the tests be free of factors that have an unfair impact upon a group of students. Therefore, an additional bias review is conducted by a separate Bias Review Committee representing each content area to be tested. Bias Review Committee members are asked to scrutinize items for potential stereotyping or other forms of bias. The purpose of the bias review is to identify any items that appear to have the potential to treat any group, regardless of ethnic background, gender, or region, differently from other groups. Committee members examine the response distribution for each of the demographic groups identified for the review. The intent of the review is to determine if members of a certain group were drawn to one or more of the answer choices for the item. If a large percent of one group selected a particular response or did not select a particular response, the item is carefully examined. Training and procedures are similar to those carried out during the Item Review meetings. The Bias Review Committee's task focuses solely on reviewing test items for potential bias after the items have been reviewed by the Content Review Committees. It is the Bias Review Committee's responsibility to ensure that items are fair to all students and that all students would have an equal opportunity to demonstrate achievement regardless of gender, ethnic background, religion, socioeconomic status, or geographic region.

Assessment Policy Advisory Committee
The Assessment Policy Advisory Committee composed of Virginia educators advises the Virginia Department of Education and the Board of Education on policy issues related to the state-mandated assessment programs. This committee was instrumental in developing the guidelines for testing students with disabilities and the guidelines for testing limited-English-proficient students in the SOL program.
Understanding Scoring
+
 What Is Focused Holistic Scoring?

Grade 8 Focused Holistic Scoring Guidelines

Focused holistic scoring is the evaluation of writing proficiency based on several specific elements or domains of writing. The writer is judged in each domain independent of the others. In Virginia's Direct Writing Assessment each paper receives three scores, one for each of the three writing domains identified: composing, written expression, and usage/mechanics. Each domain is scored holistically. The score reflects the reader's overall impression of each of the specified domains. The scorers weigh the student's relative strengths or weaknesses in each of the three domains and then assign a score point for each domain that most accurately describes the attributes of the paper. Therefore, each paper receives a separate score for composing, written expression, and usage/mechanics.

In focused holistic scoring, it is possible for a paper to receive a higher score in one area such as usage/mechanics and a lower score in another area such as written expression. Assigning separate scores assists a teacher in diagnosing a student's relative strengths and weaknesses so that instructional activities can be planned for those specific skills identified as features within the rubric-scoring guide. A focused holistic scale defines the requirements for each domain at each level of the rubric. Thus, a particular score represents not only an overall impression, but also a specific level of proficiency in the complete set of composition domains being observed.

Virginia's Direct Writing Assessment model uses a 4-point scale. Each Domain is scored independently, using the following scale:

4 =The writer demonstrates consistent, though not necessarily perfect, control* of almost all of the domain's features.

3 =The writer demonstrates reasonable, but not consistent, control* of most of the domain's features indicating some weakness in the domain.

2 =The writer demonstrates enough inconsistent control* of several features to indicate significant weakness in the domain.

1 =The writer demonstrates little or no control* of most of the domain's features.

*Control: The ability to use a given feature of written language effectively at appropriate grade level. A paper receives a higher score to the extent that it demonstrates control of the features in each domain.

As in any scoring, a range of papers is evident within each score point. In a 4-point scale there will be high 4's and low 4's, high 3's and low 3's, etc. Although trainers talk about writing assessments as having discrete score points, the assigned scores actually represent a continuum within the score points. At every score point level, there is a range of papers representing the lower, middle, and upper end of the score point. A paper assigned a score of 3, for example, might have some characteristics of a 2-level paper and some characteristics of a 4-level paper, but overall, its 3-level characteristics predominate. Because of this phenomenon, papers receiving the same score point may look very different from each other.The 3-level paper described above, for example, might be at the top end of the 3's, and look very different than a paper that has only 2- and 3-level characteristics. The second paper is still assigned a 3, but it represents the lower end of the 3's. It is important to understand the criteria used in assigning a score and not to expect papers at a certain score point always to look the same.
+
 How Should the Scale Be Applied?
  1. Look as much at what the writer does well as at what he/she does poorly.
  2. Give consideration to all features within domain.
  3. Do not contaminate the scoring of one domain by awareness of either accomplishments or errors belonging to another domain.
  4. Do not create double jeopardy, e.g., a run-on sentence in Usage-Mechanics is not also a sentence error in Written Expression, and "brite" is one misspelling no matter how often the word appears.
  5. Compare papers to the requirements of each domain, not to each other, or to student writing that you may have previously read, or to some general standard of desired literacy.
  6. Score only what is on the paper. It is impossible to evaluate what the writer might have intended to say.
  7. Note that a word that seems to have been omitted due to haste in revising, editing, or copying need not demonstrate a total lack of control over that feature.
  8. Do not score a paper on length alone. Length, in and of itself, is not a feature of any domain.
  9. Read the entire paper before assigning any scores.
+
 How Should the Rubrics Be Applied?
In learning how to apply a rubric consistently, teachers must let go of existing prejudices. One of the most difficult tasks in writing assessment is to accept the boundaries of a scoring guide when those boundaries differ from a teacher's own teaching practices or beliefs. For example, a teacher might insist upon an error-free paper in the area of usage and mechanics for classroom writing assignments while a rubric might emphasize consistent control of usage and mechanics. Assigning scores to papers in an assessment based on a rubric the teacher did not create is not to suggest teachers must apply that rubric's criteria to classroom assignments. Large scale assessments and classroom-based assessments may very well measure somewhat different skills in a different manner.

Rubrics for writing assessments are generally fairly generic. It is only in conjunction with anchor papers that the criteria at each score point begin to be clarified. The rubric provides the general description of a score point; the anchor papers are the exemplars of how those general descriptions look in practice.
+
 What Are the Procedures for Training Scorers and Scoring?

Introduction to the SOL Writing Assessment

Training of readers begins with a brief introduction to the Virginia SOL Writing Assessment in which readers are informed of the goals and purposes of the assessment program. In addition, readers are told that the assessment is an on-demand writing task in which students are given a prompt and have up to an entire school day to respond to it. Students have space in the answer booklet for notes and are given scratch paper for prewriting and drafting if they need more space. A checklist included with the writing prompt encourages students to follow the writing process. Students have four sheets of paper in the answer booklet to use for their response and are told that only what they write on these four sheets will be scored. They have access to dictionaries and may write their responses in cursive, or they may print. Following the description of the writing task given to students, readers are told that they are to consider the papers written for this assessment to be first draft writing and are to be tolerant of interlinear editing.

Training and Qualifying
A discussion of the three domains used in the scoring model: composing, written expression, and usage/mechanics follows the introduction to the assessment. Readers are introduced to the prompt and then training begins with the composing domain. Trainers first discuss the features of the composing domain as well as the general scale: 4 = consistent control, 3 = reasonable control, 2 = inconsistent control and 1 = little or no control. Score point 1 in the composing rubric is then read, and the score point 1 anchor papers are read aloud and discussed. The discussion emphasizes each paper's written annotation which uses the language of the rubric to explain the score point as it applies to that paper. The score point 2 rubric is then read followed by discussion of the anchor papers for this score point. A similar process is used for score points 3 and 4 in composing. Following the discussion of the rubric and anchor papers prospective readers independently score the papers in the Composing Training Set for the composing domain only. A similar process is used for the written expression domain and the usage/mechanics domain.

Once all domains have been discussed and all domain specific training sets scored, readers begin scoring the mixed domain sets. The purpose of the mixed domain sets is to familiarize readers with assigning scores in each of the three domains to a paper. There are three such sets of 10 papers each (sets A, B, and C), and readers are asked to assign a score in each domain to these papers. Again, scores are discussed in detail at the completion of each set.

After all of the domain training sets have been scored and discussed, prospective readers begin scoring qualifying sets. There are 4 qualifying sets of 10 papers each. Readers are required to have 70% or better exact agreement on each domain in at least one qualifying set with 60 % or better agreement in a second set. Readers who do not qualify are dismissed from the Virginia project.

Scoring
Once readers have qualified, they are assembled into teams and begin scoring papers. Each team is led by a team leader who has been selected based on demonstrated expertise in all phases of the scoring process including skill in training readers and in maintaining the organizational procedures necessary for scoring to move smoothly. During scoring, team leaders respond to readers' questions about scoring and work with readers who are having difficulty applying the rubric. A primary duty of the table leader is to "read behind" readers to ensure that they are not drifting from the scoring criteria. Team leaders also review the scoring patterns of readers throughout the project and conduct retraining as necessary.

Two readers from different teams read each paper. If the two readers disagree by more than one point in any domain, a third reader reads the paper to resolve the score.

In addition to the team leaders, the scoring contractor's scoring directors and the Virginia Department of Education review daily reader reports that show how well readers are agreeing with each other. Furthermore, each day readers are asked to score packets of papers pre-scored by the content review committee. The reader reports resulting from these calibration packets serve to identify readers who are drifting off the scale, while the discussion of the score points assigned to each paper helps to re-calibrate all readers.

Readers are monitored throughout the scoring process. Those who fail to maintain acceptable agreement with other readers or who fail to accurately apply the scoring criteria despite retraining are dismissed.
+
 What Is Range Finding and How Are Scores Assigned?
For each administration of the SOL writing test, the prompt to be used for the direct writing component is selected by the content review committee for the English Writing Test at that grade level. After the prompt is selected, range finding begins with scoring contractor staff experienced with the Virginia SOL Writing Assessment. They score several hundred papers written to the prompt by Virginia students during the field test. Following this initial scoring, contractor staff discuss the papers and make preliminary selections of papers to be used in anchor sets, training sets, and qualifying sets. Then these preliminary sets are brought to the content review committee for discussion. The content review committee may replace papers or move them to different sets. Once the committee is satisfied with the papers and the scores assigned to them, the papers are ready for final preparation before being used for training readers.
+
 What Are the Domains and Definitions?

Composing

The Composing domain includes the focusing, structuring, and elaborating that a writer does to construct an effective message for a reader. It is the creation of a product, the building of a writing intended to be read. The writer crafts his/her message for the reader by focusing on a central idea, providing elaboration of the central idea, and delivering the central idea and its elaboration in an organized text.

Features:
  • Central idea
  • Elaboration
  • Unity
  • Organization
Written Expression
The Written Expression domain comprises those features that show the writer purposefully shaping and controlling language to affect readers. This domain focuses on the vividness, specificity, and rhythm of the piece and the writer's attitude and presence.

Features:
  • Deliberate word choice
  • Selected, precise information
  • Sentence variety
  • Tone
  • Voice
Usage/Mechanics
The Usage/Mechanics domain comprises the writer's ability to form competent, appropriately mature sentences and the use of word level features that cause written language to be acceptable and effective for standard discourse. This domain includes the system of symbols and cueing devices a writer uses to help readers make meaning.

Features:
  • Sentence formation
  • Usage
    • Standard inflections
    • Comparison of adjectives/adverbs
    • Agreement
  • Mechanics
    • Capitalization
    • Spelling
    • Punctuation
    • Formatting
+
 What Are the Writing Prompts and Checklists?

Grade 8 Writing Prompt and Checklist


Think about a time when something funny happened to you or someone you know. Write about what happened.

CHECKLIST FOR WRITERS

______I planned my paper before writing it.

______I revised my paper to be sure that

______the introduction captures the reader's attention;

______the central idea is supported with specific information and examples that will be interesting to the reader;

______the content relates to my central idea;

______ideas are organized in a logical manner;

______my sentences are varied in length;

______my sentences are varied in the way that they begin; and

______the conclusion brings my ideas together.

______I edited my paper to be sure that

______correct grammar is used;

______words are capitalized when appropriate;

______sentences are punctuated correctly;

______words are spelled correctly; and

______paragraphs are clearly indicated.

+
 What Are the Rubrics?

Grade 8 Rubrics


Composing Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the composing domain's features. Full elaboration focuses the central idea both at the sentence level and throughout the entire piece by providing purposeful examples, anecdotes, illustrations, or details. Narrative organization is intact; in other modes, minor organizational lapses may occur. In all types of writing, a strong organizational plan is apparent. Any organizational lapses that occur do not significantly detract from the presentation. Several elements in the writing provide evidence of unity: appropriate transitions, a consistent point of view (e.g., not switching from "I" to "you"), a lack of digressions, and the presence of a lead and closure that is more than mere repetition of statements.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of the composing domain's features; the writer may control some features more than others. Purposeful elaboration focuses the central idea both at the sentence level and throughout the entire piece of writing. However, occasional thinness or unevenness in elaboration may occur. Narrative organization is intact. Organizational lapses may occur (in modes other than narration), but an overall plan is apparent. Unity is evidenced by the fact that few, if any, minor digressions or shifts in point of view occur. Transitions are, on the whole, purposefully used. An opening and closing, though not sophisticated, are present.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the composing domain. At this score point, major digressions may be present, or ideas compete and no one idea emerges as central. Even if a single idea dominates, the writing may be a list of general, underdeveloped statements or a skeletal plot. Often, little elaboration or organization is apparent. Although a contrived closure may be present, inconsistent control of transitions or the lack of a logically elaborated central idea prevent unity from emerging.

Score Point 1
The writer demonstrates little or no control of most of the composing domain's features. The focus on a central idea is lacking, or the piece is so sparse that the presence of a clear focus is insufficient for it to earn a higher score. Typically, the writing jumps from point to point, without a unifying central idea. No overall organizational strategy is apparent. The writing seems haphazard, and sentences can be rearranged without substantially changing the meaning. Bare statement is the norm, but even in responses that are several pages long, no purposeful elaboration is present.

Written Expression Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the written expression domain's features. The result is a purposefully crafted message that the reader remembers, primarily because its precise information and vocabulary resonate as images in the reader's mind. Highly specific word choice and information also create tone in the writing and enhance the writer's voice. Metaphors, similes, and other figurative language, if present, are purposeful. The writer repeats or varies sentence construction for effect and appropriately subordinates ideas and embeds modifiers on a regular basis, resulting in a rhythmic flow throughout the piece.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of the written expression domain's features; the writer may control some features more than others. On the whole, specific word choice and information cause the message to be clear; occasionally, a few examples of vivid or purposeful figurative language may be present. Along with instances of successful control, a few general statements or vague words might be presented; when they are, the tone and voice of the piece will flatten somewhat. Overall, subordination and sentence variety are present, but occasional awkward constructions or the lack of structural complexity may diminish the rhythm of the paper.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the written expression domain. Some specificity of word choice might exist, but mostly the message will be crafted with imprecise, bland language. As a result, the writer's voice emerges only on occasion, if at all. The selection of information may be uneven and/or consist of an attempt to tell everything that the writer knows about a topic. A relative lack of sentence variety may make the reading monotonous, and awkward constructions may be distracting enough to make the writer's meaning unclear on occasion. While a few brief rhythmic clusters of sentences may occur, an overall sense of rhythmic flow is not present.

Score Point 1
The writer demonstrates little or no control of most of the written expression domain's features. Both word choice and information are general, vague, and/or repetitive. A lack of sentence variety makes the presentation monotonous. The existence of several extremely awkward constructions may further reduce the paper's stylistic effect. The writer's lack of control of vocabulary and information prevents both tone and voice from emerging.

Usage/Mechanics Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the domain's features of usage/mechanics. The writing demonstrates a thorough understanding of usage and mechanics as specified in the Virginia K-8 SOL. The author uses capitalization, punctuation, usage, and sentence formation and applies the structural principles of spelling. A few careless errors in usage and mechanics may be present. However, the writer's control of the domain's many features is too strong for these mistakes to detract from the performance.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of most of the domain's features of usage/mechanics. The writing demonstrates a basic understanding of usage and mechanics as specified in the Virginia K-8 SOL. For the most part, the author appropriately applies both the rules of capitalization, punctuation, usage, and sentence formation and the structural principles of spelling expected of 8th graders.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the domain of usage/mechanics. Evidence of the author's knowledge of features of this domain appears alongside frequent errors. In terms of both usage and mechanics, the writer inconsistently applies the rules of capitalization, punctuation, usage, spelling, and sentence formation as specified in the Virginia K-8 SOL. The density of errors across features outweighs the feature control present in the paper.

Score Point 1
The writer demonstrates little or no control of most of the domain's features of usage/mechanics. Frequent and severe errors in the Virginia K-8 usage and mechanics SOL distract the reader and make the writing very hard to understand. Even when meaning is not significantly affected, the density and variety of errors overwhelm the performance and keep it from meeting minimum standards of competence.

Sentence Formation, Usage, and Mechanics Skills

Listed below are skills in the areas of sentence formation, usage, and mechanics (capitalization, punctuation, formatting, and spelling) which may be addressed on both the multiple choice and the direct writing components of the Grade 8 English writing test. This list is not intended to be exhaustive but to provide examples of the skills which may be addressed on the writing test at grade 8.

Standard Sentence Formation

Use complete sentences

Use standard word order

Avoid comma splices

Avoid fused sentences (run-ons)

Avoid beginning sentences with coordinating conjunctions ( e.g., and, so)

Usage

Correct use of the following:

subject-verb agreement

"I" in compound subject situations

"Me" in the objective case

a, an

them, those

good, well

pronoun antecedent agreement

adjective comparisons

adverb comparisons

adverbs instead of adjectives where appropriate ( e.g., "He played really well." instead of " He played real well.")

verb tenses (including the use of helping verbs)

plurals (-s, -es, changes in spellings)

possessives (singular and plural)

negatives (avoid double negatives)

Use of correct words (e.g., "I could have ridden the bike" instead of "I could of ridden the bike.")

Maintain tense consistency

Mechanics

Punctuation

Correct use of the following:

periods, question marks, or exclamation points at the end of sentences

periods after abbreviations

commas around interrupters (includes nouns of address and appositives)

commas in dates, series, addresses

commas with certain clauses (e.g., after introductory clauses)

punctuation in and around dialogue

quotation marks as appropriate

italics or underlining as appropriate

apostrophes in contractions and in singular and plural possessives

letter parts

Capitalization

Capitalize

the first word of a sentence

proper nouns

"I"

school subjects as needed

"Mom" and "Dad" when appropriate

 

Format

indent paragraphs or double space between them

paragraph dialogue correctly

divide words between syllables with a hyphen at end of line

 

Spelling

spell frequently used and common words correctly

spell homophones correctly

+
 What Are the Scores in the Other Domains for the Anchor Papers?

Grade 8 Anchor Set Scores
Paper ID* Composing Written Expression Usage/ Mechanics
Composing
Anchors
     
C8A01 1 2 2
C8A02 1 2 1
       
C8A03 2 2 1
C8A04 2 2 2
C8A05 2 3 3
       
C8A06 3 3 3
C8A07 3 4 4
C8A08 3 3 3
       
C8A09 4 4 4
C8A10 4 4 4
C8A11 4 4 4
Written Expression
Anchors
     
W8A01 1 1 1
W8A02 1 1 1
       
W8A03 2 2 2
W8A04 2 2 2
W8A05 2 2 3
       
W8A06 3 3 2
W8A07 3 3 3
W8A08 3 3 4
       
W8A09 4 4 4
W8A10 3 4 4
W8A11 4 4 4
Usage/Mechanics
Anchors
     
U8A01 2 2 1
U8A02 1 1 1
       
U8A03 2 2 2
U8A04 2 2 2
U8A05 2 2 2
       
U8A06 2 3 3
U8A07 3 3 3
U8A08 4 3 3
       
U8A09 3 3 4
U8A10 3 3 4
U8A11 3 4 4

*The paper ID is located in the lower left corner of the student paper.
+
 What Are Writing Assessment Issues and Answers?

How are the anchor sets compiled in Virginia?
Typically, anchor sets are compiled from low score point to high score point or vice versa. Virginia arranges anchors sets from low score point to high score point. By starting with the lowest score point level, a person who trains scorers can talk about what a student paper does rather than what it doesn't do. While this may seem insignificant, it becomes very important in helping people internalize the rubric. For example, it's very tempting for readers to say, "This paper doesn't have a good opening, so it must not be a 4." Instead, when papers are ordered from low score point to high, scorers are encouraged to consider first what a paper does successfully and to what level it rises, before considering its weaknesses.

What role does the length of a response play in writing assessments?
The length in and of itself is not a factor in scoring student responses. Rather, the concern is how complete the piece of writing is based on the criteria the prompt and rubrics set forth. Sometimes very concise pieces of writing score higher than responses that are long and rambling. In addition, a student's handwriting can make a response look artificially longer or shorter than it actually is.

Does handwriting count in writing assessments?
No. The ability of a student to communicate in writing may be affected by handwriting that is difficult to read, but professional scorers do not let that influence the score that is assigned. Writing assessments focus on how well a student communicates important ideas on paper in a logical order and a readable fashion. Penmanship is a separate skill. The only exception is a piece of writing that is completely illegible and cannot be scored.

What role does spelling play in writing assessments?
The weight an assessment places on spelling depends entirely on who developed the rubrics. It may be based on a district's expectations, or on local or state standards. In Virginia's direct writing assessments spelling is one of the features of the usage/mechanics domain. Therefore, spelling is assessed with the other features of the usage and mechanics expected at a particular grade level. Students are allowed and encouraged to use dictionaries on Virginia's direct writing assessments.

Should students practice before taking a formal writing assessment?
Students should be given guided practice for writing as often as possible. However, not every piece of writing in a classroom should be a fully polished piece of writing. A variety of writing techniques and situations helps beginning writers understand that purpose and audience come under consideration each time they sit down to write.

If a writing assessment relies on a piece of writing on demand, doesn't that violate what teachers are trying to teach students about writing as a process?
It's important for students to understand that each time they sit down to write, they must consider audience, purpose, content, and deadlines. Sometimes writers have weeks, even months, to produce a piece of writing. Other times, responses must take shape during one class period or within a few hours. This time factor holds true in the "real world" as well, as represented by project reports, research papers, and legal briefs on the one hand, and memos, essay tests, and business letters on the other. One type of writing doesn't negate another. Teachers should expose students to all types of writing and situations.

Can I apply holistic, focused holistic, or analytic scoring to student writing in my classroom?
Yes, with some cautions. The purpose of most classroom writing assignments is instructional or diagnostic.If the writing is a very first draft, it is not instructive to assign a score, even with a well-written rubric at hand. If students are writing in a workshop setting and/or writing several drafts, there generally needs to be significant teacher and/or peer input in the form of margin notes and suggestions on at least one early draft to assist them in revising and polishing their writing. Once a piece has been turned in ready to publish, using a rubric to assign a score is a quick and effective way to evaluate the writer's ultimate success.

Not every piece of writing a student produces needs to be critically evaluated. Some quick writes or journal entries, for example, may be considered exercises similar to practice routines for ice skaters or scrimmages for basketball players.

And finally, using this type of assessment in the classroom, or for a high-stakes assessment, works only if the students know the criteria beforehand. The rubric can be a valuable teaching tool. Rubrics should be shared with students so that they understand how their writing will be evaluated. The ultimate goal is student self-awareness.
Glossary
+
 General Terms Used In Writing Assessments
Refer to the Annotation Terminology Section for terms used in annotations.

adjacent scores:when two scores assigned to a writing sample are within one score point of each other; for example, a paper given a score of "4" by one scorer and a "3" by another

analytic scoring:the method by which trained readers evaluate a piece of writing in various areas and assign a separate score for each area, e.g. conventions, style, word choice, organization, topic development

anchor papers:student-produced writings that exemplify the score standards for each level on the rubric

annotation:the explanation accompanying training papers which supports the assigned score

domain scoring:the method by which trained readers evaluate a piece of writing for its overall quality in previously designated areas or domains. In Virginia, this method is called "focused holistic scoring."

draft:a preliminary version of a piece of writing

exact or perfect agreement:when two scores assigned to a writing sample by different readers are the same

field test (tryout or pilot):testing a representative sample of a student population to determine the effectiveness of assessment items

final copy:the polished copy of a student's writing which is ready for publication

focused holistic scoring:the method by which trained readers evaluate a piece of writing for its overall quality in previously designated areas. In Virginia this method was formerly called "domain scoring." The designated areas are composing, written expression, and sentence usage/mechanics

holistic scoring:the method by which trained readers evaluate a piece of writing for its overall quality

non-adjacent:when two scores assigned to a writing sample are not exact or within one score point of each other; for example, a paper given a score of "4" by one scorer and a "2" by another

nonscorable:the category into which papers fall that cannot be scored because they are illegible, incoherent, in a language other than English, off-topic or blank; the nonscorable categories differ from assessment to assessment

off-prompt or off-topic:a paper that has not responded in any way to the given prompt

overlay:highlighting on anchor papers that illustrates points made in the annotations

practice set:training set where the user must continue to select score points until the exact score is chosen

primary trait scoring:the method by which trained scorers evaluate writing based on how well the writer has fulfilled the assigned task

prompt:writing task that states the topic and purpose of the piece of writing

range finding:a process in which a group of experts score and discuss student papers in order to come to consensus about how to apply the rubric; the scored papers are usually then used to make training sets

reader:also called a scorer, a person trained to apply the holistic scoring process to student writing

reader bias:personal factors, not included in the scoring criteria or the rubric, that may affect the score assigned to a student paper

rubric:scoring criteria at each score point of the scale

scoring guide:scoring criteria used to evaluate performance assessment and to assign score points to student work; sometimes called the rubric; sometimes includes exemplars of student work

surface feature errors:errors the student made in the area of conventions (except usage) that are not immediately apparent if the paper is read aloud; for example, errors in spelling, punctuation, indenting, and capitalization

tone:a writer's tone might be described as somber, tongue-in-cheek, formal, casual, ironic, melodramatic, etc.; the tone is a result of the writer's word choice and sentence structure

umbrella statement:similar to a thesis statement although less formal, an umbrella statement consists of one or two sentences that indicate what the writer intends to do in the rest of the paper. For example, asked to write about an invention that has been important in his/her life, a writer might begin by saying, "I can't write about just one invention since so many have affected my life." This would indicate to a reader that the writer intends to discuss several inventions.

verification set:training set in which the user is immediately informed of the correct score

voice:a writer's voice comes from the elements that contribute to the overall impression of the writer's personality as it comes through in one particular piece of writing; word choice, sentence structure, organizational strategy, details, insight, tone, and approach to the topic all work together to create a writer's voice

writing process:the typical steps that are used in the development of a piece of writing, including prewriting, drafting, revising, editing, and publishing

+
 Annotation Terminology
The following terms and phrases are used in the annotations and may need further explanation for teachers who are not familiar with them. In some cases, the terms may be used in a slightly different way than other sources. Reading the explanation for these terms will help further clarify the reason a paper received the score point it did, as well as provide a sense of the many elements scorers consider as they read a paper.

audience:This term refers to the intended reader or readers for whom a piece is written. A writer can show "audience awareness" in a variety of ways. Some of these include identifying characters and places as they appear in a narrative, giving readers clear time markers so they can easily follow a sequence of events, defining technical terms or jargon with which readers might be unfamiliar, tying thoughts or events together for a reader with insightful comments, and generally providing readers with sufficient information to make the writing clear. Audience awareness is also sometimes referred to as a writer "bringing the readers along."

best fit:This term is used most often in focused holistic scoring in which a paper may have characteristics of several different score points. For example, a paper may have some features that are at a 2 level while other features may seem to be at the 3 level. The paper would receive a score of 2 or 3 depending on the score point of the rubric that most accurately describes the attributes of the paper. A scorer would also consider whether the 2- and 3- level elements were in the high or low range of those score points. Ultimately, the paper would be given the "best fit" score. It had elements of both score points but, overall, was it more like a 2 or more like a 3?

borderline papers:Within each score point there will be a range of papers that represent the lower, middle, and upper levels of that score. Papers that represent the middle level are often referred to as "solid" examples. Papers representing the extreme lower or upper levels are sometimes called borderline papers because they fall between two score points. In other words, a 2/3 borderline paper has some 2-level characteristics and some 3-level characteristics. Ultimately, of course, a scorer must assign the paper a "high 2" or a "low 3." Scorers are cautioned, however, to remember that a "high 2" is still reported back to the student as a 2 and differs significantly from a score of "low 3" which will be reported back as a 3.

central idea:This might also be called the paper's overall theme or purpose. It is governed by the message the writer would like to see the reader take away from the piece of writing. The topic is not a central idea. The central idea is the message or idea about the topic.

clustering:Clustering is an organizational strategy in which related details are placed together in "clusters" or "clumps." Clustering is generally a weak organizational strategy because little progression of ideas is evident in the paper.

cohesiveness (see unity):A paper that gives the sense of completeness with no extra details and no details missing might be said to have cohesiveness. A paper that lacks cohesiveness may have an abrupt or no ending, an introduction that doesn't connect to the body of the paper, serious gaps in the story leaving a reader with unanswered questions, or a switch in topics mid-way to a completely different topic.

control:Score points in the SOL rubrics use the term "control" to describe the writer's ability to use a given feature of written language effectively.

composing domain:This domain of the Virginia SOL Writing test includes the focusing, structuring, and elaborating that a writer does to construct an effective message for a reader.

density of errors:The shorter a paper is, the more the density of errors in conventions affects a reader's overall impression of the paper. In other words, a paper that consists of 5 sentences with a total of 12 errors might be said to have a greater density of errors than a paper three pages long with a total of 12 errors.

elaboration:A paper is well elaborated if the details, examples, and illustrations focus the central idea in a purposeful way. Elaboration occurs at the sentence, paragraph, and paper levels.

fluency:This refers to the overall rhythm and flow of a paper and takes into consideration features such as word choice, sentence structure and variation, transitions, organizational strategy, and focus. A test of fluency might be reading the paper aloud and listening to how easily and smoothly it reads. Well-controlled fluency generally means there are few or no awkward moments where the reader stumbles.

mechanics:Mechanics is a feature in the usage/ mechanics domain that includes formatting, spelling, capitalization, and punctuation.

organization:Organization is the logic structure that facilitates presentation of the central idea. This characteristic takes into account the way a piece of writing is arranged and ordered.

precise vocabulary:Words or phrases are carefully placed and purposefully chosen by the writer to resonate as images in the reader's mind.

rhythm:Rhythm is the quality of fluency in a writing. A writer's rhythm can either be pleasant and unobtrusive, or it can be choppy and abrupt; a pleasingly rhythmic paper will include sentences which are varied in length and beginnings.

selected information:Specific, purposefully chosen information helps to propel the reader through a writing. A paper lacking information, giving too much information, or repeating ideas will affect the reader negatively.

sentence formation:Sentence formation is a feature in the usage/mechanics domain that includes standard word order, completeness, avoidance of sentence faults (run-ons, comma splices) and standard coordination.

sentence-to-sentence organization:Sometimes each sentence is tied to the sentence that comes before, but there is no overall organizational strategy. These papers might seem to have an internal organization at first glance, but on closer examination, the paper often ends up in a very different place than where it began. For example: "My favorite weather is sunny because I can play baseball. Baseball is a great warm weather outdoor sport. I'm a good baseball player. I play first base, which is the same position Joe Smith plays for the Rhinos. The Rhinos had a great season last year. This year my dad got us tickets to see 3 of the Rhinos home games. The home games are great because I can take my friends and eat hot dogs and popcorn." The initial point of "my favorite weather" has been lost. Do not mistake additional sentences as always being elaboration of the topic. Sentences that can be rearranged without affecting the message demonstrate a flaw in organization.

sentence variety:A paper with sentences of various lengths, constructions, and beginnings creates a pleasant, unobtrusive rhythm for readers.

surface feature errors:These are errors the student made in the area of conventions (except usage) that are not immediately apparent if the paper is read aloud; for example, errors in spelling, punctuation, indenting, and capitalization. (See Virginia Skills Lists for items addressed at each tested grade level.)

tone:A writer's tone might be described as somber, tongue-in-cheek, formal, casual, ironic, melodramatic, etc. The tone is a result of the writer's word choice and sentence structure. For example, a piece of writing using slang or colloquialisms might be considered casual. It could be an appropriate tone for a narrative, a personal letter, or an informal essay. Sometimes, students attempt a lofty, academic tone in their writing that generally produces stilted prose since the word choices and sentence constructions may be awkward or incorrect. For example, "One's choice of a college could well depend on where one's parents attended or that of which one's own siblings," An uneven tone occurs when a student mixes casual and formal terminology. For example, "I threw myself forward down the precipice. Every nerve was shrieking, "Stop, stop. Dang it all, Stop." A flat tone is characterized by ordinary word choices, repetitive and often simple sentence structure, and an overall lack of vitality in the writer's voice. It is also sometimes called "generic" since a flat tone gives no indication of a writer's unique personality. A tone closest to the student's own generally produces the most genuine type of writing.

unity:Unity is completeness in the elaboration of a central idea in an organized way. A unified paper has no extraneous details and no details missing.

usage/mechanics:This domain reflects the writer's ability to compose competent, appropriately mature sentences (observed independently of purpose and style), to construct these sentences grammatically at the word level, and to spell, capitalize, and punctuate writing mechanically at its surface level.

usage:Usage is a feature of the usage/mechanics domain that includes inflections, agreement, conventions, and word meaning.

voice:A writer's voice comes from the elements that contribute to the overall impression of the writer's presence as it comes through in one particular piece of writing. Word choice, sentence structure, organizational strategy, details, insight, tone, and approach to the topic all work together to create a writer's voice. A writer's voice can and often does change depending on the piece's audience and purpose, although more experienced writers tend to have a similar voice in all of their writing.

written expression domain:This domain of the Virginia SOL Writing test includes those features that show the writer can purposefully shape and control language to affect readers. The features of written expression are vivid and precise vocabulary, selected information, tone, voice, and sentence variety.

Rubrics
+
    Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the composing domain's features. Full elaboration focuses the central idea both at the sentence level and throughout the entire piece by providing purposeful examples, anecdotes, illustrations, or details. Narrative organization is intact; in other modes, minor organizational lapses may occur. In all types of writing, a strong organizational plan is apparent. Any organizational lapses that occur do not significantly detract from the presentation. Several elements in the writing provide evidence of unity: appropriate transitions, a consistent point of view (e.g., not switching from "I" to "you"), a lack of digressions, and the presence of a lead and closure that is more than mere repetition of statements.
Introduction
Critical Information
Assessment Information
Understanding Scoring
Glossary
Rubrics
Introduction
+
 Welcome

Welcome to Understand Scoring-powered by Perspective

This program is designed to help you
  • gain a better understanding of the Virginia Standards of Learning Direct Writing Assessments for grade 5, grade 8, and high school
  • understand the use of rubrics in scoring student writing
  • develop consistency in scoring actual student work

Specifically, this program will
  • provide information about the Virginia Standards of Learning English Assessment
  • provide an opportunity to study the rubrics used for scoring the direct writing assessments
  • provide an opportunity to study anchor papers that illustrate each score point of each domain
  • provide an opportunity to practice scoring several sets of papers written by Virginia students
  • verify your progress by evaluating the accuracy of the scores you assign to a final set of papers (verification set)
The Critical Information sub-section below includes important information you need in order to understand the rubrics and the scoring process. It is extremely important for you to read the Critical Information sub-section before proceeding. You can review the other sub-sections at any time.
+
 Definitions of Sections

Learn About Scoring
   This section provides important information that covers many aspects of Assessment, Scoring, Virginia SOL, and a glossary
    Critical Information
    This section provides critical information about the Virginia English SOL Assessment and scoring student papers.
    AssessmentInformation
    This section provides information about Virginia SOL test development, prompts and committees.
    Understanding Scoring
    This section provides a deeper understanding of scoring. It discusses the people and process used to score Virginal SOL     writing assessments. This section also covers information about rubrics, anchor papers, alert papers, and nonscorable     papers.
    Rubric
    This section describes the criteria for each score point of the rubric for composing, written expression, and usage and     mechanics.
Anchor Papers
   This section allows you to review papers that represent each score point, with annotations and color highlights that explain the assigned scores. The anchor papers will help you understand differences between score points.
Practice Scoring
   This section allows you to practice scoring and check your scores. After scoring practice sets, you can test your progress by attempting a verification set.
Critical Information
+
 What Is Focused Holistic Scoring?
Focused holistic scoring is the evaluation of writing proficiency based on several specific elements or domains of writing. The writer is judged in each domain independent of the others. In Virginia's Direct Writing Assessment each paper receives three scores, one for each of the three writing domains identified: composing, written expression, and usage/mechanics. Each domain is scored holistically. The score reflects the reader's overall impression of each of the specified domains. The scorers weigh the student's relative strengths or weaknesses in each of the three domains and then assign a score point for each domain that most accurately describes the attributes of the paper. Therefore, each paper receives a separate score for composing, written expression, and usage/mechanics.

In focused holistic scoring, it is possible for a paper to receive a higher score in one area such as usage/mechanics and a lower score in another area such as written expression. Assigning separate scores assists a teacher in diagnosing a student's relative strengths and weaknesses so that instructional activities can be planned for those specific skills identified as features within the rubric-scoring guide. A focused holistic scale defines the requirements for each domain at each level of the rubric. Thus, a particular score represents not only an overall impression, but also a specific level of proficiency in the complete set of composition domains being observed.

Virginia's Direct Writing Assessment model uses a 4-point scale. Each Domain is scored independently, using the following scale:

4 =The writer demonstrates consistent, though not necessarily perfect, control* of almost all of the domain's features.

3 =The writer demonstrates reasonable, but not consistent, control* of most of the domain's features indicating some weakness in the domain.

2 =The writer demonstrates enough inconsistent control* of several features to indicate significant weakness in the domain.

1 =The writer demonstrates little or no control* of most of the domain's features.

*Control: The ability to use a given feature of written language effectively at appropriate grade level. A paper receives a higher score to the extent that it demonstrates control of the features in each domain.

As in any scoring, a range of papers is evident within each score point. In a 4-point scale there will be high 4's and low 4's, high 3's and low 3's, etc. Although trainers talk about writing assessments as having discrete score points, the assigned scores actually represent a continuum within the score points. At every score point level, there is a range of papers representing the lower, middle, and upper end of the score point. A paper assigned a score of 3, for example, might have some characteristics of a 2-level paper and some characteristics of a 4-level paper, but overall, its 3-level characteristics predominate. Because of this phenomenon, papers receiving the same score point may look very different from each other.The 3-level paper described above, for example, might be at the top end of the 3's, and look very different than a paper that has only 2- and 3-level characteristics. The second paper is still assigned a 3, but it represents the lower end of the 3's. It is important to understand the criteria used in assigning a score and not to expect papers at a certain score point always to look the same.
+
 How Are Rubrics Used to Score Papers?
Generally, rubrics for writing assessments are fairly generic. It is only in conjunction with anchor (exemplar) papers that the criteria at each score point begin to be clarified. The rubric provides the general description of a score point; the anchor papers are the exemplars of how those general descriptions look in practice. A good set of anchor papers attempts to represent the different ways a student might achieve a score by including papers which attain the same score point in different ways.

Because rubrics are not meant to be lists of mandatory criteria a student must meet in order to achieve a score point, the word "may" is often used in a score point description.This use of the word may encourages scorers to remember that a paper may demonstrate some characteristics of one score point and some of another. A paper ultimately should be assigned a "best fit" or most appropriate score. For example, in the 4-point focused holistic rubric used in Virginia's Direct Writing Assessment, one of the sentences used to describe a 2 in Composing is "Even if a single idea dominates, the writing may be a list of general, underdeveloped statements or a skeletal plot." While this is often the case, it is also possible for a paper that doesn't have a skeletal plot or isn't a list of underdeveloped statements to receive a score of 2 because other 2-level qualities predominate and keep the paper from achieving a higher score. One of the most important points in learning how to apply a rubric consistently is to let go of existing prejudices. When scoring writing assessments, one of the most difficult tasks is to accept the boundaries of a rubric when those boundaries differ from a reader's own teaching practices or beliefs. For example, a teacher might value a nearly error-free paper in the area of usage/mechanics while a rubric might make allowances for certain errors. Assigning scores to papers in an assessment based on a rubric the teacher did not create is not to suggest teachers must adopt that rubric's criteria to classroom assignments. Large-scale assessments and classroom-based assessments may very well measure somewhat different skills in a different manner.
+
 What Are the General Considerations for the Writing Tests?
No provisions are made for examinees to recopy their final drafts. They are encouraged, however, to revise their work and to make whatever changes they believe will improve their papers. Readers, therefore, should expect and must be tolerant of revisions and interlinear editing in these papers. To gather as much instructionally useful information as possible in its writing assessment, Virginia has adopted a scoring model that calls for evaluations in three domains - composing, style, and usage-mechanics. In this type of scoring, the observation of writing is divided into several domains (categories), each of which is comprised of various features. Each domain is evaluated holistically, with the domain score indicating the extent to which the features in that domain appear to be under the control of the writer. Thus, an awareness of the features and their use contributes to the score, but the score is a judgment of the whole domain and not simply a counting of demonstrated features. Some skilled scorers can score for all domains after one complete reading, although scorers may re-read a portion in order to make a decision about one or more domains.

Application of the Scoring Scale
  1. Look as much at what the writer does well as at what he/she does poorly.
  2. Give consideration to all features within domain.
  3. Do not contaminate the scoring of one domain by awareness of either accomplishments or errors belonging to another domain.
  4. Do not create double jeopardy, e.g., a run-on sentence in Usage-Mechanics is not also a sentence error in Written Expression, and "brite" is one misspelling no matter how often the word appears.
  5. Compare papers to the requirements of each domain, not to each other, or to student writing that you may have previously read, or to some general standard of desired literacy.
  6. Score only what is on the paper. It is impossible to evaluate what the writer might have intended to say.
  7. Note that a word that seems to have been omitted due to haste in revising, editing, or copying need not demonstrate a total lack of control over that feature.
  8. Do not score a paper on only length. Length, in and of itself, is not a feature of any domain.
  9. Read the entire paper before assigning any scores.
Nonscorable Papers
All papers, no matter how brief, must be scored, unless they are off-topic, illegible, incoherent, refusals to respond, or written in a language other than English. Only the Scoring Director is permitted to assign a nonscorable code.
+
 What Is the High School Writing Prompt Used in Understand Scoring for Virginia?

What do you think is the most important skill for students to have learned by the time they finish high school? Write about why you think that skill is most important.

CHECKLIST FOR WRITERS

______I planned my paper before writing it.

______I revised my paper to be sure that

______the subject of my paper was clear;

______everything in my paper told about my subject;

______my paper was logically organized so readers would understand my message;

______my words and information made my paper interesting to readers; and

______my sentences made sense, sounded like me, and read smoothly.

______I edited my paper to be sure that

______I used good grammar;

______I used capital letters and punctuation marks correctly;

______I let my readers know were I started new paragraphs; and

______I made my spelling correct.

______I proofread my paper to make sure that my paper was the way I wanted readers to read it.

+
 What Are the Allowable Scoring Interpretations?
When scoring writing samples, decisions must be made regarding how papers with unexpected content, organization, or structure will be scored. While some guidelines are general, other decisions must be made because a significant number of writers have chosen to respond to a specific prompt in an appropriate but unanticipated way. These "rules" generally cannot be planned in advance but come out of issues that arise during the scoring process. Teachers should consider that the following list does not necessarily represent rules that would apply to classroom assignments or even classroom assessments. They are offered here as examples of considerations of how the papers written in response to the prompts included in these training materials were scored.

High School Prompt:What do you think is the most important skill for students to have learned by the time they finish high school? Write about why you think that skill is most important.

High School Prompt-Specific Scoring Guidelines
  • In general, if it is apparent that the student is responding to the prompt, the paper should be considered on topic and should be scored according to the rubric. (Pay special attention to those papers that look as if they are refusals at first glance but are actually responses to the prompt, e.g., "Asking us to write about the most important skill students learn in high school is silly. It would be better to ask us to write about...").
  • Once a paper is determined to be "on topic," that issue should no longer influence the score assigned. Papers are not scored lower for being slightly on topic or higher for being exactly on topic. The assigned score should reflect only the student's writing ability as described in the rubric.
  • A student may choose to write about several important skills instead of just one. Such a response should be considered on topic. Depending on how skillfully the student handles the inclusion of multiple skills, the unity of the piece may be negatively affected.
  • It is not necessary for the student to explain why the skill is important.
  • The reader should not judge the appropriateness of the skill the writer chooses to describe.
  • The reader should not judge the reasons the writer gives for choosing a skill.
  • The writer need not explain how to acquire the skill.
  • The skill might be something acquired outside the classroom.
  • The skill might be concrete, e.g., how to take good notes, or a moral lesson, e.g., how to be a good friend.
  • Readers should be somewhat forgiving of naive or illogical explanations.
  • The student may state that s/he knows of no important skill learned in high school and write about why everything learned in high school is unimportant.
  • The mode the student uses should not influence the score: narrative, expository, descriptive, persuasive, imaginative.
+
 How Does Reader Bias Affect Scoring?
Reader bias refers to personal factors that may affect a reader's perception of a student's response, but have no basis in a rubric. Below are factors that affect some readers but must not be allowed to have an impact on scoring.

Appearance of Response.
The quality of the handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria.

Length of Response.The length of a student paper is not part of the scoring criteria. Readers should take into consideration only whether the finished piece seems complete and has the components required. The size of a student's handwriting can make a paper look longer or shorter on the page than it actually is.

Repetition of Response.Although readers may tire of reading several essays on the same topic,it is important to remember that for each student the response represents a unique attempt.

Offensive or Disturbing Content.If a student uses vulgar language, adopts a sexist or racist point of view, or perhaps takes a naive or narrow approach to a topic, readers should not let the student's point of view affect the score. Likewise, readers should not let a student's lifestyle or maturity level influence them either positively or negatively regarding the writing.

Alert Papers.Because the prompts are designed to have meaning to students so they can create a personal response, sometimes the paper relates situations and information that are disturbing. It is Virginia's policy to bring disturbing papers to the attention of an adult close to the student as soon as possible so that positive action can be taken if necessary. Readers are instructed to indicate to the Scoring Director any paper that is found to be disturbing and then score the paper according to the criteria.

Response to Prompt.In the classroom, there may be a "correct" response to a writing assignment. For Virginia's assessment, students are free to respond in any way they choose. There is no right or wrong "answer" as long as it is clear that the student is attempting to reply to the prompt.

Reactions to Style.A reader's own grammatical biases should not play a part in assigning a score if the student has not violated standard writing conventions. In other words, beginning a sentence with "and," the absence of a formal thesis sentence, the use of first or second person, or an informal tone are not wrong in this type of assessment. Classroom assessments often have different requirements.

Writer's Personality.Writers may come across as brash, sassy, cute, self-aware, shy, surly, flat, honest, or naive. Readers are scoring the written passage, not the writer's personality.

Reactions to Direct Writing Assessments.Some readers may approach writing assessments with their own biases in favor of one type of assessment over another. Or, they may believe it is impossible to score writing fairly using any of these techniques. The prompts and rubrics used in Understand Scoring for Virginia have been analyzed and validated by experts in the field and adhere to standard practices in state assessments. Teachers should tailor their own classroom writing assessments around local curricular expectations.
Assessment Information
+
 How Are Standards of Learning Tests Developed?
The development of each SOL test begins with the blueprint for that test. The test blueprint serves as a guide to test developers as they write test questions and construct the SOL tests. The test blueprints show (a) the Standards covered by the test and which, if any, have been excluded; (b) which Standards are assigned to each reporting category; (c) the number of test items in each reporting category and on the total test; (d) general information about how the test questions were constructed; and (e) the materials that students are allowed to use while taking the test.

Using the test blueprint as a guide, the test development contractor writes test items designed to measure student achievement of the particular Standards of Learning covered by that test. Each test item is coded to a particular SOL.

The content review committee for that test reviews each proposed test item, answering the following questions for each item:

SOL Item Match:Does this item measure the SOL it was designed to measure?

Appropriateness:Does this item appropriately measure content or skills that students in Virginia should be required to have by spring of the designated grade level or near the end of the course?

Difficulty:Is the difficulty of this item appropriate?

Fairness:Is the item free from any content that would offend or unfairly penalize students on the basis of personal characteristics such as gender, ethnicity, religion, or socioeconomic status?

If the answer to each of these questions is "yes," the item is accepted for field testing and tried out with Virginia students. Items are tried out by placing them as field test items on an SOL test. Field test items do not count toward the student's score on the test; however, data are collected on how students respond to the items. The number of field test items on a test varies depending on the test: the multiple choice section of the English Writing Test for grade 5 has 12 field test items, while the multiple choice section of English Writing Test for grade 8 and the multiple choice section of Secondary English Writing Test both have 14 field test items.

Once the field test data are collected, the content review committee reviews the items again using the statistical information collected during the field test and decides whether each item should be accepted into the item bank for potential use on a future test form. In the case of the high school tests, a separate Bias Committee reviews each item a second time for fairness.

Using items that have been approved by the content review committee for inclusion in the item bank, the test development contractor follows the test blueprint to develop test forms. These test forms are reviewed by the content review committee prior to their administration. Once the test form is administered, each test item either returns to the bank or is released as a sample test item.

+
 How Are Prompts Selected?
The process of prompt development begins with the test development contractor's developing potential writing prompts for the direct writing component of the SOL English Writing Test. According to the test blueprints, prompts are to present to the student a context for writing in the form of a question, an issue, or a hypothetical situation. The content review committee then reviews each proposed writing prompt for potential field testing.

If the prompt is accepted for field testing, it is tried out with Virginia students. Papers written to the prompt during the field test are scored by the scoring contractor. The readers involved in the field test scoring evaluate the prompt in terms of whether it would be appropriate for use in live testing. This analysis in addition to information about the number of papers assigned each score point in each domain is provided to the content review committee.

Using this information from the field test, the content review committee reviews the prompts again and decides whether each prompt should be accepted into the prompt bank for potential use on a future test form. Prior to each administration of the SOL English Writing Test, the content review committee selects two prompts to be used: one for the main prompt and one to be used as the make-up prompt for students who are not tested on the statewide writing day.

+
 What Committees Are Involved in the Development of the Standards of Learning Tests?
Content Review Committee
Each SOL test is constructed according to the specifications of the SOL test blueprint, and with the cooperation of a Content Review Committee composed of Virginia educators who have experience and expertise in the content area and grade level of the test. Working with the Department and the testing contractor, the Content Review Committees review each test question before it is field-tested. In order to make it to field test status in Virginia, the potential question must, in the eyes of the Content Review Committee, meet the following four criteria:
  1. Does the question measure the SOL it was designed to measure?
  2. Does the question appropriately measure content or skills that students in Virginia should be expected to learn by the spring of the designated grade level or near the end of the course?
  3. Is the difficulty of the question appropriate?
  4. Is the question free from content that stereotypes, offends, or unfairly penalizes students on the basis of personal characteristics such as gender, ethnicity, religion, or socioeconomic status?

If the question meets these criteria, it is used as a field test question with Virginia students during a regular administration of the SOL tests. Field test questions appear among the operational questions on each SOL test. In this way a steady stream of new test questions is available for use.

After test questions have been field tested, the Content Review Committee is reconvened to review the questions and make the same four judgments again, but this time with statistics about the field-tested questions. If the answers to the four questions are yes, then the test question is placed in the item bank for potential use in an operational SOL test form.

When preparing a test form for operational use, the testing contractor selects questions from the item bank in accordance with the blueprint specifications. Once draft test forms are constructed, the Content Review Committees are again convened. Committee members assume the task of approving or editing two forms of each grade level or end-of-course test to determine the content validity and equivalency of the test forms as a whole. While the previous committee reviews were concerned with individual questions, the focus of this review is the full operational test forms. At this stage there may be additional minor edits or revisions.

In addition to the previously discussed duties, the content review committees for the SOL Writing Tests select the prompt to be used for each administration of the tests and review and approve the materials used to train readers to score the prompt.

Bias Committee
Because passing the high school SOL assessments will be a graduation requirement, it is especially important that the tests be free of factors that have an unfair impact upon a group of students. Therefore, an additional bias review is conducted by a separate Bias Review Committee representing each content area to be tested. Bias Review Committee members are asked to scrutinize items for potential stereotyping or other forms of bias. The purpose of the bias review is to identify any items that appear to have the potential to treat any group, regardless of ethnic background, gender, or region, differently from other groups. Committee members examine the response distribution for each of the demographic groups identified for the review. The intent of the review is to determine if members of a certain group were drawn to one or more of the answer choices for the item. If a large percent of one group selected a particular response or did not select a particular response, the item is carefully examined. Training and procedures are similar to those carried out during the Item Review meetings. The Bias Review Committee's task focuses solely on reviewing test items for potential bias after the items have been reviewed by the Content Review Committees. It is the Bias Review Committee's responsibility to ensure that items are fair to all students and that all students would have an equal opportunity to demonstrate achievement regardless of gender, ethnic background, religion, socioeconomic status, or geographic region.

Assessment Policy Advisory Committee
The Assessment Policy Advisory Committee composed of Virginia educators advises the Virginia Department of Education and the Board of Education on policy issues related to the state-mandated assessment programs. This committee was instrumental in developing the guidelines for testing students with disabilities and the guidelines for testing limited-English-proficient students in the SOL program.
Understanding Scoring
+
 What Is Focused Holistic Scoring?

High School Focused Holistic Scoring Guidelines

Focused holistic scoring is the evaluation of writing proficiency based on several specific elements or domains of writing. The writer is judged in each domain independent of the others. In Virginia's Direct Writing Assessment each paper receives three scores, one for each of the three writing domains identified: composing, written expression, and usage/mechanics. Each domain is scored holistically. The score reflects the reader's overall impression of each of the specified domains. The scorers weigh the student's relative strengths or weaknesses in each of the three domains and then assign a score point for each domain that most accurately describes the attributes of the paper. Therefore, each paper receives a separate score for composing, written expression, and usage/mechanics.

In focused holistic scoring, it is possible for a paper to receive a higher score in one area such as usage/mechanics and a lower score in another area such as written expression. Assigning separate scores assists a teacher in diagnosing a student's relative strengths and weaknesses so that instructional activities can be planned for those specific skills identified as features within the rubric-scoring guide. A focused holistic scale defines the requirements for each domain at each level of the rubric. Thus, a particular score represents not only an overall impression, but also a specific level of proficiency in the complete set of composition domains being observed.

Virginia's Direct Writing Assessment model uses a 4-point scale. Each Domain is scored independently, using the following scale:

4 =The writer demonstrates consistent, though not necessarily perfect, control* of almost all of the domain's features.

3 =The writer demonstrates reasonable, but not consistent, control* of most of the domain's features indicating some weakness in the domain.

2 =The writer demonstrates enough inconsistent control* of several features to indicate significant weakness in the domain.

1 =The writer demonstrates little or no control* of most of the domain's features.

*Control: The ability to use a given feature of written language effectively at appropriate grade level. A paper receives a higher score to the extent that it demonstrates control of the features in each domain.

As in any scoring, a range of papers is evident within each score point. In a 4-point scale there will be high 4's and low 4's, high 3's and low 3's, etc. Although trainers talk about writing assessments as having discrete score points, the assigned scores actually represent a continuum within the score points. At every score point level, there is a range of papers representing the lower, middle, and upper end of the score point. A paper assigned a score of 3, for example, might have some characteristics of a 2-level paper and some characteristics of a 4-level paper, but overall, its 3-level characteristics predominate. Because of this phenomenon, papers receiving the same score point may look very different from each other.The 3-level paper described above, for example, might be at the top end of the 3's, and look very different than a paper that has only 2- and 3-level characteristics. The second paper is still assigned a 3, but it represents the lower end of the 3's. It is important to understand the criteria used in assigning a score and not to expect papers at a certain score point always to look the same.
+
 How Should the Scale Be Applied?
  1. Look as much at what the writer does well as at what he/she does poorly.
  2. Give consideration to all features within domain.
  3. Do not contaminate the scoring of one domain by awareness of either accomplishments or errors belonging to another domain.
  4. Do not create double jeopardy, e.g., a run-on sentence in Usage-Mechanics is not also a sentence error in Written Expression, and "brite" is one misspelling no matter how often the word appears.
  5. Compare papers to the requirements of each domain, not to each other, or to student writing that you may have previously read, or to some general standard of desired literacy.
  6. Score only what is on the paper. It is impossible to evaluate what the writer might have intended to say.
  7. Note that a word that seems to have been omitted due to haste in revising, editing, or copying need not demonstrate a total lack of control over that feature.
  8. Do not score a paper on length alone. Length, in and of itself, is not a feature of any domain.
  9. Read the entire paper before assigning any scores.
+
 How Should the Rubrics Be Applied?
In learning how to apply a rubric consistently, teachers must let go of existing prejudices. One of the most difficult tasks in writing assessment is to accept the boundaries of a scoring guide when those boundaries differ from a teacher's own teaching practices or beliefs. For example, a teacher might insist upon an error-free paper in the area of usage and mechanics for classroom writing assignments while a rubric might emphasize consistent control of usage and mechanics. Assigning scores to papers in an assessment based on a rubric the teacher did not create is not to suggest teachers must apply that rubric's criteria to classroom assignments. Large scale assessments and classroom-based assessments may very well measure somewhat different skills in a different manner.

Rubrics for writing assessments are generally fairly generic. It is only in conjunction with anchor papers that the criteria at each score point begin to be clarified. The rubric provides the general description of a score point; the anchor papers are the exemplars of how those general descriptions look in practice.
+
 What Are the Procedures for Training Scorers and Scoring?

Introduction to the SOL Writing Assessment

Training of readers begins with a brief introduction to the Virginia SOL Writing Assessment in which readers are informed of the goals and purposes of the assessment program. In addition, readers are told that the assessment is an on-demand writing task in which students are given a prompt and have up to an entire school day to respond to it. Students have space in the answer booklet for notes and are given scratch paper for prewriting and drafting if they need more space. A checklist included with the writing prompt encourages students to follow the writing process. Students have four sheets of paper in the answer booklet to use for their response and are told that only what they write on these four sheets will be scored. They have access to dictionaries and may write their responses in cursive, or they may print. Following the description of the writing task given to students, readers are told that they are to consider the papers written for this assessment to be first draft writing and are to be tolerant of interlinear editing.

Training and Qualifying
A discussion of the three domains used in the scoring model: composing, written expression, and usage/mechanics follows the introduction to the assessment. Readers are introduced to the prompt and then training begins with the composing domain. Trainers first discuss the features of the composing domain as well as the general scale: 4 = consistent control, 3 = reasonable control, 2 = inconsistent control and 1 = little or no control. Score point 1 in the composing rubric is then read, and the score point 1 anchor papers are read aloud and discussed. The discussion emphasizes each paper's written annotation which uses the language of the rubric to explain the score point as it applies to that paper. The score point 2 rubric is then read followed by discussion of the anchor papers for this score point. A similar process is used for score points 3 and 4 in composing. Following the discussion of the rubric and anchor papers prospective readers independently score the papers in the Composing Training Set for the composing domain only. A similar process is used for the written expression domain and the usage/mechanics domain.

Once all domains have been discussed and all domain specific training sets scored, readers begin scoring the mixed domain sets. The purpose of the mixed domain sets is to familiarize readers with assigning scores in each of the three domains to a paper. There are three such sets of 10 papers each (sets A, B, and C), and readers are asked to assign a score in each domain to these papers. Again, scores are discussed in detail at the completion of each set.

After all of the domain training sets have been scored and discussed, prospective readers begin scoring qualifying sets. There are 4 qualifying sets of 10 papers each. Readers are required to have 70% or better exact agreement on each domain in at least one qualifying set with 60 % or better agreement in a second set. Readers who do not qualify are dismissed from the Virginia project.

Scoring
Once readers have qualified, they are assembled into teams and begin scoring papers. Each team is led by a team leader who has been selected based on demonstrated expertise in all phases of the scoring process including skill in training readers and in maintaining the organizational procedures necessary for scoring to move smoothly. During scoring, team leaders respond to readers' questions about scoring and work with readers who are having difficulty applying the rubric. A primary duty of the table leader is to "read behind" readers to ensure that they are not drifting from the scoring criteria. Team leaders also review the scoring patterns of readers throughout the project and conduct retraining as necessary.

Two readers from different teams read each paper. If the two readers disagree by more than one point in any domain, a third reader reads the paper to resolve the score.

In addition to the team leaders, the scoring contractor's scoring directors and the Virginia Department of Education review daily reader reports that show how well readers are agreeing with each other. Furthermore, each day readers are asked to score packets of papers pre-scored by the content review committee. The reader reports resulting from these calibration packets serve to identify readers who are drifting off the scale, while the discussion of the score points assigned to each paper helps to re-calibrate all readers.

Readers are monitored throughout the scoring process. Those who fail to maintain acceptable agreement with other readers or who fail to accurately apply the scoring criteria despite retraining are dismissed.
+
 What Is Range Finding and How Are Scores Assigned?
For each administration of the SOL writing test, the prompt to be used for the direct writing component is selected by the content review committee for the English Writing Test at that grade level. After the prompt is selected, range finding begins with scoring contractor staff experienced with the Virginia SOL Writing Assessment. They score several hundred papers written to the prompt by Virginia students during the field test. Following this initial scoring, contractor staff discuss the papers and make preliminary selections of papers to be used in anchor sets, training sets, and qualifying sets. Then these preliminary sets are brought to the content review committee for discussion. The content review committee may replace papers or move them to different sets. Once the committee is satisfied with the papers and the scores assigned to them, the papers are ready for final preparation before being used for training readers.
+
 What Are the Domains and Definitions?

Composing

The Composing domain includes the focusing, structuring, and elaborating that a writer does to construct an effective message for a reader. It is the creation of a product, the building of a writing intended to be read. The writer crafts his/her message for the reader by focusing on a central idea, providing elaboration of the central idea, and delivering the central idea and its elaboration in an organized text.

Features:
  • Central idea
  • Elaboration
  • Unity
  • Organization
Written Expression
The Written Expression domain comprises those features that show the writer purposefully shaping and controlling language to affect readers. This domain focuses on the vividness, specificity, and rhythm of the piece and the writer's attitude and presence.

Features:
  • Deliberate word choice
  • Selected, precise information
  • Sentence variety
  • Tone
  • Voice
Usage/Mechanics
The Usage/Mechanics domain comprises the writer's ability to form competent, appropriately mature sentences and the use of word level features that cause written language to be acceptable and effective for standard discourse. This domain includes the system of symbols and cueing devices a writer uses to help readers make meaning.

Features:
  • Sentence formation
  • Usage
    • Standard inflections
    • Comparison of adjectives/adverbs
    • Agreement
  • Mechanics
    • Capitalization
    • Spelling
    • Punctuation
    • Formatting
+
 What Are the Writing Prompts and Checklists?

High School Writing Prompt and Checklist


What do you think is the most important skill for students to have learned by the time they finish high school? Write about why you think that skill is most important.

CHECKLIST FOR WRITERS

______I planned my paper before writing it.

______I revised my paper to be sure that

______the subject of my paper was clear;

______everything in my paper told about my subject;

______my paper was logically organized so readers would understand my message;

______my words and information made my paper interesting to readers; and

______my sentences made sense, sounded like me, and read smoothly.

______I edited my paper to be sure that

______I used good grammar;

______I used capital letters and punctuation marks correctly;

______I let my readers know were I started new paragraphs; and

______I made my spelling correct.

______I proofread my paper to make sure that my paper was the way I wanted readers to read it.

+
 What Are the Rubrics?

High School Rubrics

Composing Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the composing domain's features. The piece is generally unified in that all of the parts contribute to the creation of a dominant impression or idea. The sharply focused central idea is fully, but not exhaustively, elaborated with key examples, illustrations, reasons, events, or details. In all successful responses, layers of elaboration are present. Surface signals, like transitions, logically connect their respective statements into the whole of the paper. In all types of writing, a strong organizational plan is apparent. Any minor organizational lapses that occur do not significantly detract from the presentation. The writing provides evidence of unity by exhibiting a consistent point of view (e.g., not switching from "I" to "you"), a lack of digressions, appropriate transitions both within paragraphs and across the entire piece, the presence of careful logic, and a strong lead and closure.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of the composing domain's features; the writer may control some features more than others. The clearly focused central idea is purposefully elaborated with key examples, illustrations, reasons, events, or details. Occasionally, some thinness or unevenness in elaboration may occur. In all types of writing, an organizational plan is apparent. Any minor organizational lapses that occur do not significantly detract from the piece. Although there may be occasional lapses in coherence or cohesiveness, unity is evidenced by the fact that few, if any, digressions or shifts in point of view occur. Transitions are, on the whole, appropriate. The opening and closing show some skill, but not the sophistication of a 4 performance.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the composing domain. At this score point, ideas often compete, or no one idea emerges as central. Even if a single idea dominates, the paper may lack focus because of little or no elaboration. The paper may be a list of general, underdeveloped statements or the skeleton of a narrative. In the case of persuasive writing, it may consist of a few unelaborated reasons accompanied by inappropriate attempts (begging, pleading, negotiating) to persuade. Typically, the writer extends ideas with a few brief details and moves on, though chunks of irrelevant material may appear as well. Often, no more than a hint of organization is apparent. Even though an opening and closing may be present, the lack of a logically elaborated central idea prevents unity from emerging.

Score Point 1
The writer demonstrates little or no control of most of the composing domain's features. The focus on a central idea is lacking, or the piece is so sparse that the presence of a clear focus is insufficient for it to earn a higher score. Typically, the writing jumps from point to point, without a unifying central idea. No overall organizational strategy is apparent. The writing seems haphazard, and sentences can be rearranged without substantially changing the meaning. Bare statement is the norm, but even in responses that are several pages long, no purposeful elaboration is present.

Written Expression Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the written expression domain's features. The result is a purposefully crafted message that the reader remembers, primarily because its precise information and vocabulary resonate as images in the reader's mind. Highly specific word choice and information also create a purposeful tone in the writing and enhance the writer's voice. If metaphors, similes, personification, or other examples of figurative language are present, they are appropriate to the purpose of the piece. The writer repeats or varies sentence construction for effect and appropriately subordinates ideas and embeds modifiers on a regular basis, resulting in a rhythmic flow throughout the piece.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of the written expression domain's features. On the whole, specific word choice and information cause the message to be clear; occasionally, a few examples of vivid or purposeful figurative language may be present. Along with instances of successful control, some general statements or vague words may be present, flattening the tone and voice of the piece somewhat. Overall, the writing is characterized by a smooth rhythm created by the effective use of normal word order and competent variation in sentence length and complexity. An occasional awkward construction or the lack of structural complexity is not distracting.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the written expression domain. Some specificity of word choice might exist, but generally the piece is written in imprecise, bland language. As a result, the writer's voice rarely emerges. The selection of information may be uneven and/or consist of an attempt to tell everything that the writer knows about a topic. A relative lack of sentence variety may make reading monotonous, and occasional awkward constructions may be distracting enough to make the writer's meaning unclear. While a few brief rhythmic clusters of sentences may occur, an overall sense of rhythmic flow is not present.

Score Point 1
The writer demonstrates little or no control of most of the written expression domain's features. Both word choice and information are general, vague, and/or repetitive. A lack of sentence variety makes the presentation monotonous. The existence of several extremely awkward constructions reduces the paper's stylistic effect. The writer's lack of control of vocabulary and information prevents both tone and voice from emerging.

Usage/Mechanics Rubric

Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the domain's features of usage/mechanics. The writing demonstrates a thorough understanding of usage and mechanics as specified in the Virginia K-11 SOL. The author uses capitalization, punctuation, usage, and sentence formation and applies the structural principles of spelling. A few errors in usage and mechanics may be present. However, the writer's control of the domain's many features is too strong for these mistakes to detract from the performance.

Score Point 3
The writer demonstrates reasonable, but not consistent, control of most of the domain's features of usage/mechanics. The writing demonstrates a basic understanding of usage and mechanics as specified in the Virginia K-11 SOL. For the most part, the author appropriately applies both the rules of capitalization, punctuation, usage, and sentence formation and the structural principles of spelling expected of high school students. Most of the errors contained in the piece are not elementary ones.

Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the domain of usage/mechanics. Evidence of the author's knowledge of features of this domain appears alongside frequent errors. In terms of both usage and mechanics, the writer inconsistently applies the rules of capitalization, punctuation, usage, spelling, and sentence formation as specified in the Virginia K-11 SOL. Often, these papers exhibit a lack of control of tense consistency, meaningful punctuation, and the principles of spelling, thus making it difficult for the reader to follow the writer's thought. The density of errors that emerges across features outweighs the feature control present in the paper.

Score Point 1
The writer demonstrates little or no control of most of the domain's features of usage/mechanics. Frequent and severe errors in the Virginia K-11 usage and mechanics SOL distract the reader and make the writing very hard to understand. Even when meaning is not significantly affected, the density and variety of errors overwhelm the performance and keep it from meeting minimum standards of competence.

Sentence Formation, Usage, and Mechanics Skills

Listed below are skills in the areas of sentence formation, usage, and mechanics (capitalization, punctuation, formatting, and spelling) which may be addressed on both the multiple choice and the direct writing components of the secondary English writing test. This list is not intended to be exhaustive but to provide examples of the skills which may be addressed on the secondary English writing test.

Standard Sentence Formation

Use complete sentences

Avoid comma splices

Avoid used sentences (run-ons)

Avoid dangling modifiers

Usage

Correct use of the following:

subject-verb agreement

pronoun case

pronoun reference

pronoun antecedent agreement

adjective comparisons

adverb comparisons

adverbs instead of adjectives where appropriate

verb tense inflections such as -ed or use of helping verbs

possessives (singular and plural)

plural inflections (-s, -es, irregular forms)

negatives ( includes avoidance of double negatives)

frequently confused words (e.g., accept/except)

active/passive voice

Maintain consistent point of view (In direct writing, a shift in the writer's point of view is scored in composing; a shift in person such as the shift in the following sentence: "School is important to student; you have to do well to get a good job." is considered a usage error.)

Maintain tense consistency

Avoid common usage problems (e.g., lie/lay; less/fewer)

Avoid faulty parallelism

Mechanics

Punctuation

Correct use of the following:

colons

commas in dates, series, addresses

commas around interrupters (including but not limited to appositives)

commas and semicolons in sentence types

punctuation of dialogue

commas with certain clauses

apostrophes in contractions and possessives

quotation marks around dialogue and titles

italics or underlining as appropriate

letter parts

 

Capitalization

Correct capitalization of the following:

proper nouns and adjectives

academic subjects and classes correctly (e.g., history, sophomore)

title of works

I

Format

indent paragraphs or double space between them

paragraph dialogue correctly

divide words between syllables with a hyphen at end of line

Spelling

spell frequently used and common words correctly

spell homophones correctly

+
 What Are the Scores in the Other Domains for the Anchor Papers?

High School Anchor Set Scores
Paper ID* Composing Written Expression Usage/ Mechanics
Composing
Anchors
     
C11A01 1 1 2
C11A02 1 1 3
       
C11A03 2 2 2
C11A04 2 2 3
C11A05 2 2 3
       
C11A06 3 3 2
C11A07 3 3 4
C11A08 3 4 4
       
11T32 4 4 4
C11A09 4 4 4
C11A10 4 4 4
Written Expression
Anchors
     
W11A01 1 1 1
W11A02 1 1 2
       
W11A03 2 2 3
W11A04 2 2 2
W11A05 2 2 2
       
W11A06 2 3 2
W11A07 3 3 3
W11A08 3 3 3
       
W11A09 4 4 4
W11A10 4 4 4
11T29 4 4 4
Usage/Mechanics
Anchors
     
U11A02 2 2 1
U11A12 2 2 1
       
U11A03 2 2 2
U11A04 2 2 2
U11A05 3 2 2
       
U11A06 3 2 3
U11A07 3 3 3
U11A08 3 3 3
       
U11A09 4 4 4
11V06 4 4 4
U11A10 3 4 4

*The paper ID is located in the lower left corner of the student paper.
+
 What Are Writing Assessment Issues and Answers?

How are the anchor sets compiled in Virginia?
Typically, anchor sets are compiled from low score point to high score point or vice versa. Virginia arranges anchors sets from low score point to high score point. By starting with the lowest score point level, a person who trains scorers can talk about what a student paper does rather than what it doesn't do. While this may seem insignificant, it becomes very important in helping people internalize the rubric. For example, it's very tempting for readers to say, "This paper doesn't have a good opening, so it must not be a 4." Instead, when papers are ordered from low score point to high, scorers are encouraged to consider first what a paper does successfully and to what level it rises, before considering its weaknesses.

What role does the length of a response play in writing assessments?
The length in and of itself is not a factor in scoring student responses. Rather, the concern is how complete the piece of writing is based on the criteria the prompt and rubrics set forth. Sometimes very concise pieces of writing score higher than responses that are long and rambling. In addition, a student's handwriting can make a response look artificially longer or shorter than it actually is.

Does handwriting count in writing assessments?
No. The ability of a student to communicate in writing may be affected by handwriting that is difficult to read, but professional scorers do not let that influence the score that is assigned. Writing assessments focus on how well a student communicates important ideas on paper in a logical order and a readable fashion. Penmanship is a separate skill. The only exception is a piece of writing that is completely illegible and cannot be scored.

What role does spelling play in writing assessments?
The weight an assessment places on spelling depends entirely on who developed the rubrics. It may be based on a district's expectations, or on local or state standards. In Virginia's direct writing assessments spelling is one of the features of the usage/mechanics domain. Therefore, spelling is assessed with the other features of the usage and mechanics expected at a particular grade level. Students are allowed and encouraged to use dictionaries on Virginia's direct writing assessments.

Should students practice before taking a formal writing assessment?
Students should be given guided practice for writing as often as possible. However, not every piece of writing in a classroom should be a fully polished piece of writing. A variety of writing techniques and situations helps beginning writers understand that purpose and audience come under consideration each time they sit down to write.

If a writing assessment relies on a piece of writing on demand, doesn't that violate what teachers are trying to teach students about writing as a process?
It's important for students to understand that each time they sit down to write, they must consider audience, purpose, content, and deadlines. Sometimes writers have weeks, even months, to produce a piece of writing. Other times, responses must take shape during one class period or within a few hours. This time factor holds true in the "real world" as well, as represented by project reports, research papers, and legal briefs on the one hand, and memos, essay tests, and business letters on the other. One type of writing doesn't negate another. Teachers should expose students to all types of writing and situations.

Can I apply holistic, focused holistic, or analytic scoring to student writing in my classroom?
Yes, with some cautions. The purpose of most classroom writing assignments is instructional or diagnostic.If the writing is a very first draft, it is not instructive to assign a score, even with a well-written rubric at hand. If students are writing in a workshop setting and/or writing several drafts, there generally needs to be significant teacher and/or peer input in the form of margin notes and suggestions on at least one early draft to assist them in revising and polishing their writing. Once a piece has been turned in ready to publish, using a rubric to assign a score is a quick and effective way to evaluate the writer's ultimate success.

Not every piece of writing a student produces needs to be critically evaluated. Some quick writes or journal entries, for example, may be considered exercises similar to practice routines for ice skaters or scrimmages for basketball players.

And finally, using this type of assessment in the classroom, or for a high-stakes assessment, works only if the students know the criteria beforehand. The rubric can be a valuable teaching tool. Rubrics should be shared with students so that they understand how their writing will be evaluated. The ultimate goal is student self-awareness.
Glossary
+
 General Terms Used In Writing Assessments
Refer to the Annotation Terminology Section for terms used in annotations.

adjacent scores:
when two scores assigned to a writing sample are within one score point of each other; for example, a paper given a score of "4" by one scorer and a "3" by another

analytic scoring:the method by which trained readers evaluate a piece of writing in various areas and assign a separate score for each area, e.g. conventions, style, word choice, organization, topic development

anchor papers:student-produced writings that exemplify the score standards for each level on the rubric

annotation:the explanation accompanying training papers which supports the assigned score

domain scoring:the method by which trained readers evaluate a piece of writing for its overall quality in previously designated areas or domains. In Virginia, this method is called "focused holistic scoring."

draft:a preliminary version of a piece of writing

exact or perfect agreement:when two scores assigned to a writing sample by different readers are the same

field test (tryout or pilot):testing a representative sample of a student population to determine the effectiveness of assessment items

final copy:the polished copy of a student's writing which is ready for publication

focused holistic scoring:the method by which trained readers evaluate a piece of writing for its overall quality in previously designated areas. In Virginia this method was formerly called "domain scoring." The designated areas are composing, written expression, and sentence usage/mechanics

holistic scoring:the method by which trained readers evaluate a piece of writing for its overall quality

non-adjacent:when two scores assigned to a writing sample are not exact or within one score point of each other; for example, a paper given a score of "4" by one scorer and a "2" by another

nonscorable:the category into which papers fall that cannot be scored because they are illegible, incoherent, in a language other than English, off-topic or blank; the nonscorable categories differ from assessment to assessment

off-prompt or off-topic:a paper that has not responded in any way to the given prompt

overlay:highlighting on anchor papers that illustrates points made in the annotations

practice set:training set where the user must continue to select score points until the exact score is chosen

primary trait scoring:the method by which trained scorers evaluate writing based on how well the writer has fulfilled the assigned task

prompt:writing task that states the topic and purpose of the piece of writing

range finding:a process in which a group of experts score and discuss student papers in order to come to consensus about how to apply the rubric; the scored papers are usually then used to make training sets

reader:also called a scorer, a person trained to apply the holistic scoring process to student writing

reader bias:personal factors, not included in the scoring criteria or the rubric, that may affect the score assigned to a student paper

rubric:scoring criteria at each score point of the scale

scoring guide:scoring criteria used to evaluate performance assessment and to assign score points to student work; sometimes called the rubric; sometimes includes exemplars of student work

surface feature errors:errors the student made in the area of conventions (except usage) that are not immediately apparent if the paper is read aloud; for example, errors in spelling, punctuation, indenting, and capitalization

tone:a writer's tone might be described as somber, tongue-in-cheek, formal, casual, ironic, melodramatic, etc.; the tone is a result of the writer's word choice and sentence structure

umbrella statement:similar to a thesis statement although less formal, an umbrella statement consists of one or two sentences that indicate what the writer intends to do in the rest of the paper. For example, asked to write about an invention that has been important in his/her life, a writer might begin by saying, "I can't write about just one invention since so many have affected my life." This would indicate to a reader that the writer intends to discuss several inventions.

verification set:training set in which the user is immediately informed of the correct score

voice:a writer's voice comes from the elements that contribute to the overall impression of the writer's personality as it comes through in one particular piece of writing; word choice, sentence structure, organizational strategy, details, insight, tone, and approach to the topic all work together to create a writer's voice

writing process:the typical steps that are used in the development of a piece of writing, including prewriting, drafting, revising, editing, and publishing

+
 Annotation Terminology
The following terms and phrases are used in the annotations and may need further explanation for teachers who are not familiar with them. In some cases, the terms may be used in a slightly different way than other sources. Reading the explanation for these terms will help further clarify the reason a paper received the score point it did, as well as provide a sense of the many elements scorers consider as they read a paper.

audience:This term refers to the intended reader or readers for whom a piece is written. A writer can show "audience awareness" in a variety of ways. Some of these include identifying characters and places as they appear in a narrative, giving readers clear time markers so they can easily follow a sequence of events, defining technical terms or jargon with which readers might be unfamiliar, tying thoughts or events together for a reader with insightful comments, and generally providing readers with sufficient information to make the writing clear. Audience awareness is also sometimes referred to as a writer "bringing the readers along."

best fit:This term is used most often in focused holistic scoring in which a paper may have characteristics of several different score points. For example, a paper may have some features that are at a 2 level while other features may seem to be at the 3 level. The paper would receive a score of 2 or 3 depending on the score point of the rubric that most accurately describes the attributes of the paper. A scorer would also consider whether the 2- and 3- level elements were in the high or low range of those score points. Ultimately, the paper would be given the "best fit" score. It had elements of both score points but, overall, was it more like a 2 or more like a 3?

borderline papers:Within each score point there will be a range of papers that represent the lower, middle, and upper levels of that score. Papers that represent the middle level are often referred to as "solid" examples. Papers representing the extreme lower or upper levels are sometimes called borderline papers because they fall between two score points. In other words, a 2/3 borderline paper has some 2-level characteristics and some 3-level characteristics. Ultimately, of course, a scorer must assign the paper a "high 2" or a "low 3." Scorers are cautioned, however, to remember that a "high 2" is still reported back to the student as a 2 and differs significantly from a score of "low 3" which will be reported back as a 3.

central idea:This might also be called the paper's overall theme or purpose. It is governed by the message the writer would like to see the reader take away from the piece of writing. The topic is not a central idea. The central idea is the message or idea about the topic.

clustering:Clustering is an organizational strategy in which related details are placed together in "clusters" or "clumps." Clustering is generally a weak organizational strategy because little progression of ideas is evident in the paper.

cohesiveness (see unity):A paper that gives the sense of completeness with no extra details and no details missing might be said to have cohesiveness. A paper that lacks cohesiveness may have an abrupt or no ending, an introduction that doesn't connect to the body of the paper, serious gaps in the story leaving a reader with unanswered questions, or a switch in topics mid-way to a completely different topic.

control:Score points in the SOL rubrics use the term "control" to describe the writer's ability to use a given feature of written language effectively.

composing domain:This domain of the Virginia SOL Writing test includes the focusing, structuring, and elaborating that a writer does to construct an effective message for a reader.

density of errors:The shorter a paper is, the more the density of errors in conventions affects a reader's overall impression of the paper. In other words, a paper that consists of 5 sentences with a total of 12 errors might be said to have a greater density of errors than a paper three pages long with a total of 12 errors.

elaboration:A paper is well elaborated if the details, examples, and illustrations focus the central idea in a purposeful way. Elaboration occurs at the sentence, paragraph, and paper levels.

fluency:This refers to the overall rhythm and flow of a paper and takes into consideration features such as word choice, sentence structure and variation, transitions, organizational strategy, and focus. A test of fluency might be reading the paper aloud and listening to how easily and smoothly it reads. Well-controlled fluency generally means there are few or no awkward moments where the reader stumbles.

mechanics:Mechanics is a feature in the usage/ mechanics domain that includes formatting, spelling, capitalization, and punctuation.

organization:Organization is the logic structure that facilitates presentation of the central idea. This characteristic takes into account the way a piece of writing is arranged and ordered.

precise vocabulary:Words or phrases are carefully placed and purposefully chosen by the writer to resonate as images in the reader's mind.

rhythm:Rhythm is the quality of fluency in a writing. A writer's rhythm can either be pleasant and unobtrusive, or it can be choppy and abrupt; a pleasingly rhythmic paper will include sentences which are varied in length and beginnings.

selected information:Specific, purposefully chosen information helps to propel the reader through a writing. A paper lacking information, giving too much information, or repeating ideas will affect the reader negatively.

sentence formation:Sentence formation is a feature in the usage/mechanics domain that includes standard word order, completeness, avoidance of sentence faults (run-ons, comma splices) and standard coordination.

sentence-to-sentence organization:Sometimes each sentence is tied to the sentence that comes before, but there is no overall organizational strategy. These papers might seem to have an internal organization at first glance, but on closer examination, the paper often ends up in a very different place than where it began. For example: "My favorite weather is sunny because I can play baseball. Baseball is a great warm weather outdoor sport. I'm a good baseball player. I play first base, which is the same position Joe Smith plays for the Rhinos. The Rhinos had a great season last year. This year my dad got us tickets to see 3 of the Rhinos home games. The home games are great because I can take my friends and eat hot dogs and popcorn." The initial point of "my favorite weather" has been lost. Do not mistake additional sentences as always being elaboration of the topic. Sentences that can be rearranged without affecting the message demonstrate a flaw in organization.

sentence variety:A paper with sentences of various lengths, constructions, and beginnings creates a pleasant, unobtrusive rhythm for readers.

surface feature errors:These are errors the student made in the area of conventions (except usage) that are not immediately apparent if the paper is read aloud; for example, errors in spelling, punctuation, indenting, and capitalization. (See Virginia Skills Lists for items addressed at each tested grade level.)

tone:A writer's tone might be described as somber, tongue-in-cheek, formal, casual, ironic, melodramatic, etc. The tone is a result of the writer's word choice and sentence structure. For example, a piece of writing using slang or colloquialisms might be considered casual. It could be an appropriate tone for a narrative, a personal letter, or an informal essay. Sometimes, students attempt a lofty, academic tone in their writing that generally produces stilted prose since the word choices and sentence constructions may be awkward or incorrect. For example, "One's choice of a college could well depend on where one's parents attended or that of which one's own siblings," An uneven tone occurs when a student mixes casual and formal terminology. For example, "I threw myself forward down the precipice. Every nerve was shrieking, "Stop, stop. Dang it all, Stop." A flat tone is characterized by ordinary word choices, repetitive and often simple sentence structure, and an overall lack of vitality in the writer's voice. It is also sometimes called "generic" since a flat tone gives no indication of a writer's unique personality. A tone closest to the student's own generally produces the most genuine type of writing.

unity:Unity is completeness in the elaboration of a central idea in an organized way. A unified paper has no extraneous details and no details missing.

usage/mechanics:This domain reflects the writer's ability to compose competent, appropriately mature sentences (observed independently of purpose and style), to construct these sentences grammatically at the word level, and to spell, capitalize, and punctuate writing mechanically at its surface level.

usage:Usage is a feature of the usage/mechanics domain that includes inflections, agreement, conventions, and word meaning.

voice:A writer's voice comes from the elements that contribute to the overall impression of the writer's presence as it comes through in one particular piece of writing. Word choice, sentence structure, organizational strategy, details, insight, tone, and approach to the topic all work together to create a writer's voice. A writer's voice can and often does change depending on the piece's audience and purpose, although more experienced writers tend to have a similar voice in all of their writing.

written expression domain:This domain of the Virginia SOL Writing test includes those features that show the writer can purposefully shape and control language to affect readers. The features of written expression are vivid and precise vocabulary, selected information, tone, voice, and sentence variety.

Rubrics
+
    Score Point 2
The writer demonstrates inconsistent control of several features, indicating significant weakness in the composing domain. At this score point, ideas often compete, or no one idea emerges as central. Even if a single idea dominates, the paper may lack focus because of little or no elaboration. The paper may be a list of general, underdeveloped statements or the skeleton of a narrative. In the case of persuasive writing, it may consist of a few unelaborated reasons accompanied by inappropriate attempts (begging, pleading, negotiating) to persuade. Typically, the writer extends ideas with a few brief details and moves on, though chunks of irrelevant material may appear as well. Often, no more than a hint of organization is apparent. Even though an opening and closing may be present, the lack of a logically elaborated central idea prevents unity from emerging.
+
    Score Point 3
The writer demonstrates reasonable, but not consistent, control of the composing domain's features; the writer may control some features more than others. The clearly focused central idea is purposefully elaborated with key examples, illustrations, reasons, events, or details. Occasionally, some thinness or unevenness in elaboration may occur. In all types of writing, an organizational plan is apparent. Any minor organizational lapses that occur do not significantly detract from the piece. Although there may be occasional lapses in coherence or cohesiveness, unity is evidenced by the fact that few, if any, digressions or shifts in point of view occur. Transitions are, on the whole, appropriate. The opening and closing show some skill, but not the sophistication of a 4 performance.
+
    Score Point 4
The writer demonstrates consistent, though not necessarily perfect, control of the composing domain's features. The piece is generally unified in that all of the parts contribute to the creation of a dominant impression or idea. The sharply focused central idea is fully, but not exhaustively, elaborated with key examples, illustrations, reasons, events, or details. In all successful responses, layers of elaboration are present. Surface signals, like transitions, logically connect their respective statements into the whole of the paper. In all types of writing, a strong organizational plan is apparent. Any minor organizational lapses that occur do not significantly detract from the presentation. The writing provides evidence of unity by exhibiting a consistent point of view (e.g., not switching from "I" to "you"), a lack of digressions, appropriate transitions both within paragraphs and across the entire piece, the presence of careful logic, and a strong lead and closure.