ICT · NAPLAN · Teaching

How to improve student writing by 68% in one lesson or Gaming the NAPLAN writing algorithm

Improving students writing has been a priority for our school for a number years.

Our success has been mixed.

Recently I had the pleasure of working with Sarah, one of our English Coordinators with a Year 7 group where we experimented with an online computer scored essay program to improve our students’ persuasive writing.

We were very successful; a 68% improvement in one lesson. As a science teacher my ability to assist her was limited but when I crunched the data I came to the conclusion that English teaching is easy. We were very good.

Sarah assures me this instant improvement in English is not actually the norm and our short term gains may not be retained in the future.

Here is what we did.

Our school has used eWrite, a computer scored essay marking program, for three years.

eWrite is an online writing assessment for students in Years 4–10 (ages 9–15).

eWrite features advanced analysis techniques that provide instant diagnostic feedback on students’ skills across a range of writing genres including:

  • Narrative
  • Description
  • Report
  • Persuasive.

Students interact directly with the online assessment, typing into the secure system. eWrite automatically marks the students’ writing and instantly produces detailed individual and group reports, pinpointing students’ writing strengths and weaknesses.” https://www.acer.org/au/ewrite

Each year students in Australia sit the NAPLAN tests in levels 3, 5, 7 and 9. This national test of literacy and numeracy is now conducted online in half the schools in the country. The days of pen and paper assessment in Australia are numbered. All students at our school have a laptop computer and much of their writing is done electronically.

In the lead up to NAPLAN our year 7’s, 8’s and 9’s had a practice run of writing a persuasive essay under NAPLAN like conditions. The topic was to persuade people to ride their bikes to school and work.

Students were given no support or scaffolding the first time they did the task; just a 25 minute “cold write”. We emailed students their results in this format.

Dear Student Name of Year 7

This is your e-write results from your recent essay.

Score Raw score out out of a possible 28 marks 13
Band Broad band of achievement. 5
Orientation and Engagement (max 4) The writer’s sense of a reader: the capacity to orient and engage the reader through the provision of information and selection of language 2
Text structure (max 3) The presence and development of the structural elements of the required text type 2
Ideas (max 3) The quantity, quality, relevance and elaboration of ideas 1
Vocabulary (max 3) The range, precision and effectiveness of word choices 1
Paragraph (max 3) The use of paragraphs to organise and develop ideas to assist the reader to navigate the text 1
Sentences (max 4) The quality, accuracy and effectiveness of sentences 2
Sentence punctuation (max 2) The use of capital letters, full stops, question marks and exclamation marks to mark the beginning and end of sentences.  1
Punctuation within sentences (max 2) The use of punctuation within sentences to assist the reader’s understanding 1
Spelling (max 4) The quality of words used and the accuracy of spelling 2

Here is what you wrote.

People should walk and ride more

I strongly agree with this stament. why? you ask well here are my reasons why.

firstly, so many kids are sitting on their computers and eletroics playing games and eating junk food. 78% of childern are never outside, well maybe for 1 hour, but still!!! Sooo many Kids also eat junk food and thats not good for you. I will agree that once in a while it is ok, but not every second and every hour. 63% of Kids aged 6 to 14 are over wight. This is because they are also sitting on their buts texting and plaing games and never get out and do anyhting.

secondly, so many Kids get driven to school by partens. I know that some people live to far and have to drive buts no excsie. Drop them off half and let them walk. Now for the kids that live close. WALK!! or even RIDE!! Your partens pay for petral and when they drive you say 5 mins down road you are wasting their money. Now if you don’t walk or ride and make your partens drive, you could NOT get the new game thats coming out, that you really want, you know why, because you are making your partens drive you 5 MINS DOWN THE ROAD!!!! YOU ARE WASTING THEIR MONEY!! So kids please just walk or even ride.

Thridly, health is very important. Most kids are always going on their phones. Thats not good for your health. Eating junk food is not good for your health either. Exerise is always important, to health. If you are feeling over wight try and walk or even ride. Once again this ties in witth your health.

So with piece try not to play on yoiur games, stop getting your partens to drive you to school, and try and walk or even ride. Thank you for listening and try and walk or ride.

Your English teacher may give you the opportunity to redraft and resubmit this essay to improve it.

When you do NAPLAN next week work on your weaknesses as the scoring system is the same. Good luck and write well.

I suggested to Sarah that we could let students use this essay as a first draft. We’d let them redraft their essay and try to improve their score for particular criteria. I was also interested to see if we could “game” the marking algorithm. We wanted students to be more aware of how their essays are scored and if minor adjustments could lead to major improvements.

NAPLAN is apparently not computer scored but looking at how eWrite scores essays it is not difficult to see how machine scoring will be used in the future. It is important that our English teachers become familiar with the scoring system and how it works. The eWrite scoring system and the scoring rubric used for NAPLAN writing are almost identical.

To start our “Second Draft Lesson” Sarah guided the students through an essay plan on the board. She suggested that students should aim to produce a catchy introductory paragraph, three body paragraphs covering three different reasons why riding your bike was a good idea and then a conclusion.

She then asked the students to open their emailed results which included their 1st draft of the essay. We asked them to copy the essay into Microsoft Word.

Students asked about what the “bands” mean in their results. Even eWrite seems a little vague but I explained to the students that their aim should be to improve two bands. Think trend not level.

Sarah glared at me and said “Two bands is more than 2 Years of schooling. That’s pretty tough.” My response was “I think we can do it. I’ll give a prize to anyone who can improve their essay two bands.” Nothing like a bit of an incentive and extrinsic motivation (bribery).

Our students are so used to writing online with “red squiggly” highlighting every spelling mistake, they found proofreading in eWrite a challenge. I watched one girl paste her essay from the email into Word. Within seconds, 29 spelling mistakes had a red squiggly under them. She counted them and I suggested she needs to carefully edit when there is no spell checker to help. She quickly fixed her spelling mistakes as well as a number of blue highlighted grammatical errors, particularly with her use of apostrophes.

After about ten minutes of engaged editing and redrafting, we stopped students and talked about how using a range of sentence lengths makes your writing more interesting to read. We suggested that we needed to use a range of sentence lengths. I suggested students add some short sentences to their essay. “Cycling is fun.” Then a longer sentence explaining why. Off they went again for five minute varying their sentence lengths.

Our next explicit teaching was to target the criteria assessing “Punctuation within sentences.” Sarah took over at this point as punctuation is not my strong suit. “Do we have a comma in our sentences?” I suggested that the students should use a quote so they could demonstrate they know how to use “talking marks”. Apparently we don’t call them “talking marks”.

Sarah explained to her class the mechanics of using quotation marks.

I wrote on the board; Lance Armstrong said, “Cycling sometimes feels like you’re flying.” I suggested we put this quote in our essay somewhere to show that we know how to use quotation marks.

Sarah produced a nifty writing prompts sheet with hints on punctuation, vocabulary and mechanics of writing. We encouraged students to add some vivacious vocabulary to their pieces.

VCOP.PNG

“Can we think of a better adjective than “nice” here?”

We suggested, promoted and encouraged students to improve what they had written.

“Can you vary your topic sentences to make the reader want to keep reading.”

“Which persuasive techniques have you used here? What about a rhetorical question here?”

After 30 minute of reworking their pieces Sarah asked the students to paste what they had written into eWride and re-submit the essay.

There was an air of expectancy in the room as Sarah projected the class results on the board.

In what would turn English teachers world wide, green with envy, we watched as Sarah graded 25 essays against 9 criteria in less than 15 seconds. Yes, 225 separate assessments of written work before the students eyes in seconds.

“Here are your new results.”

There was genuine enthusiasm in the room. A young lady in the front proudly announced “I went up two bands!” The boy next to her said, “I went up three bands.”

The improvement in student scores was remarkable. Where we have had targeted explicit instruction on particular criteria the improvement was spectacular. The low hanging fruit, the easy wins, like paragraphing, within sentence punctuation, vivacious vocabulary and sentences are easy to improve with explicit instruction. It is not too difficult to game this aspect the scoring algorithm.

imptable

Sarah closed the lesson by trying to get students to reflect on the purpose of what we have just done.

“Why did we do that activity today?” she asked the class.

“So we get better results on NAPLAN.” said one girl.

On one level that was a pleasing response and Sarah went with it. Sarah reinforced the idea that by watching our spelling, varying our sentences, using a range of punctuation and some vivacious vocab we will “do better on NAPLAN”.

But was this our purpose?

I couldn’t help myself and had to interject.

“It is not just about NAPLAN. We want to learn to write better. We want to be able to write an exceptional persuasive piece on why we should ride bikes. We want to persuade people who read our essay. We want to be able to communicate effectively in writing so people can understand our point of view. The fact that we may “go better on the NAPLAN test” is not the main game here folks. The main game is to learn how to write well.”

The critics of NAPLAN will suggest that “teaching to the test” and trying to “game” the marking algorithm is a drawback of the assessment. If doing well on the test means you have to write better, I’m happy for us to teach to it.

Here are some examples of how eWrite grades essays. .

The marking scheme.

Orientation and Engagement (max 4) The writer’s sense of a reader: the capacity to orient and engage the reader through the provision of information and selection of language
Text structure (max 3) The presence and development of the structural elements of the required text type
Ideas (max 3) The quantity, quality, relevance and elaboration of ideas
Vocabulary (max 3) The range, precision and effectiveness of word choices
Paragraph (max 3) The use of paragraphs to organise and develop ideas to assist the reader to navigate the text
Sentences (max 4) The quality, accuracy and effectiveness of sentences
Sentence punctuation (max 2) The use of capital letters, full stops, question marks and exclamation marks to mark the beginning and end of sentences
Punctuation within sentences (max 2) The use of punctuation within sentences to assist the reader’s understanding
Spelling (max 4) The quality of words used and the accuracy of spelling

Student Sample 1

1st Draft.

There is a million reasons why we should ride and walk more. This book will tell use those reasons.

Firstly, it makes us fit, alot of Australians are just sitting around on the couch watching T.V and that needs to stop. You need some exersize and here is were you need to start walking/riding. It will make Australia alot more healthy when we are walking and riding.

Secondly, it will give us kids a longer life spane knowing that we are fit and healthy. most people are batling diabeties because we are not exersicing enough it will be devistating that we will not live long enough to see grand children or even children.

Finally, we are not socilising enough it is important to socilise too. We might have school to socilise but we won’t have other people to meet. Not every-one likes to go outside and here is a chance to.

So to sum up all I have to say is everything is important but most important is the outdoors.

Computer score for first draft.

Band OE TS ID VOC PARA SENT SPUNC PINS SP
3 1 1 1 1 1 1 1 0 2

2nd Draft.

There are a million reasons why we should ride and walk more. This book will tell use those reasons.

Firstly, it makes us fit, a lot of Australians are just sitting around on the couch (like a couch potato) watching T.V and that needs to stop. You need some exercise and here is where you need to start walking/riding. It will make Australia a lot healthier when we are walking and riding. Australia should not be a dying hazard from diabetes. 1.3 million Australians are dying from diabetes this is why we need to go outside.

Secondly, it is also good for the environment. It is already devastating that are oceans are getting intoxicated by our waste we don’t need the poor fish dying from our fuel. The plants air is get poisoned from the smell of gas and if the trees are poisoned our air is terrible that we won’t be able to breath any more, all of humanity will die. Lots of places are already suffering the consequences of bad air it should not be the same in Australia.

Finally, we are not socialising enough it is important to socialise too. Many children are dying because of bulling, and that they don’t have friends to talk to. It is bad enough that people are killing themselves because of it. We might have school to socialise but we won’t have other people to meet outside of school. Not every-one likes to go outside and here is a chance to.

So to sum up all I have to say is everything is important but most important is the outdoors and friends. Lace Armstrong said, “Riding your bike feels like you’re flying” wouldn’t you like to feel like that to?

Computer score for second draft.

Band OE TS ID VOC PARA SENT SPUNC PINS SP
6 2 2 2 1 1 2 2 1 3

Student Sample 2

1st Draft.

People should walk and ride more because its better for your body and is good exercise instend of sitting down wasting time doing nothing.

Firstly people need to walk more to get there body moving and breathing in fresh air and exploring. Walking is a slow and easy exercise and is possible for anyone and do and its free you dont need to buy a bike to ride just put on some sneakers and good walking clothes! also if your car breaks down you have to walk somewhere to be able to get signal or somthing else like another person etc. if you can walk it will be the start of a healthy fit life.

Secondly people should ride more because its good for your legs to get moving while riding your bike also its good because you can go riding in the bush or anywhere your allowed and it great to experiance things in the bush.

Computer score for first draft.

Band OE TS ID VOC PARA SENT SPUNC PINS SP
3 1 1 1 1 1 0 0 0 2

2nd Draft

People should walk and ride more because it’s better for your body and is good exercise intend of sitting down wasting time doing nothing.

Firstly, people need to walk more to get their body moving and breathing in fresh air and exploring. However, walking is a slow and easy exercise and is possible for anyone and do and its free you don’t need to buy a bike to ride just put on some sneakers and good walking clothes! Also if your car breaks down you have to walk somewhere to be able to get signal or something else like another person. If you can walk it will be the start of a healthy fit life.

In addition, people should ride more because it’s good for your legs to get moving while riding your bike also its good because you can go mountain biking in the bush or anywhere your allowed and it great to experience things in the bush. Also you can meet new people, ride with friends or even go riding and stop and have a coffee. Riding or walking is better for the environment you don’t need fuel unless its electric but if you have a pellet powered bike it doesn’t need fuel and is much better for the environment. Besides bikes don’t cause traffic jams.

In Conclusion, riding a bike or walking is great exercise and that’s why I think that you should walk or ride instead of sitting down and driving.

Here’s a quote Lance Armstrong said, “riding a bike feels like your flying”. Wouldn’t you like to experience that?!

Computer score for second draft.

Band

OE

TS

ID

VOC

PARA

SENT

SPUNC

PINS

SP

6

3

2

2

1

2

2

1

1

3

Ten students achieved the goal of improving by two or more bands. Not bad for one lesson.

 

 

One thought on “How to improve student writing by 68% in one lesson or Gaming the NAPLAN writing algorithm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s