We were one of the trial schools for NAPLAN online this year. NAPLAN is the national literacy and numeracy test in Australian schools.
Here is my motivational email to our Year 7 and 9 students and parents on the day prior to the test.
Subject: Exciting Testing Times Tomorrow.
On Mon, 14 May 2018, 5:58 pm:
Hamish will be one of the first students in Australia to complete the NAPLAN tests online.
Testing begins tomorrow.
Our College is one of a small number of schools in Australia who are trailing online NAPLAN assessment. Hamish will be part of history.
In 50 years’ time when online testing is common Hamish will be able to say “I was one of the first students in Australia to do that.”
Hamish’s experience of online testing will be more successful if the test is undertaken with a fully charged netbook and headphones. These are required for Tomorrow, Wednesday and Thursday.
We wish Hamish all the best.
Why did we volunteer for this?
We started a 1:1 Laptop program at our school in 2007. Eleven years down the track and exams and standardized tests are still done using paper, pencils, and biros. Education change moves very slowly despite what the hype sometimes suggests.
We have tried to be at the leading edge of education technology. Some of our staff would say the bleeding edge. We have insisted our curriculum and assessment are online for the last five years. Eighty percent of our staff now say they have students using their computers every lesson.
Our staff and students are obviously comfortable working in an online environment so it was a logical step for us volunteer to trial NAPLAN online. We want to help in the development of computer-based assessment as we see it as a logical step the evolution of the way we learn and communicate. It frustrates us that we have to effectively “down tools” in the senior years and get our students back on the paper, biros, and pencils.
My often used analogy is that we teach our students how to be quite proficient car drivers but then have to teach them how to ride a horse for the senior exams even though a car does the job better.
Despite our experience of online environments, NAPLAN online was not without its challenges.
Challenge 1: The level of security is over the top.
Anyone would think gaining access to NAPLAN online would also give you access to the nuclear launch codes such was the level of security. We were issued with special unique CODE cards which had to be locked in a secure room and only brought out of the days we were testing. Each login generated a unique code which had to be read from the card. I sometimes felt like we were in a James Bond film when setting up a test session.
When I sent out a tweet during a test with a photo of students doing NAPLAN online the NAPLAN police were on the phone to the school within 30 minutes telling me to take the Tweet down.
Challenge 2: Training staff on how to administer the test is time-consuming but necessary.
Each test session involved generating a unique teacher code and a set of student logins. Teachers administering the tests needed to be trained in how to start a test session, get students logged into a test session, then commence the test. They also needed to know how to pause a test if a student needed to go to the toilet or had a technical difficulty. While the system is relatively simple to operate it is not foolproof as we found out.
In a large high school when you are trying to test 200+ students at a time things can go awry. For example, a teacher who had done the training was absent on the day of the test. An untrained teacher was allocated to cover the lesson. They turned up a little late, became a little flustered, and wrote the teacher login on the board rather than the student login. The students then logged into the teacher system and crash; the session was ruined. A security breach was noticed so the test session needs to be recreated by a central admin who has access to the “nuclear missile” key card. This created a fifteen-minute delay to set up another session. Students start to get a little antsy sitting around doing nothing for 15 minutes.
Human error can destroy any system.
Pen and paper tests seemed like a good option at this point.
Challenge 3: Getting 180+ Year 7 students logged into the same website in a short space of time is not a trivial undertaking.
We did get better at it but here are the steps involved.
- Every Year 7 and Year 9 student in the school needs a copy of a special “Locked Down Browser” installed on their computer. We are fortunate that we run a fairly standard operating environment in our school and were able to push this browser out to all students in the weeks leading up to NAPLAN. We did a trial run using the locked down browsers.
- Students log in to the locked down browser and type in a unique session code. This session code was generated by the teacher supervising the session.
- Students then enter a unique username and password that is on a piece of paper. Yes, NAPLAN online is not paperless.
- At this point, NAPLAN online is like a massive “Kahoot” The number of logged in users appears on the supervisor’s screen. The supervisor can see who is logged in an who is not. When everyone is logged in the supervisor presses the “Start Test” button and everyone starts the test.
It actually worked pretty well. It is a kind of magic to stand behind 80 students who all have the “Waiting for Test to Start” screen visible. The teacher presses the go button and all the screens automatically have the test on them.
Our major issues were when students didn’t have a network or wireless connection before getting into the locked down browser. Once in the locked down browser, you are, well, as the name suggests, locked down so you can’t change or check your wireless settings. On day two we made all students get into “The Age” website so we could see they had a solid internet connection before we went into the locked down browser.
We had one large space with 80+ students in it. We were able to get over 75 of them going with no technical hang-ups but having tech support on hand is essential. Everyone got going after some reboots and a little technical help.
Challenge 4: NAPLAN Online is nowhere near as engaging as other things students can access online.
We did the writing task first.
Anyone who thinks NAPLAN is stressful for students should come and watch our students take it on.
For most of our students, they see NAPLAN as a “no stakes” test. Many of our students are well and truly on the left of the arousal curve where NAPLAN is concerned. We certainly didn’t have any on the right.
The anti NAPLAN media hype seemed to have flowed through to our Year 9 group.
One lad, after spending a massive 14 minutes of the 42 minutes allotted time on his essay, decided “I don’t want to do anymore because this is only for the government anyway.” I suggested he may like to try a bit harder for “personal challenge”,“individual satisfaction” or to “see how the first 10 Years of his formal education had him tracking compared to every other student in Australia of the same age”. He was having none of that.
Now back in the pen and paper days, this young man’s next 30 minutes would have consisted of staring out the window, or at the back of the head of the student in front. This boredom may have tempted him to read over the three short paragraphs he had written in HB pencil and add some more. Maybe he would fix up a spelling mistake or two.
Not in the online version. He’d had enough. He’s clicked the “I’m done” button. This takes him out of the locked-down browser and opens up the world of games, youtube and interesting stuff on the internet to him. I look back on the 15-Year-Old me and pose the same question, “Spend some more time on a “no stakes”, persuasive writing task or have some free time on the internet?” Yes, we didn’t have the internet when I was 15 but if we did have the internet, I would have probably made that choice of cat videos on YouTube in preference to writing an essay.
At the 30 minute mark of the 42 minute writing task my face went pale and wave of depression flowed over me as I walked from room to room seeing many of our Year 9’s playing games or watching YouTube videos rather than writing the persuasive essay that would be some indicator of how well we had taught them to write. I’m not confident our writing results will be great this year.
We didn’t see that one coming. We’ll do better next year.
Day 2 of NAPLAN online saw a change in procedure. All supervising teachers were given strict instructions to not let students goof off on their computers if they finished early.
Despite the challenges our overall experience of NAPLAN online was a positive one.
The adaptive nature of the test in fascinating. When the supervising teacher hit the go button on the maths test and the 80 screens miraculously came to life the first thing I noticed was that not all students had the same question. The test appears to be drawing from a large bank of possible questions so students next to each other were not getting the same ones.
One of the previous downsides of pen and paper NAPLAN was having the math’s paper divided into a calculator and non-calculator test. In our school, we’d always have to hunt up 20 or so spare calculators for students who didn’t have them. You don’t have to do any of this with NAPLAN online. There is a calculator on the screen for some questions and not for others. If students need a ruler to measure something in the question there was a ruler on the screen.
In the Maths test, each question had a “Read the question to me” button. Students had headphones and I noticed many of our lower literacy students using this feature for each question. Anyone who has taught maths knows that some students, particularly those from non-English speaking backgrounds, can sometimes be fine with the maths part of the question but struggle with translating English into Maths. Having the question read to them should make a difference.
After 15 questions, students were asked to check their answers carefully as they would not be allowed to go back and change them when they moved to question 16. This was where the test “adapted”. If a student did very well on the first 15 questions the next set of 15 would be harder. If the student struggled with the first 15 the next 15 would be easier. You can’t do that on a pen and paper test.
It was noticeable that for our stronger students they were running out of time near the end. They were getting harder and more complex questions that were taking longer to solve.
Students said they enjoyed that part of the challenge. We had warned them that the questions would get harder if they were doing well and that was OK. I used the high jump bar analogy with them. In the high jump, they raise the bar every time you jump over it. They keep raising the bar until you can’t jump over it. Even the best high jumper in the world fails to jump the bar in every high jump event.
How often do we raise the bar on our top students to the point where they can’t jump over it? How well do they cope when we do this?
The language conventions test also made use of headphones. The spelling words were read to students via the headphones and some questions could be read aloud.
While the training of staff, installing lock down browsers, complex security logins and some technical glitches were annoying, our overall impression of NAPLAN online is that it is better than the pen and paper version. The simple fact that we didn’t have to put out and pick up over 1000 test papers in the four days was a significant time saver not to mention a bonus for our environment.
We did NAPLAN four weeks ago and still don’t have the results. This is ridiculous. Our students are used to Moodle quizzes where we get the results instantly. I’m sure eventually the turnaround for NAPLAN results will be better with the online version. The current five-month turnaround for results is not timely or helpful.
The fact that students could type their essays was a huge plus for our students.
One day Year 12 exams will be like this. Online assessment in Year 12 is taking such a long time to eventuate.
We were glad to be at the bleeding edge in 2018 when NAPLAN was done online for the first time.
We were part of history.
We don’t want to go back to pen and paper.