Board of Studies Marking. (1 Viewer)

[xeuyrawp]

This time next week, the primary round of Board of Studies Ancient History marking will be completed.

I finished my stint on Friday, although the debriefing sessions take place next week, which I have to attend.

Seeing as though I can now discuss the marking with students, I'll open this thread as an opportunity for people to ask questions about the whole process. I think a problem is that the BOS marking process is nowhere near as transparent as it should be: There is an issue with 'leaking of information' as it may be, but I'm not exaggerating when I say that firstly, markers are constantly watched and secured by security guards, and secondly, there's nothing secret that I could really tell you about the process.

I thought it would be unethical coming on bos whilst being a marker (the possibility of me chatting with someone whose paper I might be holding the next day seemed a conflict of interest), although this became a moot point when I saw that there was literally either a guard escorting all the junior markers around, or a supervisor showing them what's happening.

None of the observation markers got to come near a paper without the senior marker sitting there.

The new batch of markers are mostly newer uni students - with me knowing 2 of them (my Egyptology tutor from sem 1, and another tutor), and recognising another 4 by face. This was only the Egypt section, so you can appreciate the huge logistics of the thing. As well as the newer markers, I saw Boyo Ockinga and, of course, Gae Callender around. Gae's very serious about the whole process, as well as being stressed about leaving for Egypt to go on a dig in 8 days.

The marking centre that I was at was at Homebush, with all HSC Ancient History papers going there. The actual markers go in to the hall, walk to their table, and sit down. A person comes with a stack of papers and places it on their right side, and the markers mark it, attach their marks on a separate piece of paper, and pass it to their left.

If there's an issue with the paper, they put it in a separate stack - things like students answering the wrong questions, attempted bribery, putting down 'fuck ancient history' a hundred times, etc. These are taken and marked by a special, separate group.

The whole process is very controlled, if you want to get up to pee or get a drink, you have to close the tops of your stacks (they're in those cardboard boxes that companies get printed material delivered in) and tell the guard where you're going.

As an observer, I walked around in a little group. We were explained the whole process, told what to and what not to do, and individually sat with markers who went through their thoughts. The markers were very nice, although I will dare say that Gae's reforms of the Egypt section markers is understandable - I think I pointed out factual errors (made up quotations and other errors) many times.

It was a bit awkward pointing out where a teacher has clearly missed an error. My tutor friend actually had to move markers once, as she felt really uncomfortable with a marker, but that's life, I guess.

It's a great experience with great pay, and I'm really looking forward to properly marking next year. It's a really unusual experience, I must say, with the whole security thing. You feel like you get no privacy: Even in your lunch breaks, you're being watched by guards - I got in trouble once for leaving the site without signing out. I almost felt like I got crash-tackled by a guard when I walked onto the site without ID, too!

The prospect of having to fail some students is a sad one, however, and one I'm not looking forward to...

 

[sikeveo]

Rob, how did you get involved in this?

 

[Lazarus]

I'll help start the ball rolling...

Any details on the 'statistical feedback' that (senior?) markers or the SOM receive progressively throughout the process? Know what they're given?

Is any reference made to marking criteria other than those published with the official exam reports?

Did you ever encounter any evidence of markers feeling pressured by external forces (e.g. media hype, community expectations)?

 

[xeuyrawp]

Rob, how did you get involved in this?

It's a reasonably long story, but the general idea is that Gae Callender, a big figure in both Egyptology in general and in the BOS came to Macquarie. I met her at the Ancient History HSC study day last year, and we talked then. Anyway, this year, I talked to her about her HSC marking, to which she said that they were having a problem with markers not knowing their shit.

I'd heard that my tutor, Camilla, had been recruited, and I basically asked Gae about the process. She said that a lot of new recruits come from Macquarie (the only uni in NSW that thoroughly teaches Egyptology). Anyway, I basically shamelessly promoted myself by a bunch of intellectual questions during her 'question time' at a conference, by talking to her at a number of papers she was presenting, and by pretending I wrote an article on her views of a Pharaoh.

After her looking at a number of my essays and articles, she agreed to let me participate observation marking of the second New Kingdom period, personalities, and sociey. Note that I did these options in the HSC and did very well at them (as I told her).

I'll help start the ball rolling...

Any details on the 'statistical feedback' that (senior?) markers or the SOM receive progressively throughout the process? Know what they're given?

Ah, this is a very touchy areas for the markers as a group. I think the time I asked this, I was given the benefit of the doubt of being a noob.

Think about it: I was part of a group of people who: Marked only one specific time period within a paper that tends to attract lower quality responses from certain areas. I don't know whether I can be blunt, but some areas are more popular in less well-off schools...

The senior markers and supervisors have to balance between making the marks standardised, and not overstandardising (if that makes sense). Technically, they can't alter marks - but that's what we're doing by setting criteria.

The 'society' sections of the paper statistically have a higher mark rate (at about 18.5/25) than the personality (the most poorly answered section, with marks at about 16/25). Senior markers do not mark more than one section a year; it is the supervisors' job to monitor how the marks are going holistically. And then, it is the marker centre supervisor to monitor how the marks are going even more holistically. (You might be noticing a trend that there are a lot of levels here...)

I have absolutely no idea what statistical feedback is given - although I would say that only occasional hard numbers are sent further up the hierachy. I think this happens because 1. all sections and options are marked simultaneously, rather than in order, so it's not until the end of the process that all marks are known across various answers,
2. although it's officially not the case, schools' papers lump together and are marked at once (you'd get lumps of like marks)
3. every section and option is at least double marked,
and 4. there are so many teams working together; I was part of the second NKE team, which has two personalities (Ramses, Akhenaten), one period (Amenhotep - Ramses), and a society (Ramesside). Each of those areas has dedicated markers - eg I could not mark Akhenaten and Ramses II in the same year.

However, I assume that the raw marks are collected. Maybe they're sent up the ladder, who knows. What I do know is that before your day of marking, you have a discussion with the supervisor who would briefly talk about some issues. Every week, the actual marker teams have a discussion - with both NKE teams talking about what they found with the papers. Like I said, there's also a debriefing that occures at the end of the primary marking round. This debriefing involves the entire marker group (all the various nations), with you filling out a huge survey with your comments on the standard of papers and advice to go in the 'notes from the marking centre' hand-out. Which, by the way, is going to be much bigger next year.

I don't suppose you'd be shocked that even though BOS goes to such great lengths of ensuring proper marking, there is no guarantee that the mark that is given to a student will not undergo severe editing before being sent out as part of their raw mark. Although, officially, no 'alligning' per se goes on at the centre, a paper just needs to be re-marked twice to be realligned.

It's important to note that every team (let alone every subject) has a different set method. Whilst I'm sure every subject is standardised in its marking practices, markers for Ancient History have a very different job to those marking Extension History (and, I would speculate, Modern History). I would even say that those marking Roman history have a different job to Egyptian history. What we discuss in the meetings is relevant only to our sections. Furthermore, our marking sheet is specific to each option.

Is any reference made to marking criteria other than those published with the official exam reports?

Yes and no. Something I don't believe I'm allowed to discuss (the only thing, I think) is what a marker actually writes on, in terms of 'the sheet of paper'. Suffice to say it's one-sided A4 with boxes - possibly something that you'd get in a humanities subject at uni.

The biggest piece of criteria, though, is the collegiality of the process. Next year, I will be a junior marker, which means that not only will every single paper that I mark be remarked (further than it usually would; ie, at least twice after me), but I can be directly scrutinised by a senior marker - as in, one can literally ask me 'please explain'.

This leads me to believe that in this process, the senior marker who gets my marks and brief comments is actually the third last person to look at the paper, with the other two markers not seeing my comments at all.

If this makes sense: if you're a noob, you're never the last person to see the paper.

If you're referring to impression marking and the like - again, that's a risky area. It's understandable that bad spelling ticks off markers. Then again, I have terrible spelling myself, so I couldn't complain. I can tell you, though, that a marker tries to objectify their marking. There's always the risk that a strange mark will attract attention from more senior markers. 'I didn't like his handwriting' as a reason will probably get you a warning or fired on the spot.

All main criteria for Ancient are set before the exam in the syllabus. The marking sheet tries to break it down into bite-sized-chunks, but I doubt anyone would say 'I didn't think they'd put that on there' when looking at the sheet.

Did you ever encounter any evidence of markers feeling pressured by external forces (e.g. media hype, community expectations)?

No. I think that any pressure is put on the higher body of the Ancient History markers who collate all the marks and figure out if there's a big enough oddity to have things re-done. That pressure is probably applied by higher up in BOS.

There's probably subtle pressure put on the marking body to establish a bell-curve, but I think this pressure is, rightly, extended to those who set the paper in the first place.

Sorry about any spelling mistakes, bad grammar, or repeating sentences, I've been having exams lately and lack of sleep kills me

 

[smurfygirl]

I don't know where you thought you were, but you weren't at HSC Ancient marking. What you are describing did not happen. You're giving students, who clearly need direction, information that is not accurate.

We do not pass papers to our left, in fact we don't pass them anywhere. The allocation of papers is controlled by the clerical staff.

All markers either currently teach HSC Ancient History, or have in the past few years ( some are on maternity, long service etc). There are no students of any variety at marking.

Non serious attempts, wrong questions etc are not marked by a separate groups. They are given to the senior marker of that section and they deal with it. It doesn't leave the group of papers it was in.

You don't have to tell the guard where you're going - that's ridiulous - it's not primary school. You can walk anywhere around homebush and are not watched during your breaks, nor do you 'sign out'. The only thing you sign is your pay details.