Correspondance between HSK score and actual correct answers

[patphilly]

Hi all,

I am preparing for the HSK 3 and noticed something a little strange. Apparently each section is given a score out of 100 regardless of the number of questions in the exam.

- the 40 questions of the listening part will convert into a score out of 100

- the 30 questions of the reading part will convert into a score out of 100

- the 10 questions of the writing part will convert into a score out of 100

At first I thought it would be easy to estimate my actual score from the mock tests:

- for example if i get 30/40 correct in listening, that's 75% of good answers, so i should get a score of 75 for that section

- and if i get 15/30 for the reading part, that's 50% of good answers, so i should get a score of 50 for that section

etc.

BUT it turns out is it not like that! And that the scores out of 100 are actually RELATIVE to an average. A score of 50 means that we are doing like the AVERAGE of test takers for this section. (as they say, the score at made so that 50 correcponds to the average, with a standard deviation of 20).

It is a bit difficult to estimate how well i would do in the actual exam. Indeed, if average people in the HSK usually do very well (let's say 30/40 in the listening part on average), then 30 good answers out of 40 would only give me a score of 50/100.

Or if people are usually struggling a lot in reading part (for example if people typically do only 10/30 on average), then my 15 correct answers would give me a much better score than 50/100.

All in all, it's kinda difficult to see the correspondance between number of correct answers and the actual scores.

Does anybody have an idea? Do the HSK authorities even disclose the raw numbers for a given test? (like, on average, people do such and such number of correct answers, or such and such score of 60/100 would require X correct answers).

[yialanliu]

That was the scoring method for the old HSK (50 with stddev of 15). However, I don't think the new hsk released how they score it. The manual doesn't have how it is derived, just tells you the max and min required to pass.

[patphilly]

OK I see, maybe this is not the same scoring system for the new HSK.

However, is there anything to suggest that they score things differently? How can we know if the scoring is simply proportional to the number of correct answers? (i.e. is it sure that 75% of correct answers give a score or 75/100?)

I find it a problem that there is so little transparency, because it makes it very difficult to evaluate oneself with mock tests.

If somebody has any information about how scoring is calculated, that would be very useful information.

[icebear]

I find it a problem that there is so little transparency, because it makes it very difficult to evaluate oneself with mock tests.

I think a Chinese approach would be to do the mock tests until you can achieve perfect scores on them

[patphilly]

Haha sure, I am aiming at the more perfect score too. But it would be quite frustrating to be able to answer 95% of questions correctly and only get a score of 70 or 80/100.

[Geiko]

Last year I passed HSK 2 with a score of 199/200. I know I made a mistake in the reading part (99/100), but there are 25 questions in that part, so 24/25 would be a 96% correct, not a 99%

[icebear]

Haha sure, I am aiming at the more perfect score too. But it would be quite frustrating to be able to answer 95% of questions correctly and only get a score of 70 or 80/100.

My impression of these types of grading systems is that they generally don't make the final score worse than what it was before adjustment. That is, if everyone got a 100/100 in absolute scores but 1 who got a 99/100, I doubt they'd fail that one person because he was the lowest mark. Generally these kind of de-meaning exercises help rather than hurt, at least when applied to rigorous tests. Just my 2 cents, I have not taken any of the HSK but I have taken plenty of tests which follow strange or seemingly opaque curving systems.

[patphilly]

That would indeed seem to be confirm by what Geiko reported... If someone can get 99/100 in an exam that contains 25 questions, it is indeed a bit odd

The only other logical explanation is if they use different weight to different questions (for example, if questions 1-5 are worth less than the following questions. etc): but even under that assumption, it is pretty impossible to imagine a weighting system that could yield 99% of points from just 25 questions...

[yialanliu]

Yea, I got a 99 on listening and there isn't 100 questions for HSK 5.

So yea, it is probably weighted with a bell curve. so it might be the same as the old hsk?

[patphilly]

OK, so maybe we can try to figure out how the weighting is done more or less. Among those of you who passed an HSK test, maybe you can report on how your scores compared to the % of correct answers you would usually get when you were practising mock tests?

For example, if you would typically get 90% of answers correct in mock tests and you felt the exam was the same difficulty, was your score higher or lower than 90/100 for that section? Then we can start to get a feel for how scores and % of correct answers are roughly related.

[edelweis]

This webpage on a hanban website (chinese.cn) claims the grading of the new HSK is proportional to the number of correct answers, except for the writing parts of HSK3 and above (detailed in the table).

Is it possible that the 99 pts come from a question they realized afterward was faulty (maybe too hard for the test, or could have several possible answers)? Or maybe even be a computer glitch?

I took HSK3 two years ago year and I don't recall finding anything strange in the grading, but then I did not reach such high levels as 96 or 99/100 for any part. I'll check later today whether the points I got for each part seem consistent with proportional grading.

[DanK]

Hello Patphilly.

Good question.

Based on research reports on the HSK system the process to generate your HSK exam marks is roughly as follows:

1) Collecting exam data. Collect answers from the student. Correct answers get a 1, incorrect answers get a 0.

2) Equating exam data. Because the questions in each exam are not the same and questions in the same test can also have very different difficulty levels, the difficulty of exams of the same HSK level will therefore vary. In equating, the HSK marks will be adjusted based on the difficulty level of the particular test and the questions within it. The widely used models for exam equating are CTT and IRT. So you can pass a test by answering lots of easier questions, or, a few very hard questions, or a mixture of both. Equating generally is not concerned about achieving a failure or pass rate but simply in making sure that each of the different tests on a given level is weighted fairly.

3) Standardization of test scores. All test takers are lined up according to their score. A normal distribution is applied taking into account that 160 is the pass mark, 0 the lowest and 300 the highest mark. Test scores that come out of the equating are then standardized according to the distribution of results by test takers. Once again, this is not about fixing a pass or failure rate, but about generating scores that reflect the percentiles of test takers.

Would agree with the other posters that a sound vocab knowledge and plenty of practice with reading and listening as well as writing, in addition to plenty of practice with mock HSK questions will give you a good chance to pass the HSK and get a decent score as well. Timing is often an issue for test takers. Practice will help with this. If a question is too hard, the best approach is to take yoru best guess and move on.

Hope this helps.

Good luck! ;-)

Regards,

Dan

[gato]

A bit convoluted. That seems to be for the old HSK.

[edelweis]

Alright I found my HSK3 results there.

听力 81.0

阅读 95.0

写作 91.0

Assuming linear grading:

听力 has 40 questions, so I got 81*40/100 = 32(.4) questions right.

Had I gotten 33 questions right, my mark would have been 33*100/40 = 82(.5).

On the other hand, with 32 right answers, my mark should have been 32*100/40 = 80.

阅读 has 30 questions, so I got 95*30/100 = 28(.5) questions right.

Had I gotten 29 questions right, my mark would have been 29*100/30 = 96(.6).

On the other hand, with 28 right answers, my mark should have been 28*100/30 = 93(.3).

Based on this and the 99/100 of posters above, my empirical conclusion is that the mark is computed as:

mark = floor((#RightAnswers + 1) * 100 / #Questions) - 1.

That is, your mark will be 1 point below what it would have been had you gotten just 1 more answer right, discarding any decimals.

[edit] I'm wondering, is this the standard method for computing grades in Chinese schools and universities?

[yialanliu]

Actually, I mis remembered. Applied for HSK 6 and found that I only got a 95 for listening on HSK level 5, not the 99 I thought I got.My bad

[patphilly]

@edelweiss: haha yeah, nice formula, it works Too bad we can't check it with your actual nb of correct answers

@DanK: thanks for the details. Are you sure this is not the procedure for the old HSK? As edelweiss mentioned on the link provided, Hanban says "各级别的客观题采用零一计分，线性转换为百分制。下表为主观题分数信息。" which although a bit convoluted, seems to suggest that scoring is proportional?

In general, I can totally understand the need to 'equate' exam data, to ensure all exams have a comparable difficulty. But then once the difficulty is equated, I don't see the need to take into account the test takers results to set scores. Because as long as the difficulty is evened out through the equating process, then all tests should provide the same average results for a large population of test-takers.

It also seems that they should just set a difficulty level and all those that clear this threshold should pass regardless of what other test takers do on average.

Other probably naive question: they have a contact address: anybody has ever tried to ask questions for clarifications to info@hanban.org?

[patphilly]

Another little related side-topic: I calculated what number of correct answers someone can expect to receive on AVERAGE if they have absolutely no clue and reply completely randomly to all questions in a test.

- for the exercises where one needs to pick whether the proposition is correct or wrong (let's call them type 1), there is 50% chance to get it right by replying randomly. So replying 10 such questions randomly, will give you 5 correct answers on average.

- for the exercises where one needs to match 5 items with 5 propositions ("type 2", let's say), the average correct answers one can get by replying randomly is 1 (the math is below if anybody is curious).

- for the exercises where one needs to pick between 3 choices A, B or C ("type 3"), then people would get 1/3 of all answers correct.

In the HSK level 1,

- listening part has 5 questions of type 1 (answering randomly gives a mean of 2.5 correct answers) -- 5 questions of type 2 (expectation is 1 correct answer) -- 10 questions of type 3 (therefore 3.3/10 correct on average). Therefore on average, somebody replying totally randomly can expect to get on average 6.8/20, or a score of 34/100.

- reading part has 5 question of type 1 and 15 questions of type 2, yielding a mean result of 5.5/20, or a score of 28%.

For HSK level 2,

- answering to the listening part randomly would give on average 12 correct out of 35 (that's 34/100)

- answering to the reading part randomly would give on average 6.5 correct out of 25 (that's 26/100)

For HSK level 3

- answering to the listening part randomly would give on average 13.7 correct out of 40 (that's 34/100)

- answering to the reading part randomly would give on average 7.3 correct out of 30 (that's18/100)

- answering to the writing part randomly would give... well pretty much nothing (unless you manage to order the 4 or 5 elements correctly, which would be less than 0.2 correct out of 10)

etc.

It's interesting that for all these 3 tests, somebody who has no clue and answers totally randomly can EXPECT as much as 34% of correct answers in the listening parts.

I wonder to what extent that would translate to a 34/100 score or whether the scoring is adjusted to take that into account. (For example, the American SAT test would subtract fractions of points for each wrong answer in order to account for random answers. Here it's not the case apparently.)

PS: and the math for the geeks: for type 2 questions (matching 5 items with 5 propositions), of course you have 5!=120 possible ways of answering randomly and only 1 permutation of those 120 will be the correct answer (giving you 5 points out of 5).

- there are 10 permutations that give you 3 correct answers (10/120, meaning it would happen 8% of the time)

- there are 20 permutations that give you 2 correct answers (17% chance)

- there are 45 permutations that give you 1 correct answer (38% chance)

- there are 44 permutations that give you everything wrong (37% chance)

That's where I get that on average, the number of correct answers you can expect is 1:

5*1/120 + 3*10/120 + 2*20/120 + 1*45/120 + 0*44/120 = 1

OK, sorry everyone for the slight digression (I could blame edelweiss for starting the math... )