How consistent have your HSK scores been?

[anonymoose]

If you've taken the HSK test more than once, how consistent have your HSK scores been?

I've just checked the results from my second time taking the Advanced HSK, and unfortunately I just missed out for the second time on getting the 9.

What really vexes me is that the areas I was weak in this time I was strong in last time, and vice versa. Last time I scored a high 10 on the writing, and this time a low pass (below the 9 level), and last time I failed the speaking, and this time scored an upper 10. I have been working reasonably consistently on my chinese in the intervening year, and would have expected an across-the-board improvement, so I don't really understand why my writing, for example, has dropped so considerably. Coming out of the exam, I actually felt my writing had been quite good, and my speaking poor, so I am rather surprised at the outcome.

Now I'm beginning to doubt the validity of the HSK. Don't get me wrong - anyone who has passed the advanced must have good Chinese, but my gripe is that there is too much variation in the scores from one test to another - in other words, one's actual chinese ability is only reflected to a certain extent, the overall score mainly being influenced by extraneous factors which would vary from day to day, and test to test. If you just happen to be 'unlucky' on test day, you end up coming away with nothing, and if you are lucky, you could end up with a 10.

Well fortunately for me, nothing depended on me passing, so no big deal. But I'm wondering whether it's worth me trying again. I've scored 9s and 10s on all of the sections before, just not on the same test. I'm now feeling I'll need to rely more on luck next time rather than any substantial improvement in my chinese.

[heifeng]

oh, from what I heard, the HSK peeps alternate the difficulty of different sections so I don't think it is that unusual for the difficulty to change. I tore up my earliest 2 exam results in a fit of rage (first one I missed a 9, 2nd one a 9) and now realized from looking below I should have torn up a 3rd one, but here are scores from the exam sheets I do still have. As you can see below, my section scores hopped all over the place but I highly doubt my Chinese changed that much during those 7 months (other than I think I actually decided to formally study 综合 prior to the 10/2007 exam, and that helped a bit, but ONLY in that section ). It's an exam, it's accuracy is questionable, at best its only good as a study tool, b/c to be honest, other than at a uni, when are you going to have an occasion to use this certificate...

4/2007 10/2007 11/2007

听力 78/B 68/C 70/B

阅读 69/B 53/- 55/C

综合 65/C 90/A 78/A

写作 84/A 71/B 84/A

口试 81/A 69/B 75B

总分 377 351 362

等级 B C B

[gato]

Do your score reports tell you what your raw scores (number of correct) are? The raw scores should be more comparable from session to session.

The scaled HSK score is based on a curve, so your score is bounded to differ from test to test, depending on who else is taking the test that day. This is more true for HSK Advanced, which fewer people take, resulting in to greater variability from session to session.

See these prior discussions about the HSK curve:

http://www.chinese-forums.com/showthread.php?p=40209#post40209

HSK Advanced

http://www.chinese-forums.com/showthread.php?p=72348#post72348

HSK paper marking question

http://www.chinese-forums.com/showthread.php?p=121609#post121609

Best way to get a HSK Level 3

[Scoobyqueen]

Might it be related to the fact that the exam is statistically dependent on how others do in the exam. So if "too many" have a nine, they need to mark some down. Maybe they have an artifical average that has to be reached.

[Don_Horhe]

But what's the point of it all, when it can be fairly simple?

[anonymoose]

from what I heard, the HSK peeps alternate the difficulty of different sections

That shouldn't make any difference, as the scores quoted have been processed in relation to everyone taking the exam. So a difficult section should be difficult for everybody, and therefore one's position relative to everyone else should still be similar.

my section scores hopped all over the place but I highly doubt my Chinese changed that much during those 7 months

Precisely. That's my point. I think this is a failure of the exam itself, because to provide a valid assessment of one's Chinese, it should come back with more consistent results. I've taught physics, and even designed physics exams. Students' scores of course fluctuated from exam to exam, but the relative position of each student was fairly consistent, so the exams were a reliable test of the ability, or at least relative ability, of the students. Unfortunately, it seems as though the same cannot be said for the HSK.

Do your score reports tell you what your raw scores (number of correct) are? The raw scores should be more comparable from session to session.

Nope. That's a highly guarded secret kept by the HSK mafia. You only get to find out the processed scores.

The scaled HSK score is based on a curve, so your score is bounded to differ from test to test, depending on who else is taking the test that day.

On the contrary, the fact that the scores have been scaled should in theory increase their consistency, as then one's score wouldn't depend on the difficulty of the paper itself.

The scaled HSK score is based on a curve, so your score is bounded to differ from test to test, depending on who else is taking the test that day.This is more true for HSK Advanced, which fewer people take, resulting in to greater variability from session to session.

I don't think this can really provide an explanation for what I've described in my initial post. Firstly, I don't know how many people take the Advanced HSK each time, but there must have been at least 100 people at my test centre, and there must be at least dozens of such centres throughout China, not to mention the rest of the world, putting the number of candidates well into the thousands. Statistically, at this level, there cannot be that large a random variation. Secondly, it may be that sittings at different times of the year attract more or less candidates from certain countries (Japan for example), affecting the overall competitiveness of the HSK from session to session, but in this case, surely all the sections would be affected in the same way. This I can understand. But I don't see why within the space of a year my writing would go from a high 10 to low 9, whilst my oral would go from sub-9 to upper 10. This would mean the first time round, most other people had relatively strong oral and weak writing, and the second time round, relatively strong writing and weak oral. I think large swings like this are unlikely.

Might it be related to the fact that the exam is statistically dependent on how others do in the exam.

See above. If one section is affected in this way, then all sections should be affected similarly.

But what's the point of it all, when it can be fairly simple?

Well, yes. I think one issue is the ridiculous time limitation with the HSK. With the reading section, it is more like a game of "Where's Waldo" where you're just scanning for the key word in the text without actually reading it. Of course, those who are good at that will perform well, but whether it tests actual reading comprehension ability is questionable.

[renzhe]

The problem with speaking and writing is that it requires human grading and assessment, unlike the multiple-choice answers which are always either correct or incorrect.

The curve won't help you there because different people are grading your essay and other people's essays. Sure, they have guidelines to follow, but in the end, such a grade will always be subjective, and will likely fluctuate between different graders.

Some standardised tests like the GRE or the ACT (or HSK Elementary/Intermediate, for that matter) only have multiple-choice questions, so they avoid this problem.

[gato]

Secondly, it may be that sittings at different times of the year attract more or less candidates from certain countries (Japan for example), affecting the overall competitiveness of the HSK from session to session, but in this case, surely all the sections would be affected in the same way. This I can understand. But I don't see why within the space of a year my writing would go from a high 10 to low 9, whilst my oral would go from sub-9 to upper 10. This would mean the first time round, most other people had relatively strong oral and weak writing, and the second time round, relatively strong writing and weak oral. I think large swings like this are unlikely.

Actually, your Japanese test taker example can explain the discrepancy you mentioned.

Japanese learners of Chinese (and Korean, as well) are very good at written Chinese but relatively poor at spoken Chinese. If your first test session had a smaller percentage of Japanese and Korean test takers than your second, then you would have gotten better "written" score on the first test and a better "spoken" score on the second. Mystery solved.

[anonymoose]

Japanese learners of Chinese (and Korean, as well) are very good at written Chinese but relatively poor at spoken Chinese

Possibly, but if we're lumping Japanese and Korean together, well, from what I've seen, they constitute more that 80% of test-takers in any session anyway.

This may be a factor, but I don't believe it can explain the whole story.

[wushijiao]

In my case, my HSK scores have been fairly accurate, I believe.

Here are my Advanced scores (spring 2006, fall 2006, spring 2007) :

聽力：56, 47, 62

閱讀：40, 54, 57

綜合：58, 42, 42

寫作：52, 65, 65

口語：63, 75, 81

Overall: Fail 269, C 283, C 307

As you can see, the biggest oddball for me is the 綜合. Does the HSK grade me unfairly there? Perhaps. But perhaps it is also of me not putting in enough work in that section, and I would hypothesize that students who study formally in classes do better at that section, on average, compared to self-studiers like myself.

but my gripe is that there is too much variation in the scores from one test to another - in other words, one's actual chinese ability is only reflected to a certain extent, the overall score mainly being influenced by extraneous factors which would vary from day to day, and test to test. If you just happen to be 'unlucky' on test day, you end up coming away with nothing, and if you are lucky, you could end up with a 10.

I'm not 100% sure if I'd agree with the concept that one person could vary from a nothing to a 10 on the same day, mainly because once one is getting into the 10 range, especially on listening, reading and zonghe, one is really starting to do go far on the curve. But I would 100% agree that on one specific day, you might miss a few questions here and a few there, and you can be thrown into a level you wouldn't normally "deserve".

So, for me at least, the test has been a fairly accurate indicator of progress, in the most general sense.

Well fortunately for me, nothing depended on me passing, so no big deal. But I'm wondering whether it's worth me trying again. I've scored 9s and 10s on all of the sections before, just not on the same test. I'm now feeling I'll need to rely more on luck next time rather than any substantial improvement in my chinese.

I think the questions you need to ask yourself are:

1) Have you learned much studying for the test, especially stuff that you may not have forced yourself to learn otherwise? (Maybe you could think of all the skills that have benefited you by studying for the test?)

2) Is the examination fee expensive for you under your current budget?

3) How much emotional stock do you put into the test? (It’s easy to say that one doesn’t give a damn, but the week or two before can be a bit stressful, at least for me).

4) Could the test be worthwhile to you in the future if you were to pass?

With that said, I think it’s worth doing. Both of my C’s could have easily been “nothings” had I done even a tiny bit worse on the 綜合. There certainly is an element of luck! So, I guess that shows that it’s always worth rolling the dice to see what you might get.

Anyway, I feel for you, and I know how demoralizing it can be to not make it. But look at the positive skills you have gained, and the areas you have improved in. I’m sure you can pass it sooner or later! 加油！！！

Edited June 5, 2009 at 03:41 PM by wushijiao
Added bold!

[Scoobyqueen]

You are right that the normalisation of the results ought to level things out (harder paper ought to be harder for everyone, therefore average scores are lower and the pass mark is lower) but since you are at the top end of the scale, you are definitely fighting against other candidates. I suspect they only ever give out a handful of 10s every year and once that quota is full, tough!

[imron]

If they only give out a handful of 10s, what does that say about 11s Maybe they don't give them out at all and just have it there to taunt serial test takers

[gato]

Take a look at the explanation for the HSK Advanced grading:

http://www.hsk.org.cn/Intro_hsk3.aspx

关于 HSK（高等）分数体系的说明

To get a 9, you'd need at least a 54-58 (scaled) score in each category. This is equivalent to the 60-70 percentile.

To get a 10, you'd need at least a 66-70 (scaled) score in each category. This is equivalent to the 85-90 percentile.

To get an 11, you'd need at least a 78-82 (scaled) score in each category. This is equivalent to the 96 to 98 percentile.

78 - 50 (mean) = 28

82 - 50 = 32

28 / 15 (sd or standard deviation) = 1.87 sd above mean equals to about 96 percentile

32 / 15 = 2.13 sd above mean, equal to about 98 percentile

See here for more on the conversion between standard deviation and percentile:

http://www.chinese-forums.com/showthread.php?p=72615#post72615

HSK paper marking question

Edited June 5, 2009 at 03:44 PM by gato

[wushijiao]

To get a 9, you'd need at least a 54-58 (scaled) score in each category. This is equivalent to the 60-70 percentile.

To get a 10, you'd need at least a 66-70 (scaled) score in each category. This is equivalent to the 90 percentile.

To get an 11, you'd need at least a 78-82 (scaled) score in each category. This is equivalent to the 96 to 98 percentile.

You have to look at each category seperatly, ie. a 70 (above 90th percentile) is needed to get in the 10 range in the listening, while only a 66 (roughly 85? ) is needed in the reading to get a 10 in the reading.

Also, the writing and oral are graded on "objectve" criteria, which, in theory, shouldn't be graded on a curve.

[gato]

Yeah, about 85, according to the chart on the HSK web site. I didn't bother calculating myself. I've revised my post above.

It's pretty hard to get above the 80 percentile in writing when more than 80% of the test takers are Koreans and Japanese.

[anonymoose]

I think the questions you need to ask yourself are:

1) Have you learned much studying for the test, especially stuff that you may not have forced yourself to learn otherwise? (Maybe you could think of all the skills that have benefited you by studying for the test?)

2) Is the examination fee expensive for you under your current budget?

3) How much emotional stock do you put into the test? (It’s easy to say that one doesn’t give a damn, but the week or two before can be a bit stressful, at least for me).

4) Could the test be worthwhile to you in the future if you were to pass?

Learning Chinese has only ever been a hobby for me. That's not to say that I don't take it seriously, but I have never set myself any attainment targets. My motivation for taking the HSK in the first place was simply to get an objective measure of my chinese ability, but the large fluctuation in scores now makes me wonder about the validity of the test. I take my hat off to anyone who has passed it, because you definitely need good chinese to pass, but as I said previously, I think getting the grade also depends on significant factors outside the control of the candidate.

I must admit that I did very little preparation specifically for the HSK. The main reason being that I ultimately care about my actual Chinese ability rather than the certificate, so I'd rather spend time improving my Chinese generally rather than HSK-specific skills. I feel that improving Chinese overall will naturally lead to an improvement in HSK performance.

I honestly don't put much emotional stock into the HSK. It would just be nice to have as some kind of milestone of achievement, but my interest in Chinese definitely does not revolve around the HSK.

[wushijiao]

It's pretty hard to get above the 80 percentile in writing when more than 80% of the test takers are Koreans and Japanese.

But only because they are better at characters, on average. Actually, think writing by hand mght be most important in the last section of the 綜合. For example, this last time I took the test, there were at least three answers I knew for sure, but just could't remember how to write.

[gato]

Right. South Korean kids are expected to know 1800 Hanja characters (write?) by the end of high school. Japanese kids are expected to know 1945 Kanji characters (Joyo Kanji) by 9th grade.

[paike]

Anon, worry not. You're making me worry with your worrying. =)

First post, been looking at the forums for a few weeks. I've been living in China for 4 years, three I've been "studying". Four months of studying, two months living on the railroad traveling and speaking. I often listen to the radio and watch spongebob sqaurepants, etc.

I've taken a load of practice tests, the middle one and it seems that my test scores hardly ever change, even after two months of intense study. Well, the first few weeks, my score shot up to the 6-7 range, but now I cant see to get any better. One week left and I take the HSK for the first time, not too worried. I'll take it in Oct and Nov, just for fun. Then take up a class in BJ or somewhere for the winter, ACK, to study for the advanced. Again, just for fun.

I've kinda dropped out of normal life and work as a teacher at Northeastern University in Shenyang, not a bad job and I only work 10 hours a week for decent pay, airline tickets and 5 months of paid vacation a year.

But, to answer the question, it seems like my scores don't change, and "studying" for the test doesn't really seem to improve them. I just need to focus on another solid text book, completely understand it and will probably see some improvement. Listening is probably my strongest section now. I listen to the radio a lot and live in the NE.

派克