Is it just me, or the HSK tests seem to be significantly harder in mainland China than the ones held abroad?

[I.carranza]

The question is, are all HSK tests created equal?

 

More specifically, HSK 6. Even-though the test might cover the same vocabulary and grammar, and that each test can be more or less difficult every month. Based on my personal experience and anecdotal evidence coming from friends and classmates, I have the feeling that HSK tests might be somewhat easier when taken abroad.

 

 

As some of you might know, in order to graduate from college or get enrolled in a Chinese university, many students need to pass HSK 6 with a certain score. In my case, I need to get more than 60 points in each section of the test, otherwise my university won't let me graduate.

 

Almost all my classmates living abroad, got more than 60 points in each section in their first attempt. Meanwhile, several classmates in China have failed to get more than 60 points in each section. We usually miss a few points in at least 1 section. Even after several attempts. 

 

So, recently my classmates and I have noticed how the foreign students in mainland China seemed to be the ones struggling the most to get more than 60 points in each section.

 

Maybe this is just us being in denial after taking the test so many times and not being able to succeed. But given the fact that tests are held in China almost every single month of the year, it almost feels as if this is a way for 汉办 to maximize profits from this standardized test. （Which I understand is normal for organizations to do this with other standardized test in the world.) See: "Why is the IELTS over-commercialised in China?")

 

Again, this could simply be wishful thinking. Us wishing that there could be another explanation as to why our classmates abroad winged the exam on their first try.

 

Maybe we are just being paranoid and not being able to accept that maybe we haven't prepared well or just don't have enough exam-taking skills.

 

But I thought it was worth mentioning. Especially because one of our Chinese teachers who was an examinator for hanban admitted that, sometimes, depending on how bad or how well test-takers perform at a specific 考点， they might grade everyone on a curve. See "grade-curving"

 

I know that maybe people don't give much thought to HSK grading system or 汉办 and its ways. But I still think this topic can generate an interesting discussion.

 

By the way, I am in no way trying to criticize 汉办 or minimize the efforts of those who took and passed the test abroad. I'm sure that, no matter where you take the exam, it'll always require your time and dedication.

 

What are your thoughts?

[大块头]

WelI, I don't think Hanban would go through the trouble of creating a dumbed-down version of the exam for international testing locations, so the only plausible scenario where such an assertion would be true is if they were using different curves to grade the exam in different locations. If the population of students taking the exam in mainland China were better at Chinese than those outside of mainland China for whatever reasons and if different grading curves were used, then the exam would be tougher to pass in mainland China.

 

Skimming the links shared here, it looks like Hanban determined one grading curve for the exam in 2013. My HSK 6 test certificate from 2016 has a percentile rank table that matches that grading curve. Does it match what's on you and your classmates' certificates?

 

[becky82]

I think we mostly assume everyone around the world is taking the exact same test, which is why there are exam dates.  However, I recall reading that Korean test takers, which make up something like 70%+ of all test takers, are normalized separately.

 

On 3/11/2022 at 2:49 AM, 大块头 said:

My HSK 6 test certificate from 2016 has a percentile rank table that matches that grading curve. Does it match what's on you and your classmates' certificates?

 

All HSK6 certificates have this exact table, no matter which month.  If you search for images of HSK6 certificates, you'll see other examples (such as here and here).

 

On 3/11/2022 at 2:49 AM, 大块头 said:

If the population of students taking the exam in mainland China were better at Chinese than those outside of mainland China for whatever reasons and if different grading curves were used, then the exam would be tougher to pass in mainland China.

 

It's possible I guess.  A test taker in China could be lumped in with a whole bunch of people from Xinjiang, who score very high.  But I don't think this is correct: I think it's based on nationality, not current location.

[I.carranza]

On 3/11/2022 at 8:01 AM, becky82 said:

a whole bunch of people from Xinjiang,

Interesting notice. I find it hilarious that, back in 2012 when they posted this, they were like: “我们认为，新HSK不存在“天花板”现象，不需要开发难度更大的考试”。Just to have them introduce 3 more levels 10 years later. Perfect example of confirmation bias.

[Publius]

Hmm, 10 years is enough to change many things. Cultural Revolution only lasted 10 years. 10 years ago Chinese was the language of the future, at least according to Hollywood (Looper). No sure how many people still believe it now.

[大块头]

After carefully reading the links that have been shared so far (and assuming the scoring system hasn't changed over the last 10 years), I don't think individual exams are graded differently based on an individual examinee's nationality or testing location. This article explains the system that Hanban uses to adjust scores to compensate for the varying difficulty of the ~8 exams it offers each year. (I wouldn't classify their system as a "curve" because it's actually rather crude from a statistical standpoint...) As Becky pointed out, a large fraction (35-75%) of the examinees each year are Korean. On average, Korean examinees have slightly lower raw scores (36.0/50 listening and 31.5/50 reading) than non-Korean examinees (37.4/50 listening and 33.2/50 reading). According to the article, Hanban calculates average raw scores for all examinees and then adds or subtracts a uniform whole number of points to everyone's final reported score based on (1) if that particular exam had a large fraction of Korean examinees and (2) how the average raw scores for that exam compare to the overall average raw scores for Korean or non-Korean examinees.

 

For example, if 74% of the people who took a particular exam were Korean and the average raw score on the listening section was 38.1/50, then the Hanban masterminds "可以考虑" subtracting 2 points from everyone's listening section scores to more closely match the overall Korean examinee average of 36.0/50. 

 

If only 33% of the people who took a particular exam were Korean and the average raw score on the reading section was 31.9/50, then everyone's reading section score may be increased by 1 point to more closely match the overall non-Korean examinee average of 33.2/50.

[markhavemann]

Seems like this has been analyzed quite thoroughly, but I thought I'd add my own experience since I've taken HSK 3, 4 and 5 in China. 

 

I've come across a reasonable number of practice questions in the actual exams. The other questions have all been really close the practice materials available.

 

I have never felt they were more difficult than the study materials available to help you prepare. Seems like it would be hard to make it easier than that abroad. Can't imagine how much easier it gets than including questions that students have potentially already answered while preparing for the test. 

 

 

[I.carranza]

Thanks for your answer. Actually, I agree with all you said. I passed HSK 3, 4, and 5 the first time I tried and with relatively good marks. But if you have seen or taken HSK 6, you'd have to admit that it's a monster of its own kind.

 

Especially because it requires you to use your logic and have some basic knowledge on Chinese semantics. Needless to say, it also requires you to see thru the ambiguity of some of the questions and answers. 

 

On a separate note, from an organizational view point, it is natural for 汉办 to make HSK 6 disproportionately harder than the rests of their lower level test (I can't blame them). Simply because it's their last exam, meaning that they don't get to have many "return customers" after 6, only "one-time customers." If you ask most people whether they would re-take HSK6 after they pass it, the answer is almost always: "no."

 

I know it's kinda pointless to talk about this when what's most important to most people is to pass the exam with a decent score. However, I think it's also important to remember that, even-though hanban's main goal might not be to profit, it certainly has bills to pay and like any other public institution, it has to ensure it doesn't become a loss-making business now that the pandemic has reduced the amount of test-takers in China and the rest of the world. And the truth is that they are not gonna make much money from the students who passed the exam, but definitely from the ones who keep on failing or getting low scores. On paper, they can say whatever they want, but behind closed doors you can't tell for sure what could be happening. The reason why I say all this (and maybe why I have a slightly cynical attitude towards this matter) is because I'm an examinator for a famous standardized language test and it's known among examinators that the test's grading system has flaws and that the "Halo effect" or "contrast effect" plus the lack of experience and/or training of the examinators can lead to improper scoring more than often.

 

On top of that, there are some stories of how some training centers which are also testing centers, correct the answers on the answers sheets (I have witnessed it myself) but only for those students who were studying the language in that specific center. (Unfortunately, the exam is not computer-based which makes room for situations like this to occur). I'm sure this is highly unlikely to happen with the HSK test and if it does, it would be a very rare isolated incident not a world-wide problem. But what I'm trying to express here is that, many things could go wrong when testing and grading, no matter what standardized exam we are talking about. To quote@Publius : "10 years is enough to change many things." 10 years ago hanban posted a notice explaining why there's no "ceiling effect" on their test. Several years later, that changed, hence the need of HSK 7, 8 and 9. Maybe in another 10 years, we will find out more.