Placement Tests and Level Tests
John
January 20, 2010 at 09:56 AM posted in General DiscussionAs promised, the new testing features are out! You can find them in the "Resources" section, and there is also a blog post about them.
Please feel free to leave feedback here.
henning
January 25, 2010 at 08:56 AM
The placement test is pretty useful for vocab repetition - primarily based on vocab recognition.
As a true "placement test" I now find it often too easy for the following reasons:
- Google Pinyin almost works as a cheat in the dictation as it suggests answers.
- In the multiple choice section you usually have (at least) a 50-50-joker when you exclude unfitting lexical categories (e.g. nouns where only a verb makes sense).
- If you have done CPod for quite a while then quite often you recognize the familiar vocab among all the strangers.
I struggle a lot more with even Basic HSK tests that require to get the details.
Don't misunderstand me: I like the test a lot and I also think it was a brilliant idea to use the existing lesson exercises for this.
We just need....even more tests. Harder ones. Testing grammar, nuances, detailed understanding.
:)
user76423
January 22, 2010 at 05:43 PM
Another test with strange things happening:
(1) Nothing typed in, but 1 point received, because the expected answer is "nothing": Snapshot
(2) The lady says "shi4", but of course it's different: Snapshot
(3) One answer is empty, and I hit it lucky: it's correct: Snapshot
user76423
January 22, 2010 at 05:18 PM
In the Newbie dictation, the woman says "jia1", so I typed in 家, and the correct answer is 夾.
Very funny.
Some expected answers are not Newbie level.
Like "的士" Taxi (only used in some parts of China)...
Or "敬禮" to salute, that's an HSK-B item...
Stupid test, or stupid student?
paulinurus
January 22, 2010 at 03:13 AM
John,
I think you need to incorporate audio, as well as, hanyi and pinyin in the tests, especially the placement test. As you know, while people would perfectly understand what is being said in Chinese, they may not be able to accurately specify the tones in pinyin.
Also, Cpod's pinyin is non-standardized. Most dictionaries and grammar books standardize their pinyins "as is" without tone changes, while Cpod's pinyin accomodate tone changes (thus lots of neutral tones in compound words) which causes frequent "fails" when we write the answers in pinyin.
If this is too much work, then alternatively I think the placement test can be scrapped. When I took the two tests, I found it not only redundant but also unreliable. I was "intermediate" in the Level Test and "newbie" in the Placement Test, mostly due to my inaccuracies in pinyin tones. Unless poddies are taking HSK or some other formal Chinese exams, who would be concerned to study pinyin tones correctly in order to figure out what level they should be studying at Cpod? The Level Test is good enough for this purpose.
calkins
January 22, 2010 at 01:02 PM
Personally, I think the inherent issue isn't with pinyin, but with taking a Chinese "proficiency test" using all pinyin. I think it'd be okay to use at lower levels (Newbie and Elementary) as long as it is accompanied by characters (like others here have mentioned). I think once you get to Intermediate and beyond, it's a lot more difficult to learn this language using pinyin only, and I believe that proficiency tests at the upper levels should be hanzi only (just my opinion, I know many will disagree).
And I think it's nearly impossible to input pinyin accurately for testing purposes, for many reasons (easy to make typos, use spaces or not with word groups?, tone change confusion, capitalization or not?, etc.). Perhaps Cpod could have a link at the top of all Pinyin-based tests, describing in detail the standard rules of pinyin and how best to adhere to those while taking these tests.
gesang
January 22, 2010 at 01:00 PM
paulinurus, actually I don't care that much... it is only a computer programm not a teacher or a Chinese I hope to understand/talk to...
If you look at your test result paper and you see most of your sentences should be marked 'right' because they ARE right (only marked wrong because of pinyin settings) just be happy: You answered very good and you learned a lot by taking the test: seeing where you could not write any answer and seeing where you are absolutely sure you know better than the computer.
I did not do the pin yin test, but using characters the tests work well..and showed me that I really have to study more vocabs if I want to take that step from Elem. to Interm.!
I am writing as much as I know in the dictation part, also if I don't understand some words. Of course the programm will give me a 'wrong' for the whole question..but for myself I know at least the first part of the sentence was perfect :)... and if it would not be a stupid computer correcting my results, but a real teacher he/she would recognize!
I wrote I was frustrated by MY results...I saw all my mistakes and what I should have known, and what I am missing... but it is fun to do the tests!
Don't be frustrated about the fact that this programm is giving you a wrong number of ..% right questions! A number given by a computer will never express what you really know anyway!
:-) 格桑
bodawei
January 22, 2010 at 12:14 PM
@Simon
Just a point about ge4, the default measure word. It is often used in conversation without the noun it refers to; if so, the fourth tone is used. Even when used with the noun.. hm, I would say from experience that it is usually fourth tone. I am learning to use it more forcefully. More experienced speakers, I am happy to be contradicted. (And I don't want to lead anyone astray.)
paulinurus
January 22, 2010 at 11:58 AM
simonpettersson,
all i can say is love for academic theory is quite a distance away from day to day practicality... take the two tests then tell us how you fared. And too, I've just given examples taken from the sentences in the lesson itself. Good luck in writing pinyin from the Expansion Sentences audio.
Looks like I might be in Gesang's company... a (forever?) Newbie at Cpod.
simonpettersson
January 22, 2010 at 11:48 AM
Paulinurus: Those all sound correct to me. bu4 is written as bu4 in pinyin, regardless of the tone after. It's rarely used as a full fourth tone in "X bu X" constructions, and "ge4" is usually uttered with a neutral tone, when used as a measure word. Also, reduplication, like "kàn kan" always has a neutral tone on the second syllable, though I might have written it without the space.
However, using a neutral tone or not varies a bit in actual usage. I haven't done the test, so I don't know how it works, but if the technology can support multiple correct answers, I'm guessing that'd be ideal.
paulinurus
January 22, 2010 at 11:37 AM
John,
Here are the neutral tones in compound words taken just from one lesson - Computer Problems And Tech Support
yǒushíhou – hou4 is 4th tone unless what?.
dǎ bu kāi, dòng bu liǎo – bu4 is 4th tone except when this and what?
nǐ shì bu shì dǎkāi le hěn duō? – shouldn’t “bu” be 2nd tone before a 4th tone, however in this Cpod sentence it is a neutral tone.
bùhuì a, jiù liǎng ge. yí, shǔbiāo yě dòng bu liǎo le. – In this sentence, ge is neutral instead of 4th tone, and bu is neutral before a 3rd tone
nǐ jiào jìshùbù de rén bāng nǐ kàn kan ba. In this sentence, one kan is 4th tone, the other is neutral.
nà nǐ kěnéng zhòngdú le. xiān shā sha dú. – One sha is 1st tone, following one is neutral. Is this the rule?
tiān a, jiǔ bǎi ge bìngdú! ài, zhēnshì fú le nǐ le.- Shouldn’t ge be 4th tone?
As for getting different results between Placement Test and Level Test, let’s hear from more poddies who have taken the 20 questions tests a few times. Did you find yourself repeatedly in Newbie placement even though you had above 50% scores in Elementary or Intermediate?
John
January 22, 2010 at 07:37 AM
Paulinurus,
I'm surprised to hear you say "Cpod's pinyin is non-standardized". We are aware of the rules of pinyin and adhere to them as best we can, particularly with regards to tone changes (a stance I sometimes have to defend because it makes some learners unhappy). I think you may be referring to a different problem: variation in the language itself. Then it's an issue of whether or not we expose students to such variation, and how early.
I'm not sure exactly what you're talking about re: neutral tones in compound words. Can you give examples? If we're wrong or being inconsistent anywhere, we'll fix it ASAP!
You're also the first to report that the placement results you're getting are inconsistent. I've heard back from a variety of testers which say they find it both accurate and consistent (Henning being one of them). I wonder if this could be a data issue, or perhaps the result of just random selection (t's possible that you could get a random selection of relatively easy or relatively difficult questions, but the chances of this go down the longer the test type)?
henning
January 22, 2010 at 06:27 AM
I have taken the Placement tests about 6 times now (1 long, 5 short tests). My results are pretty stable in the Upper Intermediate spectrum. But I never even tried the Pinyin version - I just used Hanzi.
frognotinawell
January 21, 2010 at 03:30 PM
The test is certainly a useful addition. It's shown me my weak spots, ie nearly everywhere.
Two points:
In the Newbie Level Test, the following answers were marked as wrong (yellow = CPod's answer).
4. 他是_____吗?
- A. 美国人
- B. 国家
- C. 英国
- D. 中国
Either I need to go back to Lesson One, or question 4 is wrong.
12. 我买了一件_____。
- A. 裤子
- B. 还好
- C. 你
- D. 衣服
I've checked on Google and both 一件裤子 and 一件衣服 are commonly used.
tvan
January 21, 2010 at 08:10 PM
If it makes you feel any better, I would have selected 美国人 for question #4.
frognotinawell
January 21, 2010 at 07:30 PM
Danke, henning.
I'll check with mdbg a lot more in the future.
go_manly
January 21, 2010 at 07:39 AM
When we see a problem with a question, how do we report it? After all, the number of the question in 'my' test would not necessarily be the same as the question number in someone else's test. I'm thinking of a question on my test that had the 4 options, but the question was blank.
John
January 21, 2010 at 08:49 AM
go_manly,
Unfortunately, there's no good way to do it right now. Certain things, like blank questions/answers can be identified programmatically. We have already done this, but it appears the script we used wasn't thorough enough. We'll keep refining the data and working on more efficient ways to do it.
zodboyer
January 21, 2010 at 07:01 AM
第一道题我做错了……现在很多中国人说汉语都不是很规范。
I got wrong in first tast...And now,losts of Chinese can not speak canonically.
xiaophil
January 21, 2010 at 06:52 AM
I just took the test. Fun! I do think that some of the recordings are a tad on the distorted side. However, I guess people in the real world don't sound perfect, so perhaps recordings don't have to sound perfect either.
bababardwan
January 21, 2010 at 04:33 AM
I took the long test and scored 0 out of 25 on the dictation because something came up and I had to abandon the test at the start of the dictation section and shut the computer down.I suppose there is no way to save where you're at part way through a test and finish it off later?
bababardwan
January 21, 2010 at 06:37 AM
Thanks John.That's really cool.I was in a rush at the time and didn't have time to look for a solution like that but it's great to now know it's possible. :)
John
January 21, 2010 at 06:33 AM
The test saves your answers anytime you click on a button at the bottom of the page (either "Next" or "Save"). So if you're on the last page, I think it would work to click on the little "back" arrow. I'll have to check this.
bababardwan
January 21, 2010 at 05:21 AM
Hey,I've just started to check out the blog and came across this partial answer to my question:
"Partial saving of the test, just in case you get interrupted (answers are saved when you go to the next page of the test)"
..so the question remains ..how would I have saved where I was up to?..was there something I needed to do or if I'd just exited outta there would it have automatically saved it [actually it sounds like this automatically occurs ]..and by partial you mean you can just save at the end of page one or the end of page two? ..what happens if you shut down part way through a page?
pchenery
January 21, 2010 at 04:30 AM
Excellent work on the new tests CPOD.
I scored zero out of 25 on the dictation section since there was no "convert to pinyin to tone marks" option. Is there a way around this ?
WillBuckingham
January 21, 2010 at 10:19 AM
Have you tried getting your computer IME (input method editor) to convert to pinyin tone marks? IBus for Linux can handle this easily, and I think there's Pinyinput for Windows (which I think you can find by searching the chinese-forums forums). There's also a Mac equivalent.
Have a look here for more info (I think there's a Microsoft Pinyin IME as well...)
http://en.wikipedia.org/wiki/Chinese_input_methods_for_computers
tonyjh
January 21, 2010 at 03:37 AM
The tests are great. I think they will be very useful for me.
Most of the questions were fine, but just a few seemed to have issues:
- In the multiple choice section, one of the questions was completely blank.
- On another one of the multiple choice questions, the question was not blank, but one of the choices was blank.
- On at least two "fill in the blank" questions, there was no blank to fill in. The correct answer was already shown as part of the sentence.
I used the pinyin option. I think it would be nice if there was a "convert to tone marks" button available in the dictation section. Could that be added?
calkins
January 21, 2010 at 12:14 AM
John (and team), these tests are really great...thanks!
You might already have this in the works for future tests, but what about incorporating Skritter so that there are some character-writing questions?
Again, thanks for the great resource. Nice job.
gesang
January 22, 2010 at 08:12 AM
Oh, oh... I was already frustrated about the result of my Ele-Level test...with character-writing questions I'll be definitely back to Newbie for the rest of the year ;-)...
When someone ask me how many characters I am able to write, my answer is usually: 'Well, when typing on a computer it is definitely more...'
But I think it is a good idea though - a motivation...
orangina
January 20, 2010 at 04:33 PM
哈哈!I was about to complain about the test because it is too hard when you can only see pinyin or characters but not both... (I was very good and took the character test without using Mandarin Popup to cheat. It was hard to not use it!)
But then I was placed at a higher level than I thought I would be. So now I say: Most excellent test!!!
But now no more coasting. I am not an ellie anymore. I landed squarely in intermediate, so now I have no excuses and need to get to work.
zhenlijiang
January 20, 2010 at 02:12 PM
I just took the Intermediate Level Test (scored maybe just a bit lower than I should expect). I guess the virtual graded test sheet still needs a bit of adjusting. Mine showed only the first Multiple-Choice portion in entirety. I really wanted to see the second Multiple-Choice portion as I'd made the most mistakes there but that didn't show up at all, then in the Dictation portion the correct answers were illegible, appearing with the characters piled up on each other vertically. So am still mystified re those sentences I didn't get at all.
I enjoyed the test though. And it's good timing for me, to have a test like this just one year after starting here at CPod.
John
January 21, 2010 at 08:47 AM
Could you tell me what version of IE8? We tested IE8 in house and do not get this issue.
Thanks!
zhenlijiang
January 21, 2010 at 05:25 AM
John, I was seeing a little vertical pile of the characters along the left margin for each answer (failing to line up horizontally is how it looked). I'm using IE8 too btw.
matthiask
January 21, 2010 at 02:15 AM
John, same problem here: the characters in the dictation test line up vertically instead of horizontally.
IE8John
January 21, 2010 at 01:19 AM
zhenlijiang,
What was the problem displaying the Dictation answers? The correct answers just didn't display? That's an issue I haven't seen before.
zhenlijiang
January 20, 2010 at 05:09 PM
Duh. The Multiple Choice, both portions are showing up in entirety. I wasn't looking carefully. Still seems to be a problem displaying the Dictation answers though, in the Graded Virtual Test Paper.
OK so I've taken the Placement Test now and was placed Upper Inter. I'm not updating my profile though, not as long as Intermediate lessons remain challenging enough for me. I'll just have to push myself to go through the lessons more quickly than last year and start doing more Upper Inters.
By the way I scored exactly the same (percentage) in the Intermediate-level questions in this Placement Test as in the Level Test (Intermediate) that I took earlier. Consistent test results.
go_manly
January 20, 2010 at 11:17 AM
OK, I sat the Newbie test, and have two questions.
1. tā xiànzài bù _____ jiàn nǐ.
- A. fāngbiàn
- B. zǎoshang
- C. kěyǐ
- D. bùtài
2. tā _____.nǐ děng yīxià.
- A. zài
- B. shénme shíhou
- C. huílái
- D. chūqù
The yellow is CPod's answer.
In 1, isn't my answer kěyǐ the correct one, or am I missing something?
In 2, why aren't 1, 3 or 4 all correct?
Also, it would be good to have the option of having both pinyin and characters. I don't know a lot of characters, so I need the Pinyin. On the other hand, these sentences lack context, so it would be good to see the characters sometimes.
At the top of each page, there seems to be an option to change between Simplfied, Traditional and Pinyin, but it is not active.
John
January 20, 2010 at 12:10 PM
While C is correct, it's less likely than A, because it would be saying "he/she is not allowed to meet you," which is kind of strange. Sounds kind of like the parents of a girl telling her bad-boy boyfriend that their daughter is no longer permitted to see him. Possible, bot unlikely.
Our team has had to go over the multiple choice sections of over 1300 lessons' randomly generated exercises, eliminating "extra correct answers." While they do good work, there may be a few slip-ups.
simonpettersson
January 20, 2010 at 11:51 AM
Seems to me that both A and C are correct in number one. In number two, "tā huílái" and "tā chūqù" seem to both need a "le" to be correct. I guess you could say "ta huílái", but that would be the equivalent of saying "He returns. Please wait a moment". Technically correct, but unnatural.
henning
January 20, 2010 at 10:39 AM
I love tests.
Curious: Did anyone get to 100% in the long version of the Placement test (without cheating)?
bababardwan
January 27, 2010 at 07:27 AM非常奇怪。。I just did a quick placement test and got zero.Ok,but the thing that is really strange is when I clicked on the feedback link not one of the questions in the first two pages of multichoice is even remotely like the questions I was asked and the dictation is not even close to what I heard either.Definitely some technical problem here.I'm not sure if the recent CSS thing is related or this am's server crash but I think it needs looking at.Cheers :) [sorry to have to point to even more work when you're already working so hard on this 辛苦你了John :) 】
ps the other strange thing is that it has yesterdays date as the date the test was taken and yet I only did it 5 minutes ago and never even opened the resources tab yesterday or indeed had not tried a short version of the test at any time.