Web app for checking your Mandarin pronunciation

[craig.jb]

I've created a free web app for checking your Mandarin pronunciation, available at https://accentlab.net/mandarin_words . It's currently just a prototype, but I'd love to hear your feedback and ideas! You can say any word and it will transcribe it into Pīnyīn (with tones), or you can practice HSK vocabulary.

I've been working on the program's accuracy for some time now, but the user interface on this page is pretty new and unpolished. The accuracy still isn't perfect, but I decided it was good enough to share.

There are many directions this could take as I further develop it. I could have it collect statistics on your errors on the HSK vocabulary, and then have it tell you things you need to work on, or review words that gave you trouble. I could have it export Anki flashcards. I could add interactive lessons on particular pronunciation topics, like difficult sound pairs x/sh, re/ri, tone sandhi, etc. I could add Cantonese. Let me know what might be useful to you as a Chinese learner or teacher! 多谢!

[Jan Finster]

Feedback: so far it is rubbish. It always shows "undedined".

 

Are you trying to reinvent the wheel? (Praat, etc, etc).

 

[craig.jb]

Hi Jan, sorry to hear it's not working for you. Did you click "Allow" when it asked to use your microphone? What browser are you using?

 

I don't see how Praat is equivalent, it doesn't know anything about Mandarin and can't tell you what syllable you said. I assume you use Praat to view a pitch track and try to identify tones visually, but I don't find that to be a very accurate or convenient way to practice tones.

 

When you say etc. etc. could you mention any other software that has this functionality? I know of only one, called 普通话学习，that has something similar.

[MTH123]

I love this concept! Although I'm not planning on speaking anytime in the near future, I may try it in the far future.

 

[alantin]

@craig.jb, I got it working after fiddling a little bit to get the browser to use the correct microphone.

 

 

Though the recognition doesn't seem very accurate yet, I like it! I actually asked a Chinese teacher the other day if she knew an app like this.

Maybe also offer an option to speak longer sentences too instead of only single words. Also I dislike holding the button down with the mouse to speak. How about adding a keyboard shortcut so that you could use the space bar too for example?

[craig.jb]

Thanks so much for trying it out and for the feedback, alantin!

 

I'll definitely think about alternatives to "hold and record". Initially I had a "record" button which you clicked, which then turned into a "stop" and a "cancel" button. But apparently many people are used to "hold and record" because of new social media apps, and they had trouble with it. I like the idea of a keyboard shortcut, though "space" typically means "play", so maybe I could use "return/enter" to start and stop recording. Maybe I should limit "hold and record" to mobile devices, and use record and stop buttons on desktop browsers.

 

I do want to be able to work with longer sentences. I did this already for English, French and Spanish as available from the main page. There are many challenges though. It's more difficult to accurately process running speech than isolated words. It tends to cause confusion for the user. However it's something that I plan to continue refining, and in the future I hope to have something working well for Mandarin with sentences.

 

Thanks for the kind words, MTH123! Good luck with your studies.

[feisima]

On 8/13/2022 at 3:34 PM, Jan Finster said:

Feedback: so far it is rubbish. It always shows "undedined".

 

Are you trying to reinvent the wheel? (Praat, etc, etc).

 

Hi, Jan! Maybe you didn't give your microphone permission? By the way, what browser are you using?

[Jan Finster]

On 8/13/2022 at 4:02 PM, craig.jb said:

Did you click "Allow" when it asked to use your microphone? What browser are you using?

 

On 8/18/2022 at 8:14 PM, feisima said:

Hi, Jan! Maybe you didn't give your microphone permission? By the way, what browser are you using?

 

Yes, I did on both Edge and Chrome. Still the same.

 

 

On 8/13/2022 at 4:02 PM, craig.jb said:

When you say etc. etc. could you mention any other software that has this functionality? I know of only one, called 普通话学习，that has something simila

 

Back then I was looking for such programmes and I did find 1 or 2. All quite old, but with similar features as yours. I am not trying to dodge the questions and looked through my bookmarks, but I could not find them anymore.

 

 

 

[craig.jb]

On 8/18/2022 at 2:42 PM, Jan Finster said:

Yes, I did on both Edge and Chrome. Still the same.

Thank you for the update, @Jan Finster. I will try it out in Edge and Chrome on some other people's computers to see if I can reproduce the problem. Another solution might be to select a different microphone in your browser; in Chrome, there is a microphone button to the right of the URL bar that pops up a menu for selecting the microphone. @alantin mentioned that this solved the problem for him. I'll need to look into how some other applications help their users troubleshoot these problems.

 

I've just slightly improved the error handling, so that you should see the error message "silence" or "too short", rather than "error" or "undefined". These indicate that the audio received from your browser was all silence, or that the recording was too short (less than 0.3 seconds).

 

On 8/18/2022 at 2:42 PM, Jan Finster said:

Back then I was looking for such programmes and I did find 1 or 2. All quite old, but with similar features as yours. I am not trying to dodge the questions and looked through my bookmarks, but I could not find them anymore.

I have not come across many programs for pronunciation study which actually analyze the user's speech from the microphone, especially for Mandarin. So if they're not available, I guess I'm not reinventing the wheel! Thank you for looking though.

[wibr]

Works pretty well for me (Safari on an iPhone). Would be good if you could provide some information regarding data privacy. Is the audio processing done on device or on the server, are the audio clips stored etc. . Traditional characters would also be appreciated. In order to be a useful independent study tool I think it would require a more structured approach. HSK word lists don't really provide any benefit here, it's more important to practice different combinations of sounds and tones that learners are struggling with.

[craig.jb]

Thanks so much for trying it out, and for the feedback @wibr. The audio processing is done on my server, and is not shared with third parties. It is not currently stored on the server, but in the future I'd like to store it so that I can collect data for improving my models (still without sharing with third parties) as well as providing the user with long-term statistics. Currently I have a login requirement for other languages which notifies the user of these conditions, I need to post it somewhere for Mandarin. Would you find this approach to privacy to be acceptable?

 

I can easily add traditional characters, I'll make that a priority. I'd also like to add 注音符號 (bopomofo) as an option.

 

I have also been thinking about how to make it more structured. The first thing I'd like to add is practice of tone pairs. I think this would tend to reveal some practice needs pretty clearly. I could also collect stats on initials and finals for similar directed practice. Regarding HSK lists, as an advanced student myself, I do like having an honest evaluation of my memory of words' pronunciations, as I have a few with an incorrect tone or final.

 

Thank you again for sharing your thoughts.

[roddy]

I gave it a quick try. Unfortunately, the recognition just doesn't seem to be there - which was fair enough when I was speaking, maybe my pronunciation is off. But then I tried playing Pleco pronunciation clips into the mic and it was just as far off, in some cases scoring 0/3 for initial/final/tone. 

[craig.jb]

Thank you for trying it out and for the feedback, @roddy. I'm still working on increasing the accuracy, I'll see how far I can get. I do find that I'm able to get it to recognize the majority of syllables correctly, but a few don't work right. I wonder if the way you played the Pleco clips resulted in a poor quality recording, or if you generally had background noise or some sort of distortion in your recording.

[Moshen]

Quote

I wonder if the way you played the Pleco clips resulted in a poor quality recording, or if you generally had background noise or some sort of distortion in your recording.

 

Isn't that irrelevant, though?  Won't most of your users have mediocre recording capability (or worse) and some background noise - maybe even a lot of it?

[craig.jb]

On 8/20/2022 at 8:01 AM, Moshen said:

Isn't that irrelevant, though?  Won't most of your users have mediocre recording capability (or worse) and some background noise - maybe even a lot of it?

You're totally right @Moshen, unfortunately there's only so much I can do about it. One thing I've considered is trying to measure the noise level, and give an indication to the user. Sometimes it's easy for humans not to notice the level of noise around them, because we're so good at filtering it out, but for speech processing it's a real nightmare! I've had friends want to try this program at a noisy restaurant, which really won't work very well.

[babymumu]

Good job @craig.jb! The result was surprisingly accurate from my trial. Hope it could accept customized word list later.

[craig.jb]

On 8/21/2022 at 12:29 PM, babymumu said:

Good job @craig.jb! The result was surprisingly accurate from my trial. Hope it could accept customized word list later.

Thanks for trying it out @babymumu! I'm glad it worked accurately for you. There are sometimes certain sounds that give it trouble, but it's usually pretty good. Someday I'll go back again to try to squeeze a bit more accuracy out of the acoustic model.

 

I'll definitely think about adding a customized word list. I suppose just uploading a file with one word per line in Pinyin with tones could work, or a CSV with pinyin, 汉子、definition.

[Aslaksen]

Wow! Excellent! Worked for me! Very useful. Thanks!

[craig.jb]

On 8/21/2022 at 6:52 PM, Aslaksen said:

Wow! Excellent! Worked for me! Very useful. Thanks!

I'm glad it worked for you, @Aslaksen! Thanks for trying it out.