What can't ChatGPT do now, that might change in the future?

[becky82]

There are things I want ChatGPT (or any other AI) to be able to do, but it can't or it's weak at.  However, ChatGPT is rapidly evolving.  For example, ChatGPT used to be bad at "please give me 5 example sentences where [word] is used as a verb, and 5 where it is used as a noun", but now it's quite good at this task.

 

So I want to create a kind of "ChatGPT time capsule", wherein we take snapshots of ChatGPT performing poorly, and come back in a year or so (so let's say July 2025) and see if it's changed.

 

In my experience:

 

1. ChatGPT has been awful at pinyin in the past, but it's much better now.  So I performed an ad-hoc evaluation just now, and I was unable to get GPT-4o to make a single pinyin mistake.  Once I ran out of GPT-4o quota, it switched to GPT-3.5 (I think), after which I encountered two errors, where a space was added in "Miàn duì" (面对), and ChatGPT wrote "因噎废食 - yīnyèfèishí", where the correct pinyin is yīnyēfèishí.  These issues are rather minor.
2. ChatGPT currently bad at separable verbs.  In my evaluation (GPT-3.5), I asked it to give me examples of 10 separable verbs, of which 0 were actually separable verbs.  I had to give it examples of separable verbs before it knew what I was talking about.  However, it managed to construct sentences using those separable verbs where the characters were non-consecutive.

 

I wish ChatGPT could give me book recommendations from photos of shelves at my local library, like this:

 

 

(ChatGPT currently does not support sharing conversations with images, so I copy/pasted GPT-4o's response to Pastebin.  I didn't realize how blurry my photos were [that part of the library is dark].)

 

The books it recommended are 《幸福在这里》, 《我会画画啦》, 《白色橡树》,  《希望的田野》, 《今天不说话》.  I found the first book in one of the photos (but it was too tedious to find the other four).  I asked ChatGPT to Google those books and describe the plot and reviews, but I don't think it succeeded; it gave me links to random unrelated books, and I think it just made up the plots based on their respective titles.  I asked ChatGPT to edit the photos and highlight the five books, and it drew five red squares in random places in the photos (with no obvious relation to those books).  I asked for quotes from the first one 《幸福在这里》, but it gave me quotes from another random book.

 

The other thing I think would be really useful if ChatGPT could do, is asking "in which textbook, on which page, can I find [blah]?"  Or even better would be to give it a photo of my textbooks, and have it point out which textbook, and on which page, a certain item is.  I checked if ChatGPT could do this here:

 

Quote

Where can I find the grammar pattern 愈……愈…… in the HSK Standard Course textbooks?

 

but it told me the wrong textbook.  When I explained that it's in the HSK6 Standard Course textbook 下册, it told me it's on page 29.  I checked, and it's not on page 26.  It's actually on page 147.

 

Okay, what about everyone else: what do you want ChatGPT (or any other AI) to be able to do to help with learning Chinese (that it plausibly could do in the near future), but it can't yet do?

[lordsuso]

On 7/18/2024 at 8:22 AM, becky82 said:

in which textbook, on which page, can I find [blah]?

I doubt this will ever happen, unless you provide yourself a pdf of the textbook or something that it can read, otherwise I don't see how it would be legal for it to have out-of-the-box precise knowledge of the contents of all books.

 

For me personally I wish it had a longer memory. I chat with it daily, and when I try more complicated role-play scenarios, it tends to forget after a while what we were doing, and it's hopeless to try to reference an earlier part of our conversation (This is with the free version, GPT-3.5?)

[suMMit]

On 7/18/2024 at 4:04 PM, lordsuso said:

it tends to forget after a while what we were doing, and it's hopeless to try to reference an earlier part of our conversation

Yeah, I was having this same problem. Also, the voice is also not that great on the extension I'm using. So I did a bit of searching and found this app called Univerbal. I don't think I've heard it mentioned on the forums. I've been testing it out and so far quite like it for conversation practice. It is basically set up only for conversational Chinese(and other languages), whereas with ChatGpt you need to explain to it what you want to do and as you said it gets cumbersome. Univerbal has two modes "guided conversation" and "custom conversation" where you can talk about any topic with any partner in any setting. There are various levels Beginner to Advanced. I have only been playing with it for a few days, but I like it a lot. The voice is smoother and more natural than GPT(but of course still does sound like a bot), the voice input is less clunky, and it is set up for Chinese conversation. So, I think I will use a mix of ChatGpt for doing non-conversation stuff(quizzes, stories, etc) and Univerbal for conversation practice. Seems to me a pretty powerful combination. 

 

I'm not going to go into all the details, functions and features of Univerbal, there's a bunch, you can take a look for yourself if interested and there's a free 7 day trial, here's a link: https://www.univerbal.app  It works much better on Chrome than on Safari. Maybe there are also similar Language learning AI's that are even better, I'd love to know if there are. 

 

Here's a few screen shots from a custom conversation I was playing around with. I told it : Me, store clerk, Uniqlo clothes store. It instructed me to carry out the 3 tasks on the right, and ticked each one as I completed it:

 

 

 

 

 

  

 

 

 

 

 

[lordsuso]

wow that looks amazing! I'll have a look thanks for sharing (but I think I'd rather pay for human tutors than AI)

[becky82]

I gave Univerbal a try (here's my video).  It is pretty cool!  I'd like to see what it's like in the future when it's more refined.

 

I tried using the voice input, but it corrects it as if I'm typing, so it corrects my ums and uhs and it doesn't add in the correct punctuation and then corrects that.  Still, it was good at pointing out better word choices, or when my pronunciation was imprecise, and it's fun hearing the sound when you say something correctly.  It seems to keep track of the words you use, like LingQ but for output, which is a novel feature I haven't seen elsewhere.

 

I was a bit annoyed at how much clicking I had to do with a lot of features (things like "click to continue"), and the font was rather small at times (and it couldn't zoom in much or the text would leave the screen).  And it didn't give me points, e.g., after completing a one of the "Progress" items, it stayed at 0%.  Maybe this feature isn't implemented yet.

 

It's not often your Chinese teacher is a magician and tries to teach you magic.

[Luxi]

On 7/19/2024 at 8:03 AM, lordsuso said:

but I think I'd rather pay for human tutors than AI

 

...for the time being, humans are a lot better for the planet too. AI voice uses much more power.

 

Quote

The artificial intelligence (AI) industry could consume as much energy as a country the size of the Netherlands by 2027, a new study warns.

 

[suMMit]

@becky82 I haven't had too much trouble with the Univerbal voice input. I avoid the ums and ahs and just go quiet until I think of what I want to say. If you click the microphone after speaking it will insert the punctuation, then click enter. If it has has inserted some wrong character you can just delete it out before clicking enter. I will give you that it does require a lot of clicks. I've only used the custom conversations so I don't know about the progress part. I've just figured out that if you specify a male interlocuter you get a male voice.

 

Getting back to your actual thread topic, I'd love more and better, more realistic voices. It would be awesome to be able to request : 70 year old with Sichuan accent.

 

It would be great if it could produce photos and pictures. Which AI does that BTW? 

 

As for what you mentioned re asking it about a textbook, I tried the same thing. I first asked it if it knew my textbook. It responded yes and described it. I then said could we talk about "unit 4 dialog 2" and it said it could not give me specific content from the book. But yeah, it would be great if you could download your textbook into it. Paid version can read pdfs, is that correct? 

 

 

[anonymoose]

I don't have much to add to the conversation other than that I have also noticed a vast improvement over the short time that ChatGPT has been around. As has traditionally been the case, it is often difficult to translate sentences naturally from English to Chinese or Japanese and vice versa, one, just because the structures of the languages are so different, and two, because it is often difficult to know what an appropriate translation would be without context. However, recently I have notice that the translations have become much better. I now use ChatGPT for most of my translation questions (rather than Google translate which was previously my go to translation service), and the best thing is, you can just write 'Parse and explain (paste your sentence here)' and it will break down and explain the grammar in detail. I don't think it can be overstated how much AI is changing everything.

[becky82]

On 7/20/2024 at 7:07 PM, suMMit said:

It would be great if it could produce photos and pictures. Which AI does that BTW? 

 

If you just want images, than Leonardo.AI is a decent free option (I've used veed.io for video generation).  If you want to talk with the AI about the images it generates, then ErnieBot (文心一言) can do this (this feature I inadvertently stumbled upon in this YouTube video), although it requires a Chinese phone number to use (and it seems the web version even requires a 身份证 [oh wow, I got the web version working with my green card]).

 

(It seems GPT-4o can do this too, but I haven't tested it.)

[cbaker]

On 7/20/2024 at 8:07 PM, anonymoose said:

I don't have much to add to the conversation other than that I have also noticed a vast improvement over the short time that ChatGPT has been around. As has traditionally been the case, it is often difficult to translate sentences naturally from English to Chinese or Japanese and vice versa, one, just because the structures of the languages are so different, and two, because it is often difficult to know what an appropriate translation would be without context. However, recently I have notice that the translations have become much better. I now use ChatGPT for most of my translation questions (rather than Google translate which was previously my go to translation service), and the best thing is, you can just write 'Parse and explain (paste your sentence here)' and it will break down and explain the grammar in detail. I don't think it can be overstated how much AI is changing everything.

Really agree with this as well. In addition to just 'Parse and explain' I also interrogate about certain words I'm not sure about. If I need examples, it gives me examples

[becky82]

I got ChatGPT to make a dialogue.  The audio it produces is just one voice though: it'd be great if it matched the voice to who is speaking.  I.e., two people in the dialogue = two distinct voices.

[ChristianCC]

Currently ChatGPT is not accurate enough to correct / generate natural sounding sentences specifically for learners. I tried using it for self-study using textbooks (marking grammar sections, sentences etc) and it was correct around 80% of the time. The other 20% it was blatantly wrong and either generated straight up grammatically incorrect Mandarin or said my answers were 'correct', when they were wrong. I cross referenced my findings with the answer keys and a native speaker. This may have improved in later iterations of GPT, but I have not tried it as I don't have a paid subscription.

[Luxi]

Here is one AI use I wasn't expecting. The Weixin Dushu  reader has very recently introduced AI to replace their in-App dictionary - which was rather bad and hard to read. You now highlight the character/word/phrase, click the search icon on the pop-up and get a massive amount of related information, in easy Chinese (no English, no pinyin...yet),with linked references, from an AI (not sure which). It appears as a dark-field pop-up, rather comfortable to read on an iPad. It also looks good in a Windows'browser Weixin reader - where you can use Zhonhwen in addition.

 

The text I've seen is quite readable and makes sense, the first paragraph seems enough to get the meaning (and is good language practice). It sure is an improvement on the previous rather messy dictionary.  Whatever hallucinations there may be, they can't be worse than my guessing unknown meanings. The one downside is, it probably will increase my reading time exponentially, at least until the novelty wears out.

 

[becky82]

This is the first time I've encountered ChatGPT suggesting I should ask a teacher and see what they think:

 

\