Transcrobes: Free + Open Source language learning platform (for Mandarin)

[alantin]

This is AWESOME !

 

 

@AntonOfTheWoods, I tried having the English definition there too, but found it a a little distracting right after the pinyin. I thought that it might even more awesome if was possible to place the English below the word while the pinyin stayed above it.

 

So it would look something like this:

[Jan Finster]

On 1/14/2022 at 12:14 PM, Jan Finster said:

Trying to test it with Deutsche Welle (https://www.dw.com/zh/武汉暴发疫情两周年-病毒起源陷入罗生门/a-60420872)

 

I have been waiting 10 minutes and I only see the "loading" icon rotating. Is this normal?

 

 

I tried it again, but I am back to square 1, the loading is rotating forever even though I am logged in at https://am.transcrob.es/#/ ?

[alantin]

@Jan Finster, it did that to me today too. I removed it from Chrome and reinstalled it. Worked after that.
The new settings for the glossings also appeared after reinstall.

[AntonOfTheWoods]

On 1/21/2022 at 3:00 AM, Jan Finster said:

tried it again, but I am back to square 1, the loading is rotating forever even though I am logged in at https://am.transcrob.es/#/ ?

I had proper error messaging but haven't properly reintroduced it after migrating to the new style of extensions that Google is forcing on everyone (for some good reasons...). That is a huge Pain and I will fix it by the end of the weekend for sure!

 

There are some caches that get loaded when you first start using the extension for a session (that Google clears very aggressively unfortunately) but even on slow hardware these should never take more than a few seconds (10-15, hopefully 5 or less) to load. If it takes more than that there is a problem.

 

I am also thinking about having a selection mode for external webpages, as any of the glossing options can make a real meal of menus and the like. That is something that will take a bit of effort and thought though but please be aware that you might find it ugly, particularly if there are lots of menu words you don't know yet!

[AntonOfTheWoods]

On 1/21/2022 at 1:15 AM, alantin said:

So it would look something like this:

Another great idea, particularly for advanced learners! I have added it to the issues (https://github.com/transcrobes/transcrobes/issues/20). When I get to it I would like the user to be able to choose from all of the options that make sense (like synonyms and Pinyin also, etc), with the option of putting them where they want. This is also related to another ticket, which is for the learner to be able to choose what info goes on the popup. 

 

There are a few urgent bugs and features that might take priority over this but I will get to it!

[yaokong]

I imported attached word list (imported as 6360a38f-51a8-409d-992d-fc1c2b5f32de_known-at-2022-01-21b.txt) 7 hours ago, it still says Processing "Requested" now. Is it normal for the import to take this long? Otherwise could you please check what went wrong? I obviously cannot use the service if I need to mark thousands of words as known (I did not learn according to HSK). Thank you!

known-at-2022-01-21b.txt

[AntonOfTheWoods]

On 1/22/2022 at 5:53 PM, yaokong said:

Is it normal for the import to take this long?

Hi Yaokong,

 

Sorry, I thought Gmail was putting replies into my inbox and didn't check my updates tab so missed this... Actually, you came across a case I didn't think through properly. Looking at the logs and db though, it looks like something went wrong with updating your interface - it should have shown an error after about 5 minutes. I'll dig into this too!

 

If you had have put a .csv extension, it would have imported correctly in a few seconds! Please give that a try while I fix the txt import and docs. The docs are not clear (or just plain wrong maybe...) on the txt imports.

 

Sorry about that!

 

The boring details...

I tried to import your file and sometimes it works, sometimes it fails. The "plain text" imports are going to be turned into something more useful, hopefully starting tomorrow (after adding a copy/paste text input screen). Previously .txt files were just treated as blobs of text that you can import and then get stats and lists from BUT it expects sentences. It failed because the analysis engine doesn't like really long sentences, and because you just had words it considered the whole file was just one sentence, and timed out. In any case, what you wanted was a CSV style import, which needs to have a .csv file extension. The docs definitely need improving here too!

[yaokong]

Hi Anton, 

 

Thanks for the detailed reply. I did manage to import as CSV, and created a list based on that import, setting "word knowledge" to known. I still had to reinitialise in the browser extension, and it works just fine now. I did notice some CPU (Intel 8700K) times of up to 40% (no other programs running, system otherwise idle at 1%), lasting for half a minute or even a minute on larger pages such as https://www.bbc.com/zhongwen/simp

 

I found 1 bug so far, sometimes (loading...) appears after an unknown word/phrase and the lookup eventually succeeds, but (loading...) is not removed, see attached screenshot. 

 

I will be testing your very promising service in the coming days and reported any issues I find.

[alantin]

I saw that "loading" issue too the other day.

[AntonOfTheWoods]

On 1/25/2022 at 10:41 PM, alantin said:

I saw that "loading" issue too the other day.

Yep, so did I :D. It is fixed in development but I have another MEGA change set that will include lots of new features and bugfixes coming soon.

 

While there is only one developer you can do this sort of thing :-).

 

So I am pretty happy with the copy/paste feature. You can paste up to 30k characters and it enriches it and gives you the same stats you get for imports/content. I have also transformed the .txt imports so that they now turn whatever you import into a "book". Basically it turns whatever you put in the .txt into HTML, and splits it into pages of a maximum of 30k characters. I definitely want to have lots more formats available later (and maybe even things like Word plugins, OCR image PDFs, etc.) but as long as you can copy/paste, then you should always be able to read/consume what you need to. I am also doing a lot of re-plumbing (migrating lots of stuff to redux toolkit) so it is taking longer than I thought (but this should make later stuff even faster to implement...).

[AntonOfTheWoods]

On 1/25/2022 at 8:00 PM, yaokong said:

I did notice some CPU (Intel 8700K) times of up to 40% (no other programs running, system otherwise idle at 1%), lasting for half a minute or even a minute on larger pages such as https://www.bbc.com/zhongwen/simp

 

Yeah, I am actually a systems (devops) guy, trying to do web UI development (and backend, and systems, and write a linguistics PhD!)... And I am pushing the limits of what is doable in the browser in many cases. There are heaps and heaps of optimisations but because I am doing everything myself, and teaching myself "modern web development" at the same time, I can't promise that everything will get done super efficiently, at least in the beginning. I am now starting to move things to a proper state/caching solution (redux toolkit) so hopefully I will be able to improve this soon. If you know any super smart web developers who are bored and would like to learn Chinese...

[AntonOfTheWoods]

Hi everyone. I have just pushed an update. You will get logged out and you may need to refresh a bit if anything strange happens. 

 

It might seem to be a relatively minor update but I have actually been working feverishly for almost 3 weeks now, and have rewritten a substantial portion of the user interface code. All the code to do with the popups and text enrichment was written before I knew how to write modern javascript (typescript actually), so was not very well written and was making adding new functionality much more difficult. So I did some *major* spring cleaning, and now everything is much, much better written.

 

The new copy/paste feature (Textcrobes :-D)  seems to be working quite well, and I have harmonised the reader configuration on all the interfaces (boocrobes, moocrobes, brocrobes, textcrobes). 

 

I will now (finally!) get on to the overall stats screen, the exporter and the other suggestions. Now I am much happier with the overall state of the user-facing code, things should be much quicker to implement and iterate on!

[AntonOfTheWoods]

On 1/25/2022 at 10:41 PM, alantin said:

I saw that "loading" issue too the other day.

This should be fixed now.

 

I spent the last few days fixing little bugs that (re)appeared after the rewrite but I can't guarantee that there won't be a few that I missed. Please be patient and let me know if you see any errors. I'll fix them immediately!

[alantin]

Sounds good! ?

 

On 2/14/2022 at 6:37 PM, AntonOfTheWoods said:

The new copy/paste feature (Textcrobes :-D)  seems to be working quite well, and I have harmonised the reader configuration on all the interfaces (boocrobes, moocrobes, brocrobes, textcrobes). 

Btw, what's up with the names? What is a "crobe" anyway?! ?

[AntonOfTheWoods]

On 2/15/2022 at 1:30 AM, alantin said:

Btw, what's up with the names? What is a "crobe" anyway?! ?

 

Yeah, I settled on the name well before Covid came along (2018, though I was in Yunnan at the time... ?) and decided not to change. It is actually a (reasonably) obscure sci-fi reference, coming from "microbes". The original reference is conceptually what I was hitting for, though obviously the original version is a little more sophisticated still!

[AntonOfTheWoods]

On 1/25/2022 at 8:00 PM, yaokong said:

I did notice some CPU (Intel 8700K) times of up to 40% (no other programs running, system otherwise idle at 1%), lasting for half a minute or even a minute on larger pages such as https://www.bbc.com/zhongwen/simp

@yaokongI hope to have significantly reduced the CPU usage (and network chattiness) at the cost of having slightly slower stats updates. Previously the server was pushing back lots and lots of small updates that the local database needs to reintegrate as soon as possible, which takes quite a lot of both CPU and network. Because these push updates aren't super time-sensitive, I have turned them off and just let the system pull update once every 10 minutes. Please let me know if this reduces CPU usage!

[AntonOfTheWoods]

Hi all. I have added a new stats screen "My stats" (I'll probably change to Statscrobes though :-D).

 

For the moment it has:

- Cumulative progress on words and characters known

- Daily words seen and checked

- Daily words revised in Repetrobes (or via the smiley buttons anywhere), with both nb of successes and re-revisions (aka failures :-))

 

The day stats are only being collected as of now, so you will only start to see stats appear over the coming days. It takes up to the last 6 months in data, though before we get there, there will definitely be a much more advanced stats screen that allows you to drill down into any aspect you want.

 

This was just a first go to get something out for feedback. So any feedback is more than welcome!

[yaokong]

After the update the extension stopped working correctly on some pages. E.g. I cannot get it to work on this site, it will only add glosses to the title line and the blue links in the second line, but not the main article text (see screenshot). Even after reloading the page and reactivating the extension it hangs at the same point. 

 

It works correctly on bbc.com/zhongwen (though the loading issue can still be reproduced, i.e. for some words it displays loading and that never disappears. e.g. "欧康纳(loading...)(O'Connor)")

 

CPU usage improved indeed, I hear the fans kick on less often now when using the extension.

 

[yaokong]

I just had an issue on another page (containing the same text), so I reloaded the page and activated the extension. It still only loaded glosses until about half of the page then fully stopped, so I again reloaded the page, that (assumedly temporarily) broke the extension on that particular page: nothing happens if I activate the extension, not even if I close the tab and reopen it. I will have to restart the browser now. Update: after restarting the browser the extension worked just fine on the same page, until the very end of the page.

[AntonOfTheWoods]

Thanks for the update @yaokong and sorry about the bugs. I was struggling with user interface code for a long time but now am starting to find my stride so the quality of the user experience will be a major focus for the next phase. There are some more changes (including the "loading..." fix) already in the pipeline for the extension (they take sometimes several days to approve new packages), both visual and in terms of performance.

 

The main site (am.transcrob.es) has had some serious updates already. I would be interested in what you think of the new stats interface, and any additions you might be interested in having!