Learning with texts

[laurenth]

Hello,

I did a quick search and it seems there is no thread about a tool I've been using for a few weeks now. It's called "Learning with texts", more info here: http://lwt.sourceforge.net

Basically, you load texts (and audio) into a database (copy/paste from any source), and every word of your text is displayed according to its "learnt" status, based on texts you've read previously. Click on one or more words, and you can search them in up to 3 online dictionaries, set their status, etc.

You can define your language, what dictionaries you want to use, how you want to be tested, etc.

The "test" system seems to be a kind of SRS, though I don't know if it's the same algorithm as used in Anki, Mnemosyne and co.

In a way, it may be more useful than other SRSs since the words you want to learn are displayed in the "natural" environment where they were first encountered, though there are several testing formats.

The program runs on a web server, so you have to install one locally or on the site of a web server provider, which is even better for those like me who want to do some Chinese on several computers, while commuting, etc.

I'm toying with the idea of limiting my use of Anki to character lists or specific projects, while spending more time in LWT to learn new vocab.

Maybe it's old news for some people here, but I think this program is so interesting that it deserved a thread.

L

[Guest realmayo]

Just had a look at the demo -- does it recognise words of more than one character as being individual words, or is it just character-by-character?

[laurenth]

Yes, the program recognises muti-character words. Click on a character, and there's a popup that allows you to extend the expression to up to 9 chars (and to select your dictionary).

[creamyhorror]

I'm interested in starting to use this program, but I read in a review that accessing it on a remote webserver might be slow. Does this mean the frontend isn't AJAX / written in 'live' Javascript? That's kind of a pity, since it's a perfect candidate for AJAX.

(Another really sweet feature would be seamless transfer of words/definitions to Anki. But that's probably asking for a bit too much...)

If anyone has experience in these areas, I'd love to hear about it. How do you use the app while on the go, or a location besides your main computer?

[LWT_Project]

I am the developer of LWT - Learning With Texts - http://lwt.sf.net .

LWT is a web application, so you need some kind of WAMP/LAMP/MAMP application (like XAMPP, EasyPHP, or for Mac: MAMP) installed, either on your local machine or at a web hosting service.

LWT stores its data via MySQL in a database, uses PHP via Apache to implement server logic, and uses JavaScript very heavily (via JQuery and AJAX in some cases) to implement "dynamic" stuff, that runs locally.

You can upload your current vocabulary easily via the upload function.

You can download all your LWT terms easily in TSV format or a special Anki format (suitable for use with a supplied Anki template).

I and many others (100 downloads of LWT per week) use LWT for many languages, not only Chinese. Enjoy, it's free!

[creamyhorror]

Great, thank you for the quick response. So the issue of "slowness when used remotely" is a red herring, then? Whew!

(Here's the bit about speed being sacrificed, from this review

If you are willing to sacrifice speed, you can find an online, already set-up instance of LWT for free on the fluentin3months webpage.

Anyway I'll try to get it set up on my webhost. Thanks for this awesome software!

[querido]

Hmm... for those who don't want to install the server, it looks like Benny is hosting one now.

Edit: I see creamyhorror beat me by two minutes. :-)

Edit: Installing the server was not the hardest part; there were other bits to set up too (the first language setup). But sorry, I don't have time to post about this again today.

[laurenth]

I read in a review that accessing it on a remote webserver might be slow.

I had such a problem, It was not due to LWT but to the webhoster (000webhost.com). Some statistics tracking code had been inserted into the page, with the result that my site was so slow that it was all but unusable. After I disabled that tracking code, everything was ok. You may not have such a problem with another hoster or, obviously, if you run a local server.

Setting up LWT for mandarin is not difficult. I can share my configuration page if you want, or there's a shared Google doc floating around with configuration tips for several languages.

Basically, under "My languages", under Dictionary 1 URI copy/paste e.g.: http://m.nciku.com/en/zh/detail/?query=###

under Dictionary 2 URI, e.g.: http://www.mdbg.net/chindict/chindict.php?page=worddict&wdrst=0&wdqb=###

under GoogleTranslate URI: http://translate.google.com/?ie=UTF-8&sl=zh&tl=en&text=###

under Make each character a word: Yes

under Remove spaces: Yes

And that's all.

[scoff]

Just had a look at the demo -- does it recognise words of more than one character as being individual words, or is it just character-by-character?

I tried LWT a few months ago and found manually segmenting the text to yield seriously diminishing returns (especially when I wasn't familiar enough with the text to know exactly where I should be segmenting).

The solution was to use something automated that would segment the text and insert spaces between words (I use DimSum). DimSum's segmentation isn't perfect but one of the nicer things about LWT is that you can manually choose which characters make up the "word/phrase" you are interested in learning.

In the LWT configuration I use, manually creating words out of characters that were not segmented properly makes LWT consider them multi-word terms (which is apropos for Chinese I guess). There are two ways to display these multi-word terms, and I've inserted two screenshots at the end of this post. I prefer the first (lwt.showall0.jpg) way.

[c_redman]

The lack of a built-in Chinese word segmenter is a real drawback. Even though you can manually define every word in a text in terms of multiple characters, it's quite tedious to do so. In that respect it's very similar to LingQ's handling of Chinese, at least when I tried it years ago. In theory, this kind of vocabulary management tool is supposed to make it easy to keep track of vocabulary by assuming everything is well-known that you haven't marked. In practice it's labor-intensive and hard to stay enthused after about 10 minutes of manually defining words and not making a dent in the text. Speaking of which, a built-in dictionary would also be a useful addition, so that the definitions would already be there, minimizing the extra typing.

It's great that it's open source. If I ever have the chance to add those features, I will send you a patch.

[LWT_Project]

Cit.: The lack of a built-in Chinese word segmenter is a real drawback.

You may ...

use a program like Wenlin, use "Make transformed copy" > "Segment Hanzi", and replace the " | " with a single space. Now import the result into LWT (while setting the LWT language option "Make each character a word" to No).

... or ...

set the LWT language option "Make each character a word" to Yes and import a not too big dictionary with the most important words (max. 10,000 to 20,000 words/expr.) like the list of the (new) HSK6 words or the big list of the highest level of the old HSK. e,g, from here: http://hskflashcards.com/download.php -- Now the most important words are automatically recognized by LWT. If you unset SHOW ALL you will not see the single characters and the numbers of the multi-character expressions.

Re free webhosters like 000webhost.com:

Not recommended (although shown in the documentation) due to advertisements and slowness.

[querido]

Thank you laurenth and LWT_Project.

I can't find a simple enough summary of the security issues, config settings, etc. with regard to running the server.

I'd like to verify that these settings are right for locking it down to *just this application*.

Would someone who sets this up, and who knows what he/she is doing, please comment on this?

[LWT_Project]

@querido

I have installed XAMPP, EasyPHP and MAMP as local "web" servers with the default settings. These personal web servers are only accessible in your local network (behind your router's firewall) so there is little or no risk if your firewall is configured correctly. It is difficult to say something about security risks in YOUR environment. Please refer to the help pages and forums of the package you have installed.

[laurenth]

The lack of a built-in Chinese word segmenter is a real drawback.

I use LWT to learn both single characters and multi-character words, so like the way it works.Y et, sometimes, I'm not sure how to parse a group of words; that's why I use mdbg as my second dictionary: it does a nice parsing job when you input a string of chars.

Re free webhosters like 000webhost.com:

Not recommended (although shown in the documentation) due to advertisements and slowness.

Can be annoying, yes. However, for me, the possibility of using LWT from different locations, including my phone, is a big big selling point.

L

[陳德聰]

Is anyone still using this tool? Was thinking of this as an alternative to imron's CTA in the interim while I wait for a magical Mac version to come out. But I am a bit put off by comparisons to Lingq, since that was an unwieldy mess back in the day.

[安笛 {Andy}]

@陈德聪, Yup, I am, and it's still being updated.  Like the first poster, I'm wondering how to use Anki and LWT together. I mean, what to use Anki for and what to use LWT for.

[imron]

Was thinking of this as an alternative to imron's CTA in the interim while I wait for a magical Mac version to come out.

The magical Mac version is now out

 

I'm wondering how to use Anki and LWT together. I mean, what to use Anki for and what to use LWT for.

I can't speak for LWT, but for CTA at least it can be used as a feeder program to something like Anki - that is, it can be used to analyse Chinese texts and then generate lists of words you should learn.  You then you can use Anki to do the actual studying/revision of those words.

[vivea]

Is anyone here using LWT? I installed it and set it up, but it won't use colors to highlight the words I add to "the terms". There's no pop-up translation of those words on mouse over, either, after I've added it.

 

I checked the http://lwt.sourceforge.net/testdb/, and I've got the same problem with it. So maybe this is a problem on my end or I just don't understand something about using LWT?

[vivea]

Forget it. I've found a standalone tool that copies LWT perfectly and provides no security risk, as you don't have to run a server to use it. It also happens to highlight words with colors, so no complaints anymore.

https://sourceforge.net/projects/fltr/