Is Mnemosyne unstable?

[animal world]

Disaster struck in Mnemosyne Land over the weekend and, to my knowledge, i didn't do anything to cause this calamity. On Saturday morning i had manually added a handful of cards to my "Character" category, revised the Answer on a number of other cards and then exported this category to my hard-drive (and by doing so overwriting the previous version there A bit later i did my drill of that category of 100 characters and everything was fine.

When i was done, i happen to look at the bottom right corner where, to my shock, i noticed that the number count had gone up from 2300 to 3300! I rushed to the database (Edit Deck) and noticed that all the categories were fine except for the Character one. There a screwy duplication had occurred in a random fashion. Every three, four, five or six cards a card had been duplicated. It would be shown normally and right below it both the character and the Answer would be repeated in the left side column (i.e. the Question side) and to the Answer side was pushed the character originally on the line together with the Answer that was already there.

It gets stranger still (imagine the music of the old Hitchcock shows here)!

The data wasn't at all garbled and when i undid the damage for a few cards, i noticed that the high count number for this category did NOT go down accordingly. But then when i went back to the Edit Deck i noticed that those corrections had been duplicated as a new line with the Character on the left and the Answer on the Right. But sometimes ... the answer was old, i.e. something that i had revised weeks ago!!!

Because i didn't want to lose the cards' rankings and since the last exported file showed the same garbage, i made all the corrections manually. This took five, mind-numbing hours! I was not a happy camper...

Everything is fine now but i'm still baffled what might have caused this. A few weeks ago, the system made other changes unbeknown to me but they were merely cosmetic and no big deal. Right now, i have about 8,500 entries in Mnemosyne and this database is bound to grow and grow. So, i'm a little apprehensive at this point.

My apologies for this verbose post about this but i hoped that a) another Mnemosyne user might shed some light on the cause of this problem and B) to warn others about these potential pitfalls. If Mnemosyne is indeed unstable i'll migrate to another SRS program that is more reliable.

Any ideas? Any recommendations?

[renzhe]

I don't know what might have caused your problem.

I've been using Mnemosyne for about 3 years now, and I've never had any stability or corruption problems. It's one of the most stable programs I've used.

It's still a good idea to always back up your database (as .mem file or export as .xml) before doing any changes to the database.

You could look into your .mnemosyne/backups directory too, when things go crap. There should be a recent backup in there. Might be a couple of days old, but you won't lose all of your scheduling information, and it would take less effort to recover.

[querido]

Mnemosyne has always been perfectly solid for me, but no matter how stable the program a hard drive could crash at any time. Also, in my opinion, you are unlikely to find a program with similar capabilities that is *more* stable.

I keep backups on CD/DVD.

[animal world]

It's good to hear that other people consider Mnemosyne stable as i certainly like the program. Just wish i knew what caused this weird accident. It also seems that Mnemosyne keeps some sort of garbage can because when it began to duplicate entries, at times (again totally random) it spat out entries that i had revised weeeks ago!

I tend to tinker with the Edit Deck on a daily basis. For instance, if a certain character gives endless problems, i'll revise the answer by adding more words that contain this character or a note saying not to confuse it with character x, y or z, etc. I've been good at backing up my precious Mnemosyne Database but will need to train myself to first have a look at the database before backing up to make sure that it doesn't contain a lot of garbage.

The answers to some of my entries can be quite long (occupying 5, even 10 in a few cases, lines in the Mnemosyne window). That shouldn't have caused the accident, should it? The duplications were quite random, short entries and long, detailed entries. I've looked and looked but couldn't figure out the problem

Renzhe, i believe it was you who told me that Mnemosyne slows down once the database has over 10,000 entries. How many do you have in yours and how is its speed?

[querido]

It also seems that Mnemosyne keeps some sort of garbage can because when it began to duplicate entries, at times (again totally random) it spat out entries that i had revised weeks ago!

(amateur python programmer): It doesn't have a garbage can. I think the above would be evidence that the original error (corruption of the database?) happened weeks ago, maybe.

will need to train myself to first have a look at the database before backing up to make sure that it doesn't contain a lot of garbage

(edit: The following refers to backups generally, but as renzhe said, usually ~/.mnemosyne/backups will have what you need. End edit.)

Yes, but that could get labor-intensive, and you might miss something. Make it easier on yourself by backing up by date, keeping multiple copies, because if you do find garbage you won't be certain that the *previous* backup is the fix because you might have overlooked the garbage *last time*, right? (This applies to backups generally.)

I tend to tinker with the Edit Deck on a daily basis.

Me too.

Although this shouldn't be necessary, and I'm not sure it's relevant here, it might be a good idea to do File...Save after editing the deck. I sometimes do File...Save in the middle of a big session, because in the event of a power failure, etc., the day's progress would be lost.

Just wish i knew what caused this weird accident.

I didn't want to blame you because ideally there would be perfect safeguards. But, a power failure or shutdown with mnemosyne open, with unsaved changes, could cause corruption. Aside from hardware error, that would be my guess at what happened to you.

Edited November 10, 2009 at 04:28 PM by querido

[animal world]

Thanks, Querido. I don't believe the corruption happened weeks ago as some of the entries that were duplicated showed recent edits. In other words, the duplications were all across the board without rhyme nor reason, as far as i could figure out. You're probably right that a power failure occurred and from now on i'll do Fill Save very, very, very frequently.

BTW, is there a forum dedicated to Mnemosyne anywhere? Am starting to feel slightly embarrassed by always posting my silly, little questions about Mnemosyne here

[querido]

Link to mnemosyne group!

They're working on version 2 now.

Too late for version 1, but now would be a good time for new ideas for v2.

[roddy]

BTW, is there a forum dedicated to Mnemosyne anywhere? Am starting to feel slightly embarrassed by always posting my silly, little questions about Mnemosyne here

No problem as far as we're concerned - and you seem to be getting answers, so you're obviously not the only person here using it. You might get more help on a dedicated forum I guess, but you're certainly welcome to post here.

[renzhe]

Yeah, the mnemosyne mailing list is definitely a better place when it comes to obscure bugs, though there should be enough Chinese-specific know-how on this forum too.

Renzhe, i believe it was you who told me that Mnemosyne slows down once the database has over 10,000 entries. How many do you have in yours and how is its speed?

I actually have my cards split across 4 databases: Characters, words, traditional, writing.

I prefer it that way, though the Mnemosyne author recommends keeping everything in one database.

My biggest decks are characters (4,000+) and words (~7,000). It takes a couple of seconds to load a deck, but usage speed is OK. Then again, I have a really new and really powerful computer.

On my old laptop, processing a card had a noticable delay, almost a second, though.

The next version will use a proper database, so things won't get slower.

[animal world]

Roddy, i have certainly been getting good answers here about Mnemosyne as i have on anything else (This is truly a GREAT forum!) . I merely wondered that on a Mnemosyne-specific user forums answers to my questions might be readily available as FAQ or in existing threads.

Renzhe, do you mean that you have four CATEGORIES within the ONE database? I like to have a number of different categories that i recombine when i've thoroughly mastered the words in a specific category. I find it makes the initial learning much easier. So far, the speed of Mnemosyne has been great for me.

I'm considering a new approach to reading articles, or parts thereof. Until now, looking up words and developing a word list would take up most of my reading time. At the end of the article i would have the feeling that i sorta "got it." Usually, i would also end up feeling more or less exhausted, half-dead and with a headache. What i am planning now is to select pieces for reading, quickly scan them, and do the old looking-up-in-the-dictionary and developing word lists trick. Then i put the pieces away and import those specific word lists as separate categories. When i know those words inside and out, i recategorize the words into the general word category. Only then, will i read that article for real. It should make it easier to recognize the words (I better!), focus on the sentence structure and the grammar and, hopefully, pick up some subtleties for which in the past approach i was just too exhausted to truly care about. This approach might also make reading become more enjoyable.

Querido, thanks for the link!!! Before saying at that forum a peep about the instability problems i have experienced i will first do lots of Saves as i'm doing my usual tinkering that i hadn't done until now. I certainly have some suggestions for v.2. For instance, i would like to be able to refine the search of the database, such as "only search in category X" or search "whole words exactly as requested." I also would like a feature pretty much saying "Beat me over the head with this difficult character ad nauseam because i sure keep forgetting it..."

[renzhe]

Renzhe, do you mean that you have four CATEGORIES within the ONE database?

No, I have 4 mem files.

Each one has sub-categories.

[animal world]

No, I have 4 mem files. Each one has sub-categories.

Good grief! And these you import/export every day??? Having read many of your posts, i have a good idea where you're at with Chinese. Assuming you do your Mnemosyne religiously on a daily basis, how much time does it take you? Just curious what the future's going to hold for me...

[renzhe]

No, I just use File->Load when I'm finished with a deck.

You import/export .xml files, the .mem files are Mnemosyne's internal representation. Officially, it's recommended to throw everything into a single file and then turn categories on and off, but I don't like that approach.

I try to limit my Mnemosyne sessions to around 30 minutes, maximum one hour (with distractions), otherwise my concentration fails. Nowadays, I only need 10 minutes or so, because I'm in a maintenance mode. The rule of thumb is that if the review takes too long, stop learning new items for a week or so and only do review. This reduces the load in the future.

Yesterday, I had ~40 words for review (out of 7000), ~40 characters for review (out of 4000), 5 characters to write (out of 1000) and 3 traditional characters (out of 550).

When I'm actively learning, I try to limit the characters and words to 60-70 each (for review), and the other two categories are usually light because they are not a priority.

[animal world]

No, I just use File->Load when I'm finished with a deck.

I don't see a Load on the Mnemosyne toolbar.

Renzhe, i believe it was you who reported that Mnemosyne becomes slow once the database exceeds 10,000 entries. That's not a lot and i'll definitely have that issue next year. Is that the reason you have four databases? How exactly and where do you name your various databases when you export them or whatever you do with them? So, at any one time you have 0-1 database running in Mnemosyne?

Will a card's ratings only be preserved if the card has been exported as html?

I'm just trying to anticipate future problems and how i might want to change my method now with an eye on future expansion of my happy Mnemosyne family.

[querido]

I hope no one minds if I interject. I'm eager to answer because its an easy one.

If you go File...Open, you should see one or more .mem files in there. (If not, navigate to your .mnemosyne directory. They should be in there.) Then you "Open" the .mem file, just as with a text editor you "open" a text file- just like that. These .mem files are databases. Did you know that? Open another one- just like opening another text file. Go back to the previous one, and it is just as it was before, scores, schedule, etc.

See File...Save as...? That's how you save your current database with a different name. You can also just navigate to your .mnemosyne directory with your file browser and do a rename too.

Doing the above we are not importing/exporting, we're just opening/saving/renaming existing databases, with all of the data safely inside the .mem file. Does this answer this part of your question?

[animal world]

Thanks, Querido, for your help! I knew about the .Mnemosyne file and back up the database in there to a CD periodically. In addition, i back up my "categories" whenever i make changes to them as text files to my hard-drive and then also email them to myself.

The first day i used Mnemosyne, i had a bad snafu with exporting and since then have been overly cautious. I knew Importing and Exporting and didn't seem to have much need to use anything else, let alone figuring out what you can do with those features.

[renzhe]

If you use Mnemosyne the default way, you will only ever have one database, and won't be using the Save and Load functionality. There's nothing wrong with this.

For historical reasons, I prefer opening separate databases. And yes, spreading your data like that will help with the slowdowns once you get too many cards, as you will only have a part of them loaded at any time.

On the other hand, Mnemosyne 2.0 will be out some time next year, and it will have a proper database backend and scale to millions of cards without problems, so if you're not having problems now, it's not really an issue.

[querido]

Hey... if you want to see for yourself any slowdown with version 1 with a big database, go to mnemosyne cards download and download the "20,000 Chinese sentences" collection! (If you do this, remember to be patient when importing- it could take some time.)

Having had up to 4500 cards, I've never actually noticed any slowdown.

edit: I also keep only one database, as the program author recommends. end edit.

[animal world]

As a precaution, i downloaded those 20K sentences in Anki and it's the only thing i have there. For kicks, i downloaded all of them and, thanks to many duplications, there are actually over 22K ! Anki works fine but somehow i prefer Mnemosyne.

Right now, i have about 8500 entries in Mnemosyne and everything works smoothly.