Manage Anki reviews, double productivity: like a boss!

[nicostouch]

Hey guys,

I wrote a blog post about my tips and experiences on managing Anki reviews from my 4 years learning Japanese.

Whether you're a total Anki noob or a seasoned pro it's got something for everyone so please

check it out and if you like it, share it with people you might think it will help.

http://supernewsoftware.com/reviews.php

[icebear]

In the future I'd suggest posting your link but also quoting parts/all of the article. This ensures everyone takes a look at the content and also is one step towards making your post look less self-promotional.

Manage Anki reviews, double productivity: like a boss!

Admin Edit: Actually, T&C's ask people to keep offsite quoting to minimum (with a preference for links), and only quote relevant parts rather than whole articles.

Anyone who's interested in reading more can click the link, or not otherwise.

FWIW, the OP should also note that posting just to say: "hey I've written a new blog post" is generally frowned upon. That's what RSS is for.

The best advice I've read on keeping flashcards manageable is purging your deck once you find it becoming burdensome. Once you're at a intermediate and higher level you should really be tackling native reading or A/V material for 30 minutes or more a day, which should more than handle all the SRS needs of common words. Add words you genuinely don't know, but don't bog down your list with plenty of easy words you do know.

Another tip, which I figured out on my own (but surely others know as well): I end my session once I've failed 10 cards. What's painful in flashcarding isn't the reviews that go quick and mostly smooth, but the recap at the end of all the fails, which is especially brutal if you allow yourself to have dozens and dozens of new words per session. Quitting a session once you've hit 10 fails (which will then be reviewed, in Pleco at least) makes me feel as though I'm keeping the new/hard material at a reasonable level. I also will do 3-4 sessions throughout a day, but not more, as this avoids adding too much too quickly (I add 5 new words per session).

[Guest realmayo]

The best advice I've read on keeping flashcards manageable is purging your deck once you find it becoming burdensome

I've kind of done that once, in that I reset my main deck of 12,000+ Chinese->English words and then restarted with them all as new cards, and ended up suspending or deleting around 10% of the deck as unnecessary-for-now. That was because I'd started "cheating" a bit -- when I'd been away for a while and the reviews stacked up I'd eventually just crack and hit "yes" to all 800 or so of the cards that I couldn't face answering, and after doing that a few times there were lots of cards with big intervals which didn't know.

From that experience I guess my advice would be to advise that you never do that "yes to all" thing if you do get behind a lot: instead set a gentle and realistic goal to get down to zero within, say, 10 days and no earlier. I've never seen the point of deleting an entire deck but as you say some people think it's a great idea.

I don't mind having common words in my deck: it only takes a handful of reviews and their intervals are pushed out to three or four years. But I wouldn't go to the trouble of adding a common word that I already know extremely well but for some reason had never put in my deck.

Edit: should add that the most useful thing I've found is to ruthlessly suspend leeches and after a while dribble them back into the deck as new words.

[icebear]

From that experience I guess my advice would be to advise that you never do that "yes to all" thing if you do get behind a lot: instead set a gentle and realistic goal to get down to zero within, say, 10 days and no earlier. I've never seen the point of deleting an entire deck but as you say some people think it's a great idea.

To be clear, I mean deleting an entire deck and starting from scratch, not merely reseting the scores of all words. The idea is that as a deck grows a majority of time is consumed by words that shouldn't be in there, either because they are too easy (not worth your time) or too hard/obscure (again, not worth your time). The reason for these words getting in is because most people, myself included, are not horribly discerning when adding words, especially at the middle level where consuming media is like drinking from a firehouse of new vocabulary.

Deleting everything and restarting would be similar to your suggestion of closely monitoring for leeches, although is better suited to someone that is lazy like me about deck maintenance. My threshold is that the time spent on flashcards shouldn't exceed the time spent on native content, reading or (not AND) video, in a particular day. For me that means I should never spend more than 30 minutes in a day on flashcards, and it seems around the 4-5k word mark I hit that limit and do a reset.

[imron]

As a major proponent of the delete everything and start again approach, I agree

It saves time on deck management, and as long as you are exposing yourself to enough native content through other means (reading, TV, radio etc), you'll still be doing 'repetitions' of those words anyway - but in a real life situation.

[Guest realmayo]

Can guarantee an Imron post on this topic! Only Wubi is a bigger certainty. And yes it's one approach, sure. My alternative is to suspend/delete cards that I often forget and seem unnecessary to learn.

I have two problems with the "real life as SRS" idea. First, as mentioned before, is that my reading of frequency stats suggest it can be months and months before you come across a not-very-rare word. And second, related, if you do come across that word after two months and forget it, then you probably won't see it for another two months, so will probably forget it then too.

SRS means that if you forget a word you'll see it four or five times over the next two months.

Real life: two steps forward (lots of reading), one step back (a fair bit of forgetting).

SRS: 1.5 steps forward (less time for reading because of SRS), half a step back (less forgetting).

[imron]

SRS means that if you forget a word you'll see it four or five times over the next two months.

I don't really see how we're in disagreement on this point.

I don't say never put new words in an SRS/flashcarding program, I say delete cards after the deck starts to become too large (i.e. more than about 30 mins of reviews a day).

By the time the deck is that large, many of the cards will have been seen several times over the course of a few months, and be at the point where the interval has been pushed out to 2-3 years. That being the case, it makes no real difference whether you delete them or not, because if you don't come across that word again in 2-3 years of reading/other exposure you are either a) not doing enough reading or b) the word is not very useful to you in the first place.

My experience with reading has been that even quite obscure words turn up over and over again in the most unlikely places. If real life reading is causing one to take steps back, then to me that indicates a) also not enough reading or b) not drilling the word correctly in the first place.

The second point is actually quite a common problem with SRS, because people tend to get lazy and rely on the SRS algorithm to do all the remembering for them, and so there is a tendency to mark as correct words which are only partly learnt (by partly learnt, I mean any word that you can't recognise more or less instantly in a real-life situation).

Ironically, one reason for not spending more time per word is that it dramatically expands the amount of time required to do revisions, which becomes impractical to do with larger decks. Deleting the deck solves this problem easily, and by keeping the decks to a more manageable size, it allows more time to be spent per word to get it to the point where you're not doing a fair bit of forgetting through real-life reading.

[Scoobyqueen]

This ensures everyone takes a look at the content and also is one step towards making your post look less self-promotional.

Is there a product to sell? In that case, it is better to be transparent.

With regards to the headline...actually the hardest thing about being a boss is managing people and get them to produce :-)

[icebear]

I have two problems with the "real life as SRS" idea. First, as mentioned before, is that my reading of frequency stats suggest it can be months and months before you come across a not-very-rare word. And second, related, if you do come across that word after two months and forget it, then you probably won't see it for another two months, so will probably forget it then too.

I suspect the difference of opinion is fundamental then - I know I reinforced most of my mother tongue vocabulary through absorbing natural material, and where possible hope to do the same in Chinese. I admit acquiring new material is accelerated/bolstered by SRS, but I personally see it as only a means and not the end.

[icebear]

Imron - out of curiosity, how large does a deck grow before you feel its time for a fresh start?

[icebear]

Is there a product to sell? In that case, it is better to be transparent.

The two items in his signature, one of which is erotic. Niether is strictly for sale, but I do think the post was made for self-promotion - of his blog or the services available there. Just my opinion.

[Guest realmayo]

but I personally see it as only a means and not the end

Icebear, do you think your view is unusual? Because I can't think of anyone who sees SRS as anything other than a means to an end.

Real life reading isn't SRS! It's real life reading. Real life reading is much more important than SRS

[Scoobyqueen]

Icebear - erotic reference..? I completely missed that. I guess the double productivity might also be heading in that direction. Sounds like this is a service to sell, you are right.

Looking at the write-up about how he learnt another language there is some self-praise going on there so you could be right abou the self-promotion too.

[icebear]

Icebear, do you think your view is unusual? Because I can't think of anyone who sees SRS as anything other than a means to an end.

No, I think my view is standard. I do think many people, myself included, lose the sight of the forest for the trees and end up spending too much time on SRS, and especially pay too much attention to the number of items they have in total...

[Guest realmayo]

Imron, I was referring to what happens when you forget a word. If you don't SRS, and you see a word in a book that you've forgotten, you look it up, you realise it's one you should already know, maybe you kick yourself, then you move on. If this word is not super-common then it could be a couple of months until you next see it.

But obviously with SRS a forgotten card gets reset so you'll see it several times over a few weeks. I'm suggesting that this more frequent re-seeing of a forgotten word will help you re-learn it long-term better than if you don't see if at all for a couple of months.

I also think that relying on a feeling that "gosh I only learned that word a few weeks ago and here it is again" doesn't really prove anything, it's one of those coincidences that the brain makes a big deal of like meeting someone you know randomly in a big city, but statistically it's not all that surprising that it happens sometimes, it's just that when it does happen it feels like a big deal. Sometimes you'll flip eight heads in a row. But there are lots of people who you don't bump into in the street ever, similarly the majority of words are not going to reoccur any more often than that word's placing on a frequency list would indicate.

From an older post, I found a frequency table that said the English word "uneasy" has a frequency of around one per two million. War and Peace has 500,000 words. So, it's a lot of reading in a foreign language before you see the word again.

When I referred to one step back I was referring to the natural process of forgetting words. People do forget words and if all you do is read Chinese for 30 mins a day and then look up any words you don't know, it will take a long time for the lower-frequency words to stick. That's why people historically have tried to make time to learn those words, for instance writing a short list and then spending a few minutes testing oneself on recall. SRS is another method. But look, I'd never suggest not reading. And you are not suggesting only reading. So yes, we're in agreement.

And I think your broader point is absolutely correct. If spending too much time on SRS is taking up too much time, preventing you read etc, then you need to find a way of limiting the time you spend on SRS reviews. One method is, as you say, to cut the deck. My method is to add fewer new cards on a daily basis.

[nicostouch]

I think it's an interesting topic, trying to find the right balance between SRS and actually engaging in the target language. I found the law of diminishing returns really start to kick in when my vocab deck hit 15,000 words. Every word after that it was like "sure I'm not going to forget this if I SRS it and I feel that every word no matter how rare is important but... At this point is the return really worth the effort?" Up until that point it definitely was but after that point natural SRS is a far better strategy because if you are spending a few hours per day reading/listening with a very high level of comprehension that serves to drill in and cement those native like phrases and expressions.

[Olle Linge]

I did some rough calculations on how much time I spend reviewing various sections of my 22 000 words/characters deck and it seems like I spend less than minutes per day reviewing cards with intervals over one year. This time is usually spread out through the day: waiting for the bus, waiting for a friend, waiting for the elevator.

Simply having many cards is not an argument for deleting your deck and starting over. The cards that take up time are those that you tend to forget, sometimes many times over (leeches). In general, I would say that it's meaningless to do much with cards with long intervals. What we really should care about is making sure that the cards we spend much time on are actually worth that time. Thus, I don't think deleting the entire deck is a good idea, but I think it is a good idea to delete any cards in the deck that either cause too much trouble or simply aren't useful.

I'm all for reading and listening as much as possible, I do that too, but I just want to point out that if you have a large deck that has been going for many years, size isn't the main problem because a majority of cards are reviewed once every second year and you need on average three seconds to review it.

[imron]

@icebear, my maximum size tends to be about 1,000 words (though size is not so important as total time required to review). I tend to spend quite a bit of time on any card that doesn't get instant recognition, which would explain why I have a much lower number of cards in a deck before I hit the 'revisions are taking too long' threshold. Nowadays, I'll also tend to go long bouts without doing any flashcarding whatsoever, and when I start up again, I'll also delete the old deck.

you look it up, you realise it's one you should already know, maybe you kick yourself, then you move on. If this word is not super-common then it could be a couple of months until you next see it.

Actually what happens is, I look it up, press '+', and move on. The card naturally gets added back to the new deck and will pop up regularly in revisions once more and it's no different than if I'd failed it during a flashcard review. Having a flashcard system integrated with my main dictionary reduces to zero the pain of creating and maintaining new cards. By regularly clearing out my deck, it also ensures that words going in to the deck are ones that are relevant to me at that point in time. Any word that I'm not likely to see in reading for another 2-3 years is not a word I'm interested in spending much time on learning at any particular point in time. Any word that I'm likely to encounter more regularly is going to come up again in my reading, and will be added back to the current deck if necessary when it happens.

Regarding frequencies, I'm not sure where you got that about 'uneasy', but for example the corpus of contemporary american english (the first free corpus I found in a google search, I'm happy to compare with real English if you can find a link) gives 'uneasy' a frequency of 7.33 occurrences per million, or about 1 per 136,000 words, which is significantly more common. Words appearing 1 per 2 million are those like 'quiescent', 'resplendent' and other significantly less common words.

Even so, 1 per 2 million is still not as infrequent as you might think. If you measure in terms of War and Peace, it seems like a lot (4 War and Peaces just to review the word once), but the reality is, reading 30 mins a day even at relatively slow speeds will still give this word a natural repeat within a year. Bear in mind you'll still have been flashcarding this word regularly when it first entered your deck, and assuming you haven't had any problems with it, by the time you end up deleting the deck, in all likelihood it would have a longish interval at around this point anyway.

However raw frequency statistics can be misleading because they represent a general case of the entire language. What tends to end up happening in the real world is that a person generally reads things that are of interest to them in a limited set of fields, and so words that are infrequent overall still might end up occurring quite frequently in material that you read. For example, going by the above linked corpus, the word 'orc' has an overall frequency of 1 per 7 million, which makes it seem like seeing the word again through reading will take forever (14 War and Peaces!), however if you are reading predominately sci-fi/fantasy, the word is significantly more frequent at 1 per 1.5 million. Depending on the books you read it still might be even more frequent than that (I'm sure it's more than 1 per 1.5 million in that other epic 'Lord of the Rings').

The thing is, you have no way of knowing beforehand what words are more frequent/relevant to you based on what you are reading. Deleting your deck regularly and adding words as they come up ensures that you are only spending time on words that are most relevant to you at a given point in time.

What we really should care about is making sure that the cards we spend much time on are actually worth that time. Thus, I don't think deleting the entire deck is a good idea,

But this is precisely why deleting the entire deck is a good idea! As you mentioned, cards that take up time are those you tend to forget, but there's no point spending time on them if you'll never come across them in the course of regular language use. You're better off spending time on the words that you will be coming across more regularly, but as mentioned above, you don't know which ones these are beforehand, and so deleting the deck and adding cards in as you are reading provides a hassle free way to maintain that, because deleted words are either:

• ones you already know well so it doesn't matter if you delete them
  
• ones you don't know well and that are relevant to you, and so they'll come up again soon enough in your regular reading and be added back to the deck then, or
  
• words you don't know well but that aren't relevant to you, and so it doesn't matter (and in fact is actually beneficial) if you delete them.
  

Regarding deck size, it is not the total size of the deck that matters, but the amount of time you are spending reviewing it. So I agree that simply having many cards is not an argument for deleting your deck and starting over. I would however argue that having a deck that requires more than 30 mins a day revising is a good case for deleting.

[Guest realmayo]

The one in 2 million figure I got from another post a while ago and can't remember where I got the frequency from but I'm happy to go with your list and assume (safe assumption) I got my numbers wrong. What I still want to stress is that with flashcards you know that if you forget a word you'll see it a lot over the following few months, which isn't the case just reading. But I suppose with your system you'd just add it (back) to your Pleco SRS so it would achieve exactly the same thing.

If I got up to your reading speed or had more time to spend reading Chinese, I'd seriously consider adopting your approach, or give up SRS completely.

For now, I try to be ruthless about suspending cards that I forget too often (probably an 80/20 rule thing going on here), which seems to keep reviews to below 30 minutes a day.

I also think "natural SRS" is a slightly daft phrase: reading isn't SRS, natural or otherwise, it's reading. If a sprinter decides to stop using weights he wouldn't say "now I do my benchpress naturally, while running", he'd say "I don't benchpress any more".

The other argument for SRS is hard for me to make confidently but I've always felt that words I would only see in SRS aren't strongly learned: the real neural connections will normally only be strongly made once I've seen the word "in the wild", e.g. in a newspaper or book. But that process doesn't just involve seeing the word in a book, it involves seeing it, and successfully making an effort to remember it.

Now, without SRS I might have completely forgotten that word by the time I get to it a second time "in the wild". So I'm not strengthening a connection, because that connection has ceased to exist (when I forgot it). What SRS does is artificially keep that connection alive, albeit weakly, so that when I do see it in a book again, I have something that I can reinforce.

One final thought about SRS. It's tempting to think that SRS is better than sitting with a written-out list of new vocab because it's more modern. But, if you subscribe to the newer notions that our brains are quite plastic things, maybe the old-fashioned way of learning vocab was easy for people who had to do lots of rote-learning at school, and SRS is more suited for people who didn't do much of that at school and who are used to flicking from one thing to another all the time online. The more I read about brain plasticity the more I wish I'd done more rote-learning at school and regret my rather condescending attitude to that aspect of education in China.

[li3wei1]

The main difference between SRS and a written vocab list is that SRS forces you to focus on the character. With a list, you can associate the meaning and pronunciation with the place on the list, which won't help at all. Even with a small group of hard-copy flashcards, you might have only one character with a particular radical, and you know that the character with that radical in that deck means X and is pronounced Y. Again, not much use. The beauty of SRS, especially if you leave all your cards in, is that it gradually widens the pool. You look at 吨, for instance, and it might be the only character you've seen in a while with the 屯 component, so you make that connection. But then, you might later add 纯, or if you've learned 纯 a long time ago, it will come up, and you have to learn to distinguish between the two. At the same time, you're forced to distinguish those characters from others with the mouth and thread radicals, or characters that just look similar if you squint.