How does the frequency-ordered SRS character-learning gadget really work?

What is it for?

The design of a flash card library is intimately tied to what it is meant to practice. What should go on the front of the card? On the back? What does it mean to have remembered it? Flash cards can be used to remember how words are pronounced, what they mean, how to use them, how to write them, how to translate them, and the list goes on. The design of a digital flash card system, is no different.

SRS card back screenshot

In designing our first spaced-repetition flash card activity, we chose a design geared toward a very particular use case: learning and remembering how to write characters. The motivations were several:

  1. Learning to successfully write a character requires much more attention to its structure than simply learning to recognize it on the page. One notices the repeating motifs that all Chinese characters are composed of and what makes a character distinct from other similar characters.
  2. Characters quickly fade from memory. Even native speakers suffer from character amnesia. By working character writing practice into your language maintenance routine, you can count on the spaced repetition algorithm to make sure the fraction you forget is limited.

In short, the frequency-ordered SRS gadget is to help you master writing common Chinese characters and to prevent you from forgetting them.

Try it out!
(You'll need to be signed up, which is free.)

How should I use it? (with pictures)

You will get the most out of the frequency-ordered SRS gadget if you use it as intended; the card design, the method for introducing cards, the spaced repetition algorithm and the bonus scheme are all optimized for learning how to master writing Chinese characters. I will explain more below, but first I would like to relate our idea for how it can most effectively be used.

  1. Practice writing the character or word before you look at the back of the card. You can write the characters into your hand (my preferred strategy), on a piece of paper (making it difficult to cheat), or into the air.
  2. This activity uses multi-character words as a way of reviewing the constituent characters. You can facilitate this by introducing a few new words after you introduce a new character. Whenever you introduce a new character, any multi-character words that contain that character (and only others you have already introduced) will be added to the word bank (the set words that are waiting in the wings). The word-bank word with the highest frequency is always introduced next. Since there are likely some common words among the words added when a character is introduced, you can often learn the new character in the context of some words that contain it. The number of words in the word bank is show in parenthesis on the button labeled "Introduce Word" (pictured below).
    SRS choices screenshot
  3. Try and keep a balance between how many new characters you're introducing, the number of words you're introducing, and how many reviews you are doing. At each presentation you get to choose one of these three choices (pictured above). If you introduce too many new characters and words, your review queue will balloon and it will become unmanageable. If you only spend all your time reviewing, you won't be making forward progress learning new characters.
  4. It's okay to leave a bunch of stuff in your review queue. You don't need to get your review queue back to zero each time you practice. Both the algorithm and the bonus scheme (described below) assume that you often won't do this. I often leave about 100 cards in my review queue. The algorithm is designed to prioritize this pile and is constantly adjusting the sequence to make sure you practice the most important things at the optimal times.
  5. Below the activity we show two histograms (one for characters and one for multi-character words), which give you a sense of how the cards you have introduced are distributed among review intervals. In total there are 12 review intervals, spanning from 2 minutes up to 8 months (roughly in powers of 2). Cards at high levels are ones that are reviewed very infrequently, and so it is normal for these bars to be the longest because they carry the least review burden. In general you will find these plots grow to look like the ones pictured below. You can keep an eye on these, and if the lower half of the plot is looking kind of empty (like in this picture) maybe it's time to introduce more new things rather than simply reviewing cards are at long review intervals.
    SRS plots screenshot
  6. Bump ahead anything you know well. It takes a lot of time to review cards when you're practicing writing the characters (about 15 seconds per card in our data), and there's no sense in reviewing cards more frequently than necessary. When you see something that you know well, you can use the "know it well" response (or the associated drop down menu) to bump cards up to higher interval durations and thus reduce the overall number of reviews required.
    SRS bump ahead dropdown screenshot
  7. Use the forget button when you have actually forgotten the card. This will give you a few reviews at short intervals and then return the card back to where it was at (depending somewhat on your subsequent responses). We know you will forget sometimes, and we account for this (more detail in the section "forgetting" below).

How does it work?

The sequence

There are many varieties of spaced-repetition. Here I will describe our particular algorithm so you can decide how best to use it. I will focus on the mode using the "Built-in SRS sequence." The SRS gadget can also be used with your own word lists, but this blog post will only deal with the built-in sequence. This is controlled by the following menu:

SRS word lists menu screenshot

The frequency-ordered SRS sequence consists of 2928 cards of which 870 are single characters and 2058 are multi-character words that consists of only these 870 characters. The 870 characters are ordered in a sequence that closely follows character frequencies from a corpus of text. In general, the more common characters appear earlier in the sequence, but frequency is not the only determinate. Other factors we incorporated include how many characters are derived using the character as a constituent component (or primitive), and how many other words contain that character. This way you will tend the learn:

  1. Common characters before rare ones.
  2. Structurally more fundamental characters before the more elaborate, composite ones.
  3. Characters that are the basis of lots of words before characters that are used in fewer words.

While this ordering is far from perfect, we hope that it helps make your time well spent.

The timing

There are 12 review interval durations, i.e., the minimum time between reviews of a particular card. These durations start at 2 minutes, followed by 8 minutes, 20 minutes, 1 day, and so on up to a maximum duration of 8 months. If you always select the blue button (often labeled "Got it!") then a card will tend to pass through each of the 12 levels. As I mentioned above, we encourage you to bump up cards as much as you can while still remembering 80-95%.

The review duration is not exactly the prescribed interval. Instead it is expanded or contracted by a random amount to better mix up the order of cards (so you don't memorize the sequence as a crutch). Also, cards are reviewed in order of urgency. The urgency score is relative to the interval duration, so if a card is overdue by twice it's interval then it will tend to be reviewed earlier than a card that is overdue by only 50% of it's interval, even if the calendar time of the latter review would suggest it should be reviewed first. This has the tendency to prioritize ones you're most likely to forget so you can get those reviews in sooner rather than later.

There are also two sources of bonuses that allow you to reduce your overall number of reviews without sacrificing learning:

  1. The first bonus takes advantage of the structure of the language. Remember that this activity is designed for you to master writing the characters, while learning a bunch of vocabulary along the way. If you say you remember a word, such as 朋友, then you also must remember 朋 and 友. Thus you will sometimes (randomly) get credit for either 朋 or 友 when you indicate you remember 朋友.
  2. The second bonus relates to the timing algorithm. If you're supposed to review a card after 4 days, but you don't get around to it until 8 days (because you're review queue is full of other cards that need to be reviewed more urgently), and you successfully recall the card, you might as well get credit for the 8-day interval rather than the 4-day interval. Thus there is another bonus for this occurrence which reduces your overall number of reviews (and greatly shortens the time it takes to get through the whole sequence, as I describe in the next section).

It's perfectly natural to forget some cards and we account for this in the timing algorithm. When you tap the "forgot" button, the card will be temporarily returned to the shortest review interval (2 minutes) but a few successful reviews will return it to the interval where you left off. Often this is many fewer reviews than it took to initially reach the point where you forgot it. Thus forgetting it doesn't mean you need to start from scratch. Instead, you can benefit from the few short intervals to re-solidify the memory, and then pick up where you left off.

Alternatives and supplements

Reviewing language using flashcards is a tried and true strategy with many available tools. This particular SRS gadget is just one and it may or may not be suitable for you. And even if it is suitable, it is designed for a particular learning angle (remembering how to write common characters) and there are many other ways in which SRS may be helpful for your learning efforts.

For all these other efforts, we recommend you check out at least two other tools:

  1. Anki is by far the most flexible, powerful flash card software (and it's free on certain platforms).
  2. Skritter is another excellent tool that you may want to check out.

Try it out!
(You'll need to be signed up, which is free.)

Watch the map of your Chinese knowledge populate

A major obstacle to learning a language is the feeling that you've hit a plateau. A learning plateau is the feeling that you're not making progress despite continued effort. John Pasden has a great article on this at Sinosplice, where he says:

The frustrating thing about the plateau is that you don't feel like you’re making progress when you really are.

I share John's view that the plateau is more a matter of perception than a true flattening out of progress. Progress in language learning is not something you can generally see and so we're left with the proxy of perceived progress, which might be rather far off the mark.

But maybe there are some ways to see progress?

At WordSwing, we are experimenting with ways to visualize maps of your Chinese knowledge. Here's what it looks like for knowledge of individual characters:

Character knowledge map

The colors correspond to how well you know each character and the size of the squares correspond to the frequency of the character in typical text. Thus, you can quickly spot where there are gaps (all the gray squares), or help tune WordSwing's assessment of your knowledge by toggling the color associated with characters.

We also provide a means to look at your historical maps, allowing you to see how much your maps has gotten filled in:

View historical knowledge map

And you can get an overall summary on the Dashboard, showing how many words you've rated a particular learn stage and what fraction of the typical text these comprise:

Learn stage summary
(you'll need to be signed up, which is free)

Hope to see you soon, back on WordSwing, tracking your knowledge and watching your map color in with knowledge!


Following language through the stages of learning

(updated 2016-09-08)

Learning a new word generally seems to follow a typical progression. First you notice a new word, then after you've encountered it a few times, it starts to feel familiar and you decide that maybe it's worth learning. You then more actively try and commit it to memory and perhaps try using it out in the wild a few times, and eventually it joins all the other words that you feel you've mastered.

Here's what this process looks like:

Learn stages

At WordSwing, we make this process explicit. You can place any character or word into one of these three learn stages. These learn stages have the following interpretations:

  1. Discovered - This is a word that has entered your radar. You've seen or heard it a few times and it seems worth keeping your eye on.
  2. Learning - You've continued to encounter this word, and it seems like something you'd really like to know and something you would like to actively practice.
  3. Mastered - You feel confident that you know this word well.

These learn stage ratings have two main purposes:

  1. WordSwing uses these to personalize your language practice. Specifically, WordSwing will include words in the learning stage much more frequently in the more drill-like activities, and WordSwing will assume that words in the mastered stage will be helpful context with which to learn other words.
  2. You can use these to keep track of what you've learned, what you're learning, and where the gaps are, for example by using Character knowledge map.

While practicing on WordSwing, you'll have opportunities to tune what learn stage each word occupies, for example, by using the brain tool in the drill-down pane:

Learn stage drill-down tool

The three buttons at the bottom of the above screenshot let you control whether the ratings are shown to you while you study. This can be helpful for getting a sense of how much of some text you understand or for identifying words you would like to provide ratings for but haven't yet.

You can also see on the Dashboard how many words you've rated a particular learn stage and what fraction of the typical text these comprise:

Learn stage summary

So, come over to WordSwing, and start tracking your knowledge of Chinese and reaping the benefits of personalized practice!

Hope to see you soon!

Modeling capitulation - simpler is better

We have been experimenting with some of the latest and greatest in machine learning algorithms so that your practice material is most appropriate for your knowledge and ability.

Unfortunately, many of these efforts have fallen flat on their face. Our new personalization scheme is a bit more patient.

We're still a small operation and thus the data we have collected has been a bit too limited for some of these algorithms, such as neural nets, to perform well. In response, we have revamped how study material is proposed, settling on a solution that has much better performance for new students, for whom we have collected little data.

Our new personalization scheme is a bit more patient, waiting until we have a better grasp of your knowledge before heavily personalizing the material to the map we build of your knowledge. This should reduce the rather erratic behavior that sometimes was observed with our old personalization system.

At present three activities use the new personalization scheme:

  1. Character Combination
  2. Character Similarity
  3. Recall Pronunciation

Tuning to the difficulty you want

In order to compensate for less initial automatic tailoring, we have implemented a new dial which allows you to make the practice content easier or harder, depending on your current study mood:

Difficulty dial screehsnot

The return of Recall Pronunciation

If you've been around WordSwing for a while, you'll remember a very simple activity, Recall Pronunciation, that mysteriously vanished when we redesigned WordSwing to be mobile friendly. Well, it's back!

Recall pronunciation screehsnot

It's a simple way to drill your memory for pronunciation. It features auto-suggest for pinyin entry and accepts spoken pronunciation including tone sandhi changes as well as standard dictionary pronunciation.

Thanks for you patience! And if you haven't checked out WordSwing in a while, please do! And don't forget to let me know what you think.


Page: 7 of 10

Learn a language by swinging up to ever higher levels of proficiency by effectively using the language you've learned so far.

Keep up to date