Andy Matuschak
patreon
The carrying capacity of a regular memory practice; deliberate practice and flow
🕑 Added 2020-09-30 18:07:45 +0000 UTCSlow, compounding progress is a subtly powerful force. Regular weightlifters might not perceive their progress in every session, but as the weeks go by, they’ll find they can handle loads which would previously have flattened them. Richard Hamming makes a similar observation for intellectual efforts in The Art of Doing Science and Engineering:
I had worked with John Tukey for some years before I found he was essentially my age, so I went to our mutual boss and asked him, “How can anyone my age know as much as John Tukey does?” He leaned back, grinned, and said, “You would be surprised how much you would know if you had worked as hard as he has for as many years”. There was nothing for me to do but slink out of his office, which I did. I thought about the remark for some weeks and decided, while I could never work as hard as John did, I could do a lot better than I had been doing.
In a sense my boss was saying intellectual investment is like compound interest, the more you do the more you learn how to do, so the more you can do, etc. I do not know what compound interest rate to assign, but it must be well over 6%—one extra hour per day over a lifetime will much more than double the total output. The steady application of a bit more effort has a great total accumulation.
It’s a nice heuristic, but it’s not so easy to see how to carry this out. Most knowledge work activities don’t actually compound in this way quite so reliably. But as has been described elsewhere, memory systems do seem to compound in this way. Small amounts of marginal effort yield compounding returns in one’s ability to recall a given piece of knowledge.
Most people have a regular exercise practice, a regular email practice, a regular news-reading practice. Might a regular memory practice someday become a mundane entry on that list? If so, what might the impact be? And what must have occurred to bring this change about?
We can begin by looking at some simple numbers. Say that each person is willing to review for about ten minutes a day. Even using Quantum Country’s extremely simple algorithm, we could add about 14 new questions per day while remaining under that time limit (assuming an accuracy rate of 90% and an average of 6 seconds per question). This amounts to 5100 questions per year. For reference, Quantum Country contains about 200 questions. So your practice could accommodate ingesting one Quantum Country-sized text every two weeks. The time factor scales linearly, so you could double your carrying capacity by reviewing for twenty minutes instead of ten.
I’ve previously noted that I feel most researchers in this space over-index on optimization and fancy scheduling algorithms. But in this efficiency-focused context, we can see that those choices really do matter! For instance, if we’d implemented some more accurate scheduler which reduced the average error rate from 10% to 5%, my simple simulator suggests you could add 16 questions per day instead of 14. That would come out to 730 extra questions over the course of a year—almost four extra Quantum Country-sized books. A regular memory practice exhibits outsized returns to small increases in scheduler performance.
So what? What do we imagine the impact would be if people regularly durably ingested 6,000 atoms of knowledge each year? Well, if they used this newfound ability in the way most people start using memory systems—that is, for learning unimportant trivia—such a memory practice would mostly be wasting people’s time. These systems’ promise lies in using them not for meaningless factoids but for engaging more deeply with whatever matters most to you.
As Michael Nielsen and I wrote:
One of the ideas motivating Quantum Country is that memory systems aren’t just useful for simple declarative knowledge, such as vocabulary words and lists of capitals. In fact, memory systems can be extraordinarily helpful for mastering abstract, conceptual knowledge, the kind of knowledge required to learn subjects such as quantum mechanics and quantum computing. This is achieved in part through many detailed strategies for constructing cards capable of encoding this kind of understanding. But, more importantly, it’s possible because of the way the mnemonic medium embeds spaced repetition inside a narrative. That narrative embedding makes it possible for context and understanding to build in ways difficult in other memory systems. … In some sense, Quantum Country aims to expand the range of subjects users can comprehend at all. In that, it has very different aspirations to all prior memory systems.
Apart from the aspirations mentioned there, the mnemonic medium aspires to solve the biggest outstanding barrier to regular memory practices of the type I’m describing: the challenge of writing questions. It’s quite difficult to write questions which effectively encode abstract, conceptual knowledge. Even once one’s acquired the skills involved, it’s quite taxing. Reviewing 100 conceptual questions will take me a few minutes; writing 100 good conceptual questions will take me hours. What good is a system with a carrying capacity of 6,000 questions per year if you can only spare the time to write a few hundred?
The mnemonic medium solves this problem by supplying expert-authored questions. Of course, other SRS platforms allow users to share questions, but our experiences suggest this only works well for fairly simple declarative knowledge because the questions are highly atomized. Each little question must stand on its own, presentable in any order at any time. But that makes it difficult to use the questions to effectively communicate an idea which is itself highly structured and ordered. In practice, ideas generally depend on other ideas, and emotional salience requires relating those ideas to a larger whole. By contrast, as we noted in the excerpt above, the mnemonic medium gives narrative structure and meaningful context to these questions. At least with Quantum Country, this seems to have allowed us to create an effective set of expert-authored prompts without encountering the problems which shared questions usually suffer.
But I’m not excited about a regular memory practice which consists solely of ingesting and retaining knowledge others have written. My hope is that as more high-quality books are written in the mnemonic medium, these expert-written prompts will act as a scaffold which will help people develop fluency in writing their own. We’ve received some feedback from Quantum Country users along these lines, but of course it’s too early to tell.
Texts like Quantum Country might help people learn to write their own knowledge-encoding prompts, but there are lots of other interesting ways to use a regular memory practice. I use mine to keep my burgeoning theories top of mind, to study my past decisions, to provide myself with aesthetic kindling, to modify habits, and so on. Separately, Michael’s described how one can use these systems to “see through” complex ideas. These more agentic, purposeful practices are currently inaccessible to most memory system users, but I think authored texts can scaffold users here as well.
———————————
Merlin Mann came to prominence writing about “Inbox Zero” and practices for dealing with one’s to-do lists. But over time he grew disillusioned with that culture and with the framing of his own past suggestions. He observed: look, do you need a fancy digital to-do list or a lifehacking blog to make sure that you play video games? No? Maybe that’s the problem you need to fix with your to-do list.
Sure, that’s an overstatement when taken literally—akrasia is real etc. But it’s worth asking: when we say “regular memory practice,” are we imagining an “ought-to” or a “want-to” habit? That is, is it something like flossing, which you don’t really enjoy but which you do anyway for the long-term benefits? Or is it more like the habit of enjoying a glass of wine at the end of a hard day?
When spaced repetition is working most efficiently, it feels like you’re failing all the time: each session aspires to ask you questions you’ve almost forgotten the answer to. To put it another way, spaced repetition practice which optimizes for memorization efficiency should not produce flow. It’s closer to what K. Anders Ericsson calls deliberate practice: a planned training activity focused on producing performance just beyond your present abilities, with fast-paced feedback and reflection. Deliberate practice is generally not pleasant.
But when I engage with memory prompts which are more about visualizing past experiences, or remembering the key factors for an important decision, those aren’t really about “pushing my performance level.” Those activities are closer to catechism, and they can produce flow states.
Can these two activities coincide in the same practice? The emotional experience and the objectives are fairly different! But maybe that’s OK: people like unified inbox interfaces which combine their personal and work email accounts.
I’m not sure how to thread this needle. I’m worried that a discussion framed around efficiency and carrying capacities takes us in a dutiful direction. Perhaps it’s possible to reject the oughta/wanna dichotomy by designing activities which yield enduring benefits but which also appeal emotionally, even in the moment.
For inspiration, I’ll leave you with pianist Nahre Sol, who has has many videos on her YouTube channel depicting how she makes repeated passages musically interesting through composition and improvisation during her practice. (Unfortunately, Patreon doesn't let me embed YouTube videos inline; click here to see her video!)
Comments
Andy Matuschak
For that matter: maybe an iOS 14 home screen widget displaying a rotating question is an interesting opportunity…
Andy Matuschak
Reminds me also of Kevin Simler's strategy of making his iPhone's lock screen show a photo of the book he's currently reading. If my home were Dynamicland-enhanced, I can imagine wall posters shifting to display different possibly-interesting stimuli throughout the day. As I walk around, one might catch my eye and stimulate some interaction.
Yes! I’ve discussed this with him. The last I’d heard, he had a bot that posted into his timeline… I do think there’s probably a utility to a more carefully integrated system… Twitter interests me as a site for an intervention on the theory that additional frictions make things less compelling, which is usually a cost, but in the case of an addictive service is a benefit. This makes it unlike attaching a new habit to brushing one’s teeth—where a decline in adherence would be a big negative.
Andy Matuschak
Yes! Taylor Rogalski's been running some experiments along those lines—probably you've seen? I'm interested, too, in the possibilities of integrating these types of interactions into one's ongoing working environment. Experiments with recurring writing prompts in my note-writing system seem more natural than memory review because I'm "already there." I'm not sure it's even important that it's integrated into something addictive (though that'd probably help): integrating into something you already do relatively reliably would certainly be a step up. Requiring that people adopt a habit is quite an onerous barrier!
I'm late on responding to this, but I've been interested (for reasons touched on in this post) in the idea of combining a memory practice (un-sticky, hard to maintain) into an interface that's otherwise addictive—a twitter app, for example, that alternates in memory prompts in the same feed. I can't help but imagine it like the old joke about taping buttered toast to a cat.
Now I understand, thanks! I agree with you. There seems to be a fauna of moods one can be into while reviewing. The one you describe is desirable, I wonder if it something we can induce. Sometimes I also have a hard time reviewing but without learning much, that should be avoided, but I think it's due to external causes.
Jim Beaver
thanks. I'm working with a professor who is writing another math textbook and who creates online courseware. At some point I'd like to expose him to your ideas and infrastructure. Like I said, I think you are onto something.
Andy Matuschak
Yes, that'll certainly be part of what I'm building! But I don't expect that functionality to be available to users until 2021: I'm focused on authored prompts in the short term. Until then, if you're interested in experimenting with writing prompts, I suggest you try writing them with Anki. I've written an importer from Anki to Orbit, so you'll be able to take your prompts and progress with you if you'd like to move to my system next year.
Jim Beaver
Is there a way we could start creating these prompts for things we are learning? For example, as I go through a book or set of lectures, can I create a bunch of these and then use your infrastructure to prompt me?
Evan Rocha
Actually yes, especially for the earlier review sessions. One of the reasons I like it is because I feel like my understanding often gets better over time instead of just staying the same
Andy Matuschak
Yes, we study timing data. You are indeed an outlier, but that probably just means you're thinking about the material much more carefully than most people! My guess is that you're not just working on recall: you're using the session as a context to work on correcting misunderstandings. Does that resonate?
Andy Matuschak
It's funny: as a matter of experience, I often *do* experience flow when doing spaced repetition. But I think that's happening when the algorithms are not performing well. I'm experiencing flow when I'm in a groove of trying to retrieve lots of knowledge that I've encoded durably enough to successfully remember, but which isn't so durably encoded that it's boring. But all that just means I'm not "working" very hard. If the algorithm were scheduling items to appear when I'd almost forgotten them, I would not and should not experience the "loss of reflective self-consciousness" which characterizes flow: I'd be straining to remember things and carefully looking for ways to better encode the responses I'd forgotten. Sometimes I have a session which does feel more like this, and it feels pretty clearly more valuable, though substantially less pleasant (or "intrinsically enjoyable" as Mihaly describes).
Evan Rocha
Oh wow this helps me understand why I have always struggled with deliberate practice so much. Often so frustrating. Though I never really find the Quantum Country questions too frustrating
Evan Rocha
random question, do you have timing data on the Quantum Country spaced repetition questions? I take somewhere closer to 30 minutes pretty consistently for the questions. I really hope I am not an outlier here lol.
Interesting, I've always thought about flow and spaced repetition from the perspective of memorization. While going through more internalization focused prompts or open ended questions, I haven't noticed myself getting in flow yet. In retrospection they often get me in a thinking mood. Failure is less clear in those cards and I usually take a longer time to answer, for this reasons I thought flow was not reachable. I'll pay more attention! Can you clarify why you think that "spaced repetition practice which optimizes for memorization efficiency should not produce flow"? From my, definitely superficial, understanding of flow, a right level of difficulty and short feedback loop should help getting in flow. I was writing a short post about flow and spaced repetition, I never published it because I haven't yet read Csikszentmihalyi's book and because I wanted to think more about the topic, here's a draft I was working on: https://randiisan.github.io/2020/09/12/anki-settings-for-flow.html If you get a chance to read it, I'd love to hear your opinion.
