Andy Matuschak
patreon
Prompt makeover logs
🕑 Added 2020-12-09 23:32:53 +0000 UTCMy past few weeks have been filled with plenty of interesting conversation about the prompt-writing guide and the challenges of formulating knowledge, both in the workshops and in email correspondence.
Amusingly, one of the richest discussions I've been able to observe has been José (aka Artir)'s conversation with himself as he criticizes his own recent prompts and revises them. You can see his notes here. One example:
Q: Does LDL particle size have any relation to CVD?
A: No
Problem: It’s a Yes/No question, the same question can be leveraged into an explanation qustion
Q: Why does an increase in LDL particle size not correlate with CVD?
A: Because what matters is particle count, LDL is all sufficiently small to matter for CVD.
Reading this made me realize that others would probably benefit from the same exercise! And others could benefit if you would share some of your notes and revisions as José has done. If you're game, please comment with links to your notes.
I should also mention: notes like this really help me!
As a researcher, my main "differentiator" is that I explore ideas by expressing them in real-world systems, so that I can learn from how they refract through the lens of serious use. Those observations let me improve the ideas, which let me improve the systems, and so on in a virtuous cycle. Michael Nielsen and I called this the "insight through making" loop. This approach differs from typical industry practices in that iterations are meant to express deep theoretical insights. And it differs from typical academic research by building authentic, real-world systems which provide richer fodder for generating theoretical insights. I certainly can't claim that my insight-through-making loop is operating smoothly, but that at least is the aspiration.
The prompt-writing guide has involved creating a tiny insight-through-making loop. To write it, I needed to distill a number of ideas about learning into something like a system—a framework communicable to readers. So the most rewarding part of the last few weeks has been seeing those ideas refracted through readers' interpretations, both through the workshops and through email conversation. Thank you all for that; hopefully, what I learn will translate into better theories and better systems.
Comments
Andy Matuschak
These are great insights, Matt! "Writing good prompts" involves developing better theories of what it is to understand things—what's important to know vs. what's not; what's meaningful; etc. Sounds like you've made progress there!
Andy, I'm really enjoying reading the guide to writing prompts. It's a real pleasure, so thank you. I actually just finished a major overhaul of my notes. I don't have a nice log like the above, but I thought I would share a bit about my experience in case it's useful. I've read Nielsen, done the Quantum Country, and so I think the actual way I write prompts is mostly ok, though I continue to discover more concise ways to write. But for a variety of reasons, I had fallen way behind on review and was in a position where I needed to review a couple hundred cards over a few days that I hadn't seen for months. On 2/3 of the cards, my reaction was "I don't remember, but also I don't care." My big issue was that I had misjudged the level of detail I would care about in the future. After thinking about it, I realized one of my goals with SRS is to internalize a large academic literature that is siloed over multiple domains, so that I can synthesize and draw connections between disparate works. After being away from the cards for a few months, I found I really cared about knowing different explanations, so I kept cards about those. For example, I have a set of cards about the ways evolution can cross fitness valleys, and I kept them all. But since explanations typically involve at least a bit of elaboration, I've found the average length of my cards has increased. On other cards, there was more detail than I cared about. For example, I had a card that said: Children whose parents have patents obtain patents themselves at a rate of {{c1:18}} per thousand, while children whose parents do not have patents obtain them at a rate of {{c2:2}} per thousand. I changed this to: Children whose parents have patents obtain patents themselves at a rate of {{c1: ~10x} the rate of children who do not. But many other cards I simply deleted. Instead, I just assume I can return to my notes if I ever have to write on these topics with any detail.
Andy Matuschak
Ah, great! Yes, I've found that word useful. Non-technical non-fiction is full of heuristics and mental models, which usually need subjunctives to be formulated reasonably. I do notice that "might" prompts tend to have more ambiguity issues, since it often suggests multiple candidates.
Andy Matuschak
This is great—thank you for sharing!
Same feeling for me. This is only mildly relevant, but I've noticed that the word "might" seems to open the door to significantly better prompts. Less than 1% of my prompts include it (according to a quick search), and most of those were created after discovering your notes with the prompts in them. The guide has really increased my usage of it, despite no explicit recommendation. Not sure if this is useful to you, but I thought it was interesting.
I personally call this "Flashcard Refactoring", and I actually have a series of articles on my blog / newsletter where I go in depth on particular cards in my deck and how to improve them! http://www.marknagelberg.com/flashcard-refactoring/ http://www.marknagelberg.com/tips-from-anki-flashcard-refactoring-add-enough-knowledge-to-your-deck-and-review-your-sources/ http://www.marknagelberg.com/tips-from-flashcard-refactoring/ http://www.marknagelberg.com/combating-knowledge-interference-flashcard-refactoring/
Andy Matuschak
Thanks for sharing. Yes, I definitely have that problem! It's an unfortunate pressure: usually the most easily "translatable" things are trivial things, rather than deep insights or my own ideas.
This is excellent! Unfortunately, I only have one such example: https://again.blog/formulation-fun-1/ On a somewhat related note, have you bumped up against a subtle psychological push to write things that are easily "translatable"? The closest analogy I can think of is the idea that TDD pushes you to write code that's easily testable but not necessarily optimal.
