Andy Matuschak
patreon
[Audio version] Crowdfunded research vs. the NSF CAREER grant; open-sourcing Orbit; new technical collaborators
🕑 Added 2021-05-05 05:43:11 +0000 UTCI'm terribly sorry for all the duplicate emails: Descript garbled my exports. Still learning about these "content creator" tools… anyway, here's a fixed version!
And a transcript:
Hello everyone, and happy May. I think celebrating is always more fun in video format, so I'm making this impromptu video today to celebrate a couple of exciting pieces of news with you all.
The first is that the Patreon community that you all are a part of, or perhaps you're visiting today, has hit an important milestone. I wanted to share that and some thoughts about what it might mean for crowdfunding research in general.
The second is that I'm open-sourcing Orbit today, and there’s some interesting discussion to be had about funding models, research, and that decision.
And the third is that I now have a modest budget which I'm hoping to use to involve some more people in my work.
Crowdfunding 2/3 of an NSF CAREER grant
First: this funding milestone. Now, if you were a new tenure track professor at an American research university, one of the main really career grants that you would be seeking is this grant the NSF provides called the CAREER grant. Now, it's not a sure thing—funding rates are about 15% to 25% of applicants—but it's meant to support early career researchers for about five years, at a relatively modest rate. They may need to supplement it with some other funding, but it’s enough that they can get their research going in a meaningful way. If you're at a top tier research university, you'll see pretty much all of the faculty in the sciences having one of these grants.
While it's a relatively modest grant, it is our existing institutions’ most common entry-level grant for new investigators. And so it's an interesting thing to compare against, when talking about alternative funding models.
So he exciting milestone for this community is: that we're now crowdfunding at a level about two thirds of one of these NSF CAREER grants—which is really meaningful! We're starting to come up on now substituting for this existing institution’s job in a real way. But of course there are some important differences between these grants in crowd-funding land, and the NSF-provided CAREER grant.
I thought it would be interesting to talk about those differences.
Now, one of the most important differences is really a weakness in what I've been doing so far. These career grants for new investigators are meant to include a really substantial section about your research plan, but also a really substantial education plan, and a plan for how you are going to integrate education into your research. That could mean a variety of things. It could mean mentoring post-doctoral scholars. It could just mean undergraduate education. But ideally what these grants are trying to support is a synergy between research—developing new knowledge on the one hand—and education—supporting the future generations of knowledge creators on the other hand. Part of the point of the grant is to subsidize those future researchers that you're going to be training. I don't have really any of that as part of my work right now, other than the lightweight community engagement that I do on Twitter.
It's really interesting to me to think about what it would mean to try to ramp up that component. One of the things that concerns me is that it's difficult to engage in education in a field, which has such poorly defined methodology and practice. It's not like there's a textbook or a curriculum that I could refer people to, or really like a strong preexisting tradition. Obviously there's a lot of prior art in human computer interaction and in cognitive science for the kind of work that I do. But this inventing novel cognitive-augmenting interfaces thing? We haven't, we haven't really figured out a great theory or framework for how to do that.
So I've been wary of engaging in any kind of large-scale education activities. But I do want to try to ramp that up over the coming years. So I'm investing a fair amount of effort in to trying to capture what it is that I'm doing, as I'm doing it. I’m hoping to synthesize that into something that I might actually be able to use, both to mentor other people who might want to research with me, and also maybe to publish something which might help people getting into the space. But this is a spot where I would not qualify for one of these grants with what I'm doing. You can also see this as an advantage: you can say, well, these education components in new fields are something that just slows down the research, so I have more of a position as a “research scientist,” where I just get to focus on that. I don't really have to worry about teaching and service. There are goods and bads associated with that.
And of course, hiding in all of this is the fact that graduate students and post-doctoral scholars are a source of labor. I mean that not in, not in the code-implementing way (although we'll talk about that a little later) but in the sense that there are avenues that I'd like to explore, that I just don't have the capacity to explore right now. And it would be nice to have other people doing that with me.
But I also feel there's a moral challenge to taking on students for me in this field. If I were a professor at a university and an established field, and you come to me as a graduate student, then there's this implicit deal. You work with me for a few years, and after a few years, you are going to step out as an independent investigator of your own and establish your own agenda. You’re hopefully going to be able to do that by, say, getting a faculty slot at some other university. It's a pipeline. And of course you're not guaranteed a faculty slot. In fact, there's far too many graduate students for faculty slots, but that's the understanding that you’re taking on this kind of apprentice position. You're rolling the dice and saying: I'm competing for one of these faculty slots that are available at various institutions, and funding slots for those positions, which are provided by grant makers.
Now the moral issue is: that pipeline, that path, those institutions, and those funding sources don't really exist in this space. And so if I'm taking on someone who's excited about doing independent research of their own, they can work with me for awhile and that might be helpful to both of us, and maybe push my ideas forward, and maybe help them develop their own practice. But, there's really no end state where they get a shot at doing it on their own. They'd have to create that for themselves. And arguably, they'd be put in a pretty bad position to create it by working with me as an apprentice for a few years because they're not going to be amassing capital that they could use to support themselves. Maybe they could have been developing relationships with funders or with an audience or with grant makers, but they're going to have to figure that stuff out on their own.
This lack of a next step for a student is a problem. It's both a practical problem, but also a moral problem. I feel bad for taking someone on in this way when there's not clearly a place for them to go next. So that's one key difference between these funding sources.
Another interesting difference between an NSF career grant and the Patreon crowd-funding model is that the latter is a highly continuous, incremental process.
If I'm applying for a CAREER grant, that's kind of like a “big bang” process. I'm going to spend a a hundred-plus hours on the application for that. I'm going to submit this thing and there'll be a long process, which as many as six months later will maybe give me a decision. But that decision, if it's positive, will support me for five years. And so for five years, I basically just don't really have to worry about that source of funding, I guess I have to be worried about my next grant, you know, maybe a year or two before the five years is ended, but it's this very discrete funding model.
The Patreon model, on the other hand, has this really interesting property that it's continuous and incremental and ongoing. This is both good and bad. The good part is that you don't need this really well thought through plan in order to get started and get some funding. To get an NSF career grant, you're supposed to have a pretty clear picture of how you're going to carry out this research and what you hope to achieve with it, which arguably limits the types of research you can do, or requires that you've already done a bunch of the work upfront. In fact, it's pretty common for junior faculty to apply for this NSF career grant in their second or their third year of being faculty because of that. So it's kind of a shame. The discrete nature of the NSF career grant seems to discourage—or rather it's not meant for funding highly exploratory, speculative research that the investigator can’t articulate very clearly, or where there's going to be a lot of bricolage involved.
This incremental, continuous funding is also an opportunity for people who don't yet have much standing, or even potentially much skill, to incrementally build up their funding source. Something I'm excited about is the idea that maybe you have a day job, but in your nights and weekends, you start exploring some projects that you find interesting. And maybe you can build up a small crowdfunding community around that and start earning enough money from that to make you feel like those nights and weekends are like really well invested. And so you kind of double down on it, and maybe eventually you get to this point where you can leave your day job. Now, obviously there's a big jump there, but I think that's an exciting property of the crowdfunding versus the grant model.
Another interesting thing about the continuous, incremental nature of the crowdfunding model, is that it doesn't expire, per-se. The NSF CAREER grant is for five years, and it's granted to about 300 researchers per year, but the Patreon funding is potentially indefinite. So as long as you keep doing work that people want to support—and that's both good and bad.
It's good in that you don't have to completely change up your funding model and have this all-or-nothing switch in year four of your work. But it's bad insofar as there's this feeling that you've got to keep producing, or maybe it's not as stable or secure-feeling as having all of these funds in a guaranteed way, as with the CAREER grant.
The last difference that I wanted to discuss between the NSF CAREER grant and the crowdfunding approach for research is this question of who gets to decide what to fund.
It's really different answers in the two spaces. For an NSF CAREER grant, the most important part of the decision process is what's called a merit review, wherein your proposed research is reviewed by about three peers or informed experts in the space that you propose to investigate. They will issue written comments and scores and have some discussions with coordinating officers who will then decide what to fund. It's a relatively small number of high-expertise people, and that has advantages and disadvantages. So you don't have to convince that many people, on the one hand. But there’s also a lot of problems with peer review. Perhaps these are people who are established in your field, and you're trying to take your field in a weird direction, or maybe there isn't really a field around the work you're doing. There's some concern that peer review tends to perpetuate stasis, or more conservative ideas. I think that's probably at least partially true. Certainly in my adjacent field human-computer interaction, peer review seems to promote a set of fairly conservative values and processes that would really impede my work, I think.
In the crowd funding model, on the other hand, you have lay-people or interested fans deciding that work is interesting, and you have to persuade hundreds of them, instead of just a couple. That seems harder. But then, you don't have to persuade them very hard. Merit review is kind of a high stakes thing. I think these reviewers feel like they're setting standards for a very important institutional body, so they're going to be kind of defensive about what qualifies and what what's worthy. But a patron deciding to toss you five bucks, even five bucks a month? It's just not that high stakes, by comparison. So while you have to convince a lot more people, you maybe don't have to convince them very hard. And that's interesting.
It's also interesting that they are much less likely to be, say, experts or peers or something like that. Now, obviously I do have a lot of funders who are experts and peers, and I'm grateful for you. Uh, but a lot of them are just like random internet people who find the work interesting. Which is great! This is good in that it means if you have this work that doesn't really like fit in, but other weirdos (who maybe aren't high-status or high-power) think it's interesting, then you can maybe cobble together some funding through crowd sourcing that might be difficult to achieve through traditional channels.
But a challenging part of this is: to what extent can the crowd actually evaluate research? In my case, I worry that, if I were investigating something that was like equally important, but less obviously legible to casual observers, that I would have a lot more trouble getting funded. And that's a real concern. I worry that this general effect might make crowdfunded research more boring. Or maybe more driven by fashion or mass market appeal.
Anyway, long story short, I'm really excited that this crowdfunding community has managed to go much of the way to matching our main government funding structure for new faculty in the sciences. I mean, that's really, really remarkable. I definitely would not have expected that crowd-funded research could do that two years ago.
Hopefully, it's something which can generalize to more than just me.
Open-sourcing Orbit
So all this leads us to the second topic I wanted to talk about, which is that I’m open sourcing Orbit, which is the platform I’ve been developing for the mnemonic medium, and programmable attention in general.
I'm doing that because we have reached this funding milestone.
The path that I've been on is—when I did not feel safe in my source of financial support, I felt like I had to leave my options open. Like I've spent a year working on the software, now jeez, am I going to have to like commercialize this? Am I going to have to raise VC money around this?
These are not things that I wanted to do, but I felt, well, unsafe. And so I felt like I had to leave that option open. That's why I had not open-sourced Orbit up until this point. Now I feel sufficiently safe that I am past that point.
But it's probably important to discuss: why open source this kind of thing? Something like Orbit, which is about knowledge management about your mind—I think it really wants to be open-source. There's a practical perspective on this, which is that the whole conceit is you're trying to kind of bring in these micro tasks, these pieces of knowledge from all kinds of different places. That feels, if not open-source, like it wants to be open formats, open protocol, things like this.
And then also just ideologically, it's very intimate. I think in terms of trust, I don't like to spend a lot of time putting my mind into a system that could potentially just be taken away someday. Or that has the smell of lock-in. Or that feels like some weird format that I'm never gonna be able to parse. I'm hoping that by open-sourcing, I’ll create some trust. Hopefully it'll engender some interesting mashups and experimentation.
Stepping back, I view my role as provisioning public goods. And open source software is generally in software land a better public good than closed source software. Now, there are types of software which really benefit from being commercial. In fact, the public benefits from those pieces of software being commercialized. Stripe, for instance, is an important piece of infrastructure. You could even argue that it’s like a public good. But it needs like a huge number of people maintaining it and all these operational people and there's like partnerships and stuff. All of that needs like a lot of resources to keep it running. But something small like this? I'm hopeful—though these words may bite me—that I can keep this service running with relatively little effort. I feel very anxious even just saying that. We'll see.
It is possible that long-term, the Orbit data formats and the application, and even the server (if you want to self-host)—all of them are open source, but maybe running the service is so taxing eventually that some fees are necessary in order to make sure I can hire someone to be on PagerDuty orwhatever.
I’m not really thinking about that right now. But it does come into play in the licensing model that I chose, which is kind of interesting. I haven't seen many projects do this, though a number have done things kind of like this. So what I'm doing, licensing-wise, is that all of the libraries are licensed under the permissive Apache license. And then the main application and the main backend server are dual licensed under AGPL (which is a, uh, strong copy-left license, and it’s viral over the network), as well as the Business Source License software license. It’s a relatively new software license that allows you to do whatever you want for non-production purposes. And, and after some number of years (three years in Orbit’s case), it turns into the permissively licensed Apache software.
This licensing scheme is meant to deter cheap copycat commercializations. Khan Academy had this problem where when it was open-sourced, a bunch of people downloaded the app, changed the name, put it on the app store, and charged $10. And at Khan Academy, we’d get these angry letters. It's really bad. If they're using your name, you can yell at them for trademark purposes, but sometimes they'd actually do it in a way that was totally compliant. And it just felt really bad. I don't want that. But then also, if I do end up needing to someday charge for the service model, I'd like to reserve some asymmetric power to capture some of that value as a payment for my upfront development costs. But like I said, that's not something that I'm thinking about now. It's also not something that I want to do. I’m not interested in commercializing this. It sounds boring, basically.
Along with the open source comes some interesting dangers. I have run, or been a large part of, two large source projects before. Both of them were very taxing experiences. And I do not seek to repeat that experience!
In recent years, platforms like GitHub have kind of redefined people's norm for open-source to be a project which is community run. And the community decides on the roadmap and everybody submits issues and anybody can propose features and so on. That's fine and good for a lot of projects. But it would tax me beyond my abilities for Orbit. So we're going to have something… not like that. We’ll have to articulate what it will be, but essentially the idea is that at least for the near future, this thing is primarily a vehicle for the research that I'm doing, and perhaps that collaborators will be doing with me.
So I'm going to try to be minimizing extensive open-source community interactions that are not in direct service of those research goals. Probably that will open up over time. There's more informationin the repository you can read about this.
That’s a lot of cynicism or negativity. But I do believe in, and have experienced, the positive side of open-source. Where generous contributors have delivered a lot of value, both to me and to the community, by coming in and implementing a feature that needed to get done, but that I didn't want to do. Or by identifying some horrible bug that no one knew existed but that I really cared about.
I look forward to those kinds of contributions and I really am excited to be open-sourcing the project.
In the coming weeks and months, I will probably launch some kind of mailing list for people who are interested in participating in, or following along with development decisions, roadmaps, things like this. But it's too early for like a user's list. Most of the things that people tell me that they want, or that are bad? I know! They bother me a lot too. I'm mostly not looking for that kind of engagement right now.
More technical collaborators
The final thing that I wanted to talk about is experimenting with expanding the set of people working on these tools. As we start approaching the funding level of an NSF CAREER grant, my expenses are increasingly taken care of, and maybe I can even save a little bit for a rainy day, which is nice. It makes me less nervous.
At this point, or relatively soon, marginal funds start being not for my pocket or for my mortgage, but rather for getting poured back into the project, which will almost invariably take the form of paying other people for their time working on it.
Ideally I'd love to be able to fund both technical staff who can really help me out by implementing important pieces of infrastructure and improving on my, my very rapidly written research quality code, as well as research staff who can meaningfully investigate some of the open questions and design challenges that seem to bear. And actually, if the crowdfunding model does continue to grow at its current rate, it looks like by the end of next year, I should—maybe—be able to afford to pay an intern or two, or a contractor for half of the year who's relatively affordable. Something at that level. That's pretty exciting! If we do actually continue on pace to that, it means, maybe in a year or a year and a half, that this is a model that can support not just me, but potentially others, too.
This mirrors what the NSF CAREER grant is meant to do as well. It's something that is supposed to also potentially support at least part of maybe a graduate student's stipend, or possibly stipends for undergraduates who are assisting with research. This kind of thing.
But that's a little ways away: the end of next year. And one thing that I've talked about here before is that there's this substantial challenge in switching back and forth between “research mind” and “implementers’ mind”. I find that even just trying to do the two in the same week is very difficult. So the, the model that's worked best for me is spending months at a time in one of those mindsets and then switching back and forth between.
So, for two-thirds of 2020, I was in “building things mind” with Orbit, and then actually for much of this year, I've been back in “research mind.” You've seen a little bit of that output, but more to come from that period. And now I'm starting to switch back to the “implementing mind,” but all of this is frustrating.
When I've been in research mind for awhile, as I have been now, I'll feel like: “Oh my gosh, there's all these things that I know would make this better. And it would let me answer these questions.” But I can't. No progress has been made, and months have gone by, so it feels like the project is kind of staying still. And then when I'm spending a lot of time implementing, like I was last year, I feel like there are these dozen research questions that are articulated at the start of the year, and I've made no progress on them, and I'm not actually being a researcher. I'm just like being an engineer. And that feels bad too.
Ideally, it would be nice to have some technical help before the end of next year. So to that end, a couple of exciting notes.
There've been a few volunteers who've reached out to collaborate with me on Orbit, maybe on their nights and weekends. And one of them has actually made his first commit to the repository. I'm very grateful for Ozzie’s help.
And the other exciting news there is that a small group of generous donors have made some one-off gifts to allow me to this year have a modest budget to hire a technical contractor. That's very exciting—to maybe have a little bit of professional help!
I've started the process of trying to find the right person for that. Ideally they are able to be somewhat technically independent, and take on moderately-sized infrastructure projects without a ton of oversight, which is a very difficult property to achieve simultaneous with my very modest budget for contractors at this time. I think the best candidates will be people who are already excited about the research, excited about contributing to open-source software—and are perhaps willing to accept more modest compensation in conjunction with those things. And perhaps in conjunction with getting to run some of their own ideas about related projects by me, and possibly get some mentorship there. We can do a light “barter” kind of thing.
So like I said, I’ve begun conversations with a number of people about that. But I am still on the lookout for a contract hire. And so if what I've just described—that set of impossible conditions—sounds like you or someone you know, please do send them my way.
————————
So to wrap things up, I just want to express a lot of gratitude. Really, this is a lengthy video about this bizarre funding model for research that, uh, I didn't think was possible. And it really is only possible because of you all. It really is a remarkable thing—you're doing something quite unusual here. Time will tell whether this is generalizable, or even whether it's sustainable for me personally. But I’m really grateful to be exploring this with you all.
