WEBVTT 00:00.000 --> 00:10.000 Okay, thanks very much. 00:10.000 --> 00:13.000 Thanks very much, thanks for coming. 00:13.000 --> 00:17.000 It's been really interesting, I think, being in this room today. 00:17.000 --> 00:20.000 And also, I was here in the early last year. 00:20.000 --> 00:26.000 And there's a lot of different aspects of funding of open source. 00:26.000 --> 00:31.000 And I think this aspect of how research software gets funded is a little bit different 00:31.000 --> 00:32.000 than some of the others. 00:32.000 --> 00:37.000 But it's probably just my personal view, because I think probably everybody is going to say 00:37.000 --> 00:40.000 that the way their software is funded is different than all the others. 00:40.000 --> 00:45.000 But I'm going to go ahead and start talking about this research software, 00:45.000 --> 00:46.000 specifically. 00:46.000 --> 00:49.000 I just do want to mention also that there's two co-authors, Eric, 00:49.000 --> 00:51.000 Johnson and Michelle Barker, neither of them were here, 00:51.000 --> 00:54.000 but they were contributors to the work overall. 00:54.000 --> 00:58.000 So just to start off, it's worth introducing research software, 00:58.000 --> 01:00.000 because it's a part of open source. 01:00.000 --> 01:03.000 And sometimes it's not open source, but mostly it is. 01:03.000 --> 01:09.000 There are a group of us that in 2021 end up finishing a long discussion, 01:09.000 --> 01:12.000 where we tried to define what research software meant. 01:12.000 --> 01:16.000 And we published a report on the process with a bunch of different examples. 01:16.000 --> 01:20.000 And basically what we said is that research software is source code files, 01:20.000 --> 01:25.000 or them scripts, workflows, et cetera, created during the research process 01:25.000 --> 01:27.000 or for research purpose. 01:27.000 --> 01:31.000 And people that are doing research are also using other software. 01:31.000 --> 01:34.000 They're using things like operating systems and libraries, 01:34.000 --> 01:37.000 and their software has lots of dependencies. 01:37.000 --> 01:39.000 And these are things that are used for research, 01:39.000 --> 01:42.000 but they weren't created during or with a research intent. 01:42.000 --> 01:45.000 And so we would say that these are software in research, 01:45.000 --> 01:48.000 as opposed to research software. 01:48.000 --> 01:51.000 And this varies between disciplines. 01:51.000 --> 01:54.000 So if I am a computational astrophysicist, 01:54.000 --> 01:57.000 and I'm modeling black holes, 01:57.000 --> 02:00.000 the compiler that I used to compile my C code, 02:00.000 --> 02:02.000 for example, is not research software. 02:02.000 --> 02:04.000 It's software in research. 02:04.000 --> 02:06.000 But if I'm a computer scientist, 02:06.000 --> 02:08.000 and I'm trying to build the next generation of compilers, 02:08.000 --> 02:11.000 that same compiler might be research software. 02:11.000 --> 02:13.000 So it's a little bit of a fuzzy definition, 02:13.000 --> 02:16.000 unfortunately, we've never been able to come up with a crisp one. 02:16.000 --> 02:19.000 And so we could argue about this. 02:19.000 --> 02:21.000 If anybody's interested in research software, 02:21.000 --> 02:22.000 I'm happy to talk about this, 02:22.000 --> 02:25.000 and you can tell me why this is a bad definition, 02:25.000 --> 02:28.000 and I'm won't necessarily disagree with you. 02:28.000 --> 02:31.000 But it's not clear that there's a better one. 02:31.000 --> 02:36.000 And it would say in talks where I discuss this and other people discuss this, 02:36.000 --> 02:40.000 there's often a desire to have a community effort 02:40.000 --> 02:42.000 to try to come up with a better definition. 02:42.000 --> 02:44.000 And I'd be happy if that happened. 02:44.000 --> 02:46.000 It's not helpful that it's going to do anything. 02:46.000 --> 02:48.000 It'll be much different. 02:48.000 --> 02:51.000 So if we now have research software, 02:51.000 --> 02:54.000 and if we talk about this from the point of view of the researcher, 02:54.000 --> 02:58.000 research software is usually developed by a researcher who needs it. 02:58.000 --> 03:02.000 They do it because they need it in order to solve their research problems. 03:02.000 --> 03:04.000 It's scratching their own edge. 03:04.000 --> 03:05.000 And maybe that's the end, 03:05.000 --> 03:07.000 is that they develop their software for themselves, 03:07.000 --> 03:10.000 and they solve their problem, and nothing else happens. 03:10.000 --> 03:13.000 But maybe they also archive this for reproducibility, 03:13.000 --> 03:17.000 so people in somewhere else in a few months and 10 years 03:17.000 --> 03:21.000 can do the same research and understand where these results came from. 03:21.000 --> 03:23.000 And maybe that's the end. 03:23.000 --> 03:26.000 Or maybe that researcher then says, 03:26.000 --> 03:29.000 okay, I've solved this problem, but now I have another problem. 03:29.000 --> 03:32.000 And I have to modify this software to be able to solve that next problem. 03:32.000 --> 03:34.000 I have to put in new physics. 03:34.000 --> 03:38.000 I have to put in new method of looking at a new biomedical scanner or something else. 03:38.000 --> 03:40.000 Maybe that's the end. 03:41.000 --> 03:44.000 Or maybe the software is actually going to be used by other people, 03:44.000 --> 03:48.000 which is really in some ways what we think is a community would like, 03:48.000 --> 03:51.000 not necessarily with the person that builds it like. 03:51.000 --> 03:53.000 And maybe that's the end. 03:53.000 --> 03:58.000 And if we do that, then ideally we get a community that forms around that software. 03:58.000 --> 04:00.000 We get other people that are using it. 04:00.000 --> 04:01.000 They're suggesting features. 04:01.000 --> 04:02.000 They're creating issues. 04:02.000 --> 04:03.000 They're finding bugs. 04:03.000 --> 04:06.000 Maybe they're even contributing to it with four quests. 04:06.000 --> 04:07.000 Something like that. 04:07.000 --> 04:10.000 Maybe they're just going out and telling other people that they should use it 04:10.000 --> 04:13.000 and that they should work together to build it. 04:13.000 --> 04:19.000 In this situation, there's often somebody that takes on responsibility for maintaining this software. 04:19.000 --> 04:22.000 And I know this is kind of obvious open source. 04:22.000 --> 04:25.000 But in research, it's kind of an odd thing that happens because 04:25.000 --> 04:27.000 it's somebody whose goal is to do research. 04:27.000 --> 04:31.000 And now they're not going to do research instead they're going to try to maintain their software. 04:31.000 --> 04:33.000 And so it certainly happens. 04:33.000 --> 04:35.000 But sometimes it happens consciously. 04:35.000 --> 04:38.000 Sometimes it happens kind of just along the way. 04:38.000 --> 04:42.000 Ideally it's done intentionally by somebody that's experienced and knows how this works. 04:42.000 --> 04:46.000 And they're choosing to do this at the expense of their other activities. 04:46.000 --> 04:50.000 Because they see this is the best thing for the community or the best thing for themselves. 04:50.000 --> 04:53.000 And this then ideally leads to reusable community research software. 04:53.000 --> 04:57.000 And this is, I think, where we would often like to get to. 04:57.000 --> 05:01.000 From the point of view of somebody that's really a developer that's not a researcher, 05:01.000 --> 05:04.000 but somebody that sees a common research problem. 05:04.000 --> 05:10.000 And they likely have some experience within or next to other researchers that have this problem. 05:10.000 --> 05:13.000 They develop a tool that addresses it. 05:13.000 --> 05:19.000 This is a project that I've worked on that's a workflow system named partial that Ben Clifford 05:19.000 --> 05:24.000 and the back is our maintainer for this is kind of an example of this. 05:24.000 --> 05:29.000 The developers then work with researchers to test and improve this software. 05:29.000 --> 05:35.000 They try to form a community or to develop some other method for sustainability. 05:35.000 --> 05:40.000 And this is, again, usually done intentionally by somebody experienced. 05:40.000 --> 05:48.000 And this is somebody that could do other things, but they feel like supporting this area of research and building this tool that's going to enable that is important. 05:48.000 --> 05:55.000 And again, this ideally is then going to lead to reusable community research software of everything goes well. 05:55.000 --> 06:02.000 So from the point of view of a funder, and I should say here, this is a funder who's interested in supporting the research primarily. 06:02.000 --> 06:06.000 So this is often a government agency, it can be a philanthropy. 06:06.000 --> 06:10.000 It can be a business if that research software is of interest to that business. 06:10.000 --> 06:17.000 It isn't often so much, but it certainly is in some cases in the financial industry, for example. 06:17.000 --> 06:24.000 They see multiple researchers that are working in similar areas that often would benefit from sharing software and having reusable community software. 06:24.000 --> 06:29.000 They don't want to support multiple people who are doing research in the same area. 06:29.000 --> 06:32.000 Each of whom is going to write very similar software for that research. 06:32.000 --> 06:38.000 They would much rather see that happen in a community because it's cost effective for them. 06:38.000 --> 06:45.000 And they should be supporting researchers then who choose to make their software into reusable community software. 06:45.000 --> 06:50.000 And they should be supporting developers who want to create reusable community software. 06:51.000 --> 06:55.000 And so what I'm going to be talking about is this situation, right? 06:55.000 --> 06:58.000 We're aiming to get to reusable community software. 06:58.000 --> 07:03.000 I'm not going to talk about the vast amount of research software that's built by one person for their own research, and that's the end. 07:03.000 --> 07:04.000 It never gets shared. 07:04.000 --> 07:05.000 Or it gets archive. 07:05.000 --> 07:11.000 That's important, but it's not really the focus that I have. 07:11.000 --> 07:17.000 So if we then look at the funding processes, we have the fact that resources are needed to maintain software. 07:17.000 --> 07:22.000 That's again, I think, obvious, everybody from even from all the talks that we've had already in this session. 07:22.000 --> 07:25.000 Those resources can come from users. 07:25.000 --> 07:31.000 And that's clearly the case when you have commercial software or the users are paying the developers effectively for the software. 07:31.000 --> 07:33.000 Or in theory it can come from other models. 07:33.000 --> 07:40.000 It can come from people that are contributing their time, people that are contributing small amounts of money, grant funding. 07:40.000 --> 07:44.000 These things are all more difficult, but they can happen. 07:44.000 --> 07:47.000 I'm sorry, let me go on. 07:47.000 --> 07:49.000 It can come from the developers. 07:49.000 --> 07:57.000 So somebody who's getting funded to do the research, put some of their research time into the software, and they need this software in order to do their research. 07:57.000 --> 07:58.000 They keep doing this. 07:58.000 --> 08:01.000 So it's their choice of how to do this. 08:01.000 --> 08:06.000 Or it could be that in institutions a university or a national lab or a research center, 08:06.000 --> 08:10.000 sees the software something that's really important to them and it's part of their reputation. 08:10.000 --> 08:13.000 And so they are going to put some money into supporting it as well. 08:13.000 --> 08:16.000 And that happens occasionally, but not very often. 08:16.000 --> 08:24.000 And then it can come from a third party, a funding organization that can be a government or a philanthropy or industry. 08:24.000 --> 08:32.000 And again, this is the part that I'm going to focus on is where we've got this third party activity that's bringing in the resources for the research software. 08:32.000 --> 08:37.000 And so today we have multiple grant making organizations. 08:37.000 --> 08:42.000 Just the last talk was another one that I wasn't aware of, not necessarily for research software, 08:42.000 --> 08:48.000 but potentially maybe there's a research software piece that could come out of that as well. 08:48.000 --> 08:51.000 And there's multiple projects that are seeking grants. 08:51.000 --> 08:54.000 There's probably, I don't know, thousands, tens of thousands. 08:54.000 --> 08:56.000 I'm not sure how many. 08:56.000 --> 09:01.000 Maybe even hundreds of thousands of research projects that would like to have better support than they have. 09:01.000 --> 09:09.000 So right now, each project has to be aware of which organizations exist that could support their project and write proposals to them. 09:09.000 --> 09:15.000 And again, this isn't specific to research software, but it is the main thing that happens within research software. 09:15.000 --> 09:24.000 These projects and also have to be aware of all the policies of those potential funders, including rules that they may have against duplicate submissions. 09:24.000 --> 09:26.000 And against getting multiple awards for the same work. 09:26.000 --> 09:36.000 So they have to create different proposals that are somewhat different from each other in order to avoid having these issues, even though they're really looking for funding just to do one thing often. 09:36.000 --> 09:41.000 And so this is difficult in time consuming for projects, and it's not really very efficient at all. 09:41.000 --> 09:48.000 Funding organizations, which I'm just calling funders, very often, decide which submissions they're going to support. 09:48.000 --> 09:52.000 And the means that they use for this is typically peer review. 09:52.000 --> 10:02.000 They bring together a bunch of peers, either software developers or software users, and they get them to give opinions of what's the quality of this proposal, what's the importance of it. 10:02.000 --> 10:11.000 And again, this is often a, within a peer review process, but sometimes it happens in a different way, particularly from philanthropies. 10:11.000 --> 10:17.000 Some funders, and this is particularly the case for philanthropies, will approach projects directly. 10:17.000 --> 10:22.000 So the funder says, I'm funding work in computational chemistry. 10:22.000 --> 10:31.000 It's very clear to me that this chemistry package is being used by a bunch of my researchers, I'm going to go out and reach out to the developers and ask them what they need to make sure that that package stays around. 10:31.000 --> 10:35.000 And that's really nice when it happens, but again, is somewhat rare. 10:35.000 --> 10:45.000 And it's also good for well-known and well-connected projects, and it's bad for other projects, because they don't really have a way of getting into that funding at all. 10:45.000 --> 10:52.000 And so overall, there's no coordination that happens of all these different funders, and this isn't efficient for anybody. 10:52.000 --> 10:59.000 And then, in addition, projects have dependencies, and the funder decisions often don't account for these dependencies. 11:00.000 --> 11:08.000 So they fund the application that somebody uses, but they're not aware that that application actually is dependent on a bunch of other packages that aren't research software and they're not being supported. 11:08.000 --> 11:12.000 Even though, if any of them went away, there would be a lot of problems. 11:12.000 --> 11:20.000 Okay, from the point of view of projects, so research software projects typically begin under a fixed duration grant. 11:20.000 --> 11:25.000 In the US, this is usually three to five years, maybe two years. 11:25.000 --> 11:30.000 Depends on where you are, but there's usually some duration after which the funding's going to end. 11:30.000 --> 11:35.000 And the project then has to find resources for development and maintenance when the grant ends. 11:35.000 --> 11:45.000 And sometimes the grant asks the project to give some idea about how they're going to do this at the beginning and to start thinking about it from the beginning, which is certainly better than not thinking about it until you've run out of money. 11:45.000 --> 11:47.000 But both happen. 11:47.000 --> 11:51.000 And this becomes an ongoing process for most software projects. 11:51.000 --> 11:55.000 They have grant funding that's coming in for first app. 11:55.000 --> 11:59.000 They often have some volunteers, some collaborators, some other people. 11:59.000 --> 12:06.000 And they need to do a bunch of work, and that work includes developing and maintaining the software itself, doing the documentation, and testing. 12:06.000 --> 12:16.000 And then a bunch of different community work aspects of community work like user support, supporting community development, doing outreach, things like that. 12:16.000 --> 12:29.000 Small projects have times when there's a grant, and they can do all of these things, in times where there's no grant, and they really have difficulty doing any of these things. 12:29.000 --> 12:42.000 In large projects, often can have some staff who are dedicated to sustainability, and because they have those people, they can stay in these smooth periods more often, and be more likely in these grant periods. 12:42.000 --> 12:45.000 There's been a bunch of work that's happened around software project life cycles. 12:45.000 --> 12:52.000 And so I just wanted to point out one recent paper that I was on that Yoyo Houdi led, and she's a person who's little out of this work. 12:52.000 --> 13:01.000 That proposes a life cycle model for research software projects that has the idea that projects have challenge events, and a challenge event could be a loss of funding. 13:01.000 --> 13:04.000 It could be the initial developer steps down, it could be something else. 13:04.000 --> 13:06.000 And then there's a bunch of different things that can happen. 13:06.000 --> 13:15.000 I just want to mention that there are kind of academic studies of how this process works, and what the issues are, and what things we need to be concerned about. 13:15.000 --> 13:24.000 So from the point of view of the funding organization again, the funders have to understand what's needed by the research communities that they support. 13:24.000 --> 13:35.000 So again, if I have a funder, I'll say the Chan Zuckerberg initiative that's funding a lot of biomedical work, they need to understand what software their users are using and how do they make sure that that software is still going to be available. 13:36.000 --> 13:52.000 And in fact, CCI in 2024, had a talk where they were saying a bunch of the questions that they have again as a funding organization are what tools are most frequently used by scientists in a given field. 13:52.000 --> 13:56.000 How does the use of open source compare to proprietary software? 13:56.000 --> 14:02.000 Are there emerging new tools that are replacing legacy ones to help them again understand decisions they could be making? 14:02.000 --> 14:09.000 What's the prevalent programming language? Should they be supporting the language community? For example, if there's one that's really important to them. 14:09.000 --> 14:14.000 What tools should be part of a student's computational curriculum? Can they help make that happen? 14:14.000 --> 14:19.000 And what software project should they prioritize as critical infrastructure for science? 14:19.000 --> 14:31.000 Again, not thinking necessarily about just what the scientists are using, but what are the dependencies as well? If there's one dependency that's being used by all the applications, and that's probably important for them to be supporting. 14:31.000 --> 14:35.000 Okay, and then we get into multiple funding organizations. 14:35.000 --> 14:42.000 So multiple funding organizations are working in the same spaces often, and they have alignment challenges. 14:42.000 --> 14:50.000 So sometimes they have divergent priorities within a particular space, sometimes they have different strategic goals, different geographic foci. 14:50.000 --> 14:57.000 They often have timeline mismatches. One is giving out two year grants, one is giving out five year grants. 14:57.000 --> 15:10.000 The decision making process for one may start in September and have to have awards made in December for January. The other one may be something in a different cycle entirely. 15:10.000 --> 15:14.000 They also have issues about risk of version when they try to work together. 15:14.000 --> 15:25.000 So multiple funders may be are concerned that they're going to lose control over their funding. If somebody else is also involved in that process, they also might be concerned that they're not going to have the same credit that they would. 15:25.000 --> 15:34.000 But if they work together with somebody else, the one funder might say another funder is actually as a nice mechanism. They could give them some of their money and help the community. 15:34.000 --> 15:40.000 But then would they be recognized by those projects and by the community, so maybe they're not going to do that. 15:40.000 --> 15:48.000 Maybe there's going to be some harm to the reputation if one funder does something, and that's going to basically look bad for the other funder. 15:48.000 --> 15:56.000 And then there's also issues about relationship and trust. The people that are actually doing this because the funders in some sense are people, in addition to being organizations. 15:56.000 --> 16:06.000 These people have to have to have relationships with other people and those relationships have to exist for long enough that they build up trust that they can work together and they agree on what they're going to do. 16:06.000 --> 16:17.000 There's also operational complexity. So how decision making happens may be one funder says we need to use peer review and our peer reviewers have to be in our country and another one says something completely different. 16:17.000 --> 16:28.000 So how do they actually work together? One funder says I need to have an annual report that includes these things. Another one says I need to have a report at the end of the project that includes something completely different. 16:28.000 --> 16:32.000 So what's the project going to do to meet those different requirements? 16:32.000 --> 16:37.000 Again, maybe they have different requirements on budgets and operations. 16:37.000 --> 16:44.000 They also have legal and compliance issues. So it could be that two different funders want to work together, but they can't because of government rules. 16:44.000 --> 16:49.000 That say this money can't go out of this country to this place or something else. 16:49.000 --> 16:53.000 And there may just be not the right legal frameworks to actually do this. 16:53.000 --> 16:55.000 And then there's also resource constraints. 16:55.000 --> 17:08.000 So people that are in funding organizations typically are working quite hard on what they're doing and it's challenging for them to take time away from what they're doing to try to explore how they could do things in a better way. 17:08.000 --> 17:14.000 So there are a bunch of different types of funding programs. I guess I should have mentioned before on the bottom of the slides. 17:14.000 --> 17:19.000 There's a URL on the URL as to the slides and there are also links from the fast and page for this talk. 17:19.000 --> 17:28.000 And both of those also then have links to the paper that this is based on and the paper went through some review and we're in the process of updating it. 17:28.000 --> 17:33.000 But a lot of this I'm just going to highlight that actually in more detail that's in the paper itself. 17:33.000 --> 17:42.000 So there's different kinds of funding organizations again government that can be local, that can be national, that can be multinational industry and philanthropy. 17:42.000 --> 17:49.000 Funders can work in a single discipline, they can work across disciplines, and they can work at different software stages in terms of what they want to support. 17:49.000 --> 18:01.000 So someone to support prototypes and new software, someone to support enhancement and maintenance of existing software, someone to support infrastructure and platforms, and then there's other categories as well. 18:01.000 --> 18:14.000 And then finally there's different mechanisms. So the thing that happens most often is that standalone grants are made to do the software work in research projects. 18:14.000 --> 18:25.000 But sometimes there are people that are doing research and they're along the way, they're building some software and then they can ask for a supplement to make that software public or to start to build a community around it as well. 18:25.000 --> 18:28.000 So there's a lot of different options here. 18:28.000 --> 18:35.000 If we look then at how different funding organizations can coordinate there are some examples of this that have worked. 18:35.000 --> 18:37.000 I just want to mention a few of them here. 18:37.000 --> 18:47.000 So there's an activity that the Chan Zuckerberg Initiative, the Cavly Foundation and the Welcome Trust did together that was called a central open source software for science. 18:47.000 --> 18:54.000 Three different philanthropies that used separate funding but used a coordinated proposal and review mechanism. 18:54.000 --> 19:05.000 So this was a nice example that supported a lot of projects in biomedicine and underneath biomedicine over I think six grant cycles over about four years. 19:05.000 --> 19:16.000 There's the Gordon and Betty Moore Foundation and the footprint coalition and Schmidt futures had a call that was called low cost tools for science. 19:16.000 --> 19:27.000 Where they used a centralized decision process and they basically put all the funding together and then distributed it once as opposed to the first one were each organization picked their own things that they wanted to fund. 19:27.000 --> 19:36.000 There's the kissed era consortium which is a European project that exists and does fund research software as well. 19:36.000 --> 19:42.000 European Commission clearly in terms of the horizon projects is doing this in a very centralized way. 19:42.000 --> 19:54.000 And then the Belmont Forum which has mostly been working in climate and earth science but not entirely has a way basically a mechanism for different funders to come together for particular call. 19:54.000 --> 19:58.000 Where they each put in their own funding but they coordinate on how the decisions are made. 19:58.000 --> 20:04.000 So so we do have examples of this happening we don't have a huge number of them. 20:04.000 --> 20:15.000 In terms then of the coordination the decision making so any coordination between the funders is great shared knowledge between the funders can lead to better decisions and better support. 20:15.000 --> 20:21.000 And if they can make this more active this can actually make the decisions even better. 20:21.000 --> 20:33.000 All the decision making that I've talked about so far is funder driven and I'm actually going to go a little bit different in a different path now which I think is a little bit different than anybody that I've heard talk about here so far. 20:34.000 --> 20:42.000 So so this option of top down funder driven is that the funders are deciding maybe using some some peer review or some internal knowledge. 20:42.000 --> 20:52.000 But there are alternatives and those alternatives include user driven decision making and software project decision making which I'm going to call kind of bottom up in middle. 20:53.000 --> 21:00.000 So if we imagine user driven funding decisions the idea here is that the funders want to support research. 21:00.000 --> 21:07.000 The research projects that they're funding rely on research software the funders want to support this research software. 21:07.000 --> 21:13.000 In order to implement this we could use something like DRIPS which I don't know how many people are familiar with DRIPS. 21:13.000 --> 21:16.000 Okay absolutely no one that's exciting. 21:17.000 --> 21:20.000 So you're going to learn something from this if not from me. 21:20.000 --> 21:33.000 So DRIPS is a way that users can basically register their usage and that kind of gives tokens and those tokens and can be turned into actual funding is a simple way of saying this. 21:33.000 --> 21:41.000 So basically the projects that the funders funding would say what software they're using they would divide up some virtual currency across those. 21:41.000 --> 21:48.000 The funder would then look and see which projects have the most virtual currency and those are the ones that declare the the most research projects want. 21:48.000 --> 21:52.000 And they would use that to make the funding decisions rather than using peer review or something else. 21:52.000 --> 21:58.000 Right let the community of users decide rather than rather than having the funder decide. 21:58.000 --> 22:00.000 The software project driven decision. 22:00.000 --> 22:05.000 The idea here is that research software exists in different areas. 22:05.000 --> 22:12.000 It's a chemistry, astronomy, mechanical engineering and also in tool areas like workflows or databases or other things. 22:12.000 --> 22:19.000 And if we have projects that are working there you could imagine that these projects get together and they collectively understand their state. 22:19.000 --> 22:27.000 In computational chemistry and molecular dynamics for example there's probably six or seven kind of key projects at this point. 22:27.000 --> 22:36.000 And they could say my projects in fairly good shape, this project really needs some work in this area, this project needs some work in a different area. 22:36.000 --> 22:42.000 If we had, let's say $10 million, how would we divide that up among ourselves to be most effective? 22:42.000 --> 22:47.000 And let that community decide what the right thing to do is and then tell the funder. 22:47.000 --> 22:54.000 We've tried to do this a little bit at a very low level in something that's coming out of the US Department of Energy, 22:54.000 --> 23:05.000 which supported something called the XSKL Computing Project that developed and maintained a lot of science software for XSKL Computing for about seven or eight or nine years. 23:05.000 --> 23:13.000 The Department of Energy DOE more recently created something called the Consortium Pre the Advancement of Scientific Software, 23:13.000 --> 23:18.000 with six member organizations to support that software that came out of that original bits of funding. 23:18.000 --> 23:29.000 And so each of these member organizations has a focus, some area, and they have an internal process by which they decide how to divide up that funding across that area in order to support the projects. 23:29.000 --> 23:40.000 Today the Department of Energy is supporting the overall piece, but I could imagine that those high level organizations could get together and try to make a funding recommendation as well. 23:41.000 --> 23:46.000 Okay, so getting towards the end, let me kind of recap the problem. 23:46.000 --> 23:53.000 Research software underlies contemporary scholarship. There's almost no scholarship that happens today without software. 23:53.000 --> 24:00.000 The funding pillars that support it are not well coordinated and they're not aligned to maximize efficiency and impact. 24:01.000 --> 24:13.000 We have multiple agencies and multiple organizations that provide grants, but they rarely coordinate and this creates duplicate of efforts for both them as well as for the projects that have to keep applying for these different funds. 24:13.000 --> 24:21.000 And the central issue then is really that there's a mismatch between the software projects ongoing resource needs in the limited time grant cycles that happen. 24:21.000 --> 24:27.000 And this leads to funding gaps in maintenance bottlenecks and gaps in support for the tools themselves. 24:28.000 --> 24:43.000 So what I've said then is that we could have new frameworks for more efficient coordinated funding that draw across organization communication and infrastructure sharing and these things are starting to emerge in a few places, but again not a huge number. 24:44.000 --> 24:52.000 The collaborative structures that exist show that this actually can work and the multiple funders can work together and reduce inefficiencies. 24:52.000 --> 25:04.000 And so pooled resources and co-funded calls and community different allocations can help identify software tools that have product applicability and critical dependencies across fields. 25:04.000 --> 25:15.000 And again, these new funding models and procedures have emerged and they can distribute resources and help make decisions that are somewhat better than what we've done more recently. 25:15.000 --> 25:30.000 Pure review is the thing that exists today mostly, but there's been a little bit of work in a few different areas about trying to imagine user driven ideas and project driven ideas to actually distribute this funding. 25:30.000 --> 25:39.000 So and then finally, if we did have a mix of processes that might be the right way to actually go forward. 25:39.000 --> 25:48.000 And so the fact that we have some of these now is good and it's kind of a time for funders to try more of them and a time for projects to try more of them as well. 25:48.000 --> 25:59.000 So just to conclude, if you're interested in this, there are other groups that are looking at this, the research software alliance, which is an organization I co-founded about six or seven years ago. 25:59.000 --> 26:03.000 It has something called a software funders forum that's looking at this. 26:03.000 --> 26:09.000 The global research council, which is government funders has a multilateral engagement working group. 26:09.000 --> 26:15.000 And so if you're working with any government funders, you can encourage them to join this group and look at different options. 26:15.000 --> 26:25.000 Bringing these philanthropic government funders together would help research software projects gain consistent support to be able to evolve from prototypes to sustainable infrastructure. 26:25.000 --> 26:30.000 And if you're able to get to this kind of alignment, we'd be able to reduce duplication. 26:30.000 --> 26:41.000 Strengthen the software ecosystem for better resource allocation and ensure the tools that are the vital tools for knowledge creation are robust, widely accessible and continuously improved. 26:41.000 --> 26:46.000 And I would be happy to take any questions. Thank you all for listening. 26:46.000 --> 26:50.000 Thank you for the talk. 26:50.000 --> 27:10.000 I love the idea of cooperation at both from an efficiency and efficiency up and down. 27:11.000 --> 27:21.000 Do you have any concerns that as institutions cooperate more that some built-in biases will exclude projects? 27:21.000 --> 27:36.000 And do you think that there's a model where the operational complexities and the things that are not decision making should be separated from the decision making processes so that it sort of avoids those problems? 27:36.000 --> 27:43.000 So I think there certainly is the possibility for bias. 27:43.000 --> 27:54.000 And I think where we are right now where we've tested a few of these things at very minimal scale, let's us experiment and see if that bias exists or not. 27:55.000 --> 28:07.000 I've been at University of Illinois for 10 years and I was at the U.S. National Science Foundation for four years before that as a funder and so I kind of have some knowledge of different ways that funders can experiment and can try to understand these things. 28:07.000 --> 28:16.000 And so I think those five examples that I gave in some ways are experiments and the funders then look at those and say kind of what's happened. 28:16.000 --> 28:26.000 We did what we wanted were their biases so I think it definitely is an issue but I don't think it's an insurmountable issue that we can't learn about. 28:26.000 --> 28:33.000 I'm not sure about the second part of that question that you asked. 28:33.000 --> 28:40.000 I don't think I have a good answer for that so I'm just going to I'll just go on. I think it's a good question though. 28:40.000 --> 28:47.000 Thank you for this very nice talk which point a lot of question and issues. 28:47.000 --> 29:05.000 I'm saying this myself I write some code I know to write some code and I can include code writing in my grant but as a scientist I don't how to do anything else like write it documentation packaging code and managing a community for that matter. 29:05.000 --> 29:13.000 So is there a ground dedicated specifically to filling these gaps that I cannot fill myself. 29:13.000 --> 29:22.000 Yeah, so that's a great question. So there's two different things I would mention one of them is almost at the same time tomorrow in the open research. 29:22.000 --> 29:31.000 I'm going to be giving a talk on the research software engineering movement which is kind of a group of people that have come together that are trying to do what you've said. 29:31.000 --> 29:42.000 They're often people with software engineering experience but who also understand research that worked with researchers to try to improve the software that they're writing and help turn it into good community software. 29:42.000 --> 29:58.000 So we have people that are developing with these skills. The started in the UK and expanded to a lot of the world and there's probably something on the order of 10,000 people at this point that would call themselves research software engineers primarily Europe and North America and Australia but some other countries as well. 29:58.000 --> 30:04.000 So the people exist. Some funders recognize this and have specific calls. 30:05.000 --> 30:10.000 The first call that I put up the Chan's Leckerberg lead call was exactly for this. 30:10.000 --> 30:20.000 NSF in the US has done this. The UK, some of the research councils there have done this. So it happens in DFG and Germany has done this quite well. 30:20.000 --> 30:33.000 So it happens in some places it doesn't happen in others. Again, this is part of the case that we're trying to make is to show funders where this does work and that they should be using these models to help their own researchers as well. 30:33.000 --> 30:46.000 Yes. Thank you for the talk. This is kind of a lazy question. Is there a central place to look for funding calls or is it divided out by academic areas? 30:46.000 --> 30:55.000 So there's not a good answer to that at this point. So in this top organization, Risa, the research software alliance. 30:55.000 --> 31:05.000 We tried to kind of crowdsource this and build a, to build what basically was a Google sheet that you could search through. 31:05.000 --> 31:15.000 It's very hard to do anything like this and to maintain it because all these grants, all these proposed opportunities are coming and then they end and then they may or may not get renewed in the year. 31:15.000 --> 31:19.000 Because it's really hard to keep the stuff up to date unfortunately. 31:19.000 --> 31:28.000 I'm also some of the projects I work with are working with NUMFocus, which is umbrella 501C3 charity that helps some of these research projects. 31:28.000 --> 31:37.000 And there's a person there who tries to keep a list of the ones that are of interest to the NUMFocus projects. So I don't think that there is a single answer unfortunately. 31:38.000 --> 31:45.000 Researchers generally kind of know what's going on in their country, but they don't necessarily know what's going on outside and. 31:45.000 --> 31:50.000 And we've tried to work on that a little bit, but it's it's difficult. 31:50.000 --> 31:56.000 Thank you so much for this great overview and great round of applause. 31:56.000 --> 32:01.000 Thank you.