WEBVTT 00:00.000 --> 00:13.560 Hello, welcome everyone, so thank you to the organizers, well, for organizing the 00:13.560 --> 00:18.480 Devroom first and for having me to, it's a great to be here again, it's not the first 00:18.480 --> 00:23.280 time, like three, four times, five times, I don't remember. 00:23.280 --> 00:28.920 Anyway, so my name is Ludovic Cortez, I'm a research software in January at Teneria, 00:28.920 --> 00:35.800 which is French Research Institute in Computer Science, and I've been doing stuff with 00:35.800 --> 00:42.040 package management and with HPC for a while now, in particular I've been a contributor 00:42.040 --> 00:50.840 of two gigs for a long time, yeah, a long time, so today I want to talk about package management 00:50.840 --> 00:57.000 and putting it in the hands of users, meaning in particular in the hands of users on HPC 00:57.000 --> 01:04.520 clusters, right? So, you know, I've not always been an HPC person, I don't, I don't think 01:04.520 --> 01:10.520 I'm actually an HPC person, I don't know. But, you know, when I first got introduced to HPC, 01:10.520 --> 01:15.720 like probably many people in this audience, I discovered modules, you know, environment 01:15.720 --> 01:21.320 modules, and they are great, and I'm not saying these just before Xavier's previous 01:21.400 --> 01:29.640 speaker, maintainer of environment modules. I mean, it's actually a very clever hack, it's 01:29.640 --> 01:34.040 more than hack even, because it gives you a lot of flexibility on your development environment 01:34.040 --> 01:40.520 and everything, and so everybody who uses an HPC cluster these days and even a long time ago 01:40.520 --> 01:48.840 uses modules, right? This is great. Actually, did the modules paper, the original papers from 1991, 01:48.920 --> 01:54.520 I think I checked it before, so it's been around for a while, and it's sort of timeless, 01:55.400 --> 02:01.800 people are still using it in 2026, and even there's a talk about EC, I think it's called 02:01.800 --> 02:08.040 afterwards, which is the European environment for scientific software initiative. It's still, 02:08.040 --> 02:13.720 software installation, sorry. Anyway, and it's based on modules too, so it's really, 02:13.720 --> 02:21.560 it does a great job in sit-around. But it also has some shortcomings, so in particular, if you go 02:21.560 --> 02:27.720 from one HPC cluster to another HPC cluster, well, maybe you'll have modules with the same name, 02:27.720 --> 02:33.000 like OpenMPI, but maybe they'll have a different name, and maybe you'll get different versions, 02:33.000 --> 02:37.960 or maybe they're compiled differently, you know, so you never know, or maybe you would just not 02:37.960 --> 02:45.000 find those modules on that other HPC cluster, and that's kind of annoying. And also, if there's 02:45.000 --> 02:50.520 software that's not available as a module, you'll have to go ahead and compile it by yourself on 02:50.520 --> 02:57.400 the machine, and, you know, I mean, if you're working on a complex software stack with C++, 02:57.400 --> 03:02.360 with scocals, for example, you know, things like that, you know, bidding stuff by hand on the 03:02.440 --> 03:11.480 cluster is kind of annoying, right? So, the way I saw it, as soon, pretty much as I discovered modules, 03:11.480 --> 03:17.000 that well, maybe, I mean, why don't we have it too, like, you know, like the typical free software 03:17.000 --> 03:22.280 Linux version, I was used to APT from Debian, you know, or you know, things like that. 03:22.280 --> 03:27.880 And I was like, why don't we have something like APT on clusters, right? That would be ideal. 03:28.760 --> 03:33.960 So, I mean, of course, there are a few differences, so in particular, we need more flexibility, 03:33.960 --> 03:39.240 because in HPC people just don't take, you know, software packages, other shelves, sometimes they 03:39.240 --> 03:44.520 want to customize how they build packages, right? You know, sometimes in its specific versions, 03:44.520 --> 03:51.720 etc., etc., and APT typically doesn't provide that level of flexibility. And in portability, 03:51.720 --> 03:57.080 so ideally, you know, I would like to be able to go from one cluster to another and run the same 03:57.160 --> 04:03.240 package manager command and get, you know, the same software, right? And yeah, reproducibility in 04:03.240 --> 04:09.560 the sense of being able to run the same software, you know, on different machines, and also at 04:09.560 --> 04:14.280 different points in time, because with modules, for example, if I even if I just choose one 04:14.280 --> 04:20.520 single cluster, all the time for my experiments, maybe I come back to the cluster six months later, 04:20.520 --> 04:25.880 and the modules I use are gone, or maybe there's been a great idea or reconfigured, you know, 04:25.880 --> 04:31.960 you never know, so I might find myself unable to rerun my computational experiments, and that's 04:31.960 --> 04:38.680 kind of a problem. So I thought, why don't we have a package manager for that? And well, 04:39.560 --> 04:46.920 it turns out that I had been working on Giggs as a hobby for a while, and Giggs is a package manager, 04:46.920 --> 04:52.440 and within the Giggs community, there are quite a few people actually contributing to Giggs, 04:52.440 --> 05:01.800 but also doing HBC stuff for a living, right? So with my friend Ricardo Vermous, a contributor to Giggs, 05:01.800 --> 05:07.480 at some point we thought, well, look, we have a tool, Giggs, and it actually has good properties for 05:07.480 --> 05:15.160 HBC, right, for using it on an HBC cluster, and we thought, well, maybe we should use it and promote it, 05:15.160 --> 05:20.840 as a tool for HBC clusters, and that's how we came up with this paper about having reproducible 05:20.920 --> 05:26.280 and user controls of their environments. With a package manager, that people would actually use 05:26.280 --> 05:33.640 directly, right? It turns out many people had the same idea at around the same time, right? So you've 05:33.640 --> 05:39.320 probably seen this project before a source pack is a bit counter, there are package managers, 05:39.320 --> 05:45.400 app owners, it's not a package manager, it's a container runtime, but anyway, many people 05:45.400 --> 05:50.120 felt the need to have package managers in place on HBC clusters at the time. 05:50.840 --> 05:57.080 I get back to this a bit later. So for now, I'm going to talk about Giggs specifically in HBC 05:57.080 --> 06:04.360 and what we did, what we managed to do so far. So in terms of availability on clusters, 06:04.360 --> 06:09.640 it's available in a bunch of clusters, probably not all of you don't know about this cluster, 06:09.640 --> 06:17.400 did I relatively small medium sized clusters in France and in other places in Europe, in the US too, 06:17.880 --> 06:23.320 but we have Giggs directly installed on these machines, so people can actually use Giggs directly 06:23.320 --> 06:30.920 on these machines. You might notice that there are no big clusters here, right? The T-RUN, T-Z rock 06:30.920 --> 06:36.440 clusters, meaning national scale, or European scale clusters, we don't have any of these on 06:36.440 --> 06:43.480 this list, I get back to this later. But anyway, so quick short of France, who knows Giggs already? 06:43.800 --> 06:50.680 Like the entire room, okay, I'm going to skip quickly, that's great and believe it will. 06:50.680 --> 06:56.440 So on these machines, as a user, you can go ahead, you log in and you can directly run Giggs 06:56.440 --> 07:03.160 shell, some package into LMPI Binchmarks, for example, and then you run, you know, you use Slurram 07:03.160 --> 07:10.040 and you run your command. And this particular command comes from the environment set up by Giggs 07:10.120 --> 07:14.840 shell, right? So basically, instead of doing module load, you use Giggs shell and it's kind of 07:14.840 --> 07:19.320 similar, you know, it sets up the environment and you run the command directly from there. And 07:19.320 --> 07:25.160 this is wonderful because from, you can go to any of these clusters and you go ahead, you run 07:25.160 --> 07:29.480 the same command, you get the same software environment basically. So it works pretty well. 07:30.680 --> 07:35.560 And because Giggs is a package manager, well, it knows about the packages and it knows how to 07:35.640 --> 07:40.840 rebuild those packages and to modify them in all sorts of ways. So we have what we call package 07:40.840 --> 07:46.440 transformation options. And for example, I can tell Giggs, well, please take that into LMPI 07:46.440 --> 07:53.320 Binchmarks package, between place, everywhere in the graph, up an LMPI by MPH. And it will rebuild 07:53.320 --> 07:59.720 it, if needed, and if it's already there, we'll find. And from there, I can run my benchmark, you know. 07:59.880 --> 08:07.480 Well, then we have manifest, we can store, you know, you can describe basically in a find the 08:07.480 --> 08:12.440 software environments that we want. And the cool feature is basically if you need to 08:13.400 --> 08:18.200 remember something about Giggs is that we store files and two commands, you can have a fully 08:18.200 --> 08:24.440 reproducible environment. So the Giggs is quite common here captures exactly the precise 08:24.440 --> 08:28.840 revision of Giggs you're using. And the Giggs time machine command takes that 08:29.960 --> 08:35.960 that information and reproduces the exact same Giggs, from which we're using shell to reproduce 08:35.960 --> 08:42.520 the exact same environment. All right, what about performance? We have a good tool, but if 08:42.520 --> 08:47.960 performance is not here, that's kind of a problem. And people were rightfully, you know, wonder about 08:47.960 --> 08:55.880 this in HPC circles. So first thing is MPI, we did quite a lot of work over these years, you know, 08:55.880 --> 09:01.480 about MPI packaging, because that sort of the one of the key things you want to get right. And so 09:01.480 --> 09:07.640 typically, I experienced details, but we have an open MPI package that can, that is built to support 09:07.640 --> 09:14.280 all these kinds of networks, that we actually encounter on this diversity of medium sized clusters 09:14.280 --> 09:20.520 I mentioned before. And our MPI package is able to, you know, at the time select the right 09:20.520 --> 09:25.160 back end. So if you run a slingshot machine, it's going to use that. If you run a normally path 09:25.160 --> 09:29.400 machine, it's going to use the right back end. And this is not, this is something open MPI 09:29.400 --> 09:36.920 doors for us, right? We just package it right essentially. And speaking of slingshot, well, 09:36.920 --> 09:41.640 we do, you know, there is this misconception that you're never going to get the right performance 09:41.640 --> 09:47.240 if you don't use the vendor provided MPI like create and pitch for example. And this is something 09:47.240 --> 09:51.720 we run and not asked for. This is a micro benchmark, you know, just ping pong. And we get the same 09:51.720 --> 09:57.640 performance, because there's no reason it should be different. We also run not just by micro benchmark, 09:57.640 --> 10:01.800 but also actual application. And it just work, you know, we get the expected performance 10:01.800 --> 10:07.000 because we're using, well, leapsiex side in this case for slingshot. And by the way, I'm glad we 10:07.080 --> 10:12.840 contributed to having leapsiex side available as free software, which was in the case before. 10:14.520 --> 10:21.480 So the story here regarding MPI is that I think we can bring our own MPI on the machine, 10:21.480 --> 10:28.680 right? It actually works. That's also something container people wonder about. Can we bring our 10:28.680 --> 10:32.760 MPI? Is it really going to work? Do we are we going to have the performance? Well, yes, 10:32.840 --> 10:36.920 and it works and there is no surprise, you know, you don't need to do some guest work regarding 10:36.920 --> 10:42.520 ABI compatibility, things like that. You have a world-tested software stack, you just bring it to the 10:42.520 --> 10:51.160 machine and it works as expected. That's a story. So that's one aspect of performance, but then 10:51.160 --> 10:56.040 there's also, you know, micro architecture. So we have this package transformation, 10:56.040 --> 11:02.680 dash dash tune that we can use to say, well, please build this Agen benchmark package specifically 11:02.680 --> 11:08.040 for Sky Lake. For example, that's on my laptop. And then, well, it gets rebuilt with the, you 11:08.040 --> 11:15.960 know, the dash MRH flag. And you get the expected, you know, level of performance. And likewise, 11:15.960 --> 11:21.640 now we're doing this for AMD GPUs. This is brand new. Actually, the pull request was merged like a 11:21.640 --> 11:27.560 week ago. And this is very nice. So you can say, well, I want this package, but I want it to be 11:27.560 --> 11:34.440 specifically built for this and that AMD GPUs. So the name are not really human-miniq for, right? 11:35.160 --> 11:40.920 But there's, there's documentation. It's like, am I instinct 250 something? Anyway, 11:42.040 --> 11:50.200 and this is pretty nice. So I did mention before that Giggs is not available on, on the big 11:50.280 --> 11:57.640 supercomputers, right? Like, national level European level. So how do we do? Well, we can actually 11:57.640 --> 12:04.440 use this Giggs pack command to bundle up our software stack, you know, create a bundle. So in this 12:04.440 --> 12:10.200 case, it's going to be a table, right? Because we know this software stack is well tested. So we 12:10.200 --> 12:16.200 created a table that contains Intel MPI benchmarks and all the dependencies. And then we can basically 12:16.200 --> 12:23.400 SCP it to the to the target supercomputer. We ship it entirely. We extract it and form there. 12:23.400 --> 12:28.440 We can directly run our stuff. So in this case, running the Intel MPI benchmark, 12:29.480 --> 12:35.080 you know, using all the software stack that was packaged in Giggs. So including Open MPI, 12:35.080 --> 12:41.640 including OLEDs, dependency, all these dependencies, all the way down to Leipzig. And it actually works. 12:42.600 --> 12:48.120 So in short, what we are doing now is that we're using Giggs. You can use it on your laptop. 12:48.120 --> 12:53.800 You can also use it on those tier two clusters. Giggs is already installed. But if you're 12:53.800 --> 12:59.560 targeting one of these bigers, one of these bigers, sorry, supercomputers, while you can use Giggs 12:59.560 --> 13:05.720 pack to create a table, an obtainer image, or a Docker image, and ship it directly to the machine. 13:06.360 --> 13:12.920 So it's pretty cool, but it would be even cooler if these machines were running Giggs directly. 13:14.600 --> 13:19.560 All right, there are demos and tutorials. You can look at if you're interested in how this 13:19.560 --> 13:25.800 actually works in practice. And people are actually using Giggs, I should say, in a number of different 13:25.800 --> 13:31.080 fields, you know, it's not just a geeky thing, right? People are doing research using these things. 13:31.560 --> 13:39.080 All right, so this was sort of the dream part as I see it as some of a seed because, you know, 13:39.080 --> 13:43.720 using the package manager directly on the machine. But what about reality? 13:45.320 --> 13:51.560 Well, reality is we are still using modules, as I said before, right? So if you go to the documentation 13:51.560 --> 13:57.160 of all these major supercomputers, they say that the first thing you have to do is use modules 13:57.160 --> 14:03.960 to get the officially supported software packages, right? And I was like, why is it still the case in 14:03.960 --> 14:11.160 2020-26, you know? Because SPAC, for example, supports and geeks actually supports and privilege 14:11.160 --> 14:16.760 user installs, you know, you're supposed to be able to bring SPAC or easy build on the super 14:16.760 --> 14:22.120 computer and to run directly, you know, SPAC install, etc, and get your packages there. 14:22.520 --> 14:28.680 And there is a story I keep hearing about, I note, quotas, so maybe it brings a belt to some of you. 14:29.480 --> 14:35.640 Well, if you start using SPAC install in the supercomputer, for example, then you're likely to 14:35.640 --> 14:41.480 run out of vinyl because they're the quota, right? And if you build a lot of software packages, 14:41.480 --> 14:46.040 well, maybe you're going to use a lot of this space and nine notes. And this is from a user 14:46.120 --> 14:52.200 community report of one of the French national supercomputers. So maybe that's one reason. 14:53.800 --> 14:59.080 And this is a solution to this, as I understand it, and there's going to be a SPAC talk right after 14:59.080 --> 15:04.280 me, so maybe we'll learn more about this. But anyway, one thing, a supercomputer, and it means 15:04.280 --> 15:11.720 to is that they provide, for example, SPAC or is built or both pre-configured and also in some 15:11.720 --> 15:18.360 cases, the SPAC module, they provide because it's still a module, right, is changed to the 15:19.080 --> 15:25.320 two software that the admins build with SPAC on the machine. So what that means, as I understand 15:25.320 --> 15:29.800 it is that you're not going to have to rebuild everything because this SPAC that you're going to use 15:29.800 --> 15:36.440 is changed with, you know, step that was already built. But you're still stuck with one particular 15:36.440 --> 15:42.520 version of SPAC or is built that's provided generously to you by the admins, right? You cannot 15:42.520 --> 15:49.800 just, you know, pick the version that you want. And I did a quick survey here, and, you know, 15:49.800 --> 15:56.520 some of the, so the first ones are the Euro-HPC clusters, some of them, some of them provide 15:56.520 --> 16:00.920 both, some provide just one, some provide none of them, but the typically provide 16:00.920 --> 16:07.400 obtainer or something equivalent. And the last ones are the French clusters, which are maybe 16:07.400 --> 16:14.520 a bit more conservative here. The quick show of hands, who regularly uses a super computer in 16:14.520 --> 16:23.160 this audience, right, the vast majority, who uses a package manager on super computers. 16:23.480 --> 16:28.920 Yeah, that's, that's still a lot of people. Okay, because I was curious about this, 16:28.920 --> 16:36.680 who I asked people operating a dust ray in France, so people at CNS, and they got me this data 16:36.680 --> 16:42.760 about SPAC usage. So it's, they provide two versions of SPAC on a dust ray, and this is, 16:42.760 --> 16:49.880 this is one of them, and they have statistics on who basically does module load SPAC. And that's, 16:49.880 --> 16:57.880 like, we're talking about 1% of the 1200 users of a dust ray. Again, this is just one example, 16:57.880 --> 17:03.880 maybe it's different if you go to Lumie or Jewels, I have no idea. But I do have a feeling that 17:03.880 --> 17:08.840 package managers are not so widespread, still in HPC, at least, not in the hands of users. 17:11.560 --> 17:17.240 Well, yeah, people in Seoul are using a package manager, but we can have that discussion 17:17.240 --> 17:25.400 afterwards. So yeah, all right, so so what's next? What can we do about it? Actually, I have no idea 17:25.400 --> 17:31.320 to be honest, but I can answer at least for the geek side of things. So I'm, I'm part of 17:31.320 --> 17:38.040 NAMPEX, which is the French HPC program from the French research program for HPC, and we get to work 17:38.040 --> 17:42.920 with a bunch of people in academia, mostly people developing scientific applications, typically, 17:42.920 --> 17:50.280 so users of HPC clusters. But we will see get to talk and to work with people who operate 17:50.280 --> 17:54.680 clusters, and that's kind of interesting to me, because I would like to understand, you know, 17:54.680 --> 18:01.960 what's happening? Why are they not promoting package managers more than this? And we learn from it, 18:01.960 --> 18:07.320 so, and by the way, we're hiring, so if you're interested in all this, you can check out the 18:07.320 --> 18:13.960 HPC.gigs.info page. So one thing, for example, that we learned is that from this perspective, 18:13.960 --> 18:21.800 geeks is a showstopper, because it has a build demon that runs as root, right? So if you're not familiar, 18:21.800 --> 18:27.400 while when you run a geeks command, it makes remote procedures to that build demon, that build 18:27.400 --> 18:34.520 demon might well be software or download tribute binaries, or do nothing if it's already available locally. 18:35.080 --> 18:41.880 The problem is that this build demon was running as root, the good news is that it's now fixed 18:42.520 --> 18:48.600 because, well, the unprivileged user namespaces and Linux, which is what enable this work, 18:48.600 --> 18:55.320 are now widely available, even on supercomputers, and so we developed a rootless variant of the geeks 18:55.320 --> 19:01.400 demon, and that's much better, you know, from a security perspective, for HPC CIS, that means 19:01.480 --> 19:09.800 it makes quite a big difference. And so we have some good news, thanks to this, to additional work 19:09.800 --> 19:16.040 we've been doing over a few months now, geeks is coming to one of these supercomputers to a 19:16.040 --> 19:23.640 dastore operated by CNS in France, which is, you know, one of these tier one clusters, so 19:23.640 --> 19:29.000 soonish, like in the coming months, and we'll see how it goes, I hope it's going to be fruitful, 19:29.640 --> 19:38.280 I think it will be, let's see. So I think it's time to wrap up. My impression is that we have 19:38.280 --> 19:43.560 package managers and in particular, spec and PCB, they are the stars right in the room, 19:43.560 --> 19:49.400 because they're very popular in HPC circles. They're very much used by system administrators, 19:49.400 --> 19:55.560 on HPC clusters, but they were not supposed, their audience was not supposed in my opinion to be 19:55.640 --> 20:01.160 system administrators, right? I think the goal, you know, back in 2015 when we were young, 20:01.160 --> 20:06.280 was to put those package managers in the hands of users, and there is some success according to 20:06.280 --> 20:11.400 a quick poll remaining this from, but I think we can do better, and I hope we will actually 20:11.480 --> 20:16.520 make it a reality. And this is it. We have time for questions. 20:26.760 --> 20:29.000 Questions, anyone? Yes. 20:41.400 --> 20:45.000 What's the thing? What's the barrier? 20:45.000 --> 20:50.520 The question is, what's stopping me and people from using gigs directly on the cluster without 20:50.520 --> 20:59.000 asking anyone? There are several things about this. One thing is, when you install gigs, 20:59.000 --> 21:05.720 you have to run this build-demon thing. It runs root-less now, but the problem is, if you install it 21:05.720 --> 21:09.960 by yourself on the machine while you're going to end up rebuilding lots of stuff, right? 21:09.960 --> 21:15.320 And it's going to take ages, and you're going to run out of high notes. So this is not really 21:16.360 --> 21:23.000 practical. So instead, you know, when system administrators install gigs, system-wide, 21:23.000 --> 21:29.880 then everybody can use gigs and the storage and the computational course is shared among everybody. 21:29.880 --> 21:35.720 So, for example, if I install GCC and somebody else install the same GCC, it's going to be stored 21:35.720 --> 21:40.520 once on the machine, and that makes quite a bit different. Yeah, it's much better. 21:48.520 --> 21:53.080 Ah, that's a very good question. How do we assure that people are using gigs typically 21:53.080 --> 22:00.040 correctly on the supercomputer? That's a very good question. So, I don't have a very good answer 22:00.040 --> 22:05.240 to be honest. So, for example, when I mentioned spec user and disabled user, typically system 22:05.320 --> 22:11.880 administrators provide pre-configured versions of spec or easy-built. And so, they can sort of avoid 22:11.880 --> 22:19.800 the problems, I guess, some of them. But in the case of gigs, well, I mean, what we do is provide, 22:19.800 --> 22:26.600 for example, that generic of an MPI package. So, we know, we can just ensure that it works on all these 22:27.240 --> 22:35.000 networking tokenics, but then people can still shoot themselves in the food, right? So, I don't have 22:35.080 --> 22:37.000 very good answer to that. 22:53.000 --> 22:58.040 So, the question, if I understand correctly, is how would I run proprietary software with gigs? Did I 22:58.920 --> 23:08.520 install and manage? Okay. Well, so, gigs allows you to install software, be it proprietary, 23:08.520 --> 23:13.880 or not. Obviously, I'm not a fan of proprietary software, but we have people using coulder, 23:14.680 --> 23:21.800 obviously, on HPC clusters. And so, we have coulder packages, for example. And at the command line, 23:21.800 --> 23:26.200 never, it works in pretty much the same way as any other package. You don't see the difference, 23:26.280 --> 23:29.800 except that it's good. So, it's a bit clunky, but it's not a problem. 23:37.080 --> 23:41.800 Can gigs be combined with the modules already available on the system? No, it cannot. 23:42.840 --> 23:49.640 And that's one of the fundamental differences between gigs and USB exhaust pack. USB 23:49.640 --> 23:55.000 and spec are specifically designed to allow you to combine, you know, spec-rated is a bit 23:55.000 --> 24:00.200 provided packages with modules. And not just modules, but everything that happens to be in 24:00.200 --> 24:06.840 userly, for example, which can be nice, but the downside is that if you go from one machine to another, 24:06.840 --> 24:11.000 well, you're not going to have the same modules. So, how do you, you know, deploy the same software 24:11.000 --> 24:18.040 stack, you cannot really do that, right? And so, instead gigs, you know, controls the entire software stack, 24:18.120 --> 24:24.360 and so, everything is handled by gigs, which means you can also test the software stack before you 24:24.360 --> 24:29.240 actually start choosing it on those machines. Question? 24:34.200 --> 24:40.200 Does a bill demon has to be on the same machine as gigs? No, it doesn't have to be. Yes. So, in 24:40.200 --> 24:45.800 practice it's on a cluster, there's one dedicated node for the bill demon. That's how people do it. 24:46.520 --> 24:52.200 Probably you're wondering about the security thing, right? Yes. Well, the security thing. I mean, 24:52.200 --> 24:57.560 it's some typically people on clusters installed on a separate node, dedicated node, but still, 24:57.560 --> 25:04.920 you know, there's also securities also in part about paperwork and, you know, people are more 25:04.920 --> 25:09.960 comfortable with having a rootless demon, which I mean, is also understandable generally speaking, 25:09.960 --> 25:35.960 I guess. Yeah. Yeah. Yeah. So, the time is it basically, is it a security risk 25:35.960 --> 25:41.240 users installed stuff and others use that stuff and that stuff is not secure, right? I don't think so, 25:41.240 --> 25:45.960 because to reuse the software, you have to sort of do it on purpose. Like for example, the, you know, 25:45.960 --> 25:53.240 the Intel MPA benchmarks where I customize and swap to an MPA by MPH. Unless you explicitly ask for 25:53.240 --> 25:58.440 that, you're not going to get, you know, the fine name of that binary, so you cannot, you won't 25:58.440 --> 26:03.480 end up running it by mistake, right? You have to explicitly ask for it to actually run it. 26:04.440 --> 26:09.960 I guess we have to wrap up, but we can, no, I didn't speak here. So, question. 26:17.960 --> 26:24.040 In the rootless bit lyrids installation directory is still a shadow, a new store directory, yes it is, yes, 26:24.040 --> 26:33.320 that doesn't, does the demon have right access to the new store directory? The demon 26:33.400 --> 26:39.080 has right access to the new store directory, but only the demon does. So, as a user, you cannot 26:39.080 --> 26:43.480 write into that directory, only the demon does. It's a trusted third party. 26:45.960 --> 26:48.680 Okay. Thank you. Now of a spark.