WEBVTT 00:00.000 --> 00:12.400 Welcome to my talk about reproducible builds on Android. 00:12.400 --> 00:19.200 I will show you today, let me first introduce myself, my name is Andreas Xtak Raybert. 00:19.200 --> 00:24.000 Going by the Nick Izzy, I started Izzy on Android in 2015. 00:24.000 --> 00:27.000 We are a member of the team for a couple of years. 00:27.000 --> 00:31.000 We provide APKs, built and signed by the corresponding developers. 00:31.000 --> 00:36.000 After performing some additional checks, like checking if there are clean virus total, 00:36.000 --> 00:38.000 what's inside, libraries and so on. 00:38.000 --> 00:42.000 For two years now, we also provide reproducible builds. 00:42.000 --> 00:46.000 Meanwhile, covering about two sets of the apps in our repository to be more precise. 00:46.000 --> 00:52.000 It's going to be 62.4% of our apps which are covered by reproducible builds. 00:52.000 --> 00:55.000 I will today introduce you to our reproducible builds. 00:55.000 --> 00:59.000 How they are set up and also compare with what FDroid is doing is, 00:59.000 --> 01:01.000 I'm a former FDroid main planner as well. 01:01.000 --> 01:04.000 Specific questions go to the end. 01:04.000 --> 01:08.000 If there are some understanding questions, so you lose context, 01:08.000 --> 01:12.000 I hope it's okay if you can ask them in between. 01:12.000 --> 01:15.000 I'll talk to fast, please give me a signal. 01:15.000 --> 01:17.000 I'm going to talk to fast. 01:17.000 --> 01:20.000 I'll try my best to keep down. 01:20.000 --> 01:22.000 Okay. 01:22.000 --> 01:25.000 Today, first of all, overview. 01:25.000 --> 01:29.000 The slide is also available at what is the code. 01:29.000 --> 01:31.000 Pre-tots. 01:31.000 --> 01:35.000 It's also available here for QR code, and there are a few classes at the end 01:35.000 --> 01:39.000 to scan for the slides. 01:39.000 --> 01:43.000 First, item we want to have is what are reproducible builds, 01:43.000 --> 01:45.000 and what are they good for. 01:45.000 --> 01:47.000 How do we approach reproducible? 01:47.000 --> 01:52.000 It's an easy on-droid, and what's the difference for FDroid. 01:52.000 --> 01:54.000 What are the main challenges? 01:54.000 --> 01:57.000 I can't put them up all, it would take too long. 01:57.000 --> 02:00.000 We would need two sessions, at least. 02:00.000 --> 02:04.000 What are the most frequent reasons of failed reproducible builds? 02:04.000 --> 02:06.000 Some of them also can fit. 02:06.000 --> 02:08.000 How we can fix them. 02:08.000 --> 02:11.000 What should Android app developers be aware of and take care 02:11.000 --> 02:16.000 for to give the best chances for their apps to be reproducible? 02:16.000 --> 02:20.000 And then we have a question and answers. 02:20.000 --> 02:23.000 Okay. First, what are reproducible builds? 02:23.000 --> 02:26.000 What does it mean to have a build reproducible? 02:26.000 --> 02:30.000 You as a developer, build an APK, sign it, and provide it for install. 02:30.000 --> 02:32.000 We give the source available. 02:32.000 --> 02:35.000 We take your source, try to build the APK. 02:35.000 --> 02:38.000 And if you come up with a bite identical copy, 02:38.000 --> 02:40.000 that means you reproduced your APK. 02:40.000 --> 02:42.000 We can tell everybody hey, this one is honest. 02:42.000 --> 02:45.000 They really build this APK from their source code. 02:45.000 --> 02:49.000 So that is basically what means reproducible builds. 02:49.000 --> 02:54.000 Sir Party's make sure independently that your software isn't been altered. 02:54.000 --> 02:58.000 Such increasing safety, reliability, and trust. 02:58.000 --> 03:03.000 Because people who trust you or think maybe either can trust you. 03:03.000 --> 03:07.000 Oh, somebody else could produce it, so it's really fine. 03:07.000 --> 03:12.000 We have, we make air, well that's a part of it. 03:12.000 --> 03:16.000 And make sure that the developers code always works the same way. 03:16.000 --> 03:18.000 That is the best case. 03:18.000 --> 03:21.000 Some builds are the smart sound funny, but it is a way. 03:21.000 --> 03:24.000 They are non-deterministic, but reproducible. 03:24.000 --> 03:25.000 How can that be? 03:25.000 --> 03:27.000 We build in the loop and we find the right one. 03:27.000 --> 03:32.000 There are some, how to say some errors in the build frameworks. 03:32.000 --> 03:37.000 That cause different order of bite code, which causing reproducible to fail. 03:37.000 --> 03:41.000 If you try often enough, one time lucky, you find the right one. 03:41.000 --> 03:46.000 So we can prove, okay, it's the code just working the same way if you don't need to loop. 03:46.000 --> 03:51.000 But it makes the software also more consistent because you know it works the same way. 03:51.000 --> 03:56.000 And again, it's a trust case. 03:56.000 --> 04:01.000 If it doesn't sum up and you've got some completely different one, you run a different scope on it and you see, 04:01.000 --> 04:06.000 Well, it's an entire block of code that is not present in the reproducible build. 04:07.000 --> 04:15.000 You know, maybe somebody was speaking into your code, but putting something in and poison supply chain or whatever. 04:15.000 --> 04:18.000 Or you didn't pin the dependencies properly. 04:18.000 --> 04:19.000 There's a difference. 04:19.000 --> 04:23.000 So you see it shows up things where there are improvements. 04:23.000 --> 04:28.000 You might want to add to your process. 04:28.000 --> 04:32.000 A side effect of this is if you have it really matched. 04:32.000 --> 04:36.000 Now, it comes to business case and says, we need to make sure we comply with all eye sensors. 04:36.000 --> 04:40.000 We can, because we know what is inside, it is proven what is inside. 04:40.000 --> 04:46.000 Everything is inside is mapped and we can say, if I say, if I say, if I say, if we meet that, everything is fine. 04:46.000 --> 04:52.000 Nothing was struck in by some dependency of a dependency of a dependency that violates some of the eye sensors. 04:52.000 --> 04:54.000 Everything is fine. 04:54.000 --> 05:02.000 Yeah, binary is matched, so the source code obviously has to match. 05:02.000 --> 05:05.000 I already said that before, protect your software against being compromised. 05:05.000 --> 05:10.000 That is what we had at the start point already. 05:10.000 --> 05:19.000 If the results are identical, it's protecting your software against being compromised because nobody has put something in the middle. 05:19.000 --> 05:23.000 Or taking away. 05:23.000 --> 05:26.000 The increased confidence and trust. 05:26.000 --> 05:29.000 I said that already, but there's one more point to it. 05:29.000 --> 05:31.000 I hope that is in here. 05:31.000 --> 05:32.000 Yep. 05:32.000 --> 05:36.000 It also can be used to confirm ownership in case of signing kiddo. 05:36.000 --> 05:37.000 Because we have the records. 05:37.000 --> 05:39.000 We have the records from the previous both. 05:39.000 --> 05:41.000 It says, this commit, we have both from. 05:41.000 --> 05:44.000 Is this commit, is still in your get repository. 05:44.000 --> 05:47.000 And you didn't overwrite the history so we can match that. 05:47.000 --> 05:52.000 There's an email address to the commit and we can say, yes, okay, that matches. 05:52.000 --> 05:55.000 And we can confirm your key was lost. 05:55.000 --> 06:00.000 You didn't just have somebody introduced his own stuff in materials. 06:00.000 --> 06:09.000 And overtake your repository, but it was really good. 06:09.000 --> 06:11.000 I take it that way. 06:11.000 --> 06:12.000 Okay. 06:12.000 --> 06:16.000 Let's, in overview about what we produce of the builds are. 06:16.000 --> 06:17.000 And what are they good for? 06:17.000 --> 06:19.000 There are many more use cases. 06:19.000 --> 06:22.000 Just the core ones. 06:22.000 --> 06:25.000 So how do we approach the produce of the builds? 06:25.000 --> 06:28.000 That is the Android and on FDroid. 06:28.000 --> 06:30.000 That is the Android and the progressive of the builds. 06:30.000 --> 06:32.000 Run on a complete separate track. 06:32.000 --> 06:34.000 You ship the original of AP case. 06:34.000 --> 06:36.000 Don't sign pass the developers. 06:36.000 --> 06:39.000 And in the second machine, we run the reproduce of the build. 06:39.000 --> 06:43.000 And the clients come over to you two or two to understand. 06:43.000 --> 06:44.000 We can present it to you. 06:44.000 --> 06:48.000 The clients show a green shield to say, yes, confirmed. 06:48.000 --> 06:49.000 That might be delayed. 06:49.000 --> 06:50.000 It comes later. 06:50.000 --> 06:52.000 Because runs on a separate track. 06:52.000 --> 06:55.000 At FDroid, it is integrated with the build system. 06:55.000 --> 06:57.000 FDroid ships their own builds. 06:57.000 --> 06:59.000 And if they are reproducible and not the reproducible, 06:59.000 --> 07:01.000 they ship the AP case of the developer, 07:01.000 --> 07:03.000 but only after the build process. 07:03.000 --> 07:07.000 So the build process takes a bit longer. 07:07.000 --> 07:10.000 There is, if a build fails to be reproducible, 07:10.000 --> 07:13.000 it's the AP case with an FDroid one bishop. 07:13.000 --> 07:15.000 So you get extra security. 07:15.000 --> 07:17.000 It could potentially be a problem. 07:17.000 --> 07:19.000 So we hold it back. 07:19.000 --> 07:20.000 It is the Android. 07:20.000 --> 07:21.000 You get the AP case. 07:21.000 --> 07:22.000 But you see the green shield. 07:22.000 --> 07:23.000 The shield is not there. 07:23.000 --> 07:26.000 Our clients in the future will have a feature that says, 07:26.000 --> 07:29.000 I only want to install it when it's was confirmed. 07:29.000 --> 07:32.000 So we're giving back freedom to the, 07:32.000 --> 07:34.000 because I was using our repository to say, 07:34.000 --> 07:37.000 you can decide what level of security you want for yourself. 07:37.000 --> 07:39.000 We don't tell you how to do things. 07:39.000 --> 07:40.000 We just give you the options. 07:40.000 --> 07:41.000 And you can decide. 07:41.000 --> 07:46.000 It is the Android. 07:46.000 --> 07:48.000 We use a tool called RVT lock. 07:48.000 --> 07:51.000 That's a reproducible transparency lock. 07:51.000 --> 07:54.000 We have the resets checked in the repository. 07:54.000 --> 07:56.000 And we have the locks. 07:56.000 --> 07:57.000 Also checked in is the JSON. 07:57.000 --> 08:00.000 So we can confirm that the build was run. 08:00.000 --> 08:01.000 That's the entire build lock. 08:01.000 --> 08:04.000 And we got a file hash of A. 08:04.000 --> 08:06.000 The original was the file hash of A. 08:06.000 --> 08:08.000 So the bow is met. 08:08.000 --> 08:10.000 It's a char 256 match. 08:10.000 --> 08:12.000 So it's reproducible. 08:12.000 --> 08:14.000 I've tried to use this, 08:14.000 --> 08:18.000 I've tried to offer for the builds and for the reproducible builds. 08:18.000 --> 08:20.000 But they have also second setup. 08:20.000 --> 08:22.000 Complete later. 08:22.000 --> 08:24.000 Both sites using only for tools for that. 08:24.000 --> 08:26.000 That's important for us. 08:26.000 --> 08:27.000 For both sites of us. 08:27.000 --> 08:30.000 And it is the Android. 08:30.000 --> 08:32.000 We use podmen containers with different images. 08:32.000 --> 08:35.000 We have a used flexibility that way. 08:35.000 --> 08:37.000 We can use debion bolts. 08:37.000 --> 08:39.000 The book says you don't want to jump in, 08:39.000 --> 08:43.000 no. 08:43.000 --> 08:45.000 To match environments as possible, 08:45.000 --> 08:47.000 it's którye to simply replace the files. 08:47.000 --> 08:51.000 If you use this varnthus debion Bookwall for the main. 08:51.000 --> 09:00.000 But there are very services are вид vengeance. 09:00.000 --> 09:02.900 For the last year, 09:02.900 --> 09:05.000 the boot will go for the buttons, 09:05.000 --> 09:09.160 Figure out which apps could be reproducible to signal to the developers. 09:09.160 --> 09:13.480 Hey, don't you want to set it up as reproducible build? 09:13.480 --> 09:16.120 It's not easy to switch between the two at F-Droid because if it's not 09:16.120 --> 09:19.080 reproducible, it's signed by F-Droid. 09:19.080 --> 09:23.920 So if you switch to reproducible build, everybody has to answer and reinstall. 09:23.920 --> 09:30.440 Accept yourself as a separate track, but that is not part of my talk today. 09:30.440 --> 09:33.640 Is the Android runs multiple burrows, including independent burrows. 09:33.640 --> 09:35.520 So we run two burrows ourselves. 09:35.520 --> 09:41.680 There's a third burrow run by an independent party to confirm our builds as well. 09:41.680 --> 09:42.840 So we can plug in more. 09:42.840 --> 09:49.120 If anybody of you is interested in, just meet me up at UD2O2 for details. 09:49.120 --> 09:53.440 It's easy to set up a burrow, we have even a tool for that that sets it up in five minutes 09:53.440 --> 09:58.400 and try to build the same business, all tools you need, whatever you want for it. 09:58.400 --> 10:05.040 So we can have independent burrows, proving our builds independently once more. 10:05.040 --> 10:08.160 So the developer can say, I have signed it. 10:08.160 --> 10:14.600 He has proven it was proven by that person that he has proven it right. 10:14.600 --> 10:18.640 And after all, it's currently as far as I know, only after its own builds are 10:18.640 --> 10:21.520 wasn't built, and also their vacation system. 10:21.520 --> 10:24.840 So there's a system you could also set up. 10:24.880 --> 10:29.040 I'm not involved with the details of their build system, 10:29.040 --> 10:31.800 but it's also not part of my talk today. 10:31.800 --> 10:36.040 So we have an independent verifying developer's build. 10:36.040 --> 10:38.440 We always ship the developer's build. 10:38.440 --> 10:41.160 After all, it's independently verifying developers' builds 10:41.160 --> 10:44.200 for apps established as reproducible builds that have tried. 10:44.200 --> 10:49.800 And otherwise, that other check, which could be established as reproducible builds. 10:49.800 --> 10:54.800 And they have their own ones, which they build twice to say, well, okay, it's the same outcome. 10:54.800 --> 11:00.200 So nothing in between our research, 11:00.200 --> 11:02.360 I'd be always shipping the APK from the developers. 11:02.360 --> 11:07.480 And after I'd ship the developer's APK efforts reproducible, 11:07.480 --> 11:11.440 and set up as reproducible builds, if it's not set up as reproducible builds, 11:11.440 --> 11:13.120 they ship their ones. 11:13.120 --> 11:17.480 If it is set up as reproducible, but doesn't reproduce, the ship nothing. 11:17.480 --> 11:18.240 As I said before. 11:18.240 --> 11:26.760 So, what are the challenges with reproducible builds? 11:26.760 --> 11:30.280 I can only stretch on the surface, because there are many challenges. 11:30.280 --> 11:35.480 So I want just to mention the main ones when setting up the first reproducible builds. 11:35.480 --> 11:38.880 I can go, things can go wrong later with the next release, 11:38.880 --> 11:42.760 or next to next release, we can't cover that all. 11:42.760 --> 11:50.520 But I want to mention some, also independent of the build frameworks used. 11:50.520 --> 11:52.880 It's very easy to handle Java and Kotlin, 11:52.880 --> 11:57.160 basically you just need to match the JDK version. 11:57.160 --> 12:03.680 So in most cases, it's a fashion to set up a reproducible build. 12:03.680 --> 12:06.440 Exceptions exist. 12:06.440 --> 12:08.880 More complex, the gets when you have native libraries, 12:08.880 --> 12:12.280 like the Flutter, NodeJS, React, Go, Rust, 12:12.280 --> 12:20.560 do name it, Expoh, whatever, because those frameworks and bet the build pass into the libraries. 12:20.560 --> 12:22.360 So you also need to match the build pass. 12:22.360 --> 12:24.080 You need to match the CMake version. 12:24.080 --> 12:26.120 You need to match a lot of other stuff. 12:26.120 --> 12:31.200 So it gets easier to set up a Java application, maybe in 10 minutes, 12:31.200 --> 12:35.120 15 minutes, depending on the build runs to confirm as it worked out. 12:35.120 --> 12:40.000 If you set up the Flutter version, the Flutter app for the first time takes maybe two hours, 12:40.000 --> 12:42.160 to get it right, to match that. 12:42.160 --> 12:47.440 And then you also have the developer has installed in the NDK. 12:47.440 --> 12:52.480 So libraries get stripped of the symbols, but all builders don't have. 12:52.480 --> 12:55.080 So all our stone strips, we have a different outcome. 12:55.080 --> 13:01.320 So you need to figure which version of the NDK has the developer installed on the laptop to match that. 13:01.320 --> 13:06.520 So that's getting the more complex, the more native stuff you have in. 13:06.520 --> 13:09.360 Build pass and bet it. 13:09.360 --> 13:14.240 Pass from Windowsburg, it's very, but try to match C code on whatever, on a Linux system. 13:14.240 --> 13:15.520 No, we don't put on Windows. 13:15.520 --> 13:17.840 We build on open source. 13:17.840 --> 13:24.760 So no Windows, just to scroll things out, maybe in for some, like to come in on fresh air. 13:24.760 --> 13:29.360 There are ways sometimes to get it matched by saying we want, 13:29.360 --> 13:34.680 not absolute pass, we want relative pass, doesn't always work out, gets difficult. 13:34.680 --> 13:39.000 Different version of different build SDKs produce different output, Flutter. 13:39.000 --> 13:41.720 You say, yeah, we always use latest for reproducerability. 13:41.720 --> 13:46.280 No, latest today is not the same, latest moral. 13:46.280 --> 13:53.000 So somebody, you build today, we build later than you, maybe in between there was another update from Flutter. 13:53.000 --> 13:58.400 Another one wants to check if we are doing right and build the ones later. 13:58.440 --> 14:02.000 There are two versions later already, completely different output. 14:02.000 --> 14:09.600 So you need to pin the versions even of the environments of the Flutter, exactly 3.88.8, whatever. 14:09.600 --> 14:16.320 Not just latest, or greater than between and doesn't work. 14:16.320 --> 14:25.960 Okay, the most, some I mentioned already, the links, if you download the PDF, you can follow the links. 14:25.960 --> 14:31.480 We have the first basic rule that is adding up for 80% of failures, if not kept. 14:31.480 --> 14:40.040 Always build your APK from a clean tree, clean tree, as a commit, your release tag points to. 14:40.040 --> 14:44.240 Which order you do that if you first put and then tag, doesn't matter as long as I match. 14:44.240 --> 14:49.920 And clean tree means no local changes and also no remaining artifacts from previous builds. 14:49.920 --> 14:54.120 Because they cause already different outcome. 14:54.160 --> 14:57.640 That makes up for 80% of the failures. 14:57.640 --> 15:01.720 Some of that are framework and autocorrect, but not all. 15:01.720 --> 15:04.720 Switching between Windows and Linux. 15:04.720 --> 15:07.440 This version is next version, the other one. 15:07.440 --> 15:13.720 We have to correct the linebacks, we can do that. After watching our APK, we say, okay, if we play Windows and other linebacks, 15:13.720 --> 15:16.320 it's identical again. 15:16.320 --> 15:21.880 But it's a changes with each version, because it means every time manual intervention to correct the builders have. 15:21.880 --> 15:28.760 So that causes us headache and unreaded additional time Spanish. 15:28.760 --> 15:31.720 Introducing more or less complex and forced signing. 15:31.720 --> 15:38.200 It's nice if your build setup automatically signs the APK, but we don't have your keys, and we shouldn't have your keys. 15:38.200 --> 15:41.880 We don't have your credentials, and we should never have your credentials. 15:41.880 --> 15:44.760 What we build is an unsigned APK. 15:44.760 --> 15:50.440 It's the APK is complete, we say APK sick copy. 15:50.520 --> 15:56.120 Copy over the same, as other signing stuff from the original APK, which only works if it's otherwise identical. 15:56.120 --> 15:57.920 Or the second trouble fail. 15:57.920 --> 15:59.800 That's how we test if it's reproducible. 15:59.800 --> 16:04.800 Can be copy over the signature and do we get the same head? 16:04.800 --> 16:08.320 So if you introduce complex signing, the build breaks. 16:08.320 --> 16:10.280 Oh, I can't find the signing key. 16:10.280 --> 16:11.480 Please don't present. 16:11.480 --> 16:14.680 Whatever. 16:14.760 --> 16:17.480 Building locally with stuff from setups and specified. 16:17.480 --> 16:20.680 Your build header checked in and to your repository says, 16:20.680 --> 16:24.760 I use Gradle 8.14. 16:24.760 --> 16:27.400 But local you build was at this grade online. 16:27.400 --> 16:29.640 And it doesn't matter. 16:29.640 --> 16:31.000 We get different outcome. 16:31.000 --> 16:32.600 In this case, it's very easy. 16:32.600 --> 16:36.760 It's one file that says what Gradle version was used to be needed to patch that file afterwards. 16:36.760 --> 16:38.280 But it's extra work. 16:38.280 --> 16:43.320 You have to keep it consistent to increase the chances of easy reproducibility. 16:43.320 --> 16:48.520 And third of the build timestamps are ugly, ugly, ugly. 16:48.520 --> 16:53.400 Some frameworks do that like a century, loves to do that. 16:53.400 --> 16:55.560 You build at a different time as we do with those. 16:55.560 --> 16:56.920 And that will always be the case. 16:56.920 --> 17:00.360 We want starts and closely and ends and closely as well. 17:00.360 --> 17:01.320 Can't work out. 17:01.320 --> 17:02.840 So never to build them. 17:02.840 --> 17:06.120 Definitely kills a reproducible build. 17:06.120 --> 17:09.960 Sometimes can be fixed by patching it, but a lot of extra work on this side, 17:10.920 --> 17:12.040 reproducible builders. 17:15.880 --> 17:22.280 Using non-LTS JDKs, like 22 or using 19, 17:22.280 --> 17:25.560 it's not a long time support version. 17:25.560 --> 17:28.520 The versions in between is rather tried to run stuff, which gets three 17:28.520 --> 17:31.560 bulk with the next one and doesn't work 100%. 17:31.560 --> 17:33.320 Yet and so on. 17:33.320 --> 17:39.880 So always stick with LTS versions, like a power way back 11, 17, 21. 17:39.960 --> 17:42.120 And 25, no. 17:42.120 --> 17:43.320 Let's see RTS versions. 17:43.320 --> 17:44.840 L marked RTS. 17:44.840 --> 17:46.360 We only use RTS versions. 17:46.360 --> 17:51.000 So if you come with a 22, it's like playing the lottery. 17:51.000 --> 17:54.680 You can be lucky, but usually you are not. 17:56.040 --> 17:59.000 So part of that, what under the developers 17:59.000 --> 18:02.200 up developers should do, we already had. 18:02.200 --> 18:05.800 We have this page with reproducible buildings for developers, 18:05.800 --> 18:08.120 like the basic first rule we already said, 18:08.120 --> 18:12.520 aforementioned one, and built-time, 18:12.520 --> 18:16.200 related things, like build times, and so on, 18:16.200 --> 18:21.400 because they are usually making it more complicated together reproducible. 18:21.400 --> 18:26.680 For release builds, about canaries, like just talk about the RTS versions, 18:26.680 --> 18:29.080 but you use for test builds, you don't care. 18:29.080 --> 18:34.360 But to release builds should be with a stable framework. 18:34.520 --> 18:43.000 And yeah, basically avoid the problems I just mentioned before, 18:43.000 --> 18:47.160 and that's the basic ones we would like to give. 18:47.160 --> 18:49.640 We have a couple of related links for that. 18:49.640 --> 18:52.600 We have our reproducible build documentation, 18:52.600 --> 18:55.560 including launching what you can do if it fails, 18:55.560 --> 18:57.560 how to look for arrows where you first, 18:57.560 --> 19:02.760 and which order you possess things in what is fixable, and what is not. 19:02.760 --> 19:05.880 And we have our reproducible build framework. 19:05.880 --> 19:09.800 The Arbola set up leads you to the script at set before, 19:09.800 --> 19:11.960 letting you set up a build within five minutes, 19:11.960 --> 19:15.400 with the items to specify, basic only the builder, 19:15.400 --> 19:17.960 or also the tools for checking white failed, 19:17.960 --> 19:19.720 or the repair tools, whatever you want. 19:19.720 --> 19:21.720 You can set it up in the config, and five minutes later, 19:21.720 --> 19:22.520 your builder is running. 19:26.200 --> 19:28.920 Preproducible builds should of course get a reference, 19:28.920 --> 19:32.360 because they are the ones who collect and maintain all the things 19:32.360 --> 19:34.920 around reproducible builds. 19:34.920 --> 19:37.560 Effdroid also has a documentation for reproducible builds. 19:37.560 --> 19:41.320 We want to mention here, and our clients, 19:41.320 --> 19:42.920 draw the fire and near store, 19:42.920 --> 19:50.280 shows the status of reproducible builds up at, is you enjoyed. 19:50.280 --> 19:52.360 OK, I guess I'm in time. 19:52.360 --> 19:54.920 Ah, good. 19:54.920 --> 19:58.520 You can ask any questions you have. 20:02.360 --> 20:12.920 Partly, if you include them as, what is the code? 20:12.920 --> 20:13.800 Pardon? 20:13.800 --> 20:15.080 Ah, OK. 20:15.080 --> 20:17.480 If you cover reproducible dependencies, 20:17.480 --> 20:20.920 that means we reproduced the Arbots off, 20:20.920 --> 20:23.720 but the dependencies, what is about the library's plug-in? 20:23.720 --> 20:28.600 If you have some insight as a good sub, not sub-tree, not sub-tree. 20:28.600 --> 20:30.200 I'll sub-tree, you don't think it. 20:30.280 --> 20:32.200 Sub-tree, of course, we built a sub-tree. 20:32.200 --> 20:36.120 If you have a setup, like, OK, here's a get-clown, 20:36.120 --> 20:39.560 this repository, and build a step, we covered. 20:39.560 --> 20:42.760 If you come and say, well, we have the, whatever, 20:42.760 --> 20:47.480 J&E library in there, no, that depends on the setup. 20:47.480 --> 20:50.840 We try to have everything covered, but other than F-troid, 20:50.840 --> 20:54.760 which says we want to have everything covered, or we don't do it. 20:54.760 --> 20:58.440 We say, you really should, but we don't force you. 20:58.520 --> 21:00.600 But if it's too many of these things, we say, sorry, 21:00.600 --> 21:01.560 it's not all part. 21:01.560 --> 21:05.800 It's about open source, everything should be open. 21:05.800 --> 21:07.800 Answered? 21:07.800 --> 21:08.920 All right, that's the question. 21:08.920 --> 21:09.800 Is it OK? 21:13.800 --> 21:14.600 Yeah? 21:14.600 --> 21:17.400 We used some board-pack numbers of both hardware, 21:17.400 --> 21:20.520 that we need for such a, we need some board-pack numbers 21:20.520 --> 21:24.120 about the hardware that we need for such a builder, if you want, you know? 21:24.120 --> 21:25.240 Do you cover what? 21:25.320 --> 21:27.320 Ballpark numbers. 21:29.320 --> 21:31.960 Or do you use a 10-year-old server, which is for us? 21:31.960 --> 21:33.560 Ah, OK. 21:33.560 --> 21:36.760 No, it was a 15-year-old. 21:36.760 --> 21:40.920 Actually, as I said, we build a supportment containers. 21:40.920 --> 21:42.760 We can name the hardware we have. 21:42.760 --> 21:45.240 It's running on an AMD. 21:45.240 --> 21:50.360 It's hosted by one who provides all servers and everything. 21:50.360 --> 21:54.920 So it's not something you buy at a medium market or something like that. 21:55.240 --> 21:59.080 It's not high-end enterprise hardware. 21:59.080 --> 22:01.320 It's medium software, medium hardware. 22:01.320 --> 22:05.880 But one of our builders runs on custom hardware from a local provider, 22:05.880 --> 22:08.600 just to couple together, something like that. 22:08.600 --> 22:09.400 Right. 22:09.400 --> 22:12.360 Do you do cluster-off? 22:12.360 --> 22:16.280 No, we have plans on this direction, but not yet. 22:16.280 --> 22:21.000 But if your question is, can I set it up on my own cheap laptop? 22:21.000 --> 22:23.560 Yes, you can. 22:23.560 --> 22:27.480 All reproducible based inside the containers. 22:27.480 --> 22:30.520 So the images we use, the containers we fire. 22:30.520 --> 22:34.760 For some stuff, do you need to plan for example? 22:34.760 --> 22:37.320 Yeah, it goes a bit deep inside. 22:37.320 --> 22:40.760 We have to cover CPU count and things like that. 22:40.760 --> 22:43.720 But that is a bit more extended deeper topic. 22:43.720 --> 22:49.400 I can cover that outside, but we have just one minute left also. 22:49.480 --> 22:50.440 Times out? 22:50.440 --> 22:51.960 One minute, okay. 22:51.960 --> 22:53.560 So one more question. 23:04.280 --> 23:07.080 The time stamps show up in the differs group. 23:07.080 --> 23:08.280 Pardon? 23:08.280 --> 23:08.760 Ah. 23:08.760 --> 23:17.560 If you, if you, if you, if you, if you, if you, if you expect hard times from 23:18.040 --> 23:24.360 you, if you expect hard times, then if you extract them from binaries. 23:24.360 --> 23:26.520 Of course, they end up in the binaries. 23:26.520 --> 23:29.000 That's where the differs group shows them. 23:29.000 --> 23:33.960 So we use differs group to discover them and then try to patch them. 23:33.960 --> 23:36.200 But we can't patch them in binaries directly. 23:36.200 --> 23:39.640 It's more complex to explain. 23:39.640 --> 23:43.640 What we basically do is afterwards extract them from the APK, 23:43.640 --> 23:46.680 fix the times, then put them back in the APK, 23:46.680 --> 23:49.640 and check if it's then matching after your real eye 23:49.640 --> 23:51.240 everything. 23:51.240 --> 23:54.760 Can't compress it otherwise. 23:54.760 --> 23:57.160 It's not as sophisticated, really big sophisticated. 23:57.160 --> 24:01.400 It was tricky stuff, but it's just, it's pretty much simple. 24:01.400 --> 24:04.120 Okay, I get the signal times up. 24:04.120 --> 24:05.800 I thank you very much for your interest. 24:05.800 --> 24:11.000 If you have any questions, I'm available to you at UD202. 24:11.000 --> 24:13.320 And see you at our stand.