WEBVTT 00:30.000 --> 00:43.740 Hello, my name is Roman, and I would like to present you how to reproduce a serious 00:43.740 --> 00:50.600 Bob Bach in the five minutes using the word Mi NG. 00:50.600 --> 00:59.500 Okay, a little bit about Mi, I am a occasional Linux kernel contributor, and the software engineer 00:59.500 --> 01:05.980 working for Intel and even Zion RdT features. 01:05.980 --> 01:09.740 Also, I am a regular user of Mi NG. 01:09.740 --> 01:16.500 I provided contact information, so he'll reach me if you have some questions. 01:16.500 --> 01:29.460 Okay, I will start with the introduction of CISBOT, and I am going to show you 01:30.220 --> 01:42.820 which Bach, which Bach I selected to present today, to reproduce today, and then I will continue 01:42.820 --> 01:55.020 with introducing shortly introduced to word Mi NG, and wrap up my presentation with live demo. 01:55.020 --> 02:00.540 So let's get started. 02:00.540 --> 02:12.900 CISColor is a coverage guide Linux kernel faser, and CISBOT, which is a kind of CI, that continuously 02:12.900 --> 02:23.420 uses Mi NG kernel configuration in many versions, and automatically reports, bugs and issues 02:23.460 --> 02:27.420 to different mailing lists. 02:27.420 --> 02:39.420 Also, it supports kind of dashboard, which shows list of recent bugs, let me show you 02:39.420 --> 02:48.740 bugs that I have chosen for this presentation, a lot of information, but we are interested 02:48.780 --> 02:56.740 in just several fields from this list, from this Bach. 02:56.740 --> 03:09.140 First is very interested, of course, in the kernel 3, and commit to be able to reproduce 03:09.140 --> 03:24.740 with CISBOT generates kernel configuration, so it creates it, and also it sometimes, like in this 03:24.740 --> 03:35.700 case, generate C reproducer, so to trigger this issue from user space. 03:35.700 --> 03:44.740 And here you can see that on the left, in the yellow box, crash dump is presented, and 03:44.740 --> 03:55.340 the top is there are two bugs, not two bugs, two patches provided, I selected one of the 03:55.340 --> 04:02.340 patches that already accepted into the kernel. 04:02.340 --> 04:10.420 Next, which is where it means G, where it means G is a kind of tool that significantly 04:10.420 --> 04:23.420 reduces your development cycle, and some work rules, where it means G uses, where it means 04:23.420 --> 04:36.420 G is a Python script that uses QEM, or QEM, like 3, 3, and it eliminates user space. 04:36.420 --> 04:50.100 In fact, it provides a femoral user space that simply inherits user space that your works 04:50.100 --> 05:00.780 use, it uses with VirtioFS, but it's also can use 9p as a phylboc phylboc option, so that's 05:00.780 --> 05:05.260 why it's so fast. 05:05.260 --> 05:16.460 As I note, I cannot say that this tool is covered all possible scenarios, because, for example, 05:16.460 --> 05:26.220 in some cases, you probably will not need user space at all, or you will need some minimal 05:26.220 --> 05:40.660 user space, I provide it to one of the cases in references, so next, and let me also tell 05:40.740 --> 05:49.660 what is 5 minutes, it doesn't mean that we still somehow help to build kernel in 5 minutes 05:49.660 --> 05:50.660 no. 05:50.660 --> 06:05.980 It will take so much time as usual, but VirtioFS in the cycle of build run test loop, preventing 06:06.020 --> 06:14.420 you from maintaining user space and styling kernel to VM images, and do all of this boring 06:14.420 --> 06:33.460 stuff, and now let me show you my demo setup, this is Linux kernel commit, which contains 06:33.540 --> 06:51.540 this bug, and next is, oh I am sorry, oh yeah, and this is my setup, I have here build kernel 06:51.620 --> 07:01.860 all put directory config generated by sysbot, repo generated again by sysbot, and the patch, 07:05.700 --> 07:17.420 let me briefly describe you patch and reproduce, patch is trivial, it fixes error paths, memory 07:17.420 --> 07:27.500 resource leakage on renaming some node in FS, we were not going to go deep and explain 07:27.500 --> 07:44.780 it simply removes with issue, okay, next, about reproducer, I decided to not describe it precisely 07:45.020 --> 07:55.260 because it's outdated and it's kind of messy, so instead I would like to communicate 07:55.260 --> 08:08.140 you how it works, it's quite simple and it's simply run infinite cycle unless it's detect 08:08.220 --> 08:17.900 that something happened in kernel using this game-embley model, kernel model, which is called 08:19.100 --> 08:30.460 kernel, all green, so let me show you, next, this is how our reproducer will look like, 08:31.420 --> 08:43.420 I will run a VNG in two to move the paints, first we'll show this bug in a user space 08:43.420 --> 08:52.220 because it's readed in from this debug file system, and on the second paint I will show you how it 08:52.220 --> 09:04.700 looks like in the message, so that came-in-lick reports something, okay, so let me start with life demo, 09:06.140 --> 09:19.660 here, okay, here you can see that I checked out with my kernel version and build it using 09:19.660 --> 09:33.900 C-cash, okay, now I'm going to switch to another window and here I prepared this VNG, so let's run it, 09:35.580 --> 09:45.340 yeah, now it is run, here we are, typically it takes about one or two seconds to load kernel, 09:45.420 --> 10:00.220 but we use kernel configuration that is generated by C-cash, so let me go to the second paint 10:00.220 --> 10:13.100 and enter to another window, so now we are in and let me initially clear kernel messages, 10:13.980 --> 10:28.140 generated during the boot, and I need to use so-do, yeah, okay, 10:28.220 --> 10:46.940 why it doesn't, because now it will clear the buffer, yeah, and let me show you how we 10:47.020 --> 10:56.380 reproduce our world, I already built reproducer, so all I need is to run this again, I use in 10:56.380 --> 11:09.420 pseudo-repro, so it's go into infinite cycle and now it will start executing, after, yeah, 11:10.620 --> 11:19.660 thank you, after the second execution it will show memory, at least it must, yeah, 11:20.620 --> 11:31.340 it happened, so we see that with this report, this issue, so now I'm going to 11:33.580 --> 11:45.180 turn off this VNG and rebuild the kernel, so, but before I'm going to apply this patch, 11:45.740 --> 12:01.820 get apply, okay, let me show you this patch, okay, this is it, now let me rebuild the kernel, 12:01.820 --> 12:09.900 it will take about probably two minutes and while it is being built, let me return to this 12:10.860 --> 12:20.380 slide and tell a little bit about inefficient and efficient workflows, of course, this, as I said, 12:20.380 --> 12:29.100 this tool doesn't cover all possible scenarios, for example, if you need to switch between 12:29.100 --> 12:35.900 different kernel versions that has let's say it in this way, a lot of distance between versions 12:36.860 --> 12:44.860 many files changes, of course, you will need to rebuild and it will take its time, 12:44.860 --> 12:55.820 so no single bullet, sorry, but efficient workflow implies that you use same kernel versions 12:55.820 --> 13:05.820 that over set of kernel versions that does not far from each other and this allows to 13:06.620 --> 13:14.460 work efficient, for example, if you try to apply different set of patches against this kernel 13:14.460 --> 13:22.860 for testing purposes and, or if you just hack it on the kernel and iterate it on a bug fix, 13:23.820 --> 13:41.660 okay, let me return, yeah, it's built and I'm going to rerun VNG, okay, let's wait for a second, 13:41.660 --> 13:59.020 yeah, we are in and I am trying to reproduce with bug one more time, if you remember previously 13:59.020 --> 14:07.660 it takes two iterations, yeah, sorry, it takes two iterations to reproduce, now it will take 14:08.620 --> 14:18.940 a little, it will not reproduce because it was fixed, so let's wait two more cycles and stop, 14:20.700 --> 14:27.900 so as you can see it is running an infinite cycle and it doesn't produce 14:28.860 --> 14:42.220 this error, so bug was fixed, so this is it, thank you questions, 14:42.300 --> 14:59.100 you're reproduced, I didn't really understand how that work, is this reproduced specific for this 14:59.100 --> 15:08.700 problem because you said it was generated or what does the reproduced do? You mean that if I reproduced 15:08.700 --> 15:23.420 this for, you mean the I reproduced is because if I say like bug to reproduce for, I don't know, 15:23.420 --> 15:31.100 to be able to reproduce it in five minutes or, I mean generally what does the reproduced do? 15:32.060 --> 15:42.700 A reproducer runs infinite cycle, then it's loaded, came in leaked model, let me show you, 15:42.700 --> 15:53.900 let me return back to this slide and I will reproduce this slide, it uses this file, 15:53.900 --> 16:03.500 this kernel debug came in leaked, it turns it on scan mode, then it's activate kernel paths 16:03.500 --> 16:15.580 doing some cycles to kernel and then this kernel debug came in leaked detect, memory leakage, 16:15.740 --> 16:26.540 in the kernel, this memory, this came in leaked reports, this issue and this issue, this stack 16:26.540 --> 16:36.220 trace is, it's presented in this kernel debug came in, so this file has two purposes, 16:36.220 --> 16:43.420 one for writing came in, come on street and for reporting bugs from the kernel. 16:46.540 --> 16:53.900 Other questions, so thank you, I work on Cisco, this is really cool, thank you, I want to 16:53.900 --> 17:00.140 pitch something which is maybe you know, deja view, it's a tool by also one person working on Cisco 17:00.140 --> 17:06.620 now, which is similar and it also gives you a perfect trace, so anybody is interested in this, 17:06.620 --> 17:13.900 deja view on GitHub, also you should look at. Okay, thank you. Okay, we're up, thank you, thank 17:15.820 --> 17:24.620 you. 17:24.620 --> 17:32.300 Yeah.