WEBVTT 00:00.000 --> 00:08.720 Yeah, thank you for coming to this talk as well. 00:08.720 --> 00:14.880 I'll just keep going through the slight chaos, but I'm Connor Erd, I'm from University 00:14.880 --> 00:19.360 College London and I'm here to talk about unit testing in Fortran, so I'll try and go over 00:19.360 --> 00:25.120 a bit of just my experience in his few court examples and how we can kind of automate it 00:25.200 --> 00:33.520 a little bit. So, starting off with just a bit of intro about Fortran, kind of. So, it's 00:33.520 --> 00:43.360 not working. Is it on at all? It's on, okay. Right. Hopefully it's fine. Yeah, yeah. So, for 00:43.360 --> 00:47.680 some colleagues I've got who work in Fortran, I've got a list of a few projects that 00:47.680 --> 00:51.680 they're aware of and they've worked on and what I'm trying to get across here is that it's 00:51.680 --> 00:58.400 still kind of relevant. The context that I work in is academia a lot of time and a lot of the projects 00:58.400 --> 01:02.880 work in things like fluid dynamics or weather simulations, which you can see from the 01:02.880 --> 01:08.720 UK Met Office being quite a relevant code, which is in Fortran, and we can also see what I'm 01:08.720 --> 01:13.040 trying to get across is this kind of popularity. We've got projects with quite a lot of GitHub 01:13.040 --> 01:17.840 stars which are still Fortran, but something else that I'm trying to get across is that 01:17.920 --> 01:22.240 these projects, even though they're popular, or relatively important, they often don't have 01:22.240 --> 01:27.040 unit tests, which you can see I've got a kind of ticking cross system here, so they might have some 01:27.040 --> 01:33.440 integration tests, but individual units functions often aren't tested in that way, giving 01:33.440 --> 01:39.280 that extra confidence that you think you might want for these kind of codes. So, why might that be 01:39.280 --> 01:43.280 why might it be difficult? And again, this is experienced from academia, so it might not map 01:43.280 --> 01:49.760 to someone in industry. So, in my experience, it's often legacy code, and what I mean by that is 01:49.760 --> 01:53.840 there's often if you perhaps bad practices, like a lot of global state being used, 01:53.840 --> 01:57.600 which makes it quite hard to unit test, because you've got to manage that state and make sure 01:57.600 --> 02:02.000 your tests are isolated and not interfering with the state that might affect the different 02:02.000 --> 02:08.240 tests. There's often a lack of units to test, quite bad wording there for confusion, but 02:08.240 --> 02:13.120 sort of a single function might be writing to file systems, it might be doing multiple 02:13.840 --> 02:18.880 responsibilities, and it's difficult to write that into a single nice unit test. So, that makes 02:18.880 --> 02:23.760 a difficult, and then often you've got lock into, like, specific compilers of dependencies, 02:23.760 --> 02:30.640 this might be because you start this project 10, 20, who knows how long ago, and things haven't 02:30.640 --> 02:35.200 developed with the rate that Fortune has developed, so you're locked in and I can limit the 02:35.200 --> 02:38.640 environment you can work in and the different libraries that you can connect to. 02:40.080 --> 02:44.560 And then, another point why it's difficult to test Fortune sometimes is that you get a lot of 02:44.560 --> 02:48.560 custom testing setups, so people don't necessarily use an open-source framework, they might 02:48.560 --> 02:55.040 develop something themselves, and that kind of means that the community, yeah, sure, the community 02:55.040 --> 03:01.040 won't benefit from sort of open-source contributions to a framework instead, they all have to 03:01.040 --> 03:06.000 start from the scratch again and develop their own framework. So, that limits someone thinking, 03:06.000 --> 03:10.160 oh, I might start unit testing, they don't know how to do that necessarily, which links to a 03:10.160 --> 03:14.960 lack of know-how, and again in academia, this is often the developers might be a PhD student, 03:14.960 --> 03:19.280 who already has a lot of things to be thinking about rather than learning this new unit testing 03:20.240 --> 03:26.560 way of doing things. And then, it's unclear which frameworks to choose, so what tools, and with 03:26.560 --> 03:31.920 that, we can go into what are the available frameworks or tools. To try and figure this out, 03:31.920 --> 03:35.920 you might do a little online search, and you might come across a few websites, one being the 03:35.920 --> 03:41.600 Fortune Wiki, which for a while has been sort of go to page for Fortune, and they have this page 03:41.600 --> 03:46.400 on that website, which is unit testing frameworks, but there's quite a large list here, and there's 03:46.400 --> 03:52.320 no real suggestion of which you should choose, how to use it, or why, so that can be quite confusing. 03:53.280 --> 03:59.040 If you go to the more up-to-date, perhaps more maintained website, Fortune Online, which is 03:59.040 --> 04:05.120 kind of like a new community developing slightly more modern Fortune tools, they have packages 04:05.120 --> 04:10.560 which they list, and on this page, they've got a big sort of large bin to handle things in, 04:10.560 --> 04:15.520 which is error handling, blogging, documentation, and testing, and again, this is a large list, 04:16.080 --> 04:20.320 and again, they're not all testing, and there's no real direction of what to choose, 04:21.040 --> 04:26.240 so it can be quite difficult. So with that, what have I chosen? So I have gone with something called 04:26.240 --> 04:30.800 peer-funit, and the reason for that is, there was a previous experience in our department, 04:30.800 --> 04:36.000 so I could get help when I was learning how to use it. It's open-stars, and it's open-source, 04:36.000 --> 04:40.480 sorry, and it is still being developed, which is more than you can say for a lot of the ones 04:40.480 --> 04:46.000 on the previous list, and it appears to be the most fully featured in my opinion. The most 04:46.160 --> 04:51.520 importantly, it supports unit testing, MPI, parallelized code, which is very important for 04:51.520 --> 04:58.320 Tran, because that's just a very common thing that you all come across. So what is peer-funit? 04:58.320 --> 05:04.800 It's a bit of an overview, it was developed in NASA. It is open-source, as I said, it's on 05:04.800 --> 05:09.280 GitHub, and they do accept contributions, that's the link, which there will be a QR code for this 05:09.280 --> 05:17.120 slide later. It's available under a NASA open-source license, which I believe is similar to 05:17.120 --> 05:23.360 MIT or BSD, but there are some differences, so be aware of that. It uses a pre-processor file, 05:23.360 --> 05:30.000 so it takes a specific peer-funit format and converted to Fortran, but it is very similar to Fortran, 05:30.000 --> 05:36.320 we'll see an example later, and it is trying to follow in that words J unit, and I think essentially 05:36.320 --> 05:43.120 it uses directives, which like annotations in J unit, I think, and they say essentially label 05:43.120 --> 05:49.600 the code sections for the pre-processor to understand what it needs to do. And then one last point, 05:49.600 --> 05:55.440 is it uses a lot of slightly more modern features, you often need more recent versions of 05:55.440 --> 06:01.520 compilers to get it to work. Okay, so a little bit of syntax, so this is a very simple 06:02.480 --> 06:07.360 relatively.pf file, which if you know Fortran, it is very similar to Fortran, 06:08.160 --> 06:14.240 straightly only real difference, are these the decorators, the directives, so a few things to point 06:14.240 --> 06:19.680 out is you can import pfunit first, it's funit because it's the serial version of the library, 06:21.040 --> 06:27.040 and then you label your test function, so in this case this is a subroutine, which is just testing 06:27.040 --> 06:31.600 something silly, I've written, which just doubles the numbers, doubles the integer, and then you 06:31.600 --> 06:36.160 can assert within that function using one of these directives, so that's just checking your actual 06:36.160 --> 06:42.400 output equals two, and you can do a nice message in case of a failure, so that's sort of what 06:42.400 --> 06:46.880 writing a test would look like. So I'm going to fill on the pre-processor, it's written in Python, 06:47.840 --> 06:54.320 as I said, convert stop pf to fnit, and if you build with make or see make, it will do this 06:54.400 --> 06:58.080 automatically, once you've got it in your build system setup, and we'll see how we do that in a moment, 06:58.880 --> 07:04.480 the directives for you can label a custom type, so you can parameterize tests, which is obviously 07:04.480 --> 07:10.320 very common in other frameworks and is kind of needed in my opinion for a testing framework, which is 07:10.320 --> 07:16.160 going to be worthwhile using, you can label test subroutines with the app test, as I've just said, 07:16.720 --> 07:22.400 and you can specify various assertions beyond the assert equal, which I've shown, however the 07:22.400 --> 07:26.960 documentation for this is not the best, which I'll show in a moment, but I do have a poor 07:26.960 --> 07:32.320 request to try and improve it, but what I mean is this is the documentation it looks like they're 07:32.320 --> 07:37.600 all documented, but if you scroll down there's no information, so there are definitely some issues 07:37.600 --> 07:44.400 with this, but they are trying it's a small team. Okay, so now we can look at how you integrate 07:44.400 --> 07:49.840 with make and see make, is an example from make, so this would be just within your test directory, 07:49.840 --> 07:55.840 you'll have to compile the source code as well separately, but first off you include your pre-built 07:55.840 --> 08:01.360 pfunit information, so I'm passing the path to it here, but you can do it in a bit of a nicer way, 08:02.320 --> 08:07.120 and then you include this dot mk file, that gives you access to these flags here, 08:07.760 --> 08:12.720 which are needed to build, and then it also gives you access to this function, make pfunit test, 08:13.200 --> 08:19.280 and then you name your test, here I've just called it tests, and then you use these specific 08:19.280 --> 08:26.800 variables to link up to that function, so you define your dot pfiles, you define the source objects, 08:26.800 --> 08:31.040 so your source code that you want to test, it looks a bit complex, because I'm just excluding 08:31.040 --> 08:34.960 the main program, because in Fortran, when you're doing these tests you can only have one 08:34.960 --> 08:39.760 sort of entry point, so I just exclude the source main, and then I pass in those test flags 08:39.760 --> 08:46.080 as I just defined, and something or not is that this strange syntax of test and disguise important, 08:46.080 --> 08:49.680 it is the name of the test, if I change that, I have to change the name of the variables, 08:49.680 --> 08:54.480 and that's how it picks it up, and then this last bit of boilerplate, which is an unfortunately 08:54.480 --> 09:00.960 I have to do, is to convert the F-90 files that the pre-processer will create and convert them into 09:00.960 --> 09:05.280 object files, but once you've done that it then picks everything else up, that seems to be the 09:05.280 --> 09:13.760 only bit of thing missing from the function, so now a CMake example, it's quite similar, we include 09:13.760 --> 09:18.960 pfunit using find package, like we do with make, and then we get access to a function, 09:18.960 --> 09:26.160 add pfunit ctest, we name our function, we define the source files, the test sources these are, 09:26.160 --> 09:32.160 so this is the dot pfiles again, and then we link to our source code that we want to test, 09:32.240 --> 09:40.000 again, excluding the main, and that's all you need to do for those, so then how can we now use this 09:40.000 --> 09:46.320 in a CI pipeline, for example, an automated, in this case with GitHub Actions, so I'm using 09:46.320 --> 09:51.920 containerization, most specifically Docker, so within a Dockerfile you would install and build any 09:51.920 --> 09:58.080 dependencies for your project, build the source and the test codes, and then run the test, thank you, 09:59.040 --> 10:04.320 something that can help is by wrapping, slow changing dependencies in a parent image, for example, 10:04.320 --> 10:09.200 pfunit itself, you could build that once and then just pull that parent image and have that at the 10:09.200 --> 10:13.600 start, we'll see an example of the Dockerfile, but something that can make it difficult is if you've got 10:14.320 --> 10:20.640 proprietary or software which doesn't link with say new compilers or things very well, 10:20.640 --> 10:26.960 so I'm having a problem in a current project that we rely on a licensed library which mixes 10:26.960 --> 10:30.800 on able to use it in Docker files, that's a little problem, but there is an example of me doing 10:30.800 --> 10:36.800 this in this repository, and once if someone is interested, you can have a look. Okay, so an example, 10:36.800 --> 10:42.880 Dockerfile, we start with that parent image, so I might have prebuilt pfunit in here, we copy the 10:42.880 --> 10:48.000 source code, including the tests, we then build the CMake, and to get it to work with pfunit, 10:48.000 --> 10:53.840 we specify the CMake prefix path and then that would point towards our installed version of pfunit, 10:53.920 --> 10:59.120 and we just build everything goes well, and you've got a test, and then you just run c-test, 10:59.120 --> 11:02.880 and because this is the Docker image and if those tests fail, the image won't be built, 11:02.880 --> 11:08.080 then I know that if the image is successfully built, my tests have passed, so then using this 11:08.080 --> 11:12.400 in a GitHub actions workflow, for example, if I wanted to do this on a push to main, 11:13.200 --> 11:19.520 or the default branch, a little thing is I need some permissions, because I'm storing the image 11:19.520 --> 11:24.640 in the GitHub repository, the GitHub image repository, I can't remember what it's called, 11:24.640 --> 11:29.440 but you need those to access that, and then you just run Docker build, and as long as you pass 11:29.440 --> 11:34.960 the path to the Dockerfile, if it's not the default name, then this goes green, your test is the 11:34.960 --> 11:42.080 past, so it's just a small example of how to do that. Some future work that I am going to be doing 11:42.080 --> 11:48.160 is a workshop on how to unit test fart run, which we're going to more detail, so this is going to be 11:48.160 --> 11:52.240 in the style of something called software capantry, which people may be familiar with, but it's 11:52.240 --> 11:56.640 kind of an open-source example, especially in academia of how to write lessons and workshops, 11:57.680 --> 12:03.360 but there is an available website for the software capantry, and yet the workshops on the 12:03.360 --> 12:07.600 16th and 18th of February, if you happen to be in a London, which is where I'm from University 12:07.600 --> 12:14.640 College of London, sign up, it's free, the material written so far is available at that lesson, 12:14.640 --> 12:19.920 I've also developed a repository of exercises, which will meant to complement the lessons, so you 12:19.920 --> 12:25.600 can do those in the workshop, and I'm hoping to expand to including automated testing, which I haven't yet, 12:26.800 --> 12:32.080 lintes, so there's some fart run specific linting tools, which have been developed, which are 12:32.080 --> 12:36.880 quite good, some documentation generation, including the ones specifically for fart run, which is 12:36.880 --> 12:46.560 fart, but yeah, some conclusions, so fart run, I think it's still relevant, however, it does 12:46.560 --> 12:54.960 lack some unit tests in a lot of popular projects, I recommend that it will use pfunit, again mainly 12:54.960 --> 13:03.840 for the MPI compatibility, pfunit uses the pre-processor, assertions are provided for various things, 13:03.920 --> 13:08.800 it integrates with make and see make, and you can automate your testing in a CI pipeline if you've 13:08.800 --> 13:17.360 got not to proprietary dependencies and things like that, yep, and just some acknowledgements, 13:17.360 --> 13:23.120 I am not a pfunit developer, so I want to point out the people who are, Tom Clown in Mac Thompson 13:23.120 --> 13:28.960 specifically, and these slides I learnt how to do them because of party runny, who I work with, but 13:29.040 --> 13:42.240 if you're interested in how they're made, and there's the QR code for my slides, so thank you 13:42.480 --> 14:07.840 very quick question, is one there, yes, I think that's, if anything, the main problem yes, 14:07.920 --> 14:13.120 so the question is, fart run developers, I guess in that your opinion, don't know, 14:13.120 --> 14:18.240 are often a bit older, and it's a difficult to persuade them to do unit testing, I believe that was 14:18.240 --> 14:24.320 the question, yes I found it is, slightly, although not everyone, not everyone's the same, 14:24.320 --> 14:29.520 some very keen to give it a go, and this is why I'm doing things like this talk, the workshop, 14:29.520 --> 14:34.720 some trying to ensure that it is possible, and that you can do it, and that it is worthwhile, 14:35.360 --> 14:40.000 rather than just saying, oh I do an if statement in my code, so therefore I know it works, 14:40.000 --> 14:43.840 because I've checked the value as it's running, which doesn't make sense, but that's the kind of 14:43.840 --> 14:51.040 arguments you get, so yes, I do have those problems for it. Yeah. Thank you.