WEBVTT 00:00.000 --> 00:11.440 Okay, hello, my name is Brian, and we're going to do something a little bit different. 00:11.440 --> 00:13.240 And also a little bit the same. 00:13.240 --> 00:20.320 This talk is called Keeping Spatial Scripting Sane, with Samake and Raku, something a little 00:20.320 --> 00:21.320 new. 00:21.320 --> 00:28.040 I'm a logistics engineer at Instacart in the U.S., we do grocery technology and delivery. 00:28.040 --> 00:31.160 We have a lot of fun and interesting spatial problems. 00:31.160 --> 00:35.160 I'm also a contributor to the Raku ecosystem. 00:35.160 --> 00:37.600 Anybody heard of the Raku programming language? 00:37.600 --> 00:41.400 Okay, a few people, good. 00:41.400 --> 00:42.800 First of all, I'll talk about the motivation. 00:42.800 --> 00:47.680 I'm going to talk about Samake, something that I've written to help with spatial scripting. 00:47.680 --> 00:52.280 Some examples had to extend it and then a little bit about the choice of Raku as a programming 00:52.280 --> 00:54.280 language. 00:54.280 --> 00:57.920 So if you're here for the first talk today, you sort of saw some of this about what it's 00:57.920 --> 01:04.520 like with the ecosystem of spatial data and tools, there's a lot of tools, a lot of data. 01:04.520 --> 01:05.800 It gets messy. 01:05.800 --> 01:10.040 Also, if you're writing scripts, one of the challenges is that you want to do visualization 01:10.040 --> 01:11.480 while you're writing your code. 01:11.480 --> 01:17.040 So you can't just sort of write something compile it, run it, you have to sort of iterate. 01:17.040 --> 01:22.680 Data is nice when it's local and it's small, sometimes it's big and it's far, and 01:22.680 --> 01:26.400 there are a lot of different languages. 01:26.400 --> 01:31.720 So the goal of Swahili is to be able to integrate a lot of different programming languages 01:31.720 --> 01:38.280 together at the same time, and it has some aspects of Rapples and some aspects of notebooks, 01:38.280 --> 01:45.320 like Jupiter notebooks, so yeah, so let's just drive right in. 01:45.320 --> 01:50.440 So the basic concept is like this, you have all your languages in a single file. 01:50.440 --> 01:55.160 You use two dashes to separate them into cells, so you might have some Python, some 01:55.160 --> 02:02.300 duck DB, you might have some R, anybody who's R here, you might use an LLM, first AI 02:02.300 --> 02:08.060 of the day in this talk, in this room, so these are cells, I'm going to talk a little 02:08.060 --> 02:15.120 bit about plug-ins and plug-outs for visualization. 02:15.120 --> 02:21.520 So each cell has a type like Python, and it has a name like location, and then JSON is the 02:21.560 --> 02:26.080 output format, you might generate JSON, you might generate CSV files, you might generate 02:26.080 --> 02:33.520 images, so you sort of put them all here together, and then you might want to share information 02:33.520 --> 02:39.600 between your different cells, so that can get tricky, so that's what way we introduce 02:39.600 --> 02:44.200 these angle brackets here, this is the Unicode angle bracket, they're not used in a lot 02:44.200 --> 02:48.200 of programming languages, so they probably won't get in your way, and you can interpolate 02:48.200 --> 02:54.600 Raku expressions there, so right here we find a location, generate JSON, and then take 02:54.600 --> 02:59.400 the latitude and longitude, and then put it here inside of the SQL that we can send 02:59.400 --> 03:03.080 off to duck DB. 03:03.080 --> 03:06.960 You can also put a little bit of code at the beginning, you can say, you know, a variable 03:06.960 --> 03:11.200 has dollar country, and then you can do something with it later on, so they all sort of 03:11.280 --> 03:16.280 share the same execution context. 03:16.280 --> 03:23.280 So summary, cells like this, two dashes is the beginning of the cell, the type comes 03:23.280 --> 03:30.240 at is before the colon, then you have a name, then you have an extension, the lines that 03:30.240 --> 03:36.440 start with dashes, separate the cells, the first plugin that matches the type will be used 03:36.440 --> 03:41.400 to execute the cell, and then the first plugin that matches the extension will be used 03:41.400 --> 03:46.400 to generate the data, but you can use the other types if there's multiple matches, and 03:46.400 --> 03:50.240 then you can interpolate data into your code with angle brackets, so this is basically 03:50.240 --> 03:53.760 how it works. 03:53.760 --> 03:59.600 There's an interface, so here we have a little text editor, if you don't use BI, you can 03:59.600 --> 04:05.000 also set your editor and use another one, or it can watch for changes to your files. 04:05.000 --> 04:14.720 You edit your file like this, and then you start up this console user interface, it'll 04:14.720 --> 04:19.880 highlight the parts that are getting interpolated, it'll highlight your rocko code, and 04:19.880 --> 04:24.720 then you can basically change between this, where you're editing, and then you can make 04:24.720 --> 04:29.640 it be in evaluation mode, where you'll fill in your data for you, so you can see what 04:29.640 --> 04:34.480 you're going to execute, then the bottom will have the outputs. 04:34.480 --> 04:39.960 So we run our CSV, it returns, it runs, I use Postgres here, so we could use a geography 04:39.960 --> 04:46.440 type, we are just going to buffer this point, then we can select our output file, CSV 04:46.440 --> 04:53.480 file, choose one of the plugin, plug out, that mentions it, okay, so CSV Geo, then we'll 04:53.480 --> 04:56.640 launch a web browser, and then we get to see our data. 04:56.640 --> 05:02.320 So that's how it works, so you define your input types, your output types, you kind of write 05:02.320 --> 05:07.560 some code, run some cells, share data between cells, visualize, and keep going, so let's 05:07.560 --> 05:13.760 see if we can get through a few examples in five minutes, so we're going to Geo code here, 05:13.760 --> 05:21.640 generate a CSV, then put it into Postgres, and then visualize it, so this is the what 05:21.640 --> 05:29.040 we just saw, okay, let's do a little Python and R sort of mashed up together here, so 05:29.040 --> 05:32.840 here we'll use a little Python, the slides are online, by the way, so you don't have 05:32.840 --> 05:38.640 to read everything now, so generate a bounding box with Python, and then put it into R with 05:38.640 --> 05:46.920 tidyverse, and then, and then when we, it'll basically launch the R-reple, and then you'll 05:46.920 --> 05:52.440 see that you can see the output below, and then R will spawn its own window, and you get 05:52.440 --> 05:59.200 a little map-nick tile suitable for experimenting with the tiles that you're generating. 05:59.200 --> 06:02.000 When you configure them, this is how you make a plugin, it's really simple, you have 06:02.000 --> 06:07.520 a regular expression that matches the type, and then you have a method that's called Execute, 06:07.520 --> 06:11.480 and Execute basically takes the data and runs something. 06:11.480 --> 06:16.640 There are some common ones, the ones we saw are processes and repulse of process, cake, 06:16.880 --> 06:22.400 standard in, prints just standard out, or maybe uses a temporary file, Raku has built 06:22.400 --> 06:27.680 in support for pseudo-TTYs, like, expect us, but if you don't want to use Execute, you 06:27.680 --> 06:33.560 can use this instead, so it can also do things like parse streaming, UTF8, and antsy escape 06:33.560 --> 06:40.160 sequences, so it can take care of all that stuff, and you can put a definition of a class 06:40.160 --> 06:45.360 right into the configuration file, so basically the grass, the grass plugin we saw, it 06:45.360 --> 06:50.120 just looks like this, so it's plugin, colon, colon, repule, command, grass, and that actually 06:50.120 --> 06:54.560 generates a class that has an Execute method that runs the grass command. 06:54.560 --> 06:58.920 Similarly, you can make a process like that, command is called Claude, send the arg, say 06:58.920 --> 07:05.440 you standard in, instead of a temp file, so this is actually a way of defining a class 07:05.440 --> 07:10.600 in Raku by using a roll, a roll defines an interface, and if you pass it some parameters, 07:10.600 --> 07:13.840 you can turn the interface into a class. 07:13.840 --> 07:18.120 There are a lot of plug-outs and plug-ins that come with it, I'm not going to go through 07:18.120 --> 07:22.720 all of them, but you saw a few already, the plug-outs got to work the same way, basically 07:22.720 --> 07:27.760 you define something called Execute, it gets dollar path, which is the name of the file, 07:27.760 --> 07:31.960 this is the thing that will basically show an image, you just say shell, and then open, 07:31.960 --> 07:34.600 and it'll use the system open command. 07:34.600 --> 07:43.120 So it's really easy to add things for your own tooling, and see, so yeah, so these are all 07:43.120 --> 07:47.160 the different things that are included, you know, you can generate Markdown, you have 07:47.160 --> 07:52.360 GeoJSON, if you use T-mux, there's also a way to spawn a command in a separate T-mux 07:52.360 --> 07:59.880 pane, and then it'll use the T-mux control API to send and receive data from it, so you 07:59.880 --> 08:06.240 can interact with things that way if you want to also. 08:06.240 --> 08:11.080 Sometimes people say, why are you using Raku instead of another language, I'll just give 08:11.080 --> 08:16.240 you a few reasons here, that some of which I've already mentioned, so it has really good 08:16.240 --> 08:23.320 process interactions, pseudo-T2Is built in, talked to me more at the Raku and Pearl stand, 08:23.320 --> 08:29.920 and I think that's it, and if you, for more information, check the slides, and also you 08:29.920 --> 08:32.080 can just Google for Raku, so Marky, thank you.