WEBVTT 00:00.000 --> 00:17.000 Thanks for setting up the stage, I'm Alia, I'm working on sickle rooms which is an open source framework for building data analytics applications. 00:17.000 --> 00:28.000 So I'm a software engineer mostly content, I've spent a lot of years already working on data visualization applications and data analytics. 00:28.000 --> 00:40.000 So now I'm trying to kind of automate myself, even although we now have LLM agents which can do this anyways. 00:40.000 --> 00:49.000 But yeah, the thing is that I've learned something over the years, some recurring patterns and I'm trying to build a framework which makes it easier. 00:50.000 --> 01:01.000 So sickle rooms is a React framework which helps you build applications and it's very modular. 01:01.000 --> 01:15.000 So you pick the functionality you need and yeah, and it's backed by DougDB, which is a very nice database which is invaluable. 01:15.000 --> 01:32.000 So it started with this project flow of a city which I built some years ago, where I used DougDB running in the browser to represent mobility data for cities and transportation. 01:32.000 --> 01:38.000 And this is done to better understand the troubles demands to improve the planning. 01:38.000 --> 01:52.000 And here you see like when I move the viewport zoom in and zoom out there are lots of DougDB queries running directly in the browser which prepare the data for this particular viewport and it's pretty working pretty well with DougDB. 01:52.000 --> 01:59.000 And now I work at fore square and we are building tools for geographic data analytics. 01:59.000 --> 02:15.000 So this is called the fore square special desktop, this is a desktop app, Electron, but it uses SQL rooms as it's like the framework to run everything and uses DougDB running in, it's not running in browser. 02:15.000 --> 02:23.000 In a Python backend which is like part of the Electron app. 02:23.000 --> 02:29.000 And yeah, so DougDB can be used also as a Python library because it's so embeddable. 02:29.000 --> 02:37.000 And that's what we do here and it gives like a bit more capabilities and if we were using the web as some diversion in browser. 02:37.000 --> 02:54.000 So there are some other companies using SQL rooms here for example Cosmo Graph, it's an application for application for graph and embedding visualization here you see scientific publications and it also runs that DB in browser. 02:54.000 --> 03:00.000 And also very directive different filtering capabilities and so forth. 03:00.000 --> 03:13.000 And this is a company doing transport modeling so you're basically looking at how. 03:13.000 --> 03:22.000 Basically can close the street or add a new street and see run modeling and see how the flow flows, mobility flows will change. 03:22.000 --> 03:33.000 We have to do with data data analytics and this is what SQL rooms kind of helps with. 03:33.000 --> 03:51.000 So one of the reasons I built it like data is powerful, but data alone there's this kind of dead keeping right it's too technical you need to have the capabilities to be able to access the data or you need to have software engineers could build a software for you to access the data. 03:51.000 --> 04:08.000 So SQL rooms is kind of the goal is to make it easier to make it more democratic and also I think with the advent of LLMs it's also easier now because LLMs are pretty good at generating SQL. 04:08.000 --> 04:23.000 So you don't even need to know SQL to be able to understand to analyze and data you can just express your intent as a sentence in natural language and then LLMs will generate SQL. 04:23.000 --> 04:35.000 And you can run this query and see the results and for that like the database and the results can they can stay local you don't need to send. 04:35.000 --> 04:56.000 The data to the LLMs you just express your intent it will generate SQL you execute SQL locally right so you don't it's from privacy standpoint it's pretty good what you can do because you don't need to share the data and the results with LLMs you just explain the intent maybe you share the schema so that it's easier. 04:56.000 --> 05:03.000 The schema of the database so it is easier to run of the prepare SQL and it can help you. 05:03.000 --> 05:15.000 So here's an example of which someone from the community built with SQL rooms and so it's from Amar Harolekar and he put a video on YouTube. 05:15.000 --> 05:26.000 So it's like an app running full in browser he added data with like 20 million records which is remains in browser to duck db and then he asked something. 05:26.000 --> 05:42.000 LLMs to generate a query generate a SQL query it run in browser and then SQL rooms can generate a chart based on that it's actually also LLMs generate a specification of the chart which we can run in browser. 05:43.000 --> 06:03.000 Yeah so I'll SQL rooms have room in it's title so what is a room it's like somewhat like a chat room but intended for data analytics so you have the data in this room you have visualizations it's like a self-contained workspace and you can work alongside. 06:03.000 --> 06:17.000 And AI systems or multiple users at least I'm working on that enable collaboration that's why why why that db is a good fit it's because it's made for analytics it's columnar. 06:18.000 --> 06:33.000 It's embeddable as I mentioned before it's a run in browser and low inference if you can run it in browser you don't need to set it set up the cloud of to run it it's single note it's not distributed so. 06:33.000 --> 06:46.000 But it's it's a limitation but at the same time it's it's a good thing because you own your data if you run it locally you don't need to share it with cloud providers also you don't have the round trip to run the queries right so. 06:46.000 --> 07:00.000 You don't send the data over the network and like if you run it in browser and user browser it's kind of you get horizontal scaling for free every new browser adds compute to your to your overall system. 07:01.000 --> 07:13.000 And privacy standpoint from the privacy standpoint it's a good thing so in SQL rooms if you go to the website there is an example of offline use you can run. 07:13.000 --> 07:31.000 They up and install it as a PWA right look which you can done locally on your computer you can use opfs which that'd be supports to store the data locally and you can even connect local a system by allama. 07:32.000 --> 07:35.000 So what what's the problem provides. 07:35.000 --> 07:44.000 So it's like it's right now it's dependent on a bug DB we assume that we are using that to be although we want to support more. 07:44.000 --> 07:48.000 Databases in future like we have a data connector. 07:49.000 --> 08:00.000 Abstraction already so when you start with an application it will like give you functionality for loading data schema management and query management. 08:00.000 --> 08:04.000 There are UI components which are like. 08:04.000 --> 08:08.000 Useful for data center caps like table schema tree query editor. 08:08.000 --> 08:15.000 I'm other integrations with various visualization libraries Vega which we saw for the charts. 08:15.000 --> 08:20.000 Most of the creature cost most for graphs and capital for just special data. 08:20.000 --> 08:23.000 And there is an UI system. 08:23.000 --> 08:28.000 So it's a very modular toolkit this is an example of. 08:29.000 --> 08:40.000 Well built with the sequeroos you have the schema editor query schema schema of your query editor you can add data you can run queries and you see the results. 08:40.000 --> 08:55.000 And so the thing is that it's very modular and the most tricky part to to modular is the state right what if you want it to be able to pick the functionality you need for your particular app. 08:55.000 --> 09:03.000 And then combine it in a state storage in a store which you can access from anywhere in your application. 09:03.000 --> 09:10.000 So this is what the sequeroos allows you to do by using two-stunt slices. 09:10.000 --> 09:24.000 So basically for every module you want to add to your application you just added two-stunt slice and then this will be part of your store and accessible from anywhere. 09:24.000 --> 09:34.000 So the room store you need to initialize it and you add this slices which you want to pick. 09:34.000 --> 09:44.000 You can also decide to add some data to the database, create it or you can add it later. 09:44.000 --> 10:07.000 Then you can basically wrap your your entire application into room shell component and pass it to the store which you initialize and then in every component within that hierarchy you can access anything on the store run SQL queries and do something with results. 10:07.000 --> 10:09.000 All running browser. 10:09.000 --> 10:18.000 So yeah and to compose different slices in your store you just like added to your store. 10:18.000 --> 10:23.000 So here I'm adding SQL editor then this functionality will be available in my app. 10:23.000 --> 10:35.000 And I have access to it where this kind of hook from components and you can extend it with your custom functionality as well of course. 10:35.000 --> 10:54.000 So collaboration more interesting so coming soon I'm working on notebook implementation and canvas module and canvas is actually a really great fees for collaboration. 10:54.000 --> 11:04.000 So kind of this is like a 2D and 2D notebook you have also cells which can be queries or visualizations and you can like arrange them on a 2D surface. 11:04.000 --> 11:10.000 And I'm working on a CLI tool so which you can run. 11:10.000 --> 11:18.000 Just like it will be like a Python module you can just run with UVX without having to install it. 11:18.000 --> 11:32.000 And then you can pass it the name of the database file and it will be a WB file on your computer which you can like both the data will be safe there and the state of your application. 11:32.000 --> 11:46.000 So then you rerun it it will like be your kind of project file and you'll have a reproducible artifact which you can send to someone or you can if you run this same server. 11:46.000 --> 11:53.000 And on server you can basically other people can collaborate with you on same project. 11:53.000 --> 12:11.000 So to enable this collaboration I'm working on CRDT module for which will be also like a slice you add to your store and so it's close to working. 12:11.000 --> 12:27.000 I have a PR hope to lend soon so here we specify like the local storage we use so that we can so it can still work offline right so we can stay have a copy of the state in index to be in a browser. 12:27.000 --> 12:32.000 Here we pass the connector to the sync server and the URL. 12:32.000 --> 12:44.000 And this this is the important part we pass mirrors which is like specification what part of the state we want to sync right we don't want to sync everything on a state. 12:44.000 --> 12:48.000 We need to specify which parts and to which data structures. 12:48.000 --> 13:02.000 So I have I've had some experience using yjs so I built this tool for collaborative map drawing in the past so I'll consider yjs first but then I learned about. 13:02.000 --> 13:15.000 And I chose this because what they have we they have this sort of a lot of CRDT mirrors module which is basically. 13:15.000 --> 13:20.000 This way of expressing the mapping between your original data structures. 13:20.000 --> 13:28.000 And this is what I need because because. 13:28.000 --> 13:35.000 Secular room is kind of a framework and I don't know in advance all the state which will be in your application. 13:35.000 --> 13:42.000 You decide right you you can add your custom state so we need a mechanism to express like what parts of the state. 13:42.000 --> 13:51.000 We map to what kind of certain structures and this might be generic must be generic enough this is what this. 13:51.000 --> 13:57.000 A lot of mirror module enables. 13:57.000 --> 14:03.000 So this was one reason and another reason was that I followed the work of Marimo who knows about Marimo notebooks. 14:03.000 --> 14:11.000 A few people so it's like a somewhat similar project and they they recently migrated away from wages to. 14:11.000 --> 14:17.000 To Laurel because they had some issues with wages. 14:17.000 --> 14:21.000 So yes there's a Python server you can run. 14:21.000 --> 14:33.000 Which will run both.tb database right it will rapid in an HTTP server and you can access it via web sockets and it will do the thinking. 14:33.000 --> 14:42.000 So yeah here I have a short demo of the canvas so I start the server here and then I open to to web clients right so. 14:42.000 --> 14:51.000 This is the canvas I can create a query and there are the client sees it immediately around the query there but I don't see the results here because the results are not synced. 14:51.000 --> 14:56.000 Right we don't need to sync them we want to be explicit like when. 14:56.000 --> 15:04.000 The user actually execute the queries so we picked what part of the state we sync and it's not everything. 15:04.000 --> 15:11.000 Yeah but you basically see you can change the data and the other clients will see it. 15:11.000 --> 15:25.000 So yeah so this is the basic like architecture which I'm working on now so there is a server right right now which will have the copy of the CRDT will. 15:25.000 --> 15:35.000 So there will be instance maybe some AI agent orchestration which will have access to the same.db and the CRDT state and can modify it. 15:35.000 --> 15:42.000 So we can have agents training both on the clients and on the server and multiple clients right. 15:42.000 --> 15:44.000 Sorry. 15:44.000 --> 15:52.000 One minute oh okay but I'm a thing almost here sorry so yeah the data is not part of it so this can be a femoral for instance. 15:52.000 --> 16:04.000 I'm considering to use the cloud fair containers for this which can be like we can spin it out on demand and this will be like one session shared by multiple users. 16:04.000 --> 16:14.000 And we'll shut we can shut it down after all users disconnect but the data can stay separate and we can just load it on demand. 16:14.000 --> 16:17.000 So here's also for interest but we'll be the next step. 16:17.000 --> 16:19.000 So this is the last slide. 16:19.000 --> 16:22.000 Just the goals of the project. 16:22.000 --> 16:25.000 Make it easier to build data and asics apps reduce data. 16:25.000 --> 16:29.000 Get keeping stay open and independent. 16:29.000 --> 16:31.000 This is my LinkedIn. 16:31.000 --> 16:33.000 This is Sigorums.org. 16:33.000 --> 16:35.000 Is there time for questions? 16:35.000 --> 16:36.000 Yes. 16:36.000 --> 16:38.000 Okay thank you first. 16:39.000 --> 16:49.000 Thank you very exciting to see what's possible in the browser nowadays that definitely wasn't possible like five or ten years ago. 16:49.000 --> 16:50.000 So really cool. 16:50.000 --> 16:53.000 Any questions from the room? 16:53.000 --> 16:55.000 All the way back there you know. 16:55.000 --> 16:58.000 If you do your question I'll repeat it for the recording. 16:58.000 --> 17:10.000 Would you go into a little bit more detail why why genius would not work for your use. 17:10.000 --> 17:13.000 So it would work for my use. 17:13.000 --> 17:14.000 Sure. 17:14.000 --> 17:17.000 Why wouldn't why jazz work for my use. 17:17.000 --> 17:27.000 I think it would work but I just chose a lower certainty at the moment for convenience because they have this mirror functionality which allows you to express. 17:27.000 --> 17:34.000 How you map your application state into serity data structures in a very convenient way. 17:34.000 --> 17:37.000 So for me that was the main reason.