WEBVTT 00:00.000 --> 00:13.800 So, hello. I'm here to speak about data and Kubernetes, state of the search. I'm 00:13.800 --> 00:20.680 Matthias, I'm from Belgium, and I work at Blenscale. To start off, like, what is Kubernetes? 00:20.680 --> 00:25.000 Like, we're in a database room data. We most of us will understand. Kubernetes is an 00:25.000 --> 00:31.440 open source framework for management, deploying, and scaling of containerized applications. 00:31.440 --> 00:37.640 And then containerized applications, it's something that you pack together with all the 00:37.640 --> 00:42.840 dependencies and everything it needs to be lightweight, to be self-contained, and to be 00:42.840 --> 00:48.320 portable between different machines. Because in Kubernetes, you most likely will 00:48.320 --> 00:55.320 separate storage from compute. So, that brings us to stateless versus stateful. In the 00:55.320 --> 01:02.080 stateless world, each request is independent, and it's an isolated transaction. Think 01:02.080 --> 01:07.920 about, like, an XP web request, they don't know about each other. They don't store any 01:07.920 --> 01:13.400 history while a stateful service is something where you want that history to be stored 01:13.400 --> 01:19.000 and to be retrieved. And so, in many cases, you'll have that storage component attached 01:19.000 --> 01:27.000 to your compute. Something about Blenscale, the company I work in, it was found in 2018 01:27.000 --> 01:35.600 as a by-the-founders of Blenscale, where the Vites developers originally, that left Google 01:35.600 --> 01:41.000 when Google decided that the workload at YouTube was running, need to go to a different 01:41.000 --> 01:49.200 product. And then, in 2021, we changed our business strategy to become database service 01:49.200 --> 01:56.360 product, where we basically were running managed Vites in the cloud for people. And since 01:56.360 --> 02:02.080 last year, we also run postpress. And important for us is that everything we do as a database 02:02.080 --> 02:10.600 company runs a Kubernetes. So, as we are in the business of saving people's data in the cloud, 02:10.600 --> 02:17.520 we are expected to store the things. Not everyone runs on the black hole engine, which 02:17.520 --> 02:25.760 is a little nice growth hook. And database can become quite large. So, we are in operation 02:25.760 --> 02:33.000 side of the database. So, we do backups and we do failovers. And as the data grows, that 02:33.000 --> 02:39.040 become all very slow. And when you run in the cloud, you will most likely be running with 02:39.040 --> 02:45.840 the cloud block storage, like EBS in Amazon, or persistent disks in Google, or any other. 02:45.840 --> 02:52.160 And these storage solutions are all network attached, which makes them very slow, high latency 02:52.160 --> 02:58.200 for every right operation and every operation that you need to search for. So, what we 02:58.200 --> 03:02.960 decided to do somewhere last year is we decided to run everything on the ephemeral storage 03:02.960 --> 03:11.400 that the cloud instances offer. It's available both in Amazon and in Google. But it's 03:11.400 --> 03:18.320 not meant to be stateful, because it's a very fast storage. That is used for scratch-based, 03:18.320 --> 03:24.920 like if you have a processing work line that needs to store intermediate states. You 03:24.920 --> 03:32.800 will store them on the instance storage, and then you can continue processing. If you 03:32.800 --> 03:38.280 lose that data, you can just restart your process. Well, if you run a database, you don't 03:38.280 --> 03:45.520 want that. So, this storage is lost when you destroy the machines, so if you shut it down, 03:45.520 --> 03:50.680 that's storage is lost and you cannot get to it anymore. So, we name the product that 03:50.680 --> 03:57.560 we developed from this plan scale metal, because we are running on the ephemeral storage 03:57.560 --> 04:02.920 in the instances in the cloud. And this make your database run really fast, but virtually 04:02.920 --> 04:08.040 unlimited IOPS. There is a limit to the IOPS that these devices can have, but with the 04:08.040 --> 04:14.560 workloads that we have been running, you cannot reach that limitation. We will do a multi-dase 04:14.560 --> 04:21.260 deployment, minimal 3AZs for a production workload to ensure that we can deal with a full 04:21.260 --> 04:26.740 AZ-outage. And for the durability guarantee of your data, we will use my SQL stemizing 04:26.740 --> 04:32.440 replication, or if you're using postgres. We will have at least one synchronous replica in 04:32.440 --> 04:39.020 another AZ. And we require the cross AZ acknowledgement in the storage in the stemizing 04:39.020 --> 04:44.360 to make sure again that we can deal with the full AZ-outage. We make daily full backups 04:44.360 --> 04:50.480 and verification to the object stores on the cloud. So, every backup to restore the previous 04:50.480 --> 04:55.840 backup and then validates that is valid, that you can replicate from it again. And if we 04:55.840 --> 05:03.320 need to do an upgrade to the database or any sort of rolling operation, we will first search 05:03.320 --> 05:11.880 new instances. So, you can keep the durability of your data before we will remove any 05:11.880 --> 05:18.880 old instances. And that was it.