WEBVTT 00:00.000 --> 00:08.800 And thank you everybody for having us today. 00:08.800 --> 00:11.480 So I'm Steve Fan, Steve Fan Caron. 00:11.480 --> 00:14.160 And with me, it's Anahllo, and I don't have a touchman 00:14.160 --> 00:17.120 diva vacer, who will show you live demo on the robots 00:17.120 --> 00:18.560 in the end. 00:18.560 --> 00:22.400 And so of course, with me, are Michel Scogov, the black robot 00:22.400 --> 00:24.920 you see here, which is a new app key, and cookie, which 00:24.920 --> 00:29.600 is the yellow one, which we can talk about afterwards. 00:29.600 --> 00:31.760 All right, so before talking about these robots 00:31.760 --> 00:33.560 in particular, I want you to share with you, 00:33.560 --> 00:37.040 like one point of view on open source robots today. 00:37.040 --> 00:39.440 I like open source robots. 00:39.440 --> 00:41.200 This is a good place to say that. 00:41.200 --> 00:44.360 I'm trying to curate a list of open source robots. 00:44.360 --> 00:46.240 If you know some that are not on the list, 00:46.240 --> 00:48.960 please feel free to add them there. 00:48.960 --> 00:53.080 And I just wanted to kind of take a look back as to why, 00:53.080 --> 00:55.600 also we can now we are in a better position 00:55.600 --> 00:59.560 to do open source robots than we were in the past. 00:59.560 --> 01:03.240 Here are a quick view of some of the projects. 01:03.240 --> 01:08.120 NASA opens sourced a robots, and they're Apache 2.0, 01:08.120 --> 01:13.360 which is like a rover that has excellent documentation. 01:13.360 --> 01:16.320 They're also educational open source robots, 01:16.320 --> 01:19.040 like the robot soccer kits that are used for education 01:19.040 --> 01:19.640 purposes. 01:19.640 --> 01:21.680 It's actually quite fun for kids in mid schools, 01:21.680 --> 01:26.960 for instance, to control these robots in Python. 01:26.960 --> 01:29.880 And more recently, there was the Open.mini, 01:29.880 --> 01:32.520 not meant to resemble any particular robot, 01:32.520 --> 01:33.960 but it has been heavily reproduced, 01:33.960 --> 01:38.920 like more than 50 times, which in my opinion is amazing. 01:38.920 --> 01:42.880 And this is a bipedal robot, also a bipedal robot. 01:42.880 --> 01:43.960 And everything is open source, 01:43.960 --> 01:46.560 you can train reinforcement learning policies for it, 01:46.560 --> 01:49.520 UV run train, and it will train one for you. 01:49.520 --> 01:52.240 It's also very cute, et cetera. 01:52.240 --> 01:56.320 And so I was wondering, so I've been doing robotics 01:56.320 --> 01:58.480 for a bit over 10 years now, and I wonder, 01:58.480 --> 02:00.960 like, how come we have this kind of a comprehensive 02:00.960 --> 02:02.560 explosion of open source robots now? 02:02.560 --> 02:04.040 Like, how did we get there? 02:04.040 --> 02:07.040 How was it and what changed? 02:07.040 --> 02:08.960 And this is just one point of view, 02:08.960 --> 02:13.120 but looking back at 10 years ago. 02:13.120 --> 02:15.400 So personally, I've been working in academia. 02:15.400 --> 02:18.000 And in academia, we were working with very expensive robots. 02:18.000 --> 02:19.920 So this humanoid, you see here, 02:19.920 --> 02:23.760 costs north of 300,000 euros. 02:23.760 --> 02:25.600 It's expensive, you only have it in the lab. 02:25.600 --> 02:27.560 And when you want to operate it, there is a whole process, 02:27.560 --> 02:28.600 because you don't want to break it. 02:28.600 --> 02:30.960 So you have a crane, et cetera. 02:30.960 --> 02:32.880 There were open source softwares already, 02:32.880 --> 02:36.240 but the hardware was not open source at all. 02:36.240 --> 02:41.880 Then, I think one big change came with Bencats master thesis, 02:41.880 --> 02:45.160 who introduced the QDD, what we call now QDD actuators, 02:45.160 --> 02:47.560 or the appropriate safety of actuators. 02:47.560 --> 02:51.200 So the combination of a brushless motor and planetary drive 02:51.200 --> 02:52.360 train. 02:52.360 --> 02:55.000 And it was also fully documented on Benz blog, 02:55.000 --> 02:56.280 which is a great blog to read. 02:56.280 --> 02:58.680 If you haven't, it's really a pleasure. 02:58.680 --> 03:01.880 And it helped reproduce the actuator heavily. 03:01.880 --> 03:05.680 Reproduced, including by I'm going to cite one company, 03:05.680 --> 03:08.080 because I think they've been doing an amazing job. 03:08.080 --> 03:10.320 It's called MJBOTs, it's an American company, 03:10.320 --> 03:13.240 and they do open source motor drivers, 03:13.240 --> 03:16.040 and also integrated actuators. 03:16.040 --> 03:18.760 And so these robots, basically, were made possible, 03:18.760 --> 03:21.800 because we could buy the electronics and the actuators 03:21.800 --> 03:24.960 from that company and then develop software on top of them. 03:25.560 --> 03:27.760 And MJBOTs does thing open source as well. 03:27.760 --> 03:31.080 So the actuators, the secode of the motor controllers, 03:31.080 --> 03:35.800 you can access their driver board is a hatch. 03:35.800 --> 03:38.960 You can put on the Raspberry Pi, which we do here. 03:38.960 --> 03:42.480 You have the firmware, you have C++ Python libraries, 03:42.480 --> 03:44.240 and the supports is implemented. 03:44.240 --> 03:46.760 The state of the art technology, known as Discord, 03:46.760 --> 03:49.040 where you just pop in and ask a question, 03:49.040 --> 03:51.680 and the government jables, it's just there, 03:51.680 --> 03:53.960 and then it's directly there. 03:53.960 --> 03:57.400 All right, so these took us to upkey wheel bipeds, 03:57.400 --> 04:01.360 which are one way to apply these actuators 04:01.360 --> 04:04.840 and motor driver boards, and we wanted to make it 04:04.840 --> 04:06.640 as a wheels biped. 04:06.640 --> 04:11.080 So it's kind of a in-between the mobile robot 04:11.080 --> 04:15.000 with a fixed base or the car, and a humanoid robot. 04:15.000 --> 04:18.400 So you have, in less actuators, 04:18.400 --> 04:20.240 you have the problems from both worlds. 04:20.240 --> 04:23.680 So you need to balance, you need 04:23.680 --> 04:25.080 to balance, because you have wheels. 04:25.080 --> 04:27.360 So if you don't do anything in your forward direction, 04:27.360 --> 04:29.720 in the sagittal direction, it's kind of fall. 04:29.720 --> 04:31.240 So you have to do active balancing. 04:31.240 --> 04:33.920 So if you want to do research, 04:33.920 --> 04:36.160 you still have the same research questions 04:36.160 --> 04:38.920 that you need to solve for humanoids. 04:38.920 --> 04:40.120 But it's only six joints. 04:40.120 --> 04:44.720 So when it breaks, well, maybe you have to replace a motor. 04:44.720 --> 04:46.720 I mean, if it falls, it can fall on the ground 04:46.720 --> 04:47.560 and stand back up. 04:47.560 --> 04:50.000 We added the bumpers in one iteration, 04:50.000 --> 04:52.480 but even if you don't have the bumpers, it's OK. 04:52.480 --> 04:55.000 If it falls on you, OK, it's going to bruise you, 04:55.000 --> 04:56.000 but that's it. 04:56.000 --> 04:59.200 Happened in the past. 04:59.200 --> 05:01.680 And then, yeah, so it's kind of a small form factor, 05:01.680 --> 05:05.120 but it allowed us to do plenty of things. 05:05.120 --> 05:08.040 So the goals of the project was to be fully open source. 05:08.040 --> 05:10.960 So the hardware and electronics are open source already. 05:10.960 --> 05:13.120 We build on that, and then the software we build on top 05:13.120 --> 05:15.160 is also fully open source. 05:15.160 --> 05:17.400 These hybrid locomotion capabilities, I mentioned, 05:17.400 --> 05:19.920 so worst of both worlds. 05:19.920 --> 05:22.920 We want it to be reproducible and accessible. 05:22.920 --> 05:24.280 So let's parts. 05:24.280 --> 05:28.160 I mean, don't use too many actuators. 05:28.160 --> 05:31.880 Try to make it a bit easier so that people can build it. 05:31.880 --> 05:36.560 And then the two outcomes for us are education and research. 05:36.560 --> 05:39.360 So for education, we can use it as a way 05:39.360 --> 05:42.920 to teach model predictive control, so more on the control side, 05:42.920 --> 05:46.160 and also reinforcement learning, so the newer techniques. 05:46.160 --> 05:49.440 And we build on open source libraries for that. 05:49.440 --> 05:51.320 And then, for the research, also, 05:51.320 --> 05:54.480 it's something that's cheap for a research lab, 05:54.480 --> 05:59.840 but that allows you to run experiments with new technology. 05:59.840 --> 06:02.400 So I'm going to talk about the software first, 06:02.400 --> 06:04.080 and then Vanolstein and Etienne can show you 06:04.080 --> 06:06.400 more about the hardware. 06:06.400 --> 06:10.040 So the software, I have to say, doubt that first, 06:10.040 --> 06:11.560 it's not using Ross. 06:11.560 --> 06:14.400 This is we kind of wanted to make something 06:14.400 --> 06:15.800 that you can pip install, and it 06:15.800 --> 06:19.960 could just run directly on your machine. 06:19.960 --> 06:22.360 So as a matter of fact, if you want to try it out, 06:22.360 --> 06:25.240 you can get Chrome, go there, and you 06:25.240 --> 06:26.360 will run the example. 06:26.360 --> 06:28.560 It's going to fire up a bullet simulation, 06:28.560 --> 06:31.640 and you can apply forces to the robot, et cetera, directly. 06:31.640 --> 06:35.760 So we want it to be something that beginners can directly 06:35.760 --> 06:36.480 access. 06:36.480 --> 06:37.880 I'm not saying this is not possible in Ross. 06:37.880 --> 06:41.680 I don't know how to do it, but it might be possible. 06:41.680 --> 06:44.880 And then we want to use it in Python 06:44.880 --> 06:47.200 to stay in a simple world. 06:47.200 --> 06:49.840 So develop new behaviors in Python, 06:49.840 --> 06:52.120 unchecked with all the mess you can do, 06:52.120 --> 06:54.520 and then the low level that communicates with the actuators, 06:54.520 --> 07:00.120 runs in C++, because the MJBot API was in C++. 07:00.120 --> 07:03.360 There is IPC between the two, and every time there is IPC, 07:03.360 --> 07:05.080 actually, we log everything. 07:05.080 --> 07:08.160 So they exchange dictionaries, and these dictionaries 07:08.160 --> 07:11.440 are fully logged, so that then we can use them 07:11.440 --> 07:15.840 for learning from real robot data, for instance. 07:15.840 --> 07:18.120 And I wanted to mention the gymnasium API 07:18.120 --> 07:19.600 we're going to come back to that. 07:19.600 --> 07:21.680 Interface to different physics simulator. 07:21.680 --> 07:24.320 There are actually some connections with the presentation 07:24.320 --> 07:26.160 we had before on the Zebo. 07:26.160 --> 07:28.960 We also had the questions of what physics simulator 07:28.960 --> 07:33.760 to use, and we're going to come back to that as well. 07:33.760 --> 07:36.720 So if you want to try it out, feel free to get clone, 07:36.720 --> 07:39.320 and run. 07:39.320 --> 07:41.600 This should work. 07:41.600 --> 07:45.400 Gymnasium API, we wanted to use a standard API 07:45.400 --> 07:46.800 from reinforcement learning, also, 07:46.800 --> 07:50.880 because it fits well with the control, standard control. 07:50.880 --> 07:57.080 So if you go to gymnasium, they have a bunch of standard examples, 07:57.080 --> 08:00.440 like the card pool here, and it works basically, 08:00.440 --> 08:03.760 you make an environment to expose your system to a user, 08:03.760 --> 08:06.000 and it should have a reset function and a step function. 08:06.000 --> 08:09.200 So initialize and execute. 08:09.200 --> 08:11.080 And we stick to it. 08:11.080 --> 08:15.080 So if you want to switch from the card pool to an app key robot, 08:15.080 --> 08:18.080 you only change, if you look at the diff, 08:18.080 --> 08:20.760 you change the name of the environment, 08:20.760 --> 08:24.840 and you change the observation instead of being a position 08:24.840 --> 08:27.320 for the card pool is going to be the pitch angle, 08:27.320 --> 08:29.360 for instance, for an app key. 08:29.360 --> 08:31.200 And that's it. 08:31.200 --> 08:34.400 So we hope that this makes it easier for newcomers. 08:34.400 --> 08:36.280 They can just come, look at the examples, 08:36.280 --> 08:37.640 and it's basically the same. 08:37.640 --> 08:40.160 If they had reinforcement learning during their education, 08:40.160 --> 08:44.440 for instance, it's going to sound familiar. 08:44.440 --> 08:47.760 And then, this allows us also to connect 08:47.760 --> 08:51.040 to stable-based lines, for instance, 08:51.040 --> 08:54.000 or reinforcement learning libraries in general. 08:54.000 --> 08:56.000 So reinforcement learning libraries, 08:56.000 --> 09:00.800 I'm going to show you an example with stable-based lines here. 09:00.800 --> 09:05.680 And essentially, what they will want to do, 09:05.680 --> 09:08.080 so they will use the simulator to generate samples. 09:08.080 --> 09:11.840 And then you will use these samples to train your neural network 09:11.840 --> 09:15.840 that will commend your robot, and that's called the policy. 09:15.840 --> 09:19.080 And so the way it works here is that it spins multiple instances 09:19.080 --> 09:21.880 of the robot, as many as your CPU can handle. 09:21.880 --> 09:26.320 And then it's going to apply PPO, which is an RL algorithm here, 09:26.320 --> 09:28.360 to train a policy for the robot. 09:28.360 --> 09:29.720 And this is going to take some time. 09:29.720 --> 09:31.600 So I'm not going to let it fulfill. 09:31.600 --> 09:35.080 This particular training would take about maybe 10 minutes 09:35.080 --> 09:38.400 on my laptop. 09:38.400 --> 09:40.560 If I were not on battery, I would leave 09:40.560 --> 09:42.640 it running in the background, but here. 09:42.640 --> 09:44.920 And it would be really slow here, because I put the GUI 09:44.920 --> 09:48.120 in practice, you'll never show the GUI. 09:48.120 --> 09:50.920 And we interface with RL-based lines to do with Zoom, 09:50.920 --> 09:55.400 which was also provided by the developer's stable-based lines. 09:55.400 --> 09:58.320 And basically, if you have a German variable for your robot, 09:58.320 --> 10:01.880 you can go to this library, connect it to your environment, 10:01.880 --> 10:03.520 and ask it to train policies for you. 10:03.600 --> 10:05.440 And it will do it directly. 10:05.440 --> 10:08.080 So, oops, sorry for that. 10:08.080 --> 10:10.600 In the previous version, we implemented that logic ourselves, 10:10.600 --> 10:12.560 and then we figured out someone did it better than us. 10:12.560 --> 10:15.840 So now we use that. 10:15.840 --> 10:17.720 So stable-based lines is one. 10:17.720 --> 10:21.240 There is also this nice contribution that came 10:21.240 --> 10:25.080 from Magic Crystal, who is someone else in the academic community, 10:25.080 --> 10:26.760 using MJ Lab. 10:26.760 --> 10:31.000 And so this is another physics simulator, 10:31.000 --> 10:32.360 which was mentioned before as well. 10:32.360 --> 10:35.720 And we'll do a local warp, the GPU version. 10:35.720 --> 10:38.160 And this one can run more environments in parallel, 10:38.160 --> 10:39.880 because it runs on the GPU. 10:39.880 --> 10:43.080 And in today's technology, this is faster. 10:43.080 --> 10:48.520 So I will show you not the training this time, 10:48.520 --> 10:50.560 but if you want to try it out, also, 10:50.560 --> 10:54.800 mark made it really simple that here, this is a script, 10:54.800 --> 10:56.400 but the script is just because I'm afraid 10:56.400 --> 10:57.800 I will forget the command line. 10:57.800 --> 11:00.480 You'll be run Python soon, right? 11:00.600 --> 11:03.280 And here, so he prepared this, 11:03.280 --> 11:04.920 where basically here, you have a module 11:04.920 --> 11:07.040 of simulation that's running live. 11:07.040 --> 11:09.000 And then from the keyboard here, 11:09.000 --> 11:13.720 I can command it to go forward or backwards or turn. 11:13.720 --> 11:16.800 And you can see that that policy actually 11:16.800 --> 11:19.800 learns to use not only the wheels, 11:19.800 --> 11:22.000 but also the rest of the legs. 11:22.000 --> 11:26.200 So it's actually much smoother than the policy we are going 11:26.200 --> 11:29.680 to throw in the end, which is going to keep the legs straight. 11:29.680 --> 11:32.280 And this one also is a neural network policy, 11:32.280 --> 11:35.360 so that was trained with a large amount 11:35.360 --> 11:37.080 of simulated data in mujoko. 11:39.840 --> 11:41.560 And so as well, if you have your own robot, 11:41.560 --> 11:45.040 or you have other use cases, so MJ lab app 11:45.040 --> 11:47.480 issues use how to apply it to this robot, 11:47.480 --> 11:50.840 but if you want to use other robots, you can use MJ lab, 11:50.840 --> 11:53.440 make a gymnasium, or basically, 11:53.440 --> 11:55.960 even they don't use gymnasium, I think, actually. 11:55.960 --> 11:58.240 Connected to MJ lab and use it to train policies 11:58.240 --> 11:59.640 for your robots. 12:00.560 --> 12:04.160 That's it, about reinforcement learning. 12:04.160 --> 12:08.280 So I wanted to kind of share some examples of things 12:08.280 --> 12:10.280 that worked and didn't work in the project so far, 12:10.280 --> 12:13.840 so the project has been running since 2022, roughly. 12:15.000 --> 12:17.760 We have been very happy with open sourcing everything. 12:17.760 --> 12:19.600 Again, very easy to say here, 12:19.600 --> 12:21.600 but this means people will just pop up. 12:21.600 --> 12:23.320 Basically, I didn't expect 12:23.320 --> 12:25.200 and I think nobody expected really 12:25.200 --> 12:27.080 in advance what would work or not. 12:27.080 --> 12:29.920 So for instance, some of the things that didn't work, 12:29.920 --> 12:32.840 I expected at first that people would want to reuse 12:32.840 --> 12:36.360 the kind of C++ interface to communicate with MJ 12:36.360 --> 12:37.320 about actuators. 12:38.440 --> 12:41.440 No, and that made the code more complex to build, 12:41.440 --> 12:43.040 because every time we needed to change that, 12:43.040 --> 12:46.040 then you had the cascade of projects to update that was a mess. 12:46.040 --> 12:47.800 So eventually we just merged everything 12:47.800 --> 12:51.640 into a single project as much as we could, and that helps. 12:51.640 --> 12:55.200 So it's simpler for us, and nobody really cared anyway, so far. 12:56.200 --> 12:59.440 We open sourced the hardware, the 3D printed parts. 12:59.440 --> 13:02.160 Some people would design it entirely, so you know. 13:02.160 --> 13:04.760 I mean, we don't know in advance what's going to work 13:04.760 --> 13:08.440 or not for other people, so very happy about open sourcing. 13:08.440 --> 13:10.680 We were very happy about switching to Pixie. 13:11.680 --> 13:12.880 Oh yeah, in the things that didn't work, 13:12.880 --> 13:14.840 I should mention the Wi-Fi in the network. 13:14.840 --> 13:18.320 I mean, this sounds trivial, but actually, 13:18.320 --> 13:19.640 yeah, we had trouble with that. 13:19.640 --> 13:23.240 This robot has drew all like it sets up in access points, 13:23.240 --> 13:25.360 and it tries to connect to the Wi-Fi. 13:25.360 --> 13:27.160 We made it too complicated with the Wi-Fi. 13:27.160 --> 13:28.840 That's my fault. 13:28.840 --> 13:32.120 But then when we switched to Pixie, you had this tool called Pixie Pack, 13:32.120 --> 13:35.120 so it will prepare your whole Python environment for you, 13:35.120 --> 13:38.920 and upload it to the robot, whether it has internet or not. 13:38.920 --> 13:40.680 That was hugely helpful as well. 13:40.680 --> 13:42.920 So Pixie also was cool. 13:44.200 --> 13:47.560 And then the model predictive control, which we are going to show you in the end, 13:47.560 --> 13:50.760 that was cool as well to be able to run model predictive control. 13:50.760 --> 13:55.760 It, the main benefit is that it has two parameters. 13:55.760 --> 13:57.960 So when we switched to a different robot, 13:57.960 --> 14:00.600 even these two robots, they have the same structure, 14:00.600 --> 14:02.880 but they have different masses, different geometry. 14:03.880 --> 14:05.760 The MPC just works as is. 14:05.760 --> 14:08.520 You basically need to update the wheel radius. 14:08.520 --> 14:10.520 That's a biggest parameter. 14:10.520 --> 14:13.760 Everything else, we don't need to adapt. 14:13.760 --> 14:17.520 So that was very convenient to make new robots. 14:18.480 --> 14:21.760 And yeah, and things that didn't work here, 14:21.760 --> 14:26.760 we also tried to reuse the same code for the rear robot, 14:26.760 --> 14:30.160 and for simulation testing, and for reinforcement learning, 14:30.160 --> 14:31.360 that was too greedy. 14:31.360 --> 14:32.360 Now we have the couple. 14:32.360 --> 14:34.760 For reinforcement learning, you live in a different world, 14:34.760 --> 14:36.440 you train a neural network, 14:36.440 --> 14:39.240 and then just take this neural network and run it on the robot. 14:39.240 --> 14:41.160 That was much simpler. 14:42.560 --> 14:47.400 Anyway, so some applications that have been tried with these robots, 14:48.360 --> 14:50.800 adding an articulated head. 14:50.800 --> 14:52.480 So if you look on the top of the robots, 14:52.480 --> 14:54.640 and for the new version, even on the front, 14:54.640 --> 14:57.800 we have a four-screw pattern, so that people can put add-ons. 14:57.800 --> 14:59.520 So basically, you 3D print a part, 14:59.520 --> 15:00.600 and then you can put the camera, 15:00.600 --> 15:02.560 you can put the handle, et cetera. 15:03.560 --> 15:06.800 So one project did an articulated head, two extra motors, 15:06.800 --> 15:10.120 and the application was tracking a narooka marker. 15:10.120 --> 15:12.280 So someone moves around in the room is a marker, 15:12.280 --> 15:13.640 and the robot just follows you. 15:14.600 --> 15:18.400 We've been applying it to estimated contacts. 15:18.400 --> 15:19.880 That's more like a research question, 15:19.880 --> 15:23.080 but this robot does not have a wheel to ground sensor, 15:23.080 --> 15:25.120 so you estimate it indirectly. 15:25.120 --> 15:28.720 We've been using machine learning to try to improve 15:28.720 --> 15:30.760 on that contact detection. 15:31.960 --> 15:36.600 And then Vanhonten, around here, worked on obstacle avoidance, 15:36.600 --> 15:38.280 and so he can tell you if you're interested 15:38.280 --> 15:40.640 about how to use, so Gaussian splatting 15:40.640 --> 15:43.040 to generate images for your simulation. 15:43.120 --> 15:46.760 So they are photorealistic, as far as a robot is concerned, 15:46.760 --> 15:49.160 and reinforcement learning to train a policy, 15:49.160 --> 15:51.720 and then we can use the camera so that if an obstacle 15:51.720 --> 15:54.720 comes in the field of view, the robot will correct the velocity 15:54.720 --> 15:58.840 that the user inputs, but basically, this is something 15:58.840 --> 16:00.760 that with an implicit model, 16:00.760 --> 16:02.800 it doesn't have an explicit geometry of the environment. 16:02.800 --> 16:05.600 We can talk with Vanhonten if you're interested. 16:05.600 --> 16:09.680 Just hand it over to you, and then in the pipeline, 16:09.680 --> 16:11.680 we want to execute some more dynamic motions, 16:11.680 --> 16:14.680 like pre-recorded motions that people can design offline, 16:14.680 --> 16:18.320 and then run on the robot, so like jumping would be cool. 16:18.320 --> 16:23.600 And then also stream line the way to use vision 16:23.600 --> 16:26.240 when you move, not just proprioceptive, 16:26.240 --> 16:29.880 so these two robots here have cameras, for instance, 16:29.880 --> 16:33.000 that will be developed more in the future. 16:33.000 --> 16:35.880 And now I will turn to the hardware, 16:35.880 --> 16:37.880 so maybe I will give like a high level of review, 16:37.880 --> 16:40.720 and then you guys can take over with a demo. 16:40.720 --> 16:42.880 So for the hardware, we try also to make it easier 16:42.880 --> 16:44.240 for people to reproduce. 16:44.240 --> 16:46.560 So we have a bit of manual with pictures, 16:46.560 --> 16:50.840 never enough, but some pictures, a bit of material, 16:50.840 --> 16:52.440 3D printing parts, I mean, 16:52.440 --> 16:56.920 get repository using LFS for the free CAD parts. 16:56.920 --> 16:59.920 Now, thanks to these guys now, the parts are designed 16:59.920 --> 17:04.080 with free CAD, it was initially blender, my bad. 17:04.080 --> 17:06.640 And yeah, there are variants of these robots. 17:06.640 --> 17:09.480 So I will show you, so cookie here is an app key T, 17:09.480 --> 17:12.560 which was designed by Mika and Mathieu. 17:12.560 --> 17:14.520 And then this, that you see on the right, 17:14.520 --> 17:18.120 is actually a complete redesign of the robots hardware 17:18.120 --> 17:20.440 by Martin Krizo, because when he built it, 17:20.440 --> 17:23.760 he was like, oh, I actually have my own preferences, 17:23.760 --> 17:25.280 so I would just redesign everything. 17:25.280 --> 17:28.280 So he re-improves again and shared it. 17:28.280 --> 17:31.120 This is also it's shared on shape, 17:31.120 --> 17:34.120 and you can find it in the readme of Marx project. 17:34.120 --> 17:38.480 So quickly about the electronics, when you build the robots, 17:38.480 --> 17:41.920 so the here is a battery sure, the battery plug. 17:41.920 --> 17:45.120 We connect the standard, like a re-obi battery 17:45.120 --> 17:48.800 to an MJ bot's PowerDisk board, 17:48.800 --> 17:53.480 and that will then supply power to the Raspberry Pi via the hat. 17:53.480 --> 17:59.120 And then this Pi is connected to the motors via twisted cables 17:59.120 --> 18:04.120 for it uses the KNFD protocol to communicate with the actuators. 18:04.120 --> 18:05.800 And then we don't go beyond that. 18:05.800 --> 18:08.120 I mean, the MJ bot's API takes care of us 18:08.120 --> 18:10.680 for us of sending packets to the motors 18:10.680 --> 18:12.520 and handling them. 18:12.520 --> 18:15.560 You send the packet, you get back the sensory measurements, 18:15.560 --> 18:16.200 et cetera. 18:18.800 --> 18:20.960 And yeah, just in pictures. 18:20.960 --> 18:22.960 So we didn't plan anything for the hardware, 18:22.960 --> 18:24.520 so it's not like a linear progression. 18:24.520 --> 18:27.480 Of course, it's a tree of what people did. 18:27.480 --> 18:30.320 So here you have various robots, rookie, 18:30.320 --> 18:31.480 Michel Sorgoff. 18:31.480 --> 18:33.680 This is Nadia Fedor. 18:33.680 --> 18:37.920 This one is a name, Michel Sorgoff B's cookie, 18:37.920 --> 18:41.680 and then this one is called Blueberry Service. 18:41.680 --> 18:44.680 And yeah, we hope it continues. 18:44.680 --> 18:47.640 So cookie here, you will see in the end, 18:47.640 --> 18:49.880 it goes in the direction of adding more features. 18:49.880 --> 18:53.400 So it's a cooler robot to use, 18:53.400 --> 18:57.880 and for instance, adds charging of the robot directly onboard. 18:57.880 --> 19:00.120 So you don't need to do it separately. 19:00.120 --> 19:03.200 It has bumpers, it has a screen on the robot. 19:03.200 --> 19:06.200 So you can interact with it directly without your computer, 19:06.200 --> 19:07.600 but then it's more complexity. 19:07.600 --> 19:09.360 So it's harder to reproduce. 19:09.360 --> 19:13.040 And even when it breaks, I know it will take time 19:13.040 --> 19:16.360 to open things and find the cables again, et cetera. 19:16.360 --> 19:22.040 But when it works, it's a pleasure to use. 19:22.040 --> 19:25.360 And for upkey V2, I will switch over to you guys, 19:25.360 --> 19:27.520 because the two designers of upkey V2 are here. 19:27.520 --> 19:28.360 So. 19:28.360 --> 19:30.840 OK, so very well. 19:30.840 --> 19:34.600 So I'm using a team for these two years now, 19:34.600 --> 19:36.280 and they made it. 19:36.280 --> 19:40.040 And so we had some, like, real-time one things. 19:40.040 --> 19:42.960 They've been, they've been working for upkey V2. 19:42.960 --> 19:45.760 At some point, we wanted to do a computer design, 19:45.760 --> 19:47.120 just great time. 19:47.120 --> 19:50.760 Because the previous part were design blender. 19:50.760 --> 19:54.200 So to iterate on a part and modify it, 19:54.200 --> 19:55.680 it was not easy. 19:55.680 --> 19:57.720 And with Freakhead, we could use 19:57.720 --> 20:00.040 parametric parts to modify it easy. 20:00.040 --> 20:02.440 And I don't know if we want to improve, 20:02.440 --> 20:04.880 like, increase the length of the legs or something. 20:04.880 --> 20:06.080 We can do it. 20:06.080 --> 20:10.480 It was also wanting to make the robot less complicated 20:10.480 --> 20:14.360 in the complete other direction than upkey T, I think, 20:14.360 --> 20:18.600 I believe, it's much simpler. 20:18.600 --> 20:21.520 We had way too many parts for now on the main parts, 20:21.520 --> 20:24.480 or one torso, when the one lower leg and two 20:24.480 --> 20:26.320 because two sides. 20:26.320 --> 20:30.600 And so it's a much simpler robot, much easier 20:30.600 --> 20:32.920 to operate with, because everything is accessible, 20:32.920 --> 20:36.560 even the cables are not internal, but they're now outside. 20:36.560 --> 20:39.400 And yeah, improve the durability of the robot too, 20:39.400 --> 20:43.160 because we had issues of tbs and females breaking. 20:43.160 --> 20:46.400 And so the new redesign, which has a sort of fog design 20:46.400 --> 20:49.960 for the apparent lower leg, makes it much much stiffer. 20:49.960 --> 20:53.160 For instance, it hit a wall at some point. 20:53.160 --> 20:55.720 And we can see here. 20:55.720 --> 21:00.160 At the part, slightly broke, but it's not a complete failure, 21:00.160 --> 21:04.160 and which could not have been possible in the previous design. 21:04.160 --> 21:06.280 I think we can launch the end school. 21:06.280 --> 21:07.840 Yeah. 21:07.840 --> 21:08.880 Let's end with a live demo. 21:08.880 --> 21:11.000 And if you guys have questions, feel free to. 21:15.000 --> 21:19.800 Demo effect might pop in. 21:19.800 --> 21:21.760 So what you are going to see here 21:21.760 --> 21:24.360 is the default MPC controller 21:24.360 --> 21:28.880 that we're also distributing in the robot package. 21:28.880 --> 21:30.520 Yeah. 21:30.520 --> 21:31.320 And we have a leash. 21:31.320 --> 21:33.840 Yeah, we have the little leaf, just in case. 21:33.840 --> 21:36.320 So if we don't have time for the demo now to switch 21:36.320 --> 21:38.360 to the next presentation, you would probably see us around 21:38.360 --> 21:41.240 also today with the robots. 21:41.240 --> 21:43.520 So don't hesitate to, yeah. 21:43.520 --> 21:45.000 Don't hesitate to interact with a happy tool. 21:48.200 --> 21:48.480 Yeah. 21:48.480 --> 21:51.040 And so the fact that it has legs is for, 21:51.040 --> 21:52.440 come and take a visit for crouching, 21:52.440 --> 21:55.480 but hopefully in the future, I hope we can 21:55.480 --> 21:58.880 climb these stairs, for instance, that would be cool. 21:58.880 --> 21:59.680 But we will see. 22:02.160 --> 22:04.280 Maybe I should showcase a bit. 22:04.280 --> 22:06.440 Maybe you can stand here. 22:06.440 --> 22:08.800 If you want to try later on to come and pull it 22:08.800 --> 22:12.680 and see how much it can balance, feel free to. 22:12.680 --> 22:16.000 And so this is the MPC, it's a QP that solves the real time 22:16.000 --> 22:19.760 on the Raspberry Pi with a reverse gradient matrix 22:19.760 --> 22:24.440 to have the legs be able to bend and go lower. 22:24.440 --> 22:28.440 And you can feel the perturbate and integration. 22:28.440 --> 22:29.280 Exactly. 22:29.280 --> 22:29.920 And it's still open source software. 22:29.920 --> 22:32.000 So the QP solver is called ProxQP. 22:32.000 --> 22:32.960 It's open source. 22:32.960 --> 22:36.080 And yeah, the inverse kinematics and MPC 22:36.080 --> 22:37.640 are also open source libraries. 22:37.640 --> 22:41.200 If you are interested, we can talk about it 22:41.200 --> 22:43.000 or you can find it from the project report. 22:45.560 --> 22:47.280 All right, thank you very much for attention. 22:47.280 --> 22:48.120 Thanks. 22:48.600 --> 22:49.120 Thank you. 22:57.120 --> 22:59.280 I think we might have time for one question. 22:59.280 --> 22:59.960 If you're OK. 23:05.600 --> 23:07.080 Thanks a lot for presentation. 23:07.080 --> 23:09.040 Can you quickly talk about the reward function 23:09.040 --> 23:10.800 you are using in reinforcement learning? 23:10.800 --> 23:11.320 Yes. 23:11.320 --> 23:13.960 Variables are you trying to optimize during it? 23:13.960 --> 23:14.600 Yeah, sure. 23:14.600 --> 23:15.880 Thank you for this question. 23:15.880 --> 23:17.800 So indeed, when you do reinforcement learning, 23:17.800 --> 23:19.800 all when you do the QP, you have to define the objective 23:19.800 --> 23:21.680 function. 23:21.680 --> 23:25.160 In reinforcement learning, the reward has three terms here. 23:25.160 --> 23:27.960 The main term is you take the point at the top of the robot 23:27.960 --> 23:30.200 and you want to keep it where it should be. 23:30.200 --> 23:33.040 So basically, you have a reference point 23:33.040 --> 23:35.120 and you want to stay there, which means if you're 23:35.120 --> 23:36.680 tilting, you have to tilt back. 23:36.680 --> 23:39.960 If you're translating, you have to translate back. 23:39.960 --> 23:42.120 This is the same in MPC and RL. 23:42.120 --> 23:45.480 And then in RL, we also add actions smoothness 23:45.480 --> 23:46.800 for regularization. 23:46.800 --> 23:48.640 And the action here is a ground velocity. 23:48.640 --> 23:52.280 So we try to minimize ground velocities. 23:52.280 --> 23:54.960 And there is probably a third term that I forgot. 23:54.960 --> 23:56.200 Velocity dumping, maybe? 23:56.200 --> 23:57.720 I'm not sure. 23:57.720 --> 23:58.760 It's in the code. 23:58.760 --> 24:00.520 It's in the code. 24:00.520 --> 24:04.600 It's right here. 24:04.600 --> 24:05.200 I can check. 24:05.200 --> 24:05.640 I mean, it's good. 24:05.640 --> 24:08.360 Yeah, yes, yes, yes, yes. 24:08.360 --> 24:10.160 OK, it's somewhere in the code. 24:10.160 --> 24:11.040 Thank you, thank you. 24:11.040 --> 24:11.360 Thank you. 24:11.360 --> 24:11.840 Thank you. 24:11.840 --> 24:17.840 All right, thank you very much. 24:17.840 --> 24:18.840 Thanks. 24:18.840 --> 24:26.840 Thank you.