WEBVTT 00:00.000 --> 00:17.000 Welcome, everybody. Welcome back to the Tool Docks Room. For the next talk, we're going 00:17.000 --> 00:25.960 to have Daniel Beck, who's a documentarian and documentation consultant. I've met you in 00:25.960 --> 00:33.320 the past life when you were working for the MDN project. Daniel is going to be talking 00:33.320 --> 00:39.840 about MDX, which has been one of the most popular talks also last year. So, very much looking 00:39.840 --> 00:45.120 forward to it and I'll hand it over to you, Daniel. 00:45.120 --> 00:54.220 Thank you. Can you all hear me well? Excellent. There are so many of you that care about Mark 00:54.220 --> 01:01.980 Tom. I'm so sorry, my condolences. So, I want you to imagine that you're writing 01:01.980 --> 01:07.580 a blog post. This is where you're on website. You don't play by anyone else's rules here. 01:07.580 --> 01:13.900 You just get to open your text editor. You get to start writing and words are flowing. I want 01:13.900 --> 01:19.300 you to imagine that you need to reference a chart and provide some explanatory text. That is 01:19.300 --> 01:25.700 you want to insert a figure. I want you to imagine what is the next thing you're going to 01:25.700 --> 01:31.300 type? What are the actual keys you're going to press X? Do we need to... 01:31.300 --> 01:36.300 What's that any better? 01:36.300 --> 01:59.300 Let's try that. That is better. There we have it. What's the next thing you're going 01:59.300 --> 02:04.300 to type? Well, you think about that. I think it's like history of Mark down that we can look 02:04.300 --> 02:08.860 to as maybe inspiration. Neil Dash recently wrote this history of Mark down and talks about 02:08.860 --> 02:15.340 it as this open source success story. A couple of hackers built this thing, released it into 02:15.340 --> 02:21.100 the world and now we live in this world of Mark down. Everyone's writing Mark down. He writes 02:21.100 --> 02:25.300 that the single biggest source inspiration from Mark down syntax is the format of plain text 02:25.300 --> 02:31.300 email. Unset there is that the format of plain text email was never set up. There were 02:31.300 --> 02:40.300 no rules and so our world of Mark down inherits that kind of lawlessness. Really, when 02:40.300 --> 02:44.300 we have to like confront this thing of like one of the next characters we're going to type, 02:44.300 --> 02:52.300 we have lots of options. I want to know how many of you type three columns in your mind. 02:52.300 --> 03:00.300 Okay, one person, fantastic. How many of you type curly-based percentage sign? I got to 03:00.300 --> 03:07.300 get a couple. How many of you type import figure component from figure component.js? Yeah. 03:07.300 --> 03:12.300 But somehow we live in this world where Mark down plurals have proliferated, where like the original 03:12.300 --> 03:17.300 mark down or common mark, they don't provide figures so these extended markdowns have flourished. 03:17.300 --> 03:22.300 And so these end up being things that real people have type. So I've done this three columns in a row. 03:22.300 --> 03:27.300 And when I type this out, I cannot help but think of chopped onions. First of you kind of squint. 03:27.300 --> 03:30.300 I think you can kind of see it. Is it the chop part of the onion and the un-shot part of the onion? 03:30.300 --> 03:37.300 Second, onions are delicious. If we're going to have a bunch of Mark down flavors, this is a pretty good mascot. 03:37.300 --> 03:43.300 And third, this makes me cry. When we extend Mark down, we take lots of hits. 03:43.300 --> 03:50.300 We take hits to readability and writeability. So readability, this is kind of ugly to look at at just a source level. 03:50.300 --> 03:56.300 At a writeability level, if I'm an author and I am in a new environment, I have to ask myself what extensions are here, 03:56.300 --> 04:03.300 how does this play with the markdown that I am familiar with, what white space is surprisingly significant. 04:03.300 --> 04:06.300 It makes it less secure and less reliable. 04:06.300 --> 04:13.300 MDX is really powerful, but it's also very difficult to lock down. You're just like one author from away, 04:13.300 --> 04:18.300 from like an ill-advised copy and paste from an LLM to a cross-site scripting attack. 04:18.300 --> 04:24.300 Reliability takes a hit. Even if it works, you've got this massive set of dependencies. 04:24.300 --> 04:27.300 You're not, it's not this one, Pearl script that produces Mark down. 04:27.300 --> 04:35.300 It's a presentation of Mark down. It's got this massive JavaScript and NPM dependency. 04:35.300 --> 04:42.300 An operability, interoperability, takes it. This is not going to play nice with a GitHub or my text editor. 04:42.300 --> 04:51.300 And I'm going to get locked in. I can't use Pandok to print this out. I can't truly move between content management systems. 04:51.300 --> 05:00.300 And so I relate a lot to poor percentage here. I am a writer and a programmer, and that means I know lots of ways to use a semicolon. 05:00.300 --> 05:06.300 And in my consulting and freelancing career, I have helped teams build reliable docs, tools and processes. 05:06.300 --> 05:11.300 And several times, I have warned about using a Mark down variant. 05:11.300 --> 05:16.300 And all the one time I've seen that go on to blow up in someone's face, often mine. 05:16.300 --> 05:21.300 And so at that one time where it hasn't, it just hasn't happened yet. 05:21.300 --> 05:26.300 So I can tell you, never extend Mark down, but I know you won't listen, and that's fine. 05:27.300 --> 05:35.300 But wouldn't it be nice? Wouldn't it be nice if Mark down already included an extension point that plays nice with existing tools that doesn't require changing a parser? 05:35.300 --> 05:39.300 That's a little familiar, a little readable, a little portable. 05:39.300 --> 05:42.300 I've got good news for you. There's three of them. 05:42.300 --> 05:45.300 I'm going to tell you about each one. 05:45.300 --> 05:49.300 There may be there's more, but like these are three that I know that are pretty suitable. 05:49.300 --> 05:52.300 I'm going to show you a contrived example of each. 05:53.300 --> 06:06.300 I'm going to assume you're going to be building some kind of artifact from your markdown that you're not concerned that much with the source representation that you are building website or producing something on artifact. 06:06.300 --> 06:11.300 Then I'm pretty sure what I'm going to talk about a kind of tip or tip for working with them. 06:11.300 --> 06:15.300 The first are bare links to treat them as short codes. 06:16.300 --> 06:22.300 In this example, an author wants to insert a newsletter sign-up form in the middle of the prose. 06:22.300 --> 06:27.300 You've reached this point in the article, you want to drop in a newsletter sign-up form. 06:27.300 --> 06:29.300 On the left, I could do it with MDX. 06:29.300 --> 06:34.300 I could have this component that I, this HTML looking thing, that I placed somewhere. 06:34.300 --> 06:36.300 On the right, I've got a bare link. 06:36.300 --> 06:39.300 I just have a paragraph break, and then I'll link this. 06:39.300 --> 06:44.300 Depending on your markdown configuration, you might not even need the angle brackets here. 06:44.300 --> 06:48.300 The upside to this is before I've done anything, this link is clickable. 06:48.300 --> 06:53.300 I can click on it in my editor, and it will open up a webpage where I can do something. 06:53.300 --> 06:56.300 I can sign up for a newsletter. 06:56.300 --> 07:00.300 The same cannot be said for this MDX. 07:00.300 --> 07:03.300 This plays nice with other tools. 07:03.300 --> 07:10.300 So for instance, GitHub's markdown rendering is going to show me a link that I can click on. 07:10.300 --> 07:13.300 And the implementation story here is pretty straightforward. 07:13.300 --> 07:18.300 We find a paragraph that contains a link that we know and we turn it into something else. 07:18.300 --> 07:23.300 This is a lot less complicated than all of the many things that MDX is doing. 07:23.300 --> 07:26.300 Downside here is pretty limited customization. 07:26.300 --> 07:29.300 If you're going to go this route, you kind of have two options. 07:29.300 --> 07:33.300 One is to have lots and lots of URLs that do things. 07:33.300 --> 07:39.300 So you can have a sort of vanity URL that points to your newsletter sign-up form that's also itself a hint to your tooling to make 07:39.300 --> 07:41.300 show the big newsletter sign-up form. 07:41.300 --> 07:43.300 You can also do this with query parameters. 07:43.300 --> 07:47.300 So this is one of those things where you probably only want to do it for URLs that are under your control. 07:47.300 --> 07:49.300 You probably don't want to be passing. 07:49.300 --> 07:54.300 So YouTube, arbitrary URL parameters. 07:54.300 --> 07:57.300 If you're going to do this, please implement it defensively. 07:57.300 --> 08:00.300 You're going to want to transform the HTML. 08:00.300 --> 08:03.300 After you've converted it from markdown to HTML and then do stuff to the HTML. 08:03.300 --> 08:08.300 Find that paragraph that contains the A tag and do that. 08:08.300 --> 08:18.300 To the extent that you're able to avoid tampering with markdown processing and the very, very complicated rules there on all the better. 08:18.300 --> 08:21.300 The second extension point here are code fences. 08:21.300 --> 08:23.300 You can do fancy code fences. 08:23.300 --> 08:26.300 On the left here, we have a GitHub flavored markdown table. 08:26.300 --> 08:27.300 I hate writing these. 08:27.300 --> 08:29.300 I hate maintaining these. 08:29.300 --> 08:32.300 They're kind of nice to look at, I guess, in their source format. 08:32.300 --> 08:35.300 But I think that's the main thing going for them. 08:35.300 --> 08:41.300 On the right here, we have a code fence containing CSV. 08:41.300 --> 08:44.300 So in this situation, I've got some configuration information. 08:44.300 --> 08:49.300 I've probably dumped it out of a spreadsheet or database anyway. 08:49.300 --> 08:51.300 Here, I just paste it into the code fence. 08:51.300 --> 08:55.300 And then the general strategy here is we coordinate off some arbitrary text. 08:55.300 --> 08:58.300 We flag it as rich content and we promote it into something else. 08:58.300 --> 09:01.300 And the trick here is in the InfoStream. 09:01.300 --> 09:06.300 Where you see the three back ticks and CSV. 09:06.300 --> 09:08.300 This is the start of the InfoStream. 09:08.300 --> 09:10.300 That first token is really important. 09:10.300 --> 09:12.300 That's what gets you that syntax highlighting in your editor. 09:12.300 --> 09:13.300 We want to respect that. 09:13.300 --> 09:19.300 But lots of tools will just accept whatever you want on the rest of that line. 09:19.300 --> 09:20.300 And they don't care. 09:20.300 --> 09:25.300 So in this case, we can flag it as a pretty table and render it out as a table. 09:25.300 --> 09:28.300 Just like we would for this GitHub flavored markdown. 09:29.300 --> 09:32.300 Forges like GitHub and Codeberg do something quite like this. 09:32.300 --> 09:35.300 So GitHub uses this trick to render mermaid diagrams. 09:35.300 --> 09:38.300 Where if you tell it it's mermaid, it gives you the mermaid diagram. 09:38.300 --> 09:40.300 It doesn't actually show you the source. 09:40.300 --> 09:43.300 They don't use the InfoStream to control this. 09:43.300 --> 09:47.300 But it's pretty close to what I'm suggesting here. 09:47.300 --> 09:50.300 Another cool trick you can do with this is markdown in markdown. 09:50.300 --> 09:56.300 So on the left side, we have this weird block notation to say that, 09:56.300 --> 09:58.300 this set of paragraphs is in set. 09:58.300 --> 10:01.300 It's going to decide from the rest of the text. 10:01.300 --> 10:04.300 On the right, we can do the same thing with a code fence. 10:04.300 --> 10:06.300 And flag it as markdown. 10:06.300 --> 10:12.300 And then alert our tools that we actually want to turn that into something else. 10:12.300 --> 10:15.300 And all the way, we get the nice syntax highlighting. 10:15.300 --> 10:23.300 We don't see these chopped onions in, like say, the GitHub rendering. 10:24.300 --> 10:27.300 The upside to this whole approach is it plays really nicely with tools. 10:27.300 --> 10:30.300 I don't know about you, but a lot of code fences. 10:30.300 --> 10:34.300 They just kind of show up and you get a nice gray box with some, you know, 10:34.300 --> 10:36.300 monospace text, great authoring. 10:36.300 --> 10:38.300 It's also a similar story. 10:38.300 --> 10:39.300 I know how to do this. 10:39.300 --> 10:42.300 I don't have to learn anything new. 10:42.300 --> 10:45.300 The downsides here do exist. 10:45.300 --> 10:50.300 You've got issues, if you have automatic formatters or spell check. 10:50.300 --> 10:54.300 Other things that might need special attention depending on the actual, 10:54.300 --> 10:57.300 like, contents of the code fence. 10:57.300 --> 11:01.300 These onions aren't insurmountable, but they do require some attention, 11:01.300 --> 11:04.300 depending on your specific needs. 11:04.300 --> 11:07.300 And the implementation story is more complicated. 11:07.300 --> 11:10.300 Like it's that in a second. 11:10.300 --> 11:14.300 My big tips for taking this approach is to separate your concerns. 11:14.300 --> 11:19.300 So use that info string as the control for display. 11:19.300 --> 11:23.300 And then reserve the contents of the code fence for content. 11:23.300 --> 11:30.300 You don't want to find yourself breaking a special comment format inside your code fence 11:30.300 --> 11:33.300 to control this kind of thing. 11:33.300 --> 11:40.300 To the extent that you're able to, again, avoid tampering with the markdown parsing. 11:40.300 --> 11:45.300 It's better if, for instance, you have a tool that gives you a document tree out of markdown, 11:45.300 --> 11:49.300 and then you transform that tree and then convert into HTML or something like that. 11:49.300 --> 11:53.300 Again, you don't want to be fighting markdown itself. 11:53.300 --> 12:00.300 And exercise restraint, it is very tempting to invent some custom DSL that goes inside that code fence. 12:00.300 --> 12:05.300 And then you're not that far removed from something like MDX anyway. 12:05.300 --> 12:09.300 The third extension point is HTML itself. 12:09.300 --> 12:14.300 It is tempting to think of writing HTML in markdown as a kind of failure, 12:14.300 --> 12:20.300 but it is not in contrast to something like restructured text or ASCII doc. 12:20.300 --> 12:26.300 Markdown just fundamentally assumes a certain sort of HTML-ishness to it. 12:26.300 --> 12:28.300 You don't have to invoke it. 12:28.300 --> 12:30.300 Specifically, you can just start writing HTML. 12:30.300 --> 12:32.300 This is an expected behavior. 12:32.300 --> 12:38.300 And HTML itself is really sensible, there's data attributes, there's custom elements. 12:38.300 --> 12:41.300 There's lots of things you can do to make HTML your own. 12:41.300 --> 12:49.300 And so here, I have, on the left, Github's Adminitions and Tax. 12:49.300 --> 12:53.300 This one is really hard. 12:53.300 --> 12:57.300 It looks like a blockcode, it's not a blockcode. 12:57.300 --> 13:01.300 It has completely different indentation rules from a regular blockcode. 13:01.300 --> 13:04.300 This is really complicated from an offering perspective. 13:04.300 --> 13:07.300 I'm not even sure what they're doing in terms of implementation. 13:07.300 --> 13:09.300 It's kind of scary. 13:09.300 --> 13:10.300 It's really just like it. 13:10.300 --> 13:15.300 On the right, we've got either data attribute or custom elements to do something similar. 13:15.300 --> 13:17.300 This is a contripe example. 13:17.300 --> 13:19.300 I'll show you a better one in just a moment. 13:19.300 --> 13:22.300 But first, I'm using this one to acknowledge that there's a real downside here, 13:22.300 --> 13:25.300 which is the offering experience is poorer. 13:25.300 --> 13:29.300 The rules around HTML in markdown are somewhat complicated, 13:29.300 --> 13:33.300 especially when you have differences between block level, HTML, 13:33.300 --> 13:38.300 HTML that appears on its own line versus HTML that appears in line. 13:38.300 --> 13:42.300 It's also a poor reading experience. 13:42.300 --> 13:45.300 It's not so nice to look at. 13:45.300 --> 13:48.300 And it's not as tool-friendly as other options. 13:48.300 --> 13:53.300 So for instance, the forges will escape some HTML, 13:53.300 --> 13:58.300 it just disappears or it'll render it out as a literal HTML. 13:58.300 --> 14:01.300 It's not ideal. 14:01.300 --> 14:04.300 That said, this pain is very familiar pain. 14:04.300 --> 14:06.300 I know a bit about writing HTML. 14:06.300 --> 14:08.300 You probably know a bit about writing HTML. 14:08.300 --> 14:12.300 That's a skill that you can actually transfer between projects. 14:12.300 --> 14:16.300 It's not ideal, but it's something that you can carry with you. 14:16.300 --> 14:18.300 HTML's really powerful. 14:18.300 --> 14:20.300 We have lots of ways to do stuff. 14:20.300 --> 14:25.300 And HTML is going to probably outlast your career. 14:25.300 --> 14:31.300 I cannot say the same thing for your weird little markdown extension. 14:31.300 --> 14:36.300 But I said that forges will escape this kind of stuff. 14:36.300 --> 14:44.300 And this is actually a kind of a nice property of working with raw HTML in markdown. 14:44.300 --> 14:47.300 You can use it in particular for things that are hits. 14:47.300 --> 14:51.300 Things that you don't necessarily need to see but make the presentation nicer. 14:51.300 --> 14:55.300 So in the left, we've got this crammed-down syntax that uses the chopped onion. 14:55.300 --> 15:01.300 You know, curly brace thing to apply a CSS class to a paragraph. 15:01.300 --> 15:07.300 And then on the right, I've used a custom element to do the same thing. 15:07.300 --> 15:10.300 Now, a tool might well hide that styling hit. 15:10.300 --> 15:14.300 I might see nothing in like a rendered version of that markdown. 15:14.300 --> 15:15.300 But that's okay. 15:15.300 --> 15:16.300 It's still readable. 15:16.300 --> 15:18.300 I can still know what's going on there. 15:18.300 --> 15:24.300 And I don't see this kind of ugly crammed-down specific syntax. 15:25.300 --> 15:29.300 If you're going to take this approach, it can be a little bit threatened. 15:29.300 --> 15:32.300 There's this sort of like impulse of like, okay, now I need to build all the tools. 15:32.300 --> 15:38.300 Like, I editor needs to completely, you know, simulate, you know, the full environment of like building it. 15:38.300 --> 15:39.300 Like, no. 15:39.300 --> 15:49.300 Like often really what you need is a little bit of CSS to give yourself the clue that you've done the right thing. 15:49.300 --> 15:59.300 So while your editor's markdown preview might not let you, you know, implement custom markdown parsing or something, 15:59.300 --> 16:06.300 it will probably let you give it a little custom CSS where you can apply some styles and throw a red box around something important. 16:06.300 --> 16:10.300 Again, I remind you implement this stuff defensively. 16:10.300 --> 16:14.300 If you have to transform things, do it at the level of HTML. 16:14.300 --> 16:16.300 There's lots of nice tools for working with HTML. 16:16.300 --> 16:19.300 There are rather fewer nice tools for working with markdown. 16:19.300 --> 16:24.300 Please do things to the HTML. 16:24.300 --> 16:29.300 If you need to help authors, you kind of need to help authors in this scenario. 16:29.300 --> 16:34.300 I do acknowledge that this is like a tougher authoring environment. 16:34.300 --> 16:43.300 And so to the extent that you're able to give, to give authors hints about say what the valid element names, 16:43.300 --> 16:55.300 what the valid attributes are, the value values, that can be all the difference between this being a really hard thing to work with and something that's just like that is fine. 16:55.300 --> 16:59.300 And sometimes you're going to have to chop some onion. 16:59.300 --> 17:02.300 Some use cases are really well justified. 17:02.300 --> 17:11.300 If you've got some interactive tool that is just got some pros sprinkled into it, that is a use case where MDX is probably the right thing to do. 17:11.300 --> 17:14.300 And I'm not going to judge you on that. 17:14.300 --> 17:19.300 In line markup is really really hard. 17:19.300 --> 17:25.300 There are some things where I don't have a better idea of like how to approach that. 17:25.300 --> 17:35.300 I don't know if you've all seen like discord has some markdownish inline syntax for spoiler text. 17:35.300 --> 17:39.300 So you can hide some, you can obscure some text. 17:39.300 --> 17:45.300 And like, hey, I don't know, I don't think I've got something better there. 17:45.300 --> 17:51.300 And legacy variants, you probably have extensions that already exist in your world. 17:51.300 --> 17:53.300 You've got to cope with that. 17:53.300 --> 18:01.300 And if you do, whatever the reason that you have to accept a markdown variant, you can give in gracefully. 18:01.300 --> 18:05.300 I think the strongest strategy for handling this is going to be with polyglot authoring. 18:05.300 --> 18:09.300 So you might default to common work in the kind of general case. 18:09.300 --> 18:17.300 And then ask authors to deliberately invoke a markdown variant to opt into that thing. 18:17.300 --> 18:23.300 So if you do find yourself, you know, years later you have to migrate to a new content management system or something like that. 18:23.300 --> 18:31.300 You've at least somewhat isolated the more complicated content that needs extra attention. 18:31.300 --> 18:33.300 You can also use off the shelf variants. 18:33.300 --> 18:43.300 So, you know, if something is widely used, it makes sensors pick that tool up instead of trying to invent your own. 18:43.300 --> 18:45.300 Again, messing with markdown is really challenging. 18:45.300 --> 18:49.300 It has a very complicated rules, especially around indentation. 18:49.300 --> 18:51.300 Like, leave it to someone else if you're able to. 18:51.300 --> 18:55.300 And if you must build your own, look to what has come before. 18:55.300 --> 19:03.300 Have a nice look around and talk to writers, documentarians. 19:03.300 --> 19:07.300 Maybe this is like a good place for me to do a little self promotion. 19:07.300 --> 19:11.300 I've worked with a lot of teams to develop authoring experiences that do not make you cry. 19:11.300 --> 19:15.300 And I'm available even for very short consultations. 19:15.300 --> 19:17.300 You just need someone to do a reality check. 19:17.300 --> 19:19.300 I can help with that. 19:19.300 --> 19:25.300 But, but I really know, like the general thing here is like you can check in with people who have been there before. 19:25.300 --> 19:33.300 You can, in particular, ask and listen to what writers have to say about the experience of trying to communicate with the thing that they're trying to communicate. 19:33.300 --> 19:37.300 So, I want you to think back to that exercise we did the beginning, 19:37.300 --> 19:41.300 where I had to imagine, like, what are you going to type next? 19:41.300 --> 19:47.300 This is a great way to find a way forward when you do think you need to extend Markdown. 19:47.300 --> 19:51.300 So, you can look at what writers are doing already. 19:51.300 --> 19:55.300 You can ask them to experiment with, like, potential. 19:55.300 --> 19:57.300 Like, what would you fill in this blank with? 19:57.300 --> 20:01.300 And so, the right extension, the right syntax, or maybe you just will work around, 20:01.300 --> 20:03.300 may already be right under your nose. 20:03.300 --> 20:09.300 And so, on that note, I invite you to go forth and shop here onions. 20:09.300 --> 20:11.300 Thank you. 20:11.300 --> 20:19.300 Thank you.