On Text Editor Plugins
Hey. It's been a bit.
I think I should just accept that these posts are probably always going to be months apart. Anyway, about editor plugins...
The Editor
Well, I guess I should first start with the editor. It's a project I've been building on-and-off for ~6 years now. It's been re-written at least 4 times (honestly it's probably more, I've forgotten) in 4 different languages (Zig, C, Rust, Odin). The current implementation is in Odin and is actually pretty capable if you don't mind some unergonomic parts. It started as a VIM clone but along the way turned into what my ideal VIM experience would be. It kinda takes inspiration from everything (Kakoune, VS Code, Helix) and it's really been a joy to work on. After I got all the core systems built and could start actually adding features to the editor in in the editor the future paths of what to work on became a little too much. It's very easy to get distracted on what to work on in it if you aren't keeping track somehow (even just a todo.md is better than nothing).
However one thing I wanted to make sure was a good experience with the editor was the plugin system. I've seen far too many implementations (not necessarily in text editors) treat plugins as second-class citizens where the app has internal utilities that plugins don't have access to and either have to re-implement them or just flat out can't be done via a plugin. I've also seen this pendulum swing the other way where plugins are first-class and core systems in the application are actually just plugins, but only the barest primitives are given and no higher-level abstractions exist to make plugin development swift. Obviously, with many things, the balance is somewhere in the middle.
The First Implementation
An earlier version of the editor had two different plugin systems. The first one involved loading DLLs at runtime and the second was a lua runtime. The DLL system was exactly what I wanted. You could build whatever you wanted and even include third-party libraries without a hassle, and if I found something annoying to build or lacked an editor API for, I would add it. The only problem was that it required a developer to compile plugins for every architecture and operating system. That problem pretty much made me abandon that idea, which led me to integrating lua into the editor. It seemed like the perfect choice since lua is practically the defacto plugin language that everybody uses. Plus it's really easy to add it to a project. I got pretty far with the lua implementation and at one point even the whole UI for the editor was defined as an internal plugin. However, I had a different issue when it came to lua. It wasn't a fault of lua itself, but the unavoidable fact that you have to have a stable API (of which I didn't really have) and you have to have bindings for that API. So when I made a change to the internal API I essentially had to re-implement that change again. This is always the case with any API obviously, but it's what made me rip out both plugins systems all-together until I had solidified my API a little more.
Another Go Around
After several months of taking a break from working on the editor, I got back around to fixing some major minor issues, and cleaning up the internal API a little. It was at that point that I thought I should try my hand again at implementing a plugin system, knowing before hand that there would most likely still be API changes that would take place. I think keeping that in mind when building a plugin system though is very important and will change your design decisions when building the bindings (such as properly keeping the bindings separate from the editor code itself).
My main goal with the plugin system was always to enable a developer to write in whatever (compiled) language they wanted. I don't have a well thought out reason for this goal, it's just something I've set out to do. I also had a new goal as well...which is sorta in conflict with the first one. I want the plugins to remain portable across systems. As you may know, compiled binaries and portability are very orthogonal concepts. However, WebAssembly has been rising in support for a while now, and I thought it was the perfect use case. Of course, in the spirit of the entire project, I was going to build the WASM runtime from scratch...that is, until I read the atrocity that is the WASM specification1. After foregoing that idea, I still thought a virtual machine was a good thing to experiment with so I started looking at RISC-V. Thankfully, since RISC-V was initially created as part of a university course on learning about instruction sets, the specification is very detailed and more-or-less easy to grasp. I do have some faults with the way they present glossary of instructions, but it's infinitely more understandable than the WASM spec.
Anyways, I spent some time implementing a basic and naive RISC-V virtual machine with a couple linux syscalls implemented so programs can print to the console (you can find it here as well as a vendored version in the editor repo). Once I got that done I then started to integrate it into the editor. I've currently gotten as far as plugins being able to call & define editor commands (that show up in the command palette).
I've realized at this point however, that I still have the same problem with the API interface that I had before. In order for the plugins to make calls to the editor (with the way I'm currently doing it), it involves too much tight coupling to the plugin system and some special casing of certain functionality. This to me is a sign that I need to re-design my internal API or even the data structures I'm using2.
So that's where I am at currently with this project. It has so many fun puzzles that crop up in places you wouldn't expect, and really makes me appreciate even the <textarea> I'm typing this post into right now. You can be (reasonably?) sure about seeing future posts about this topic.
Database Update
Unfortunately I haven't not progressed on the database stuff, my work has been a bit busy for me to think about that project. I haven't abandoned it though.
Leaving this as a footnote to not detract from the main point of the post but...why in the world does a simple virtual machine have the most arcanely written specification. If you actually want people to implement your virtual machine, at least make an op-code table that developers can reference where you don't have to jump between EBNF definitions and vaguely worded instruction definitions. Maybe I'm just dumb, but I can't even find out exactly how the stack works.↩
A funny idea also popped up into my head about refactoring the editor to run entirely in the virtual machine and get rid of the plugin boundary entirely. I'm ashamed to admit that I have actually considered this option with a little more seriousness than what it's worth. It would be a super fun thing to do though...↩