Emit multiple packages in codegen tests. Exericse as plugins.
Using golang's plugin feature, we can... well, *do* this. To date, testing codegen has involved running the "test" in the gen package to get it to emit code; and then switching to the emitted package and _manually_ running the tests there. Now, running `go test` in the gen package is sufficient to do *everything*: both the generation, and the result compiling, and we can even write tests against the interfaces and run those, all in one step. There's also lots of stuff that becomes possible now that we can easily generate multiple separate packages with various codegen outputs: - Overall: everything is granular. We can test selections of things, rather than needing to have everything fall into place at once. - Generally more organized results. - We can more easily inspect the size of generated code. - We can more easily inspect the size of the compiled result of gen! (Okay, not really. I'm seeing a '.so' file that's 4MB is coming out from 200sloc of "String". I don't think that's exactly informative. Some constant factor is thoroughly obscuring the data of interest. Nice idea in theory though, and maybe we'll get there later.) - We can diff the generated type for variations in adjunct config! (Probably not something that will end up tested, but neat to be able to do during dev.) Doing this with go plugins seemed like the neatest way to do this. It's certainly not the only way, though. And in particular, I will confess that this will probably make developing from a windows host pretty painful: go plugins aren't supported on windows. Mind, this doesn't mean you can't *use* codegen or its results on windows. It just means the tests won't work. So, someone doing development _on the codegen itself_ would have to wait for the CI server to run the tests on their behalf. Hopefully this is survivable. (There's also a fun wee wiggle in that loading a plugin has the requirement that it be built with the same "runtime". The definition of "runtime" here happens to include whether or not things have been built in "race" mode. So, the '-race' flag disappears from our CI config file in this diff; otherwise, CI will get the (rather confusing) error "plugin was built with a different version of package runtime". This isn't really worrying to ditch, though. I'm... not even sure why the '-race' was in our CI script in the first place. Must've arrived via cargo cult; we don't _have_ any concurrency in this library.) An alternative way to go about all this would be to have the tests for gen invoke `go test` (rather than `go build` in plugin mode) on each of the generated packages. It strikes me as similar but worse. We still have to invoke the go tools from inside the test; we'd have *more* work to do to either copy tests into the gen'd package or else generate calls back to the parent package for test functions (which still have to be written against interfaces, so that they can compile even when the gen isn't done, as it indeed isn't when you freshly check out the repo -- exact same as with the plugin approach); and in exchange for the extra work, we get markedly worse output ('go test' doesn't nest nicely, afaict), and we can't separate the compiling of the generated code from the evaluation of tests on it, and we'd have no possibility of passing information via closures should we wish to develop table-driven tests where this would be useful. tl;dr: Highest cost, uglier, and worse. No matter which way we go about this, there *is* a speed trade-off. Invoking the compiler per package adds at least a half second of time for each package, in practice. Worth it, though. And on the off chance that this plugin approach does burn later, and we do want to switch to child process 'go test' invocations... the good news is: we shouldn't actually have to rewrite very much. The everything-starts-from-NodeStyle-and-tests-against-Node work is exactly the same for making the plugin approach work, and will trivially pivot to working fine in for child 'go test' approaches, should we find it necessary to do so in the future. So! With our butts covered: a plugin'ing we shall go! Some of the code here still needs cleanup; this is a proof of concept checkpointing commit. (The real thing probably won't have such function names as "TestFancier".) But, we do get to see here: plugins work; more than one in the process works; and they work even when the same type names are in the generated packages. All good.
Showing
schema/gen/go/fresh_test.go
0 → 100644
Please register or sign in to comment