Introduction to Metaprogramming in Nim
101 points by vbit 9 years ago | 11 comments- bluejekyll 9 years agoI like the macros definitions. Those are very elegant. I use macros in Rust, but I don't find them very intuitive, or easy to get right. They are ridiculously powerful, but it feels like learning a new language, like there are three languages in Rust, safe, unsafe, and macros (don't get me wrong I love Rust, and enjoy it the more I use it).
Out of curiosity, could someone who's written in both languages, give their opinion on the differences between the two macro systems?
- pathsjs 9 years agoOther examples of things one can do with metaprogramming: async/await http://nim-lang.org/docs/asyncdispatch.html#async , pattern matching http://andreaferretti.github.io/patty/ , for/do comprehensions https://github.com/vegansk/nimfp/blob/master/tests/fp/test_f...
- jboy 9 years ago... define decorator-like Nim pragmas that generate Python C-API boilerplate code for decorated Nim procs, to auto-generate type-safe, exception-safe, refcount-safe Python bindings for Nim? https://github.com/jboy/nim-pymod :)
- pathsjs 9 years agoYes, I also wanted to add your library, but I was not sure how it works and to what degree it uses metaprogramming. But I am glad that you mentioned it! :-)
- pathsjs 9 years ago
- jboy 9 years ago
- jboy 9 years agoFor anyone unfamiliar with Nim, I'll give a quick summary of Nim generics, templates & macros, in terms of C, C++, Java & Lisp:
* Nim generics = parametrization-by-type of procs (functions) or other type definitions. Happens at compile time. Like Java generics or C++ templates, except that it uses [ ] rather than < >.
* Nim templates = direct textual substitution of code at the call-site at compile-time, like the C preprocessor, except that: 1. It operates upon a parsed Nim AST rather than plain-text code; 2. It's (by default) hygenic; and 3. The syntax is the same as regular Nim language syntax (in contrast to the crippled C preprocessor syntax).
* Nim macros = compile-time evaluation of code to perform side-effects, one of which may be inserting new code at the call-site. Nim macros are most like Lisp macros. An invoked Nim macro receives a parsed Nim AST as a tree data-structure, and is able to traverse & manipulate that AST, or create & output a new AST. When evaluating macros, the Nim compiler runs the macro code in a compile-time Nim interpreter, so macros can invoke any other functions, allocate data-structures, etc. And again, the syntax is the same as regular Nim language syntax.
[There's another Nim language feature that I really like, which I think is worth mentioning here: the `const` keyword, to define constants. Nim provides `var` to define variables that are read-write storage boxes, and `let` for single-assignment storage boxes. `const` is like `let`, except it's evaluated at compile time. This means you can evaluate arbitrarily-complicated expressions (including function calls) at compile time, obtaining the result as a constant of the appropriate result type, which can then be inlined at all usage-sites -- just as if you'd entered the literal value directly in your code.]
With this background in place, I can finally get to my main point:
When I'm getting excited about Nim to friends, I tell them that I think Nim macros are the best tradeoff between an expressive Python-like syntax & powerful AST-based, Lisp-like macros.
You see, no-one would dispute that it is very elegant to use regular function syntax to operate upon homoiconic code as a data-structure. And no-one would dispute that operating upon a pre-parsed AST is superior to crude text-concatenation (like in the C preprocessor). But as a commenter on HN pointed out in a recent thread about Lisp:
"""So, the real question is why did such a magical language lose to the upstarts that all appeared in the late 80's and early 90's: Perl, Python, Tcl, Lua, etc. Answer: files, strings, and hash tables. All of those languages make mangling text pathetically easy. Perl is obviously the poster-child for one-liners, but all of those make that task pretty darn easy. Lisp makes those tasks really annoying. Just take a look at the Common Lisp Cookbook for strings: http://cl-cookbook.sourceforge.net/strings.html""" -- https://news.ycombinator.com/item?id=11700176
Dense language syntax is beneficial because it enables brevity for frequently-occurring operations. For example: string indexing, slicing & especially regex matching, if you do a lot of text-processing; inline arithmetic operators if you do a lot of arithmetic; and array operators, if you do a lot of matrix processing.
Nim's macros combine a dense (Python-like) language syntax with powerful AST-based macros, enabling you to traverse & manipulate the AST just like any other data-structure.
- justinlivi 9 years agoAs someone who's never coded in either go or nim, this seems to me to be the exact opposite ideology of golang. The metaprogramming is very cool, but I imagine sharing a code base that utilizes it heavily is a nightmare.
- lqdc13 9 years agoIt can be or it can make it easier.
For example, decorators and context managers can do very similar things but make things easier in many cases in Python.
- honewatson 9 years agoThe difference is you or your team have the option to use the magic in moderation vs no magic at all.
- adrianratnapala 9 years agoI think only time can tell if that sort of argument works. Your team's habits matter, but so does the rest of the ecosystem. Strangers with libraries too central to ignore might end up forcing inscruitable templates and macros on you.
As far as I can see, the Python community has done a pretty good job of using magic sensibly. The Ruby and C++ communities, not so much.
- honewatson 9 years agoYes time will tell. Its basically like giving someone Jedi powers with the risk they could fall to the dark side while Golang is like Han Solo just getting things done in a much less sophisticated yet practical manner.
- honewatson 9 years ago
- sesteel 9 years agoTo be fair, "magic in moderation" is subjective. It may seem to be moderate until you have a new dev that is struggling to spin up on the codebase.
- adrianratnapala 9 years ago
- lqdc13 9 years ago