Silkenweb Example: Hackernews Clone

Onyx, a new programming language powered by WebAssembly

197 points by bkolobara 1 year ago | 86 comments

crabmusket 1 year ago
I would really like if the language reference[1] had a "rationale" or "principles" section describing the decisions at a high level. At the moment it dives straight into the minutiae of syntax, built-in types, etc.
The language marketing insists it is "data-oriented", even in the footer of every page. But I couldn't find any more detail on this.
My favourite example of this is Austral, which has a detailed rationale[2] that justifies the choices made and the tradeoffs. It really helped me decide whether I was interested in the language.
Odin, cited in the blog post, has a shallow rationale on its FAQ page[3], but seems to have had more time/maturity to document specific "data-oriented" features like built-in SoA[4].
[1] https://docs.onyxlang.io/book/Overview.html
[2] https://austral-lang.org/spec/spec.html#rationale
[3] https://odin-lang.org/docs/faq/
[4] https://odin-lang.org/docs/overview/#soa-data-types
- brendanfh 1 year ago
  Thanks for the feedback! I agree my documentation is very lacking in that department. I hope in the next few days to have a page in the documentation that describes my rationale, and the tradeoffs made during development.
- ReleaseCandidat 1 year ago
  Roc also is a great site on how to present a "new" language. Specially the sections that explain what they mean by being "fast", "friendly" and "functional".
  https://www.roc-lang.org/
- dwenzek 1 year ago
  Thanks for these links!
  Austral rational documentation is indeed really interesting. My preference goes into another direction (notably toward expression-based languages), but it's refreshing to read well explained design decisions.
  The section about linear types is also a gem to this regard. I would even consider this as a reference source to introduce the motivations and principles behind the Rust type system and borrow checker. Only the syntax would have to be made a bit more rusty.
  [1] https://austral-lang.org/spec/spec.html#rationale-linear-typ...
  - mst 1 year ago
    > For many people, semicolons represent the distinction between an old and crusty language and a modern one, in which case the semicolon serves a function similar to the use of Comic Sans by the OpenBSD project.
    That's a delightful sentence, and in context of how the design makes opinionated choices based on its goals entirely reasonable.
    (I don't know if I'll agree with all of those choices after I've thought about them sufficiently to be comfortable having an opinion, but I do very much appreciate that they're explicit, deliberate, and explained)
  - crabmusket 1 year ago
    Same with the section on capability security!
- DoesntMatter22 1 year ago
  To go along with that I really need a "Why should I use this" type of thing.
  Similar to what you are suggesting.
solarkraft 1 year ago
It seemed inevitable that a new language would choose WASM as a runtime.
We get a bytecode runtime to rival JVM and CLR, but with a diversity of stake holders (no single vendor trying to push a language or enterprise offering), it's JITed even on Apple devices and on the damn web! Pretty amazing.
What makes me excited about this: Getting to poke around, at runtime, my compiled .NET C# code using PowerShell (in Snoop) was pretty eye-opening to me. The "standard" interoperability between PowerShell (Core?) and .NET objects as well, of course.
It feels great to have a common runtime making calling out to other languages so seamless. Yet it seems to me like it never reached the popularity it deserved, but maybe it will get another try, with a lot more enthusiasm (no more Oracle/MS brake-foot).
- pjmlp 1 year ago
  Except the detail that it is mostly driven by Google, given Chrome's market share.
  - nolist_policy 1 year ago
    Wrong, Google drove NaCl for a long time (portable LLVM bytecode) but then Mozilla countered with asm.js which kickstarted and heavily influenced the development of Webassembly.
    - pjmlp 1 year ago
      Wrong, because you are only focusing on what lead to WebAssembly 1.0, almost a decade ago, not what happened afterwards the Web turned into ChromeOS.
IshKebab 1 year ago
Looks neat. Some obvious basic questions that I imagine most people would wonder:
1. Is it value based (C++, Rust, Go, TCL) or reference based (basically everything else).
2. How is memory freed? GC?
3. If GC how do you deal with the fact that Wasm GC is still in progress?
4. What about concurrency? Is it single threaded?
5. How does error handling work?
6. Does it support modern type features - Option<>, sum types, etc.
7. It's imperative, does that mean things that should be expressions (e.g. if/else) are statements?
- brendanfh 1 year ago
  Those are all great questions. I will add to the docs to explain more details later, but for now:
  1. It is value based, like C. There is obviously passing by pointer, but you do that explcitly.
  2/3. Memory is manually managed. To make this easier, like Zig and Odin, everything that allocates should take an allocator as a parameter. This allocator can be the general purpose heap allocator, or it can be an arena, ring buffer, or even tracked allocations that can be freed all at once. More details on that to follow in the documentation.
  4. It does have multi-threading support, thanks to the atomics WASM proposal. The standard library currently has support for basic multi-threaded primitives like mutexes, semaphores, and condition variables. Note, there is currently not async/await or any kind of concurrent execution.
  5/6. It does have modern type features like sum types. It has Optional(T) and Result(Ok, Err) out of the box. That is the preferred method of error handling.
  7. It is mostly statement oriented, but there are features like Elixir's pipe operator (|>) and if/else (value if condition else otherwise) expressions.
  - cutemonster 1 year ago
    About 2/3, does the compiler / language somehow help me remember to free memory I've allocated, or avoid double frees? Are there smart pointers or sth like that? (I've been coding lots of C/C++ :-))
- norir 1 year ago
  8. What does blazingly-fast build times mean quantitatively? How does it scale with project size?
  - brendanfh 1 year ago
    The largest project I have in Onyx is currently around 30000 lines (not all that large I know). That project on my Intel i7-1165G7 laptop compiles in about 80ms.
    There are currently no large Onyx projects that can really test how this number scales, but I would guess the growth is not linear. So for an off the cuff estimated, I would say a million line project could compile in about 4-5 seconds.
    Also worth noting that the entire compiler is currently single-threaded. So that number could get much better.
    - titzer 1 year ago
      That's impressively fast. Well done!
kemitchell 1 year ago
Clearly Wasmer wants to do WebAssembly. But for a whole new, general-purpose programming language, what's the case for that target, versus, say, LLVM intermediate representation?
- brendanfh 1 year ago
  Onyx is entirely independent from Wasmer. It is a separate project developed by me, over the past 3 years. It does however use Wasmer for running on Windows, MacOS, and optionally on Linux.
  I started making Onyx because I simply wanted to create my own programming language for the fun of it. I choose WebAssembly as the target for my compiler because it was relatively easy to write my own full-stack solution, instead of relying on something like LLVM. That language evolved over the years until I had something I felt was worth sharing. When the Wasmer team reached out to see if I wanted to do a collaboration blog post, I said yes.
  The main reason I didn't go with LLVM was simply because I wanted to write everything myself (for fun) and to have a super-fast compiler. LLVM is not the fastest backend, but it is understandable for all the optimizations that it does.
  - digdugdirk 1 year ago
    Very cool project! If I wanted to try it out in a repl based environment (maybe like a jupyter notebook?) where/how would you recommend I get started?
    - brendanfh 1 year ago
      Onyx does not really support a REPL environment at the moment because of the way it is compiled. Its execution cannot be paused in the same way as Python or JS. If you want to give it a try without committing to installing anything, I would recommend the Onyx Playground at https://try.onyxlang.io
soulbadguy 1 year ago
Well it's Friday and i have some time to kill. Before i start, creating a language is hard and i am just making random comment on the internet.
From the "why webassembly" section :
> When making a programming language, you need to decide how your programs will actually be executed.
In abstract maybe, and obviously some language features might make an interpreter or a compiler harder/easier to make. But in general and in practice the language semantic and the "execution mode" can be treated independently. Some of the languages mentioned even have multiple style of execution (AOT, JIT, interpretation , compilation etc...)
> Onyx does not rely on libraries like LLVM to generate its code, because WASM is so simple to target.
In term of correctness maybe. It will be interesting to see what happens as performance and more complex code pattern need to be supported. Most language evolve some type of high level representation between the AST and they target of choices for language specific transformation and error reporting (SIL,clangIR and HIR? not sure about that last one).
>My strategy was to wrap libwasmer.a, the standalone library version of Wasmer, into my own custom WASM loader, to allow imported functions to be linked against native libraries
Doesn't this both limit the portability and introduce potential security risk (and thus negating the whole point of wasm) ?
I think the inter reaction with the outside world is what WASI and the other is trying to address.
bnchrch 1 year ago
I'm in no way against a new language! Love programming languages, particularly ones that push boundaries and challenge assumptions!
(Great work on that front btw Onyx Team)
But! When I see powered by WebAssembly. I can't see a good reason to choosing a unique language when any language in theory can target WASM/WASI.
In other words Javascript/Python is looking at your lunch and is very hungry.
- orangea 1 year ago
  When I see "powered by WebAssembly", I take it as a sign that the language is going to be designed around WebAssembly's VM interface rather than the implicit standard of the C memory model and POSIX-like API which basically all popular programming languages are designed for. Even languages like Python have the concepts of stdin/stdout/stderr, a filesystem, threads, and a single globally accessible heap baked into their design. It would take a lot of work to amend Python's design to be able to easily use features that are somewhat unique to WebAssembly like the various things a .wasm file can import or export. As it stands, languages like Python can be run on WASM, but they use inelegant hacks in order to be able to replicate the execution environment features that their interpreters require such as malloc, and usually it is assumed that the target will be WASI. A language that is designed for WASM from the ground up would make it so that WASM can be used to its fullest potential as a lightweight execution environment that makes extremely minimal assumptions about what interfaces are going to be provided to the program.
  - brendanfh 1 year ago
    That is exactly the way that I see it too.
    While Onyx does have core library support for doing standard POSIX things like stdin/stdout/stderr, file operations, and networking, Onyx can be used in an entirely different environment without any of these things. In fact, the standard libraries are (almost) entirely separate from the compiler (i.e. the compiler makes no assumptions about what the standard library is), so if the standard library does not suite your use case for one reason or another, you can write your own. It will be a bit of work, but there would be nothing in your way.
    - soulbadguy 1 year ago
      > Onyx can be used in an entirely different environment without any of these things. In fact, the standard libraries are (almost) entirely separate from the compiler (i.e. the compiler makes no assumptions about what the standard library is), so if the standard library does not suite your use case for one reason or another, you can write your own. It will be a bit of work, but there would be nothing in your way.
      Isn't that true for most native languages as well ? C++ have multiple stdlibs.Rust,C and c++ can be used against bare metals.
      The orthogonality of the std lib and the compiler and language semantic (maybe at the exception of the types) seems pretty common these days.
  - soulbadguy 1 year ago
    I don't think i fully agree. Yes it might be possible to design a programming language that have a better synergy with wasm than the one we currently have, but IMO this won't have a particularly big impact on dev prod or performance. wasm was design to be the target of current languages, so in a sense it does capture a lot of things that already exists or can be mapped easily to existing languages. Designing against wasm might create a case of overfiting. But only the future will tell.
    - orangea 1 year ago
      Here is a specific example, again using Python, of how a programming language not designed for WASM might not be ideal: suppose you are writing a Python script which is to be compiled to WASM and needs to import a function. How do you specify that in the script? There could be some Python function like `wasm_import("my_function", "u32", ["u32"])`, which allows the programmer to specify a function to be imported... and the problem with that is that in Python everything normally happens at runtime, but the compiler which is turning your script into a .wasm file needs to know in advance what functions you are going to import. So you face a dilemma: either you mess with the fundamental assumption of Python that everything happens at runtime, or you add new syntax which allows specifying metadata, or you use magic comments, or do some other hacky thing.
      This might seem strange to those who are used to today's dominant WASM paradigm, where the programmer usually controls both the VM and the .wasm program — such as website developers who create a .wasm file and then also write the Javascript to load it into the browser. But one of the most exciting things about WASM, to me, is the potential for domain-specific execution environments in which the code which loads and executes the WASM is written much earlier and by a different entity than whoever writes the WASM.
- crabmusket 1 year ago
  Serious question, in what use cases do you see JavaScript especially (but also Python) competing in the "on top of WASM" space?
  Any time I see a language that compiles directly to WASM my instinct is that it will compete with Zig, Rust or C/C++/emscripten.
  - orangea 1 year ago
    In the "on top of WASM" case, I imagine that the advantage of Python and JS will be exactly the same as the "not on top of WASM case". Namely, mainly development convenience at the cost of performance.
    - crabmusket 1 year ago
      So it's scenarios where the deployment target has to be WASM, and these languages are chosen to be used on top of it. Like some of these new FaaS clouds which are targeting WASM-only deployments, or e.g. Fastly edge compute?
      That makes sense, the mention of JS got me thinking about browsers which is where I spend most of my time.
- gumballindie 1 year ago
  Add syntax to the list - it’s horrible. The target audience is more used to the basic syntax of javascript.
  - sakjur 1 year ago
    I quite like its syntax. I also don’t think the target audience is beginners, this seems like a passion project someone is making for themselves and people who have a similar taste to them.
    - gumballindie 1 year ago
      > passion project someone is making for themselves and people who have a similar taste to them.
      From that angle i quite like it. Personally I also like the syntax.
royjacobs 1 year ago
How optimized is the generated code? One of the benefits of using LLVM is the amazing set of optimization passes, after all. Will Onyx have to reimplement a lot of these optimizations?
- brendanfh 1 year ago
  The compiler does not have many optimizations currently implemented. Currently, the compilers just uses constant folding and generally reducing unneeded operations if possible. There are plans to do function inlining and better tree-shaking, but those are a ways out.
  The "nice" thing about WebAssembly is that the job of generating optimized machine-instructions is up to the WASM embedder, which Wasmer is doing very well right now. Onyx does have a second runtime that is currently only implemented for x86_64 Linux, that does have some more optimizations, but it does not compile to native code; just to an interpreted byte-code.
rishav_sharan 1 year ago
> Fast Compilation Onyx's compiler is written entirely in C and features incredibly fast compilation. The web-server for this website was compiled in 47 milliseconds.
Ok, that got my attention. Its refreshing to see a new language which doesn't takes 10 seconds to build a Hello world example.
Couple of questions;
1. The Windows installation is a bit tedious. Any plan to support Scoop for installing Onyx?
2. Given that gamedev seems to be a focus, with the raylib integration, how is the performance compared to something like c. Will a comparable onyx program get me to at least 50%+ of the performance of c? I haven't used webassembly much so I am curious how this works out.
I was just thinking of creating a new project to try out Raylib 5. So your post has perfect timing for me!
Best of luck to your language. Looks lovely in the online playground. Now going to set it up on my device
ReleaseCandidat 1 year ago
> To navigate these limitations I came up with a way for any WebAssembly embedder, like Wasmer, to interact with native system components. ... I added a custom section to the WASM binaries output by Onyx that specifies which native libraries it wants to link against.
So the main reasons for using WASM and all of it's advantages are lost. It's neither sandboxed nor self contained any more, but slower than using "native" compilation and still missing libraries (compared to JVM, CLR, Beam,...). Don't get mé wrong, I know about the advantages of WASM for the language implementor (although I'd use WASM GC), but as a user, but I don't know why I should use Onyx.
kettlecorn 1 year ago
Can Onyx's compiler be compiled to Wasm to itself run in the browser?
It's niche, but I've thought it'd be cool if there were a mini Wasm-based game building tool in the browser, but that requires a Wasm language with a compiler that can run in the browser.
- brendanfh 1 year ago
  Currently, it does not. But it is written in C with a reasonable separation between the OS things and the compiler things, so compiling with Emscripten should be possible without too much modification.
  This is similar to what the Onyx Playground currently does. To do that I just POST your project files to a server, save everything to disk, run the compiler, delete everything, and send the resulting WASM file back to the browser, to be run in a WebWorker.
pyrossh 1 year ago
Nice name analogous to ruby.
From wikipedia,
Onyx - A banded variety of chalcedony, a cryptocrystalline form of quartz.
Ruby - A clear, deep, red variety of corundum, valued as a precious stone.
- Alifatisk 1 year ago
  Is Crystal also analogous to ruby?
  - pyrossh 1 year ago
    Ahh yes. Totally forgot about crystal.
    Crystal - A solid whose atoms (or molecules) are arranged in a repeating pattern. Crystals are found naturally or can be made artificially.
    Someone should use Diamond for the next programming language, really like the wiki definition.
    Diamond - is a carbon crystal formed under heat and pressure.
pyrossh 1 year ago
I've been trying to build a programming language targeting only wasm but never seem to get the time to work on it.
Like the syntax, mainly the pipe operator |>
A few questions,
1. Any plans to use binaryen to optimize the wasm output?
2. How is memory management handled? Are you going to use wasm-gc?
3. How does the C FFI work? Do you convert the WASM types to C types?
bobajeff 1 year ago
> When linking, I find the libraries on disk, load them, call a function in each dynamic library that returns a list of procedures that can be used when linking. This external linking enables Onyx to use any native C library, from Raylib and OpenGL, to PostgresQL and OpenSSL.
This is really cool. I wonder if there is any overhead using native libraries from wasm like this. My intuition says it does similar to node-ffi and JNI but I hope it's not as bad.
- brendanfh 1 year ago
  I haven't directly measured any throughput or latency numbers. I know it is relatively fast, but I know it is not as seamless as I would like.
  Currently, for WASM to call any imported function, its arguments have to be formatted in a particular way that is specified by the wasm-c-api. Then it can invoke the imported function. Only after that can the arguments be translated back into the native calling convention in order to call the real native function. I don't know how that compares to node-ffi and JNI, but I imagine its not as slow.
  I have a piece of documentation I need to write that will explain how calling native libraries works from Onyx, so that should be able to go into more detail.
spuz 1 year ago
I am getting network errors loading some paths on the onyxlang.io website. That's not a good advertisement for Wasmer.
- syrusakbary 1 year ago
  Oh no! Do you mind providing the paths where the errors occurs or the requests ids returned by Wasmer, so we can investigate further?
  Thanks!
  - spuz 1 year ago
    I think the requests that were failing were pretty random actually but I didn't get any response from the following. After a few minutes the request would timeout with an error NS_ERROR_NET_HTTP2_SENT_GOAWAY in shown Firefox.
    https://onyxlang.io/
    https://onyxlang.io/playground
    The second one is a 404 but I wasn't even seeing that earlier (this is linked from the first page of the docs).
    Currently, I don't have any issues but the problems I was having lasted at least a few hours which is why I mentioned it.
    - syrusakbary 1 year ago
      Thanks a lot for the info, I'll relay to the team to investigate further
WhereIsTheTruth 1 year ago
Does it support hotreload?
To me that's the piece that's missing in all these new languages
Being able to hot patch code saves you from having to recompile everything and reload the state of your program, saves lot of time
I was excited when Zig announced planning it, but it's been like 4 years now and they went radio silence unfortunatly
- brendanfh 1 year ago
  That is a planned feature because I really want that too! I believe it should be possible and not take years to develop, especially because Onyx does have its own WASM runtime that I can completely control how and when certain things happen, like pausing for a hot reload. This runtime is currently Linux only, but I should be able to address that relatively soon. I don't know if it would be possible with Wasmer as the runtime, without Wasmer making severe modifications to its code generation and runtime. Hot reload is a tricky problem, but I believe for the developer experience, it is worth trying.
insanitybit 1 year ago
How did you find the experience of compiling to wasm? Do you build an IR first at all or just ast -> wasm?
- brendanfh 1 year ago
  I found it relatively easy. I planned a lot of things out of whiteboards and notebooks, but the actual code writing was mostly straight forward. For Onyx, I went straight from AST to WASM. The trickiest part was probably deciding how non-primitive things would be represented in WASM, like structures and tagged unions. But all of that came later after I had the basics down. I started with the simplest little compiler that could only compile binary operations on integers, and basic function calls. Then I expanded the compiler outward once I had the underlying data structures and code paths figured out.
adiabatty 1 year ago
Since the guy who makes this is in the thread and I'm not the first to say stuff about the website…
Seems cool but I was trying to read one of the examples on <https://onyxlang.io/> and the carousel changed pages, throwing me off.
Would you consider not using a(n auto-advancing) carousel with code examples?
Not unrelated: <https://shouldiuseacarousel.com> (short answer: "no")
- brendanfh 1 year ago
  I totally agree the auto-advancing carousel was a bad idea. I wanted to add something with movement to the page, but that was definitely not the right idea. The carousel has been removed!
- 1 year ago
CyberDildonics 1 year ago
I'm not sure a specific compilation target that is meant to be as simple and generic as possible makes a new language a good idea.
New languages mean you start all over and wipe away decades of tools, knowledge, libraries, development environments, package managers, workflows and now people are in the wild west again. To justify that you have to have some enormous benefits.
ipsum2 1 year ago
Not to be confused with Onnx, a popular language for machine learning models, that is also commonly used with WebAssembly. Even the logo looks similar.
https://onnx.ai/
https://cloudblogs.microsoft.com/opensource/2021/09/02/onnx-...
- FireInsight 1 year ago
  Not to be confused with Fedora Onyx, the budgie version of atomic Fedora.
  https://fedoraproject.org/onyx/
  - ketralnis 1 year ago
    Not to be confused with Onix https://bulbapedia.bulbagarden.net/wiki/Onix_(Pokémon)
  - tripdout 1 year ago
    Not to be confused with Onyx, the son of rappers Playboi Carti and Iggy Azalea.
  - cutemonster 1 year ago
    Not to be confused with Onyx the gemstone. Or the other way around
  - mike_hock 1 year ago
    Not to be confused with Onyx the clothing store.
  - JoshStrobl 1 year ago
    Funny to see Onyx be mentioned in the wild! :D
- egberts1 1 year ago
  Not to be confused with Onex Hacking Tool.
  https://github.com/MasterScott/onex
- 1 year ago
koube 1 year ago
nit: seamless not seemless.
Actually TIL seemless is actually a word but it's a synonym for unseemly.
- 1 year ago
ivanjermakov 1 year ago
I have irrational anger towards := assignment operator, probably because of Pascal course in high school.
Interesting if anyone else have such stories regarding language syntax and features.
- cgh 1 year ago
  In Go, when you export function names they are capitalised. Eg MyFunc() would be exported and myFunc() is not. This always looks so jarring to me and I’ve avoided Go largely because of it, which must rank as one of the more snowflake reasons to avoid a programming language.
  - ivanjermakov 1 year ago
    Yes, I hate it. And in general, capitalized function names break mental model of inferring token kind by its name. In my world, capitalized token is always a type, lowercase is a variable (or property, function, etc).
    Popular counter-argument is that it can be fixed with syntax highlighting, but for me it's a flaw in the language syntax.
  - mst 1 year ago
    For brave service in the forces of self-awareness, I hereby award you https://trout.me.uk/medal.jpg
    May you wear it with pride.
- CooCooCaCha 1 year ago
  It bugs me too. Mainly because I don't see anything wrong with "=", plus it's shorter and more intuitive.
  - brendanfh 1 year ago
    The example in the blog post could have been written a little bit better, but in Onyx ":=" is actually a combination of two operations: defining a new variable with ':', and assigning the value with '='. If the type can be inferred on the right hand-side of the assignment, you can leave out the type between the ':' and the '=' and it effectively becomes a single operator. An alternative way to write that could have been 'input: str = "111 110 121 120";'.
    To assign in Onyx, you only have to use '='.
  - xigoi 1 year ago
    As a mathematician, I see everything wrong with using “=” for something other than equality. Especially when writing something like x = x + 1.
- topikk 1 year ago
  I also have this irrational anger, but as a result of PL/SQL. It's also an objectively hideous operator that is inconvenient to type.
- solarkraft 1 year ago
  I have an irrational (?) hate for having to declare variables at all because it constantly hassles me while testing and copying/modifying code.
- acheong08 1 year ago
  Coming from Go, I got used to :=. I would’ve preferred if it defaulted to const though
  - cutemonster 1 year ago
    Coming from Pascal, I like := :-)
- 1 year ago
onyxlangfanboy 1 year ago
[flagged]