Given the amount of testing we (and other parties) have done, and given the strong theoretical foundation of our algorithm I’m pretty confident we’d do well in many production environments. If you do find any performance issues though, we’d love to hear about them!

Hosts should then be synced using PTP or a proper NTP local stratum (just get a proper GNSS source for each DC if you have then funds).

https://tsn.readthedocs.io/timesync.html

Deploy chrony to bare metal servers wherever possible.

(a) the inherent risks in implementing a protocol based on trivially spoofable UDP that can be used to do amplification and reflection

and

(b) emergent resonant behavior from your implementation that will inadvertently DDOS critical infrastructure when all 100m installed copies of your daemon decide to send a packet to NIST in the same microsecond.

I'm happy to see more ntpd implementations but always a little worried.

NTP is worth moving to a memory safe language but of course it's not the single most critical thing in our entire stack to make memory safe. I don't think anyone is claiming that. It's simply the first component that got to production status, a good place to start.

NTP is a component worth moving to a memory safe language because it's a widely used critical service on a network boundary. A quick Google for NTP vulnerabilities will show you that there are plenty of memory safety vulnerabilities lurking in C NTP implementations:

https://www.cvedetails.com/vulnerability-list/vendor_id-2153...

Some of these are severe, some aren't. It's only a matter of time though until another severe one pops up.

I don't think any critical service on a network boundary should be written in C/C++, we know too much at this point to think that's a good idea. It will take a while to change that across the board though.

If I had to pick the most important thing in the context of Let's Encrypt to move to a memory safe language it would be DNS. We have been investing heavily in Hickory DNS but it's not ready for production at Let's Encrypt yet (our usage of DNS is a bit more complex than the average use case).

https://github.com/hickory-dns/hickory-dns

Work is proceeding at a rapid pace and I expect Hickory DNS to be deployed at Let's Encrypt in 2025.

Most of the security bugs we hear about don't cause random crashes on otherwise healthy machines, because that tends to get them noticed and fixed. It's the ones that require complicated steps to trigger that are really scary. When I look at NTP, I see a service that:

- runs as root

- talks to the network

- doesn't usually authenticate its traffic

- uses a bespoke binary packet format

- almost all network security depends on (for checking cert expiration)

That looks to me like an excellent candidate for a memory-safe reimplementation.

I'll flip the question around, why not start at ntpd?

Of course, there are many other services that could be made memory safe, and maybe there is some sort of right or smart order in which we should make our core network infrastructure memory safe. But everyone has their own priorities here, and I feel like this could end up being an endless debate of whatabout-ism. There is no right place to start, other than to just start.

Aside from memory safety though, I feel like our implementation has a strong focus on security in general. We try and make choices that make our implementation more robust than what was out there previously. Aside from that, I think the NTP space has had an under supply of implementations, with there only being a few major open source implementations (like ntpd, ntpsec and chrony). Meanwhile, NTP is one of those pieces of technology at the core of many of the things we do on the modern internet. Knowing the current time is one of these things you just need in order to trust many of the things we take for granted (without knowledge of the current time, your TLS connection could never be trusted). I think NTP definitely deserves this attention and could use a bunch more attention.

The answer is that there is observability functionality which provides its results as JSON output via a UNIX socket[0]. As far as I can see, there's no other JSON functionality anywhere else in the code, so this is just to allow for easily querying (and parsing) the daemon's internal state.

(I'm not convinced that JSON is the way to go here, but that's the answer to the question)

[0] https://docs.ntpd-rs.pendulum-project.org/development/code-s...

https://packages.debian.org/bookworm/net/ntp

https://www.vox.com/2015/5/23/8647095/kool-aid-jonestown-fla...

They really do appear to be all in on avoiding memory leaks from C/CPP:

> Over the next few years we plan to continue replacing C or C++ software with memory safe alternatives in the Let’s Encrypt infrastructure: OpenSSL and its derivatives with Rustls, our DNS software with Hickory, Nginx with River, and sudo with sudo-rs. Memory safety is just part of the overall security equation, but it’s an important part and we’re glad to be able to make these improvements.

It seems like a really challenging endeavor, but I appreciate their desire to maintain uptime and a public service like they do.

Rust is really good for correctness. Take "Hello, World", the obvious toy program. Someone tried giving it various error states instead of (as would be usual) a normal happy terminal environment. In C or C++ the canonical "Hello, World" program terminates successfully despite any amount of errors, it just doesn't care about correctness.

The default Rust Hello World, the one you get out of the box when you make a new project, or you'd show people on a "My First Rust Program" course, will complain about the errors when they happen. Because doing so is correct.

It's the New Jersey style. The priority for these languages was simplicity of implementation. It's more important that you can cobble together a C compiler easily than that the results are useful or worthwhile. This contributed to C's survival, but we pay the price until we give it up.

TFA's second sentence explains the facts of the matter along with the flavor of Kool-Aid they stock:

> The CA software itself is written in memory safe Golang, but from our server operating systems to our network equipment, lack of memory safety routinely leads to vulnerabilities that need patching.

Because it is a fact. An obvious, obvious fact to anyone who has been working in this ecosystem for any amount of time.

> Drinking the Rust kool-aid by the sound of it.

Would you say the same thing if they'd instead decided to use golang, zig, nim, ocaml, etc.? If not, maybe consider that your emotional stance around Rust is coloring your judgment.

Our modern systems are built on a house of cards where security is concerned. I agree that the "RiiR" meme is tiresome and dumb, but I'm tired of seeing report after report of new vulnerabilities found in foundational libraries and programs. The majority of those vulnerabilities are of the type that Rust won't even let you compile. Languages like C and C++ have their place, but for most applications, there are safer alternatives that don't require harsh compromises or significant trade offs.

True, there are plenty of alternatives for many of those networking services, not necessarily Rust.

However I don't think the pace of updates really changes anything in terms of tool chain security. If Rust decided to go to a 36 week release cycle, each release would just have 6x as much stuff in it. If you can't keep up reviewing N changes in a 6 week release cycle, moving to a 6*X release cycle will not help you review N*X changes.