They could have just use any old less powerful chip, like, whatever has at least 3B performance is fine, and used the money saved for stuff we actually need.

Maybe ditch those awful micro HDMIs, it's kind of a nasty connector and USB C exists.

Onboard lithium charging is a few cents. Better yet, onboard LTO charging, but it seems nobody does that yet. Why just stop at an RTC battery when you could have a few minutes of real backup power to help stop SD corruption?

While you're at it, add an onboard I2C FRAM or battery backed SRAM.

Instead of NVMe why not hire a few devs to go around fixing stuff that can't run well from the SD card?

But I agree that the price to get a happy Pi is pretty harsh compared to a tiny PC.

I find it hard to believe that it can be worse than RPi 3, which could try to draw up to 15W => 3A under the highest loads, while USB 2 could officially only go up to 2.5 W = 0.5A at 5V, resulting in countless corrupted SD cards. (15 W being the minimum maximum for USB C.)

The tooling for light software development and hobby embedded hardware development has been great so far.

True, but its voltage can be used as an estimate. Most PC MB still around monitor the supply voltages, some also of the battery feeding the RTC. If you have the (Debian) package 'lm-sensors' installed, look for

    sensors[<pid>]: Vbat:           +2.91 V  (min =  +2.70 V, max =  +3.63 V)  ALARM

You should do it every year or two...

I don't follow! How does trusting DHCP with IP address automatically mean I should trust it with the current UTC time?

Time requires higher degree of trust than IP.

I may not care what my IP address looks like but I might care a lot about what the current UTC time is and I might want this to come from a more trustworthy device.

DHCP can provide the IP address of one or more NTP servers to consult for time queries. I use this feature so that all of the machines on my LAN (at least the ones that can be configured to ask for and/or use this information) have the same general idea of what time it is.

[0] See subsection 8.3 here: <https://www.rfc-editor.org/rfc/rfc2132#section-8>

I don’t know how widely this DHCP RFC is supported though.

Anyway, I solved these problems with sbts-aru, but maybe it's interesting for you to test if you haven't already done this.

PPS turned up quickly on Wikipedia, so that one is readily answered

> PPS signals have an accuracy ranging from 12 picoseconds to a few microseconds per second, or 2.0 nanoseconds to a few milliseconds per day based on the resolution and accuracy of the device generating the signal.

https://github.com/semuconsulting/PyGPSClient

I've always been amazed that the Foundation never mandated the use of validated brands or types of SD cards.

Not that it matters much to me, I've had to retire all of my Pis earlier than the 4. It seems that all of my Raspberry Pis prior to the 4 all had some sort of design issue with their power regulation. After a couple years of constant operation, they just start locking up randomly around once every day or so. I've had everything from a Pi 1 through 3 exhibit this behavior. I spent WEEKS troubleshooting this and conclusively ruled out wonky SD cards, power supplies, connected devices, proximity to other equipment, and every other factor I can think of. I found many threads with other people reporting the same issues but the Foundation refuses to acknowledge it as a problem. I only run Pi 4's now, and am gradually switching over to Intel N100-based systems for low-power operation.

Is my clock more likely to be accurate after NTP.org got signed? It looks like it's less likely, on average.

If memory serves though, Raspbian used to not even have `fake-hwclock` by default and even more Pis would end up with this "wildly wrong time near the epoch and can't bootstrap DNS/NTP" failure mode. You'd sometimes also see it with VMs too (esp if they were doing a pure-software clock that ran slower than realtime, instead of patching clock calls to the real hardware clock on the host).

I use an external RTC with a Raspberry Pi for some sensor systems I work with. Its purpose is to give a reasonable chance of having the correct time for recording data in remote locations where no internet connection is available at startup. For a Pi that only provides network services (and thus always needs a working internet connection), I probably wouldn't think to bother.

That was actually the easier hardware platform, because it either had good NTP sync or it didn't reach our C&C servers.

The difficult one was the platform that had an RTC. Those would slooowly drift out of sync if NTP was broken. Slowly and silently.

Why all the complex solutions when it's so easy and cheap now to install a stratum #1 time server, even a Pi 3 will suffice. It works on a Pi Zero, but then you only have a wifi link.

The sbts-au project make's it incredibly easy to setup. git lone and then install. One command, it's that simple.

Then the real solution might involve ensuring good clock sync. Using a GPS synched time source is good for this, such as this one :) https://github.com/hcfman/sbts-aru

— Restoring more sane shutdown timestamp from file instead of just using 1980.

— Making blind single use ntpdate request to some public server to set local time after the network is available, but before any other application uses it. There might be appropriate hooks in your init system if you don't want to make it an independent step.

— The same, but using some local time source. It is not widely known that even plain old Windows desktops, while not having a complete NTP server, can be trivially configured to announce time over some form of SNTP. If you have any computer with a real clock running nearby, you can probably rely on it. Next step router is also a good potential candidate.

As for compartments and access control, you can try to use basic VLANs (one for ntp traffic, and only it, and one for VPN) or maybe some two-step network configuration process (get time with one configuration, reset and proceed with other services).

All of that is not a problem, the problem is what to do when something fails. Time server becomes unavailable, intermittent internet connectivity, server you use for pings is available, but others are not, etc. You can choose to stop completely if there is no time source, and wait, but that might render the system unavailable for remote configuration. You can try to restart the device or restart the network router automatically (though only a fixed number of times, restart loop will surely break something), but that won't help if the problem gets fixed some time later. You can also use a fallback configuration that doesn't depend on correct time (e. g. only start an ssh server so you can connect to it).

Do you have something on the LAN side that could run an NTP server? I'm thinking you could add your laptop to your Pi's list of NTP servers, then just start that service up temporarily as-needed to give the Pi a time that's good enough, then it can get actual correct time sync from somewhere else once Wireguard is working.

I’m actually setting up a router with a rpi4 right now, only using Fedora IoT instead of OpenWrt. It’s a bit more assembly required lol

Multi-booting PCs with Windows and Linux over the years, I have seen the time sync problem go from nonexistent to show-stopper.

For devices that you only need to connect to the internet occasionally or sporadically (so that's what you do), that's where I noticed it most.

Linux sets the RTC to UTC, then when you reboot to Windows, Windows uses the RTC as local time. Which for me is 5 or 6 hours different from UTC, depending on Daylight Savings Time.

Plus when Daylight Savings comes around, each OS wants to make a 1 hour correction the first time you boot it after that date. With multi-booting this can add up to more than one hour difference too.

Didn't used to be so bad, if you were a few hours (days or longer for many PCs) away from the actual time, things did not fail for this reason.

Eventually only one hour off was OK for a while, now nope.

There are just so many more obstacles to smooth reliable operation, and weak links in a more extensive chain that must remain perfectly strong. Otherwise the chain is broken, you are disconnected, and the weak link is too many sections away from you now to be within reach.

Complete perfection is required more so than ever, while at the same time, your efforts to approach perfection are being made more difficult.

Unless someone is actually aiming to do a time coordinated task with others, you really need to know the time, or even the day-of-week to do stuff, because most human things in world aren’t actually occurring in small time sensitive windows.

While systemd-timesyncd contains a workaround for this, chrony does not seem to.

Plus the time to learn a few modprobe commands. Perhaps run a few commands echoing numbers to random sysfs files. Perhaps learn WTF a 'Device Tree Overlay' is - a concept that's completely absent on x86 systems. You see, this is an 'I2C' device, so it doesn't plug and play like USB and PCIe devices do. It is much simpler, and hence for some reason more difficult. Perhaps your device tree overlay needs to be complied? Or maybe not. Perhaps you'll edit your modules file and your kernel command line. Of course that won't involve update-grub, why would you think that?

Don't forget to periodically check on your settings, just in case an OS update or something has disabled one of those settings.

Oh and of course you won't be able to fully test this over SSH, if the network is up NTP will have set your clock whether the RTC is set up right or not. As we've learned from the article, if you run into clock problems you might not be able to connect remotely.

Then simply invent a time machine so you can do all of this in the past, before the pi SD card fails and it drops off the network.

(And you can do the same thing were you write the time to the RTC before you have the correct time)