Ask HN: If everyone at your company vanished how long would the lights stay on?

13 points by BoGoToTo 2 years ago | 30 comments
First: No, I do not work at Twitter. Second: I think about his question from time to time and wonder what it is like elsewhere.
  • tjohns 2 years ago
    My building uses motion-controlled lights. I think they switch themselves off after about 15 minutes of no motion... so about 15 minutes.
    • vba616 2 years ago
      Lights? Where we're going we don't need lights...

      https://en.wikipedia.org/wiki/Lights_out_(manufacturing)

      • BuildTheRobots 2 years ago
        > FANUC, a Japanese robotics company, has been operating as a lights-out factory since 2001. Robots are building other robots at a rate of about 50 per 24-hour shift and can run unsupervised for as long as 30 days at a time. "Not only is it lights-out," says Fanuc vice president Gary Zywiol, "we turn off the air conditioning and heat too."

        Enslave them, treat them badly and leave them alone for weeks at a time with the express intent of increasing their own numbers. What could possibly go wrong.

        • fragmede 2 years ago
          Meanwhile, the temperature controls inside a data center are ridiculously extreme compared to the heating/cooling system in use at my house. Those computers' environment is taken care of way better than the inside of my apartment is.
          • tomcam 2 years ago
            Username checks out
          • Riverheart 2 years ago
            And now I want to watch Event Horizon
            • vba616 2 years ago
              Hmm...isn't it close to a quote from Back to the Future? Is there any connection between the movies?
              • qbrass 2 years ago
                Event Horizon had a similar quote to the BttF one. "Where we're going, we won't need eyes to see."
          • LinuxBender 2 years ago
            I would add to that question: If every datacenter was power-cycled at the same time and employees were not allowed to log in, how long until customers could use your site? i.e. The Ultimate Chaos Monkey Test
            • nickm12 2 years ago
              I work at Amazon. The internet would be in trouble. I know very little about the fulfillment side of things, but I would expect things to stop there pretty much immediately. There is of course automation, robots, etc. but also many people in the loop and once they vanish I'd expect things to enter some kind of passive failure mode. Hopefully these failures would be isolated from the rest of the services.

              The more interesting question is how long AWS and retail services would continue running and how they would fail. I don't think anyone could say. All people worldwide vanishing in an instant isn't a realistic scenario to plan for—if it happens, then the world has bigger problems and in any case, who would be around to execute the recovery plan?

              Still for the sake of the thought experiment...I'm assuming in this scenario the world sees fit to keep supplying the data centers with power and other utilities, but no one has access to the buildings, tries to get in, bomb them etc. In this case, I would expect things to fail gradually, then suddenly and catastrophically, in the period of a few days to a few months. Failure should happen mostly independently by availability zone.

              • nicbou 2 years ago
                I'm the only one here, and it's a simple content website.

                The domain has just been renewed for 5 years. The VPS is paid by credit card, and is set to auto-renew. This should last until the card expires in early 2027. There's money in that bank account so it should be fine.

                Then Cloudflare will serve the website for a bit longer, probably a month or two.

                After that, only archive.org copies of the website will remain, but at least the pages are self-contained (by design) and will run just fine when served from an archive.

                So until about 2027.

                There would be issues though. Information would grow outdated, links would break, mailboxes would get filled, etc.

                The tax office would probably cause problems if I stop doing bookkeeping though. My tax advisor could delay their wrath, but eventually they'd freeze my account and payments to the web host would stop. I give it a few months, tops.

                • postultimate 2 years ago
                  It's a radioluminescent paint company, so about six and a half thousand years.
                  • yuppie_scum 2 years ago
                    Answering from the cloud perspective.

                    While I have seen some cloud infrastructures running on life support with minimal supervision for years, AWS tends to shut down, deprecate or migrate backing services every now and then.

                    ASGs stop autoscaling. RDS forces version upgrades. EBS volumes go poof. Kube APIs vanish. Upstream AMIs, docker images or apt packages get moved to different repo URLs. Sooner or later some auto upgrade or scaling group would go awry and take out a persistent data store or cause services to fail.

                    There’s also the possibility for an unpatched vulnerability to get exploited.

                    I give it about 3-6 months, depending on the architecture pattern.

                    • rospaya 2 years ago
                      > AWS tends to shut down, deprecate or migrate backing services every now and then

                      Fairly rarely. For Azure that might be a week, not months.

                      • happyrock 2 years ago
                        This. There is constant maintenance work needed to keep things online even in a steady product state.
                      • jelavich 2 years ago
                        Servers run until vendor contracts get terminated.

                        But actual lights? In shared office bldgs it actually depends on the renter's requests to the cleaning service. I've worked many places where the crew would turn the light on/off when entering/leaving (though would be shocked seeing me working in the dark); but I've worked a place or two where they requested all lights stay on all night, because "when mgmt shows up early in the morning they can't find the light switches next to the door" (made up quote but you get it)

                        • nitwit005 2 years ago
                          Our users are all internal, so I expect it'd last a long while given no inbound requests.

                          The data sources we use do change format from time to time, so I assume one of those would break eventually.

                          • jhoelzel 2 years ago
                            Well I pride myself by creating self healing, easy to operate kubernetes clusters. Soooo as long as the hoster provides me VM's or Nodes and would not kick me out because of the invoice, i recon until a critical system update is needed.

                            So maybe 2-5 years? :D

                            with k3s the cluster iself can autoupdate and with the right settings it can update the system too.

                            PS: running does not mean its secure, just means that it wont die right away :D

                            • whateveracct 2 years ago
                              Something goes wrong on business day 1 without anybody to respond to customer issues and touch the database.
                              • lolc 2 years ago
                                I'd expect between a week and a month of operation before one of our central systems clogs. After that, many services would still respond to requests but provide stale data. Further two months I'd assume billing issues would bring most everything down.
                                • hackerman123469 2 years ago
                                  Probably not for long considering the electricity will be cut off if it's not paid. None of the services run etc. are located within the company building, so presumably most of those things would run just fine indefinitely.
                                  • bryan_w 2 years ago
                                    The big thing would be disk failures followed by cert expiration. A handful of months, tops
                                    • c22 2 years ago
                                      Our landlord is pretty lax, so maybe 3-4 months. If our users get creative and work together they could probably keep the thing running indefinitely. Our website will work for 10 years, but it's just static and informational.
                                      • iamthemonster 2 years ago
                                        I work in oil and gas production. It's actually a very interesting question. We could possibly operate without a single bit of human input for a couple of weeks if everything goes well. I'm not sure how much electrical grids are reliant on human input, but there's no reason our gas supply would necessarily disappear immediately.

                                        It really makes me think that apocalypse / societal collapse scenarios are likely to be more survivable than we think, particularly in Year 1.

                                        • bombcar 2 years ago
                                          They'd stay on until someone turned them off, we don't have timers on them.

                                          Oh, you didn't mean the actual building lights ...

                                          • IronWolve 2 years ago
                                            Well, production, shipping would stop. Servers/services would likely stay running as long as power was on.
                                            • _-----_ 2 years ago
                                              Forever, or at least until the credit cards expire and our hosting/domain payment fails.
                                              • rolph 2 years ago
                                                ive often wondered toungue in cheek if google data centers are fullof dust,cobwebs,mummified remains, and a rackfarm humming away keeping the [lights] on, according to a cost-benefit curve analysis, labour efficiency algorithm. /s
                                                • jelavich 2 years ago
                                                  I've been in a couple "lights out" datacenters and there always seem to be electrical/cooling/backups engineers in and around running regular checks. Which is great! Also services for cleaning public spaces.
                                                  • tomcam 2 years ago
                                                    Come on, there are very few mummified remains at most Google data centers. I did the last count, and it was barely 40%.