Rim/Blackberry tales – reply all
159 points by cloudedcordial 7 months ago | 79 comments- 486sx33 7 months agoSome nice nostalgia for me
-In high school we walked to that exact KFC for lunch and would discuss the previous nights antics playing StarCraft broodwar.
-I used to fix computers (professionally) at a store on the same street as that gas station as an after high school job
-In Dec/jan 2010 I worked 18 hours a day laying floors in the new RIM buildings at Philip/Colombia. A friend’s dad did a lot of the furniture moving. Both of us made over $4000 a week in our early 20s
-Now out of those 4 buildings I think black berry only has two floors of one building
-Waterloo has seen serious decline since the death of RIM
-Not sure it will ever come back, most people including myself left years ago.
-there has been a serious condo tower boom, but that sucks for “walkability” and it’s radically changed the area
-if you attended university in Waterloo in the 2000s and lived off campus, wherever you lived is likely gone and there is a condo tower there now.
- kspacewalk2 7 months ago>-Waterloo has seen serious decline since the death of RIM
>-Not sure it will ever come back, most people including myself left years ago.
I think you overestimate Waterloo's decline just a tad, perhaps your perception being coloured by leaving it. I assure you, it's thriving and in many ways better than the early 2000s when I went to UW. Including the condo boom you mention, though I'm puzzled why you think this somehow hurts walkability.
But yes, Lester street is unrecognizable and every single house I lived in between 2002 and 2007 is gone.
- wizee 7 months agoAgreed 100%. Waterloo has grown substantially over the last 15 years, and is generally thriving. The condo boom and general increase in density has only increased walkability. Most of these condos have ground level retail too. The light rail is also a nice addition.
As RIM/Blackberry declined, a whole ecosystem of startups emerged started or staffed by ex-RIM folks. The universities have also grown substantially.
- wizee 7 months ago
- morkalork 7 months agoThe wild thing about the condo boom in Waterloo is 77% of units are owned by investors. It truly exemplifies the mess that is the housing market. Rentors can't break into it and homeowners are doubling, tripling up on properties.
- cloudedcordial 7 months agoI was in the area around 2010 working for someone else. Adding to your bullet points:
- The pool business near the single-digit RIM buildings had more business than they could do. Many folks wanted swimming pools at their homes.
- Various eateries such as the sandwich shop mentioned in the article made decent money during the height of Blackberry.
- People skipped starter homes and bought single houses as their first homes. Some real estate agents waited outside of some buildings during bonus was announced.
- marssaxman 7 months ago> People skipped starter homes and bought single houses as their first homes.
This is a terminological distinction I am not familiar with; what is a "single house", and what is the difference between a starter home and a first home?
- creaturemachine 7 months agoMaybe they consider a townhome or apartment condo as a starter home, which is true in the current market, but 25 years ago it wasn't uncommon to buy a detached as your first home.
- tempest_ 7 months agoThey mean a single family home which might be described as a "detached house"
In the GTHA (including Waterloo) there is no such thing as a starter home any more, which in the past meant "small detached house, probably needs some work". The only thing they build now is very small 500 sqft condos and very large 3000+ sqft houses.
- creaturemachine 7 months ago
- marssaxman 7 months ago
- dmuth 7 months agoYour username is nice nostalgia for me. :-)
- nazcan 7 months agoDensity has hurt walkability?
- dddddaviddddd 7 months agoI have no specific experience with Waterloo, but sometimes towers follow a Corbusian ideal of a tower surrounded by nothing; or worse, a tower surrounded by high-speed roads/highway — essentially a stacked bedroom community with no walkable amenities.
- cldellow 7 months agoFWIW, I live in the region and disagree with OP's characterization of "serious decline" and "most people have left".
I went to school here from 2003-2008, moved away and moved back in 2011.
The area's population has increased by ~20% since 2012 (~the death of RIM, according to its stock price). In 2011, it got regional train service to Toronto. In 2019, it got a local light rail train.
The university area that the OP seems to be referring to is, IMO, more walkable and bikeable now than before. Some of the towers are mixed use, with ground floor retail.
The city is definitely quite different from the early 2000s, though.
- cldellow 7 months ago
- toast0 7 months agoA dense housing boom could hurt walkability if it replaced mixed low density retail and housing and if there was no compensating retail boom near housing.
First floor mixed use retail can address this, but sometimes those spots sit vacant because of cost or other issues with rhe space.
- 486sx33 7 months agoI guess my definition of walk ability includes seeing interesting things and being able to walk towards them. The condo towers block the sun and you can’t see beyond them walking down the road.
My post was a bit too negative I suppose, some people probably like the tall buildings surrounding you when you’re walking, I mean it works for New York City right.
- 486sx33 7 months ago
- dddddaviddddd 7 months ago
- StrictDabbler 7 months agoI'm glad to see WCRI is still there, though it's hard to claim it's fully "off-campus".
- Waterluvian 7 months agoDid you go to WCI too? This is all so nostalgic for me.
Though I grew up in Waterloo and lived at home, yeah, the city sure has grown a ton. I moved away to raise a family.
- cloudedcordial 7 months agoUsername checks out haha!
- 486sx33 7 months agoYes. Grade 9 in 99 :)
- Waterluvian 7 months agoCool. One year behind you. So we went at the same time.
I want to say small world but after enough time it’s bound to happen, eh?
- Waterluvian 7 months ago
- cloudedcordial 7 months ago
- kspacewalk2 7 months ago
- easton 7 months agoThat random “J” at the end of the messages brings me back to mail circa 2010. As I recall, iOS also didn’t render Outlook’s smileys right, leaving a bunch of Js in mail from my Mom.
(For the ones who ‘missed’ it: https://devblogs.microsoft.com/oldnewthing/20060523-10/?p=31...)
- jareklupinski 7 months agofor a long time, I thought people were just really into giving me a single letter nickname
- SoftTalker 7 months agoLots of email clients still don't.
- jihadjihad 7 months agosame here J
- nehal3m 7 months ago!!!1one
- jareklupinski 7 months ago
- Lammy 7 months ago> Blackberry at the time was kind of an “everything goes, whatever it takes” free for all. And this damn the torpedoes full steam ahead attitude was pervasive everywhere.
> RIM Job
I love that this was actually the URL for their careers page in this era: https://web.archive.org/web/20101122175558/http://rim.jobs/
- tgsovlerkhgsel 7 months agoI don't know why these articles (and many comments here) always make it sound as if the people replying were idiots. For many, or I'd rather say most in this case, it's clear that they know exactly what they're doing - having some fun given the opportunity that presented itself.
- cruffle_duffle 7 months agoReply all storms are some of the most amusing things to witness and joke about with your colleagues. Sadly I haven’t seen a good one in a very long time :-(
- cruffle_duffle 7 months ago
- arjvik 7 months agoPoor Sumit B. He never did anything - it was his manager!
- cloudedcordial 7 months agoSumit B forever has a story to tell at dinner parties.
- cloudedcordial 7 months ago
- ano-ther 7 months agoThis happens often enough (even MS itself broke their internal Exchange servers).
And every story seems to end with admins having to improvise. Am curious: (why) isn’t there a “kill reply-all chain” button as a feature?
(The article explains that this didn’t work for RIM because of BB’s architecture, but for Exchange?)
- eddieroger 7 months agoIn theory, mail should be stateless, so what would they use to do that? Some clients and servers understand headers for thread identifiers, and then it would be possible to (in theory) zap that id, but all it takes is one client not supporting it for that to be out the window. The article kind of points this out - Exchange had a mitigation, but the clients got the notification and were able to reply because by then the mail was on the client (pushed).
- wiredfool 7 months agoThe search term for MS for is bedlam 3.
- RandallBrown 7 months agoI worked at MS in 2012 and we had another email storm that shut down almost everything. I remember not being able to do any work at all because the network was so slow I couldn't even edit files (the internal source control system required getting a lock on the file to be able to edit them).
- RandallBrown 7 months ago
- philipwhiuk 7 months agoThe standard fix is to massively limit who can send emails to DLs - which is an Exchange config option.
- eddieroger 7 months ago
- teddyh 7 months agoThe story of Bedlam3: <https://rodneymbliss.com/2013/10/17/i-survived-bedlam3/>
- insane_dreamer 7 months ago> it’s estimated that over 15 million emails were generated in the space of about an hour.
wtaf?
- indrora 7 months agoEmail storms and nuclear meltdowns have the same rough physics.
If 1% of emails are blind replies from people out of the office, but are tracked by the originating sender (not the DL) since OOO messages in Exchange get sent to the DL (as well as the originating sender) and soon enough you have Exchange sending a new OOO email per reply from a new reply-er. Your typical company the size of Microsoft is going to have probably... 3-5k people out of the office at any one given time (the lingo is "OOF", or "out of facility")
OOOs can also trigger another OOO, which can in turn cause a new OOO to spawn.
Soon, you've got OOOs reply-all'ing to OOOs and drenching a DL... and an exchange forest.
- indrora 7 months ago
- insane_dreamer 7 months ago
- lproven 7 months agoThe world needs a driving licence for email. It would mandatorily include use of plain-text and bottom-posting.
- afandian 7 months agoEurope has one. I’ve got one. I am qualified to use 2004-era Microsoft software, and have a certificate to prove it… somewhere.
(Edit: since I got mine it’s acquired the word ‘international’ but lost the word ‘driving’. Swings and roundabouts.)
https://en.m.wikipedia.org/wiki/International_Computer_Drivi...
- beng-nl 7 months agoI used to think the same but I caved when I started using web interface email clients. Let’s face it, the world has moved on, new generations are online, and we are the wrong ones now.
- lproven 7 months ago> I caved when I started using web interface email clients
It's one of the things I like about Gmail. It does plain-text and bottom quoting just fine.
> Let’s face it, the world has moved on, new generations are online, and we are the wrong ones now.
NEVER GIVE UP! NEVER SURRENDER!
https://mygeekwisdom.com/2014/03/15/never-give-up-never-surr...
- speerer 7 months agoI don't understand why this isn't configurable. Why can't the basic data structures (of one email after another in serial) be displayed in either order depending on mail client settings?
Admittedly I haven't looked into it because I'm perfectly fine with top posted emails. But I routinely sort files in my directory. Why not emails in a displayed thread?
- bregma 7 months agoIt's people quoting text, not threads of messages.
The ability to semantically parse text to determine what order paragraphs should be displayed in to suit the tastes of the individual reader is a very recent development. Or, rather, will be soon. Maybe not very soon.
- thworp 7 months agoAs the other comment mentioned, the email body contains the entire quote chain. The way clients accomplish threaded display is a combination of:
- parsing the unstructured email body and looking for quote levels, html formatting and printed email heads
- parsing certain headers like message-id, in-reply-to, dkim sig
- looking for sections of the message body in the inbox
This is done because there is nothing in the protocol to cleanly accomplish what you want. Even if there was, you could not rely on it at all. Doing anything with email is a gigantic PITA, you sometimes get emails where the msg-encoding header doesnt match the body's encoding, html in the plaintext section and other fun things.
Since nobody really cares about the RFC and just does their own thing, there is no chance at improvement.
- bregma 7 months ago
- immibis 7 months agoIf the entire email is being replied to, I can just read that email, which is displayed immediately before the current one in a threaded display. Why should I have to scroll past another copy of that email?
However, the original email is included as a convenience in case my MUA doesn't support threaded display or it's a mailing list I joined after the original email was sent or any other reason why I might not see it. That's why there is quoting at all.
Nobody top-posts when using selective quoting because obviously it's different.
- lproven 7 months ago
- bigstrat2003 7 months agoI hate both of those things. I guess this means war, please look for my envoy bearing the formal declaration of war.
- lproven 7 months agoYou are wrong.
The biggest and most successfull FOSS project of all time is coordinated entirely by email. These are the rules:
https://subspace.kernel.org/etiquette.html
Again: you are wrong.
- bigstrat2003 7 months agoI don't give a shit what the Linux kernel does. It doesn't change the fact that I think plain text is ugly, HTML mail has no drawbacks I care about, and bottom posting is just plain weird.
> Again: you are wrong.
No, you.
- bigstrat2003 7 months ago
- lproven 7 months ago
- csmattryder 7 months agoI'm teaching the chapter, "Why is everyone signing off with J? A crash course in email from Windows users"
- xattt 7 months ago> bottom-posting
Immediate and indefinite suspension of email license.
- GJim 7 months agoA: Because it messes up the order of things.
Q: Why is top posting bad?
- gknoy 7 months agoYou jest, but I've noticed that the conventions of "newest on top" vs "newest on bottom" is _seriously confusing_ for some people that I help navigate tech stuff. I don't know how to describe the heuristic for:
- New text conversations show at the _top_ of the list of conversations - New messages are at the _bottome_ of a conversation - New emails are at the _top_ of your email client (?) - and now you remind me that email replies can be both at top and bottom (:
It feels arbitrary, but I suspect this is due to the heritage of paper, where newer things are on top of the pile, but in a given document, newer text tends to be added at the bottom/end. (it's a stretch :))
- immibis 7 months agoOnly if you ignore that I just read the previous message.
In reality it's:
Q: Why is bottom posting bad?
Q: Why is bottom posting bad?
A: Because it repeats itself.
Q: Why is bottom posting bad?
A: Because it repeats itself.
Q: Isn't this also a problem with top posting?
Q: Why is bottom posting bad?
A: Because it repeats itself.
Q: Isn't this also a problem with top posting?
A: No, because the reader skips to the next message when they get to the quoted part.
- gknoy 7 months ago
- GJim 7 months ago
- cloudedcordial 7 months agoA local technical college has a dedicated course on "Slack at workplace" (or a paraphrase of the title).
- sixstringtheory 7 months agoHopefully they’re discouraging students from sending lots of fragmentary messages interspersed with superfluous nonsense like “hey” “i mean” etc
- sixstringtheory 7 months ago
- afandian 7 months ago
- chamanbuga 7 months agoI vividly remember this incident and more like it. I was coached by my manager to never reply all to an email addressed to a larger distribution group because of this exact reason. Not only were you annoying people, but everyone's email ended up getting delayed for an hour or two.
Can someone explain to me why the backlog would happen? Why they didn't have systems to protect from such a basic DOS attack?
- canucker2016 7 months agofrom the first Exchange Reply-All email storm, a dev who worked on the Exchange server, https://techcommunity.microsoft.com/blog/exchange/me-too/610...:
It didn't fix the problem completely from what I recall, there were smaller versions of Bedlam at MSFT. I've heard that some branch of the US Dept. of Defense created their own Bedlam storm a few years back. So they had to layer in a few more guardrails to prevent another reply-all from getting out of control.An Exchange email message actually has TWO recipient lists – there’s the recipient list that the user sees in the To: line on their email message. This is called the P2 recipient list. This is the recipient list that the user typed in. There’s also a SECOND recipient list, called the P1 recipient list that contains the list of ACTUAL recipients of the message. The P1 recipient list is totally hidden from the user, it's used by the MTA to route email messages to the correct destination server. Internally, the P1 list is kept as the original recipient list, plus all of the users on the destination servers. As a result, the P1 list is significantly larger than the P2 list. For the sake of argument, let’s assume that 10% of the recipients on each message (130) are on each server. So each message had 100 recipients in the P1 header, plus the original DL. Assuming 100 bytes per recipient email address, this bloats each email message by 13K. And this assumes that there are 0 bytes in the message – just the headers involve 13K. So those 15,000,000 email messages collectively consumed 195,000,000,000 bytes of bandwidth. Yes, 195 gigabytes of bandwidth bouncing around between the email servers. ... So what did we do to fix it? Well, the first thing that we did was to fix the MTA. And we tried to scrub the MTA’s message queues. This helped a lot, but there were still millions of copies of this message floating around the system. To prevent anything like this happening in the future, we added a message recipient limit to Exchange – the server now has the ability to enforce a site-wide limit on the number of recipients in a single email message, which neatly prevents this from being a problem in the future.
Here's one reference, https://www.theregister.com/2023/02/14/us_army_reply_all_sto..., though I thought they had one back in the 2010s.
- canucker2016 7 months agoAnother storm hit MSFT around the start of the pandemic, https://forums.theregister.com/forum/all/2020/03/26/microsof...
- canucker2016 7 months agoMaybe the last fix for the reply-all email storm problem (at least for Exchange servers)?
see https://www.theverge.com/2020/5/10/21253627/microsoft-reply-...
- canucker2016 7 months ago
- canucker2016 7 months ago
- canucker2016 7 months ago
- Daub 7 months ago> Blackberry at the time was kindof of an “everything goes, whatever it takes” free for all. And this damn the torpedoes full steam ahead attitude was pervasive everywhere. Red tape and beuaracracy was loathed by everyone.
Ironic as RIM became known for its cripplingly dense beuaracracy and red tape.
- brazzy 7 months agoNot surprising though that the negative consequences of one extreme can lead to an overreaction in the opposite direction.
- brazzy 7 months ago
- setheron 7 months agoEvery company has lore like this; I remember when I joined Amazon hearing how someone would create SEV-1 tickets because their phone didn't work (Jeff B. would get personally paged for all SEV-1 at the time)
- 7 months ago
- shepting 7 months agoI was working there that fall as one of my co-op work terms!
I recall the onboarding tour around the testing rooms which were essentially giant Faraday cages. There was a print-out on the door exhorting employees to CLOSE THE DOOR! when you come or go. Apparently it was a semi-monthly occurrence where someone would accidentally leave the door propped open and the nightly tests on upcoming devices would make real 911 calls to the local dispatchers as the E2E tests on physical hardware were running.
- tverbeure 7 months agoI love reply all events and have fond memories of the adventures of the Fluke Meter at GlobeSpan (long defunct.)
It started when one good soul sent out a worldwide email asking "Who has the Fluke meter?" and after the first person replied "It's not here!", the rest of the world reacted in kind.
It took about a day for the storm to die down.
- 7 months ago
- dmalik 7 months agoHaha I was working there at the time. I don't remember how many emails came in but well over 100.
Emails were so abused there though. I would get over 100 a day that were work related. Think Slack over email.
- philipwhiuk 7 months agoYeah, I'm pretty sure this isn't the only time it happened - I don't recall Sumit but I do recall others.
I worked in the NOC and you quickly learned to basically ignore every non-personal email sent before you were on shift that you weren't directly copied on. If it wasn't your shift and it wasn't handed over, it wasn't important.
- dmalik 7 months agoYa this definitely happened a few times.
- dmalik 7 months ago
- philipwhiuk 7 months ago
- brazzy 7 months agoI once had that happen to me with spam. A spammer had set up full-fledged mailing list software to send spam. With the reply function active!
I'm sure you can imagine the rest.
- minkeymaniac 7 months agoSome more of these: https://en.wikipedia.org/wiki/Email_storm
- hnthrowaway0328 7 months agoA few years ago we got the same thing: a guy sent a email to finance showing his bonuses and somehow sent it to everyone. Then everyone knows his bonus and salary.
- tbojanin 7 months agothis is one of the funniest tech stories ive heard!
- 7 months ago