It's the benefit of how Apple does product ownership. In contrast to Google and Microsoft.

I hadn't considered it, but AI convergence is going to lay bare organizational deficiencies in a way previous revolutions didn't.

Nobody wants a GenAI feature that works in Gmail, a different one that works in Messages, etc. -- they want a platform capability that works anywhere they use text.

I'm not sure either Google or Microsoft are organizationally-capable of delivering that, at this point.

That's a little premature, let's try not to be so suckered by marketing.

I think the private compute stuff to be really big. Beyond the obvious use the cloud servers for heavy computing type tasks, I suspect it means we're going to get our own private code interpreter (proper scripting on iOS) and this is probably Apple's path to eventually allowing development on iPad OS.

Not only that, Apple is using its own chips for their servers. I don't think the follow on question is whether it's enough or not. The right question to ask is what are they going to do bring things up to snuff with NVDIA on both the developer end and hardware end?

There's such a huge play here and I don't think people get it yet, all because they think that Apple should be in the frontier model game. I think I now understand the headlines of Nadella being worried about Apple's partnership with OpenAI.

The most important question to me is how reliable it is. Does it work every time or is there some chance that it horribly misinterprets the content and even embarrasses the user who trusted it.

https://www.theguardian.com/us-news/2024/apr/16/house-fisa-g...

Two features I really want:

“Position the cursor at the beginning of the word ‘usability’”

“Stop auto suggesting that word. I never use it, ever”

"Can you meet tonight at 7?" Me "oh yes" Siri "No you can't, your daughter's recital is at 7"

It's these integrations which will make life easier for those who deal with multiple personas all through their day.

But why partner with an outside company ? Even though it's optional on the device etc, people are miffed about the partnership than being excited by all that Apple has to offer.

Given that this will apparently drop... next year at the earliest?... I think it's simply quite a tease, for now.

I literally had to install a keyboard extension to my iPhone just to get Whisper speech to text, which is thousands of times better at dictation than Siri at this point, which seems about 10 years behind the curve

Yup! The hardest part of operationalizing GenAI has been, for me, dragging the "ring" of my context under the light cast by "streetlamp" of the model. Just writing this analogy out makes me think I might be putting the cart before the horse.

No-one is hitting anything out of the park, this is just Apple the company realising that they're falling behind and trying to desperately attach themselves to the AI train. Doesn't matter if in so doing they're validating a company run by basically a swindler (I'm talking about the current OpenAI and Sam Altman), the Apple shareholders must be kept happy.

I also expect it to fail miserably on names (places, restaurants, train stations, people), people that are bilingual, non-English, people with strong accents from English not being their first language, etc.

I did not see the announcement. Can Siri also send emails? If so then won't this (like Gemini) be vulnerable to prompt injection attacks?

Edit: Supposedly Gemini does not actually send the emails; maybe Apple is doing the same thing?

- Proofread button in mail.

- ChatGPT will be available in Apple’s systemwide Writing Tools in macOS

I expect once you'll get used to it, it'll be hard to go without it.

I can't think of something less exciting than a feature that Gmail has supported for a decade.

Overall there's not a single feature in the article that I find exciting (I don't use Siri at all, so maybe it's just me), but I actually see that as a good thing. The least they add GenAI the better.

Yeah but what about people going to the wrong airport, or getting scammed by taking fake information uncritically? "Well it worked for me and anyway AI will get better.". Amen.

We've been building this up for some time, this tiny universe is the most common thing for me to respond to "will you tell me a story?" (something that is requested sometimes several times a day) since it is so deeply ingrained in both our heads.

Yesterday, while driving to pick up burritos, I dictated a broad set of detailed points, including the complete introductory sequence to the story to gpt-4o and asked it to tell a new adventure based on all of the context.

It did an amazing job at it. I was able to see my kid's reaction in the reflection of the mirrors and it did not take away from what we already had. It actually gave me some new ideas on where I can take it when I'm doing it myself.

If people lean on gen ai with none of their own personal, creative contributions they're not going to get interesting results.

But I know you can go to the effort to create and create and create and then on top of that layer on gen AI--it can knock it out of the park.

In this way, I see gen AI capabilities as simply another tool that can be used best with practice, like a synthesizer after previously only having a piano or organ.

Context will be more important when the gift itself is easy.

ai spam, especially the custom emoji/stickers will be interesting in terms of whether they will have any reusability or will be littered like single-use plastic.

Same thing for your kid, the kid likes both stories, gives 0 shit that you used GenAI or sat up for 8 hours trying to figure out the rhyme, those things are making YOU feel better not the person receiving it.

Remember this ad? https://www.youtube.com/watch?v=sw1iwC7Zh24 12 years ago, they promised a bunch of things that I still wouldn't trust Siri to pull off.

I can't but feel all of this super creepy.

Also no competitor is going to be as good at integrating everything, as none of those have as integrated systems.

i might have missed it but there has not been much talk about guardrails or ethical use with their tools, and what they are doing about it in terms of potential abuse.

Apple also has a long track record of "you're holding it wrong". I don't expect an amazing AI assistant out of them, I expect something that sometimes does what the user meant.

To be fair, this was just the keynote -- details will be revealed in the sessions.

They repeated this so many times they've made it true.

I mean they have great PR, but in terms of privacy, they extract more information from you than google does.

Notice what's missing? A photorealistic style.

It seems like a good move on their part. I'm not that wild about the cartoon-ification of everything with more memes and more emojis, but at least it's obviously made-up; this is oriented toward "fun" stuff. A lot of kids will like it. Adults, too.

There's still going to be controversy because people will still generate things in really poor taste, but it lowers the stakes.

I think it shows the context for the information it presents. Like the messages, events and other stuff. So you can quickly check if the answer is correct. So it's more about semantic search, but with a more flexible text describing the result.

I bet that’s going to be the case. I think they added the servers as a stop-gap out of necessity, but what they see as the ideal situation is the time when they can turn those off because all devices they sell have been able to run everything locally for X amount of time.

I am worried at the infinite ability of teenagers to hack around the guardrails and generate some probably not safe for school images for the next 2 years while apple figures out how to get them under control.

This can be never. LLMs fail fast as you move away from high resourced languages.

The way it works is that when the on-device model decides "this could better be answered by chatgpt" then it will ask you if it should use that. They described it in a way which seems to indicate that it will be pluggable for other models too over time. Notably, ChatGPT 4o will be available for free without creating an OpenAI account.

Being best in class for on-device AI is a huge market opportunity. Trying to do it all would be dumb like launching Safari without a google search homepage partnership.

Apple can focus on what they are good at which is on device stuff and blending AI into their whole UX across the platform, without compromising privacy. And then taking advantage of a market leader for anything requiring large external server farms and data being sent across the wire for internet access, like AI search queries.

If the system doesn't say "I'm gonna phone a friend to get an answer for this", it's going to stay either 100% local or at worst 100% within Apple Intelligence, which is audited to be completely private.

So if you're asking for a recipe for banana bread, going to ChatGPT is fine. Sending more personal information might not be.

More specifically "is openai seeing my personal data or questions?" A: "No, unless you say it's okay to talk to OpenAI everything happens either on your iPhone or in Private Compute"

No, but they said it'll be available for audit by independent experts.

They specifically stated it required iPhone 15 Pro or higher and anything with a m1 or higher.

https://en.wikipedia.org/wiki/Apple_A17

Per the comparison table on that page, the "Neural Engine" has double the performance in the A17 compared to the A16, which could be the critical differentiator.

Siri can now be that assistant, that summarises or does things, that would instead make you go through various screens or apps. Feels like it rescues clutter, not increases it to me imo

And good news! You can clear your homescreen too fully from all icons now =)

This does look like a real-world implementation of the concept promoted by Rabbit. Apple already had the App Intents API mechanism in place to give them the hooks into the apps. They have also publish articles about their Ferret UI LLM that can look at an app's UI and figure out how to interact with it, if there are no available intents. This is pretty exciting.

At https://openadapt.ai we rely on users to demonstrate tasks, then have the model analyze these demonstrations in order to automate them. The goal is similar to Rabbit's "teach mode", except it's desktop only and open source.

1. Yes, App Intents feel like the best version of a LAM we'll ever get. With each developer motivated to log their own actions for Siri to hook into, it seems like a solid experience.

2. Image Gen - yeah, they're pretty nasty, BUT their focus on a "emoji generator" is great. Whatever model they made is surprisingly good at that. It's really niche but really fun. The lifelessness of the generations doesn't matter so much here.

3. Polish - there's so much polish, I'm amazed. Across the board, they've given the "Intelligence" features a unique and impressive look.

Say more? it's just the media thing about intellectual property rights?

Followed by maybe search engines once it gets to a certain level of quality (which we seem to be a bit far from).

Then either desktop or home(alexa).

This is the curse of small-language models. They are better suited for constrained output like categorization. Using them for email generation takes... well, that takes courage.

Thankfully, there is an option to use GPT-4o for many of the text generation tasks.

The output is almost worse, dripping in a passive aggressive tone.

All of these "make your life easier" features really show that no tech is making our lives simpler. Task creation is maybe easier but task completion doesn't seem to be in the cards. "Hey siri, summarize my daughters play and let me know when it is and how to get there" shows there's something fundamentally missing in the way we're living.

- So far, the quality has been very hit or miss, versus places where I intentionally invoke generative AI.

- I'm not ready to relinquish my critical thinking to AI, both from a general perspective, and also because it's developed by big companies who may have different values and interests than me.

- It feels like they're trying to get me to "just take a taste", like a bunch of pushers.

- I just want more/better of the right type of features, not a bunch of inscrutable magic.

It's not unlike the first spreadsheets. Sure, they will some day benefit the entire finance department, but at the beginning only people who loved technology for the sake of technology learned enough about them to make them useful in daily life.

Apple has always been great at broadening the audience of who could use personal computing. We will see if it works with AI.

I think it remains to be seen how broadly useful the current gen of AI tech can be, and who it can be useful for. We are in early days, and what emerges in 5-10 years as the answer is obvious to almost no one right now.

This barely scratches the surface on how much AI integration there's going to be in the typical life of someone in the 2030s.

After a random update my bank's app has received AI assistant out of blue to supposedly help their clients.

At first I was interested how these algorithms could enhance apps and services but now, this does indeed feels like shoving AI everywhere it's possible even if it doesn't makes any sense; as if companies are trying to shake a rattle over your baby's cradle to entertain it.

Aside above, I was hoping that after this WWDC Siri would get more languages so I could finally give it instructions in my native language and make it actually more useful. But instead there are generated emoticons coming (I wonder if people even remember that word). I guess chasing the hottest trends seems more important for Apple.

I'll go back to a dumbphone before I feed the AI

I can’t help but think it’ll get worse with AI

Mostly I see no point in things like email self hosting if half my contacts are on Gmail and the other half on Microsoft.

My suggestion (as someone that tried to escape for some time) is to build a private system for yourself (using private OS and networks) and use a common system to interface with everyone else.

What's a promise from Sam Altman worth, again?

Their brand is equally about creativity as it is about privacy. They wouldn't chop off one arm to keep the other, but that's what you're suggesting they should have done.

And yes, I know generative AI could be seen specifically as anti-creativity, but I personally don't think it is. It can help one be creative.

In 2024 they don't have to wiretap anything. It's all being sent directly to the cloud. Their job has been done for them.

However, to me, the off-device bit they showed today (user consent on every request) represents a strategic hedge as a $3T company.

They are likely buying time and trying to prevent people from switching to other ecosystems while their teams catch up with the tech and find a way to do this all in the “Apple Way”.

If you don't specifically activate it, it won't do shit.

Of course we've only seen examples from an overly produced hype/propaganda video, but it looks to me of yet another example of Apple taking products and making them usable to the masses

Who cares how your flight information shows up at the right time in the right place? the only thing that should matter is that it does.

https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked...

> Sherlocked as a term

> The phenomenon of Apple releasing a feature that supplants or obviates third-party software is so well known that being Sherlocked has become an accepted term used within the Mac and iOS developer community.[2][3][4]

There's a term for that, it's called being Sherlocked: https://www.howtogeek.com/297651/what-does-it-mean-when-a-co...

I don't use Grammarly, really, but I think at least that one is more automatic?

This was my first thought when I saw Rabbit r1 - will all of us become backend developers just glueing various API between legacy services and LLMs? Today seems like another step in that direction.

When I ask Siri Pro what I'm doing on the weekend, she plans a dinner with a mix of friends and compatible strangers. Any restaurant is fine: the food is going to be personalized anyway.

  >> - incoming a billion developer app functions/APIs

So any _real agent_ which has full access to all Apps can still blow Siri out of the water.

Your phone won't do anything else. For 99% of people, they pick up their phone, AI will just decide what they want to see. And most will accept it.

Someday everyone in the room will all pick up their phones when they all ring at once. It will be some emotional trigger like a live feed from a school shooting. Everyone in the room will start screaming at the totally different experiences they're being presented. Evil liberals, clueless law enforcement, product placement being shown over the shooter's gun. You'll sit horrified because you returned to a dumbphone to escape.

That will be the reality if this AI assistant stuff isn't checked hard now. AI is getting better at addiction an order of magnitude faster than it's getting better at actual tasks.

But that’s in a perfect world.

Even to this day, post ChatGPT, I still can’t imagine how I would ever use this AI stuff in a way that really makes me want to use it. Maybe I am too simple of a mind?

Maybe the problem is in the way that it is presented. Too much all at once, with too many areas of where and how it can be used. Rewriting emails or changing invitations to be “poems” instead of text is exactly the type of cringe that companies want to push but it’s really just smoke and mirrors.

Companies telling you to use features that you wouldn’t otherwise need. If you look at the email that Apple rewrote in the keynote - the rewritten version was immediately distinguishable as robotic AI slop.

You know I hadn't considered that and I think that's very insightful. Thank you

The last time our IRS wanted sth from me, they just e-mailed me, I replied and the issue was solved in 5 minutes.

Oh, and you don’t need any paper ids within the country - driver license, car registration and official citizen id are apps on your phone, and if you don’t have your phone when say police catches you, you give them your data and they check it with their database and with your photo to confirm.

In essense, we've saved 50 lives a year by avoiding certain mistakes with better record keeping and killed 5000 since the medical queues are too long due to busy doctors so people don't bother getting help in time.

Doctors have exams, residencies, and limited licenses to give out to protect their industry. Meanwhile, tech companies will give an engineering job to someone who took a 4 month bootcamp.

And despite that it's still your family doctor.

I fully agree with your vision. It's obvious once laid out in words and it was a very insightful comment. But the incentives are not there for other industries to automate themselves.

If someone can do that more productively with Gen AI, do you care?

Google has a few different features to handle making calls on your behalf and navigating phone menus and holds.

The only thing I'd add: I don't think the responsibility for lack of automation is solely on these other industries. To develop this kind of automation, they need funds and IT experts, but (i) they don't have funds, especially in the US, since they aren't as well funded as IT industry, (ii) for the IT industry this kind of automation is boring, they prefer working on AI.

In my view, the overall issue is that capitalism is prone to herding and hype, and resulting suboptimal collective decision-making.

https://forums.macrumors.com/threads/do-m4-ipad-pros-with-8g...

One reason could be future AI models.

I'm not sure if this has been verified independently, but interesting nonetheless and would make sense in an AI era.

People do stuff that they claim implements it using trusted, "tamperproof" hardware.

What they're ignoring is that not all of the assurance is "cryptographic". Some of it comes from trusting that hardware. It's particularly annoying for that to get glossed over by a company that proposes to make the hardware.

You can also do it on a small scale using what the crypto types call "secure multiparty computation", but that has enormous performance limitations that would make it useless for any meaningful machine learning.

Any data sent to 3rd party AI models requests your consent first.

The details will need to emerge on how they live up to this vision, but I think it's the best AI privacy model so far. I only wish they'd go further and release the Apple Intelligence models as open source.

Anyway it seems like a small subset of Siri queries utilize ChatGPT, the vast majority of functionality is performed either locally or with Apple's cloud apparently.

They were also pretty explicit about planning to support other backend AI providers in the future.

I believe they will just provide an interface in the future to plugin as a Backend AI provider to trusted parties (like the search engine) but will slowly build their own ChatGPT for more and more stuff.

Also, it seems that most of Siri's improved features will still work without it (though perhaps less well in same cases) -- and therefore Apple is not fully dependent on it.

That’s the only case I can think of where it’s an external tech you’re making requests to, usually it’s things like Rosetta made out of Apple IIRC but integrated internally

Are they though?

I just setup ubuntu 24 for my son to play games and it's comparatively a very unpolished experience. I'm being very polite when I say that.

I also missed the part of the linked article where it says that my Mac is going to take a screenshot every few seconds and store it for three months.

EDIT: Yes, I'm wrong.

Who is to say they aren't eventually going to replace the OpenAI integration with an in-house solution later down the line? Apple Maps was released in 2012, before that they relied on Google Maps.

> Notes can record and transcribe audio. When your recording is finished, Apple Intelligence automatically generates a summary. Recording and summaries coming to phone calls too.

So the functionality exists, maybe just not in the Voice Memos app?

Oh boy. Someone is going to make a lot of money in court finding people who did this.

Some general OS rethinking is overdue. Or maybe Android is ready for this? Haven't looked into it since they made development impossible via gradle.

Despite this negativity the announcements were better than expected, rebranding AI is bold and funny. But the future will belong to general Agents, not a hardcoded one as presented.

That's part of the differentiation between (on the phones) the Pro and non-Pro these days with the Pro getting all the new stuff, and non-Pro getting a partially improved (and partially degraded, usually wrt the camera) Pro from the prior year.

More seriously, I think the SDK they've teased is only really intended for making their ready-made features integrate with your programs. If you want complete control, you'd probably have to write it the old way, or integrate it with some existing local LLM backend.

We all experience our own reality individually.

https://en.wikipedia.org/wiki/Cottingley_Fairies

“Added ‘start the chronometer’ to your reminders”

Difference I suppose with Apple is they agree not to scrape your inputs.

Plus it’d be weird UX for all your devices to get worse because the iPad in a drawer somewhere finally ran out of power.

https://x.com/markgurman/status/1800198524031906258?ref_src=...

I'm quite confident in the ability of openAI to provide a great usable LLM tech, but much less so of apple. All the demo they've shown in the WWDC could just fall flat if the tech really isn't working well enough in practice. I guess we'll just have to wait and see..

Sam: https://x.com/sama/status/1800237314360127905

Siri reviews the request and decided if it can respond on its own or if it needs ChatGPT. It then pops up a dialog asking if it is OK ti send the request to ChatGPT. It will not be the default LLM.

They gave more space and reverence to Ubisoft.

Apple specifically doesn't want to know your shit, they're jumping through weird hoops to keep it that way.

It would be a LOT easier just to know your shit.

You just say "hey siri, ask chat gpt..." and then it will

I do agree that the extra tap is a bummer for anyone that wanted ChatGPT baked into the OS, even easier to access than it is in the ChatGPT app.

I am not entirely sure I will ever actually allow it to connect to ChatGPT for privacy reasons, but having the option when it can't be handled another way is nice.

I imagine this is more a stopgap until more and more of this can happen locally anyways. Especially since it sounds like Siri determins when it should reach out.

A tale as old as markets.

Literally one of the demonstrations in the Apple Intelligence part of the keynote was "7am alarm", which creates an alarm for 7 AM.

How the hell is it underwhelming?

More seriously(?), I thought "cool" is just what everybody writes to show appreciation when they have nothing else to say.

Except it doesn't do that. The ChatGPT integration is via Siri and opt-in (you ask Siri something, it prompts you to send that prompt to ChatGPT). The rest of the LLM and ML features are on device or in Apple's cloud (which is not OpenAI's cloud). The ChatGPT integration is also, by their announced design, substitutable in the future (or you'll be given a set of systems to select from, not just ChatGPT). They are not sending all data on your device to OpenAI.