Apple are also going to great lengths to attempt to protect user's privacy when their new features need more GPU/NPU power than is available on the device. The dedicated server hardware and infrastructure (Apple Private Cloud Compute) they've built out to minimise the privacy risks are impressive.

https://security.apple.com/blog/private-cloud-compute/

It seems unlikely many of the other companies will attempt anything like this.

In reply to a post by andynormancx:

It seems unlikely many of the other companies will attempt anything like this.

Its also unclear how many businesses using Mac / iPhone will trust AI tech, lots of IT Directors / CIOs are worried about liability.

I know about the security stuff, but thanks for the link.

Now, because of the way I think, what is Apple getting out of all of this? They are not charging for use, not yet anyway, and if they don't use the data., what are they getting out of it?

Do they think it will sell more devices? I don't think most people are that bothered about AI to be honest.

I have my Echo units dotted around the house, and all they are used for mainly is to tell me the time, turn on lights or the coffee machine, not that the coffee machine thing is working at the moment. Technology er. Listen to music, ask it to do some sums when I can't be bothered to pick up the calculator. Now and again I will ask it for some information.
I am pretty sure most people do the same thing with their echo units, if that will change when they update it, I don't know.

The only useful thing I see from this AI stuff they want top shove on my Mac is maybe the writing tools, but I already use language tool.

As with any investment they make in their platform, they hope it will make more people buy their devices and use their services than if they didn't build out this functionality. And they hope it will make existing customer stick with them.

In reading the rest of this comment, understand that I am far convinced how useful all the large language model stuff will be in the long term. I use it extensively for technical stuff, as when you use it with a good understanding of its limitations, in areas where you have enough understanding to spot its mistakes, it can be very effective.

Apple AI covers a lot of different and varied features.

Writing tools - this is all the "summarise this", "make this more formal" stuff which uses Apple LLM models and also "Compose text" which uses ChatGPT to generate completely new text from prompts

I can definitely see quite a few people using those features, not everyone is confident in their writing so I can see it having some appeal. There are also subtle but unmissable (not Clippy) prompts to use it, so I think there is a chance people will actually discover these features.

There will of course be news stories of people who claim to have got sacked because Apple AI summarised something for them and made a significant mistake (which it definitely will do, it is an LLM).

Apple Mail - the mail app attempts to identify important messages, it summaries messages when viewing them in a list and it has "smart reply" stuff to attempt (basically fancy auto complete for suggesting whole replies to emails). It also has automatic filing of emails into categories (like GMail has done for a long time),

I think plenty of people will use all of that, though as with all LLM stuff it remains to be seen if Apple can crack things like making the summaries accurate, they have a long way to go from what I've seen so far. I think a lot of people will use the automatic categories, if Apple make it work well (for me it hasn't, but I've seen other people who love it).

Messages - has conversation summaries and "smart reply" stuff

Safari - webpage summaries, can be fairly handy, but few people will ever use it because it is two taps deep...

Phone and audio recordings - transcripts and summaries of those transcripts. Looks handy, but I expect few people will ever notice they can record a call or record audio in Notes (the age old discovery of new features problem)

Image Playground - generates [censored] images with a very Apple UI (mainly there to try and stop you from making images of famous people or making porn/other content some people find upsetting). The sort of thing that people might use for a few weeks and then get bored

Genmoji - lets you create your own emoji with a prompt

Many people love emoji, I can see this getting lot of use (more message threads where you have to guess what people actually mean 😢)

Image Wand - lets you generate an image from a sketch you have made.

I can see this being actually useful for people, if Apple can get it to generate good quality images.

Photos - they have used it to make the search far, far better. You can also use a prompt to create a memory and they've added a reasonably ok generative image clean up tool.

Plenty of people will use the search and the clean up tool (for those who actually realise they can edit their photos)

Notifications - this is what made the headlines with clumsy and inaccurate summarisation of BBC New app notifications.

Plenty of people will leave this enabled, lets see if Apple can actually make it give accurate summaries...

Siri - the current changes to Siri are mostly UI. But it now uses the ChatGPT to answer some stuff inline and it will also for some queries hand you off to ChatGPT proper.

People aren't going to be impressed with Siri in 18.3, because it doesn't have the potentially transformative changes that Apple announced last year.

I've probably missed some stuff.

-------------------

When you say "I don't think most people are that bothered about AI to be honest", I think you are probably right. They probably aren't bothered about the nebulous concept of "AI".

But they will be bothered about actually useful or fun new features that are enabled by "AI" (if they do actually turn out to be useful/fun). Apple AI is many things beyond just a chatbot.

And on the subject of Siri, at the moment is appears to be universally recognised as being worse and Alex and Google Assistant (though in my experience Alexa on my aging Echo Show is just as bad as Siri).

The changes Apple have planned for Siri, if they can actually make them work, would be game changing. Obviously Goole/Amazon/OpenAI/etc are also working to to the same things.

The changes are roughly:

- combine the current basic "turn on this", "set a timer" interface with something akin to asking ChatGPT questions
- giving Siri access to various data on your device (contacts, emails, messages, data in other apps)
- giving Siri the ability to take actions within any app that chooses to make itself available to Siri

If they can actually make that work well, then you Siri could actually be that dreamed for (by some at least) digital assistant that understand your personal context:

"Siri, what time does my aunts flight get in ?"

"Her email from Tuesday says she gets in at two this afternoon. Her flight is with BA, arriving at terminal 5"

"How long will it take me to get there"

"With traffic the drive to Heathrow will be around 45 minutes. Her flight is due in an hour from now, do you want me to message her and let her know you are leaving soon ?"

"Yes please"

"Ok, I've messaged her and I've put a route into Maps for you. Don't forget that last time you went to Heathrow you got a parking ticket for stay in the short stay car park for too long"


"What is the longest you can stay in short stay ?"

"60 minutes"

and as you are driving to the airport

"I have an update on your aunt's flight, looks like it is going to be 30 minutes late arriving. Do you want me to plan a route to the medium stay car park ?"

"Yes"

Who knows if Apple (or anyone else can actually pull this off), if they do I think the EU is going to insist they make all the hooks into iOS to do it available to everyone else as well...

So, in summary:

More caveats than a software license agreement.
More caveats than a used car salesman’s promises.
More caveats than a reality TV contract.
More caveats than an online dating profile.
More caveats than a magician’s trick reveal.
More caveats than an insurance policy’s terms and conditions.
More caveats than a politician’s campaign speech.
More caveats than the ingredients list on a diet snack.
More caveats than a Wi-Fi hotspot in a coffee shop.
More caveats than a time traveler’s warning.
More caveats than a genie’s wish-granting manual.

(ChatGPT's finest work, humorous phrases is not an area it shines in, though the genie one is ok, I'd have gone with "More caveats than a genie’s instruction manual")

Edited by andynormancx (Sat 01-Feb-25 09:17:42)

In reply to a post by zyborg47:

Now, because of the way I think, what is Apple getting out of all of this?

Trying to lock people into their ecossytem so they buy a replacement Mac, iPhone, iPad, and upsell to Vision Pro, and of course all the services AppleTV+, iCloud+, Fitness+ etc. The competition in AI is hot, and most have really good desktop apps for Mac (even before Windows) such as ChatGPT.

what are they getting out of it?

If they pull it off, reputation, it can be done privately, you don't have to freely give your data to the likes of Meta or Google.

I don't think most people are that bothered about AI to be honest.

In corporate it is the biggest thing for 15 years. In the home most people are ignoring it except for picture edits.

I have my Echo units dotted around the house,

Echo isn't AI, and Amazon has lost billions on the Alexa experiment, they have no use in business.
https://arstechnica.com/gadgets/2022/11/amazon-alexa...

The only useful thing I see from this AI stuff they want top shove on my Mac is maybe the writing tools, but I already use language tool.

In my company we are already determining we can give people many hours of drudgery back; by getting the computer to do the tedious work. This is for complex jobs such as Software Development. The same technologies that underpin Apple's current home user AI tools are capable of doing other things, the Transformer Model / LLM.

In reply to a post by jchamier:

Trying to lock people into their ecossytem so they buy a replacement Mac, iPhone, iPad, and upsell to Vision Pro, and of course all the services AppleTV+, iCloud+, Fitness+ etc. The competition in AI is hot, and most have really good desktop apps for Mac (even before Windows) such as ChatGPT.

Is AI really going to lock people into the ecosystem? The majority of people I know that have an Apple product only have an Iphone and have no intention of getting any other Apple products. Okay, it is only a tiny amount of people in the world, but I wonder how that expands to the rest of the world. I know one person who have a mac like me, love macs, had them for years she has, but will never buy a iPhone and one person that has mac, iPhone, Ipad and what ever else.
Me, I have no interest in buying an iPhone, too expensive, and I don't like IOS.

I realise AI is hot, and it is being pushed onto us, but do Joe public think it is hot?

what are they getting out of it?

If they pull it off, reputation, it can be done privately, you don't have to freely give your data to the likes of Meta or Google.

In corporate it is the biggest thing for 15 years. In the home most people are ignoring it except for picture edits.

Yes, corporate, until it goes wrong. We have AI now to keep an eye on stock, it is not great, it goes belly up now and again, and when it does work, it gets things wrong. The old system, well not classed as AI, did a better job, mainly because it had someone putting in information about forecasts, IO mean how much a product is going to sell, instead of a computer trying to work it out.
oh, you sold that many parsnips last week, so you are going to sell that much this week. Then you get a load of parsnips delivered, that is going to take ages to sell through, then a few weeks later, you don't get any parsnips at all.
Just using Parsnips as an example.

We have also got so-called AI scales, in that they can tell what is put on it, more hassle than the old ones. Okay, that is not all to do with them being AI, but they seem to be less reliable than the ones we had before and the camera need cleaning every hour or so. No idea how it gets so dirty


The best one was the old spring scales that hang up when I first started there, but with people being pushed to use Self scan and those silly handsets, it is not possible to have them now.

Echo isn't AI, and Amazon has lost billions on the Alexa experiment, they have no use in business.
https://arstechnica.com/gadgets/2022/11/amazon-alexa...

It has to have some sort of AI, because it needs to know where to get the data and what to look for. It may not be AI in the ChatGPT form. What amazes me is how quick it is, I just asked it where is my home city, and it came back with the answer less than a second, well after it said good morning.

I know Amazon are losing money with it, that is why they are working on a new AI system with Claude, but they have problems with it at the moment. They will charge for that and hope people will go for it. I expect a fair few might. I doubt I will, it does what I need.

The whole idea of Alexa was to get people to order using it, but I have never used it for that, even stuff I buy often.

In my company we are already determining we can give people many hours of drudgery back; by getting the computer to do the tedious work. This is for complex jobs such as Software Development. The same technologies that underpin Apple's current home user AI tools are capable of doing other things, the Transformer Model / LLM.

This is corporate and not home use and is fine as long as it don't muck up. I think even if Joe public leave AI enabled on their computer or phones, it don't mean they will use it. I had a email on my Android phone from Google, saying i can now use Gemini, I have not really used Google Assistant, so no chance of me using Gemini, I like how they, like Apple and Microsoft, enable it as default, NOT.

Alexa is the default voice assistant thing on my phone as it is more useful, or would be if my Switchbot hub has not decided to play up.

I think at some point all this AI stuff is going to come back to bite us, if it is not already. The amount of energy that is needed to keep these systems running is enormous, can't be doing much good to the environment.

My main problem is the pushing of it onto people, putting it as default. Just have a notice come asking if you want it enabled.

i know what it covers and thanks for the long post

But some of us don't want it, and yet it is being pushed more onto us.

The sudden rise of generative with LLMs and diffusion image generation has really messed up the terminology around all of this.

What the scales are doing wouldn’t have been described as being AI before. We’d have talked about it using a machine learning image recognition model. It is almost certainly using a neural network based image recognition model, like many technologies have been doing for decades.

They might be a pain as an employee, as a customer I find the new scales an improvement on tapping around on those terrible resistive touch screens to find the right product. Also as a customer the hand held scanners are a great improvement on putting everything lose in the trolley, then have to queue to have a teenager throw it all down the checkout and having to frantically pack it into bags.

It is entirely possible that your new “AI” powered stock system doesn’t actually have much AI in it and it is just not a good system…

Taking about hand held scanners, Sainsbury’s do need to have a rethink about how they handle them at Christmas. I had a random check and I had missed scanning one single item (72p worth in a trolley load worth £220). Which meant a complete rescan, done in the trolley because there was no where else. But then the systems were overloaded in some way, that meant the rescan couldn’t be transferred to the self service til to pay. Which meant another full rescan into the real til in the self service area. But we had to wait 15 minutes for the transfer to fail first. All while the queue of people needing a full rescan was building.

Took me forty minutes between finishing shopping and paying. On the plus side, I did my best to help the staff with the scanning, I was polite about it all and they were seeming to manage to be unstressed about the situation. And I’ve queued over 30 minutes at Christmas just to go through normal checkouts.

It is all a bit different from when a teenage me was part of the launch staff for the first southern ASDA, back in the 1980s. If you were paying with a cheque you had to take your cheque guarantee card and a driving licence/passport/council rent book to the customer services and get a stamp put in your cheque book. And the protocol if the tils went down (powered by a couple of IBM XT machines) was that if you were currently being checked out, a member of management was to look at each trolley load, estimate the price and then halve it (as far as I know that protocol only ever got used one for about 10 minutes).

What Alexa does also wouldn’t have been described as AI until the last year or two, it would have been talked about as using machine learning (and even then only for parts of its implementation). It will be using multiple neural network based models to listen for the wake word and then to covert the spoken words to text.

Everything else past that point very probably didn’t start by using machine learning/AI to get the job done. The first versions of Alexa/Siri/Google Assistant likely just used a big pile of largely human written to parse the text and understand the intent of the request.

No doubt they are now using at least some machine learning models on that parsing/intent determining side now. But I don’t believe any of them are yet using anything like ChatGPT to do that side of it.

Using a LLM like ChatGPT to do the things that Alexa/Siri/Google Assistant do, while also being able to answer questions like ChatGPT does, is still a developing field of research. If you hear people talking about AI and agents, that is the problem they are trying to solve.

Amazon don’t care whether you use Alexa to order or not. They care whether the presence of Alexa in your home/on your device makes you more likely to order.

Our Echo Show is endlessly trying to come up with new ways to show ads, which are always enabled by default. It is about time I replaced it with an old iPad running a Home Assistant dashboard.

I didn’t just post that to tell you what it was.

It was to demonstrate that Apple AI isn’t a single thing, but a whole series of very different features. Some of which seem likely to be popular and most which would have been enabled by default in the past without anyone making a comment, because they wouldn’t have had the scary “AI” attached to them.

In reply to a post by andynormancx:

The sudden rise of generative with LLMs and diffusion image generation has really messed up the terminology around all of this.

True,

What the scales are doing wouldn’t have been described as being AI before. We’d have talked about it using a machine learning image recognition model. It is almost certainly using a neural network based image recognition model, like many technologies have been doing for decades.

I don't disagree, but the system still has to recognise the difference between say a parsnip and a banana, sure it has to be in the database, and it can't just go around the net looking for a parsnip, shame really, it may do a better job.

They might be a pain as an employee, as a customer I find the new scales an improvement on tapping around on those terrible resistive touch screens to find the right product. Also as a customer the hand held scanners are a great improvement on putting everything lose in the trolley, then have to queue to have a teenager throw it all down the checkout and having to frantically pack it into bags.

I spent 10 minutes yesterday trying to get a set of those scales working, the roll has run out, but it is such a pain to reload and since I don't do it very often, more so of a pain. I was not even on the department but was pass it.

I did not mind the old scales, it was not as if you were typing on the screen, just pressing a picture. I can understand why they changed, it is to stop people fiddling the system, or cut it down anyway. With the old system, someone could print a ticket out for something that was cheaper that what they were buying, the new scales more or less stops that, but there are still ways to put a cheaper ticket on an item if you really want to do that. Easier to get away with it these days with so many selfscans. Scan and shop is a bit of a risk if you get checked.

It is entirely possible that your new “AI” powered stock system doesn’t actually have much AI in it and it is just not a good system…

I don't know, we were told it was coming months ago and when it did, it went belly up almost a week later, and it keeps having problems, it still needs input from us, which is why gaps are scanned often. No doubt some of that is because of shrink, unknown loses. Theft, breakages that are not scanned or shortages from the depot that is not logged or known about, that short of thing leads to what is known as shrink.

This new system is supposed to look at weather, how much is being sold, time of year and other stuff. I don't think you can beat human input into this to be honest. Technology don't work for everything.

Taking about hand held scanners, Sainsbury’s do need to have a rethink about how they handle them at Christmas. I had a random check and I had missed scanning one single item (72p worth in a trolley load worth £220). Which meant a complete rescan, done in the trolley because there was no where else. But then the systems were overloaded in some way, that meant the rescan couldn’t be transferred to the self service til to pay. Which meant another full rescan into the real til in the self service area. But we had to wait 15 minutes for the transfer to fail first. All while the queue of people needing a full rescan was building.


Took me forty minutes between finishing shopping and paying. On the plus side, I did my best to help the staff with the scanning, I was polite about it all and they were seeming to manage to be unstressed about the situation. And I’ve queued over 30 minutes at Christmas just to go through normal checkouts.

Then write or email their customer service. I don't use it myself, I know very little about it. I do see people being checked and think that I can do my shopping and get out before they do, so I don't see the point, also, there is a need for a loyalty card to use it.
If a customer asks me about them, I normally direct them to the customer service or to someone on checkouts. The last time I told a customer that I knew nothing about them and told them to go to customer service, I was taken into the office by a manager. By all accounts that was the wrong thing to do. I should have taken them to the customer service.

It is all a bit different from when a teenage me was part of the launch staff for the first southern ASDA, back in the 1980s. If you were paying with a cheque you had to take your cheque guarantee card and a driving licence/passport/council rent book to the customer services and get a stamp put in your cheque book. And the protocol if the tils went down (powered by a couple of IBM XT machines) was that if you were currently being checked out, a member of management was to look at each trolley load, estimate the price and then halve it (as far as I know that protocol only ever got used one for about 10 minutes).

What Alexa does also wouldn’t have been described as AI until the last year or two, it would have been talked about as using machine learning (and even then only for parts of its implementation). It will be using multiple neural network based models to listen for the wake word and then to covert the spoken words to text.

Everything else past that point very probably didn’t start by using machine learning/AI to get the job done. The first versions of Alexa/Siri/Google Assistant likely just used a big pile of largely human written to parse the text and understand the intent of the request.

No doubt they are now using at least some machine learning models on that parsing/intent determining side now. But I don’t believe any of them are yet using anything like ChatGPT to do that side of it.

Things have changed and not always for the better. i miss the days when you could just go into a shop, do your shopping, wait at the checkout and be served pretty quickly, because they had enough checkouts and staff and not these stupid selfscans that fail. Well some fail. The ones in Aldi are pretty good, even if the beep is pretty loud on them
Now, we have to put up with people in front with loyalty cars and not only that the supermarkets want people to use apps, which takes longer as people fiddle around with their phones. This is one of the reason why I tend to shop at Aldi more these days, no silly loyalty cards.


U

sing a LLM like ChatGPT to do the things that Alexa/Siri/Google Assistant do, while also being able to answer questions like ChatGPT does, is still a developing field of research. If you hear people talking about AI and agents, that is the problem they are trying to solve.

And it may bite them on nose at some point.

Amazon don’t care whether you use Alexa to order or not. They care whether the presence of Alexa in your home/on your device makes you more likely to order.

Our Echo Show is endlessly trying to come up with new ways to show ads, which are always enabled by default. It is about time I replaced it with an old iPad running a Home Assistant dashboard.

I don't have an echo show, I will not have a camera connected to the net in my house and the show has one. I know my phone have cameras, but most of the time they will either see the ceiling or not see anything at all as it is on the table.
I do have a webcam, that I use for discord now and again, but it is not permanently plugged in.

i had a muck around with home assistant on a Dell thin client machine i got from Ebay, that is what I got it for, but it don't support my smart thermostat, so i decided not to bother wiht it. Just got to find a use for the Dell now.

As for Apple AI and Sequoia itself, i reinstalled Sequoia and turned off the AI right away, and so far i have not had anything from Apple, saying, please use our AI, but then I am not signed in to the Apple account as it is now called.