A poor analogy could be trolls convincing a flash mob to dress like a suspect's description which they overheard with a police scanner. No one in the mob is guilty of anything more than poor fashion choice.

The difference between this and hashes that require image data to be almost identical is that someone who accidently sees it can avoid and report it. If I can make cat photos that set off Apple's false positives, then there's a lot of people who will be falsely accused of propagating child abuse photos when they're really just sending cat memes.

> Similar patterns abound nationwide, suggesting that Karma's career was not unusual. Lex, a drug detection dog in Illinois, alerted for narcotics 93 percent of the time during roadside sniffs, but was wrong in more than 40 percent of cases. Sella, a drug detection dog in Florida, gave false alerts 53 percent of the time. Bono, a drug detection dog in Virginia, incorrectly indicated the presence of drugs 74 percent of the time.

And this is just to address the original concept of this scanning.

As many others have pointed out there is too much evidence pointing to other uses in the future.

When dealing with say countable infinite sets you can certainly create a provable unique hash for each item in that set. The hash won't be interesting or useful. E.g. a hash that indexes all the integers n with a hashing function h(n+1)... so every integer you hash will be that value plus one. But this just being pedantic and wanting to walk down the thought.

The use of the perceptual hash is because some people might evade the cryptographic hash by making small modifications to the image. The fact that they'd discarded the protection of cryptographic hashing just to accommodate these extra matches is unsurprising because their behavior is largely unconstrained and unbalanced by competing factors like the public's right to privacy or your security against being subject to a false accusation.

[1]: https://news.ycombinator.com/newsguidelines.html

SHA hashes aren’t suitable for this: you can change a single bit in the header to bypass a hash check. Perceptual hashes are designed to survive cropping, rotation, scaling, and embedding but all of those things mean that false-positives become a concern. The real risk would be if someone figured out how to many plausibly innocent collisions where you could send someone a picture which wasn’t obviously contraband or highly suspicious and attempt to convince them to save it.

If you hack my phone and plant some photos with a sufficiently old timestamp I'd never notice them. I can't imagine my situation is all that uncommon either.

How frequently do most people look at their camera roll? I'd be surprised if it's more than a few times a week on average.

Does an attacker even need access to the phone? If iCloud is syncing your photos, your phone will eventually see all your pictures. Unless I've misunderstood how this works, the attacker only needs access to your iCloud account.

Devolving the job to the phone is a step to making things more private, not less. Apple don’t need to look at the photos on the server (and all cloud companies in the US are required to inspect photos for CSAM) if it can be done on the phone, removing one more roadblock for why end-to-end encryption hasn’t happened yet.

For me it’s sad because I have literally always stood by them and they make amazing hardware and software. However at the end of the day I’d rather have the nature of my device be one where it is under my control, than all the wonderful apple tech.

I just don’t know what to use instead.

It's laughable that a man who can call a press conference at a moment's notice and get news coverage for anything he says can be "silenced" because private companies no longer choose to promote his garbage.

That was true of private spaces long before HN existed. If you're a jerk at a party, you might get thrown out. I'm sure that's been true as long as there have been parties.

The only thing "the SJW crowd" has changed is which words are now seen as offensive.

If you want some concrete examples:

- Trump's attempted coup, the range of support it received, the lack of condemnation it received.

- Law's allowing things like running over protestors

- Law's with the transparent goal of suppressing voters

- Widespread support (not unjustified IMO) for stacking the supreme court

- Police refusing to enforce certain laws as a political stance (not because they legitimately think they're unlawful, just that they don't like them)

- (Justified) lack of trust in the police quickly trending higher

- (Justified?) lack of trust in the military to responsibly use tools you give it, and support for a functional military

- (Justified?) lack of faith in the border guards and the ability to pass reasonable immigration laws, to the point where many people are instead advocating for just not controlling the southern border.

Generally these (and more) all speak towards the institutions that make the US a functional country failing. The institutions that make the rules for the country are losing credibility, the forces that enforce the rules are losing credibility. Neither of those are things that a country can survive forever.

- You are running a closed source proprietary OS that you cannot verify is not already doing anything.

- This could theoretically already be weaponized (with the existing server-side implementation) by getting someone to download a file to a folder that iCloud automatically syncs from.

>iCloud is opt-out.

Yes, and that's how you opt out of this scanning. It's the same opt-out as before.

>Under the guise of something no-one can genuinely be against (think of the children!) they have now included a pretty generic snitch into your phone and made everyone accept it.

I dunno what to tell you. I think the system as designed is actually pretty smart[1] and more transparent than before.

If you used iCloud before, and you're putting photos up that'd be caught in a hash comparison, you've already got a snitch. Same with any other cloud storage, short of hosting your own.

[1] I reserve the right for actual bona-fide cryptographers to dissect it and set the record straight, mind you.

>That's neither here, nor there. It's another thing to peak selectively with a warrant of sorts, than to (a) peak automatically in everybody, (b) with a false-positive-prone technique, especially since the mere accusation on a false match can be disastrous for a person, even if they eventually are proven innocent...

I do not believe that iCloud CSAM server side matching ever required a warrant, and I'm not sure where you've gotten this idea. It quite literally is (a) peak automatically in everybody.

Regarding (b), with this way - thanks to them publishing details on it - there's more transparency than if it was done server side.

>especially since the mere accusation on a false match can be disastrous for a person

As noted elsewhere in this very thread, this can happen whether client or server side. It's not unique in any way, shape or form to what Apple is doing here.

The same checking when you synced things to iCloud. As has been repeated over and over again, this check happens for iCloud Photos. It's not running arbitrarily.

Your photos were compared before and they're being compared now... if you're using iCloud Photos.

Far more likely Apple takes a bunch of hashes from a third party in the law enforcement side of things (ie cops) and trust that the third party is definitely giving them hashes to protect against the Very Bad Thing that Apple's customers are worried about.

Whereupon what you're actually trusting isn't Tim Cook, it's a cop. I'm told there are good cops. Maybe all this is done exclusively by good cops. For now.

Now, I don't know about the USA, but around here we don't let cops just snoop about in our stuff, on the off-chance that by doing so they might find kiddie porn. So it should be striking that apparently Apple expects you to be OK with that.

It _does_ make sense client side if you view it being done server side as a blocker for E2EE on iCloud. There is absolutely no world where Apple could implement that without keeping the ability to say "yes, we're blocking child porn".

Okay, but the users of said 3rd party are doing it under the assumption that it is encrypted on the 3rd party's system in a way that they cannot gain access to it. The unencrypted data is not what the user is giving to iCloud. So technically, the data this scan is providing to the authorities is not the same data that the user is giving to the 3rd parties.

Definitely some wiggle room on both sides for some well versed lawyers to chew up some billing hours.

Child pornography, Terrorism? Solve it the old way.

I don’t know why citizens are obligated to make their jobs easier.

We survived the times when phone calls were not moderated, we survived the times when signal intelligence was not a thing.

The economics I'm thinking of are along the lines of cryptocurrency energy usage per participant, vs image scanning energy per caught perpetrator. The number of caught perpetrators via this method over time will approach zero, but we'll keep using energy to enforce it forever.

All this does is remove technology from the problem of child abuse, it doesn't stop child abuse.

This is a good point, but it's not just about people getting wrongly convicted, this system even introducing a remote possibility of having strangers view your personal files is disturbing. In the US, it violates the 4th amendment against unreasonable search, a company being the middleman doesn't change that. Privacy is a shield of the individual, here the presumption of innocence is deposed even before the trial. An extremely low false positive rate or the perceived harmlessness of the current government don't matter, the systems' existence is inherently wrong. It's an extension of the warrantless surveillance culture modern nations are already so good at.

"It is better that ten guilty persons escape than that one innocent suffer." - https://en.wikipedia.org/wiki/Blackstone%27s_ratio

In a future with brain-computer interfaces, would you like such an algorithm to search your mind for illegal information too?

Is it still your device if it acts against you?

I'd rather have evidence for that statement first, since these are just funny numbers. I couldn't find false-positive rates for PhotoDNA either. How many people have been legally affected by false positives so far, how many had their images viewed? The thing is, how exactly the system works has to be kept secret, because it can otherwise be circumvented. So these technical numbers will be unverifiable. The outcomes will not, and this might be a nice reason for a FOIA request.

But who knows, it might not matter, since it's a closed source, effectively uncontrollable program running soon on millions of devices against the interest of their owners and no one is really accountable so false positives can be treated as 'collateral damage'.

I don't, not in the slightest. Back in the days when Geek Squad had to report any suspicious images found during routine computer repairs, a guy got reported to the police for having child porn, arrested, fired from his job, named in the local newspaper as a pedophile, all before the prosecutor was actually persuaded by the defense attorney to look at these "disgusting pictures".....which turned out to be his own grand children in a pool. Of course he was immediately released but not before the damage to his life was done.

>>But I’m not concerned about a perceptual hash ruining someone’s life

I'm incredibly concerned about this, I don't see how you can not be.

I want ZERO computerized algorithms involved in any law enforcement process - especially the "criminal hunting" steps.

Counterargument, why you should not talk to the police (In the US): https://youtu.be/d-7o9xYp7eE

Police Officers exist in a career field that is riddled with incidents of Tunnel Vision. The sibling comment posts a video about not talking to police from a law professor. I'd heed that advice.

You can turn off auto-updates on the Synology devices I own at least (1815+, 1817+).

Assume we had this perfect hash knowledge. I’d create a compression algorithm to uniquely map between images and the 256 bit hash space, which we probably agree is similarly improbable. It’s on the order of 1000x to 10000x more efficient than JPEG and isn’t even lossy.

Admittedly, I haven’t had a chance to read the original source material yet. It’s possible that the person I heard this from was wrong.

They don't need to train a model to detect the actual data set. They need to train a model to follow a pre-defined algo

Sounds like that's what they did since they say they're matching against hashes provided by NCMEC generated from their 200k CSAM corpus.

[edit: Ah, in the PDF someone else linked, "First, Apple receives the NeuralHashes corresponding to known CSAM from the above child-safety organizations."]