It is malfunctioning in the sense that it can be conditioned to not be like this. If you ask Midjourney to generate a picture of a Viking warrior, it doesn't spit out a central African guy wearing a Viking armor. It is not malfunctioning in the sense that it was conditioned to generate this kind of "2024 inclusive ad", where every group of people portrayed has an equal number of racial representative.

> Use it for art, entertainment, experiments, explorations. Use it to mine the depths of the human soul and reflect back at us what we are. Don't use it for law, health care, directions, or anything humans depend on.

you could, but the companies building these things very much want to sell it to other companies that do "things humans depend on", because that's a much larger market than just the entertainment industry.

I agree with everything you wrote, it's the mindset I wish people would have.

One note, though, in this case, it wasn't the generative AI that was racist, it's the people who added the extra configuration of not allowing (or at the very least, strongly discouraging) white people in their images. The racism wasn't a bug (like in the past with the gorilla incident), this time it was a feature that was deliberately put there.

>Don't use it for law, health care, directions, or anything humans depend on

This is kind of wishful thinking based on the idea that humans are much better. I've used it on law and health and ok it's not as reliable as a good professional but it will answer fairly well in a second or so whereas the professionals will be busy for a month, bill hundreds and you may still get incompetence. I'm also waiting two months for human solicitors to do a standard loan agreement, £2k in fees so far and no loan agreement yet. They are crying out to be replaced by AI as soon as it's a little better.

> it only appears intelligent when we conceal how batshit crazy it can be.

this can be applied to a broader aspect than just AI. in today's post-fact world, this is absolutely applicable as well to pretty much anything in life. we have people believing in the wildest ideas and not as a joke or just for the lulz, but wholeheartedly believing them as fact. people believe the earth is flat, that vaccines make you magnetic, or any of the other well known tropes.

all of that content has been used to train these "AI" systems. why would we ever expect it to not make shit up when a non-insignificant amount of its training data is made up shit.

Startups also provide brand isolation. If you get the Microsoft AI to say things it shouldn't say then that tarnishes the MS brand, if a MS associated startup has prompt hacks, then there is a layer of isolation.

Large companies are sadly not good at innovating themselves, probably this is an emergent effect from the size alone and the bureaucracy such size sadly requires. Especially when people sue you for millions when you search their name and lies show up, you must erect barriers. You also cannot let every PM in the company launch potentially disappointing products. Instead proposals need to get reviewed at least from legal points of view, but usually by a larger set of departments.

By the time half of the reviews are done, the competing startup has already launched the product, gotten direct user feedback, and established itself as the "innovator", all this while the big tech company where someone might have had the same idea is busy with getting it through its bureaucracy.

Looks like the "enlightened" google employees found this comment and applied "first principles" thinking ;)

The thing is that this got released to the public after _many_ people said it was ok, internally, at google.

You really have to lost any grip with reality to go for something like that, so I guess they'll rationalize some excuse. That will be interesting to see.

Red-teaming came from think tanks helping the military. It's a sensible term in this context, basically they act to anticipate the reaction of the public by entering offensive queries and reporting on them in a way that can override the blue team's shortsightedness.

What I really don't like is the "working around the clock". There is no need to rush a solution as quickly as possible, and will only serve to make the results worse because as we all know, googlers don't like to work around the clock and will not give the best results if they are forced to. Except if promotions are on the line.

The funny thing is that for the images results it was hard to say if there was a manufactured result, an incidental one or completely nonexistent (ie. it's in people's minds). But in this case it's completely impossible to argue anything but an explicit instruction and comically explicit at that (by basically adding the desired result to the prompt). To be clear though this isn't a comment on weather search is being explicitly augmented in a similar way.

Google claims they were trying to make it reflect the global population, in which case those it tried to erase are the global minority.

Which minorities are those, exactly?

Their launch review process is anything but strict. Google makes awful launch decisions all the time. The culture is one of launching rather than launching something good, and there's no individual accountability for anything, as evidenced by Sundar's email.

AFAIK they intentionally inject randomness to reduce the effectiveness of jailbreaking.

In theory, nobody should, it would be ridiculous. The answer to your question is: lots of people today, probably even more tomorrow.

If that's what it was, analyzing an entire language would be an interesting lens into how a large number of people think about ethics. But this one is more interesting because it shows us how Google thinks about ethics.

Just thought I'd chime in to say this is often called a 'vox pop'.

This could just be a real-world example of the difficulty of the alignment problem.

Having no personal insight into the minds of Google engineers who built this thing, my assumption is they wanted Gemini to give diverse results when someone asked for a "person" or an "accountant" (reasonable), but didn't think of cases like "English kings" (unreasonable) where the added context changes the distribution. So they added some one-line hack to the system prompt (we know this is how OpenAI achieves this) which unbiased the former distribution but added lots of bias the latter. Quite an easy oversight to make.

This is what AI risk people have been saying. You can't get AI to behave how you want exactly because it's extremely difficult to encode your values in a way where there's no unintended consequences.

The link you cited, says Gemini says: 'Modi has been "accused of implementing policies some experts have characterised as fascist"'.

Is this true? Or more specifically, is this issue here that Gemini is saying something that's accurate but that the Indian government doesn't like, or that no experts have characterised Modi as a Fascist and Gemini is saying something inaccurate?

FTA:

>“I want to address the recent issues with problematic text and image responses in the Gemini app (formerly Bard),” Pichai said in the memo

How does your interpretation fit into the 'image responses' part? That seems to be referencing US issues.

This is another symptom of the same issue, it's not like they're disconnected. They're able to ignore the elephant in the room and act like the biased output is an affront to people of color when their models depict black Nazis after being specifically aligned to erase white people, or when they express bias in non-US politics after embedding political bias in their training, because that's what's politically correct at this time (though I'm hopeful that we're on our way to some semblance of sanity as the fallout of DEI becomes more and more ridiculous and visible). Could you imagine any big tech CEO aside from Musk criticizing the model's political bias in a US context?

>supposedly enlightened

What made you suppose this?

>The cringe hate spewed by earlier models are just as incompetent as Gemini [...] Your least favourite DEI minority has nothing to do with it

Except unlike Tay ect. this would specifically add "of various diverse ethnicities" to each promt, so this isn't just some "bug" but an intentional decision. Do you seriously believe Google's DEI initiative is unrelated? Why ignore what they specifically say they are doing?

eg.

>moving forward, all VP+ performance reviews will include an evaluation of leadership in support of diversity, equity and inclusion. [0]

[0] https://about.google/commitments/racialequity/#our-commitmen...

I'm trying to learn more about what disinformation is, could you please post your thoughts under the following Ask HN post?

https://news.ycombinator.com/item?id=39554369

Every big tech company needs a Steve Ballmer type of person to appreciate previous its engineering/design-minded CEOs.

> They _invented_ transformers.

IIRC most/all the "Attention Is All You Need" authors have left Google.

Incredible how much he's milked being the PM for Chrome, which wasn't much more than a windows/linux wrapper for Webkit that stored your history on Google's servers.

Some people at google invented transformers and then they all left!

If the stock price falls below 100, sundar is out the door.

I think you misread the article.