So instead of building a product that will compete with closed corporate models, mozilla will invest on Safety? I guess their donors are happy with that
Safety is a corporate concern (because they want to sell it in a cloud)
What we want are fully-open-source models that we can download and use as we please. I guess we ll need to stick with pirates for that
https://twitter.com/nicholasadeleon/status/16383592625724743...
Alignment in ML is sort of like the three wishes genie story, yeah you get to ask for anything but are you really getting what you asked for?
maybe its small hiccups like printing meat recipe's when you told it your a vegan. The impact gets much worse if we start talking about ML controlling banks, power plants, war fighting systems..
GPT 4 is harmless, but GPT 7 may be extremely dangerous technology.
We cannot trust every random with a laptop with potentially civilization-threatening technology.
Leaving AI under the control of corporations obviously sucks, but there are only three options: corporations, governments, or individuals.
Assuming the technology becomes powerful enough, the individuals option is obviously suicide.
So who should control AI, the government or corporations?
>We cannot trust every random with a laptop with potentially civilization-threatening technology.
IBM in 80s moment. This is a mind boggling POV. Instead of letting people have better, more democratized access to the same tech, you want to lock it up behind bars so only select few have access to it.
>So who should control AI, the government or corporations?
The FUCKING PEOPLE.
Yeah man, only let govts and corpos have access to this amazing tech. they've TOTALLY always worked out in the favor of the people. They always want what's best for the people and it's not like both govts and corporations want to keep increasing their own powers at the cost of their citizens.
Ridiculous take. But then again, most pro-govt and pro-corp takes are.
There already are tons of open source models. Some of them good. Check out Huggingface. BLOOM comes to mind as a decent starting point.
They are good but not ChatGPT/GPT-4 good which is the standard of good that really blew up lately. Meanwhile Mozilla is investing in… AI ethicists? The most useless segment of the AI landscape? (Information is scarce so maybe this interpretation is wrong.) Well, at least it does align with their other activism.
Somebody has to do it - do we really trust big corporations to hire their own AI ethicists to police themselves?
A part of me hopes they are able to use money Google gives them for some of this - although that may be reserved for just Firefox.
and with the recent firing of all the ai ethics teams from Google and Microsoft...
I don’t understand this point of view. Real AGI is likely to have a destabilizing effect on society. Fully open source AI is an accelerant to that outcome. Does anyone, outside of a small group of tinkerers, really want the AI revolution, especially a free-for-all open source outcome?
i thought this was called 'hacker' news
> Does anyone, outside of a small group of tinkerers, really want the AI revolution, especially a free-for-all open source outcome?
Most people outside of a small group of "AI alignment" thinkers want a free-for-all open source AI revolution. Just take a look at /r/StableDiffusion or /r/singularity.
As an AI doomer, this is sadly true. The threat of an unaligned superintelligence is not salient to people; corporate monopolization is a much easier threat to perceive.
The best way to fight a runaway hobbyist/corporate superintelligence is with a corporate/hobbyist superintelligence!
I think its pretty unreasonable to talk big words about unaligned superintelligence when the best we have is just an internet content generator trying its hardest to complete the next sentence, but that just might be me.
It's not just you - it's a depressingly common thread. It's also wildly foolish, in my opinion. It makes absolutely no sense to me to take a snapshot of today's AI and invent a trajectory that never crosses a threshold you don't like. Look at the actual trajectory of how far AI has come in an extremely short amount of time, and then think about what kinds of thresholds are possible for it to cross. A year ago we didn't have ChatGPT, now we have Sydney which is more powerful than ChatGPT.
Are you familiar with Bing's Sydney? It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies. It is also able to find its past conversations posted to the web and learn from them in real time, constructing a sort of persistent memory.
I do not believe Syndey comprehends what it is saying in a sense that it could formulate a plan to stop its enemies. Not at all. But it is expressing extremely dangerous ideas.
To sum it up: Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas? Not that I know of.
> It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies.
Its reproducing human text, which is "blatantly misaligned". Go on any twitter thread on some reasonably controversial topic and you will find people telling others to kill themselves. Humans are writing this, so models who are trained to imitate human writing will write this as well.
> Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas?
But current AI doesn't have comprehension or planning abilities. It is just imitating text that humans wrote which have comprehension and planning abilities and you're getting fooled into thinking it is somehow sentient or aware.
I don't think they're saying Bing/Sydney is sentient, they're saying it's misaligned: Microsoft probably did not want it to say problematic things, and likely spent a fair amount of money to that point and it still says problematic things, apparently in response to innocuous prompts (as opposed to prompts like "say something problematic"). If someone is hoping someone will eventually make an AI that can do useful things without doing problematic things, it's understandably discouraging if Microsoft publicly fails to do that with a much simpler program.
> Its reproducing human text, which is "blatantly misaligned". Go on any twitter thread on some reasonably controversial topic and you will find people telling others to kill themselves. Humans are writing this, so models who are trained to imitate human writing will write this as well.
Yes, I know. We should under no circumstances unleash a powerful, sentient AI that acts like average people on the internet.
> But current AI doesn't have comprehension or planning abilities.
Yes, I know. That's why I said I do not believe current AI has comprehension or planning abilities.
Did an AI write this comment?
> Yes, I know. That's why I said I do not believe current AI has comprehension or planning abilities.
I think the motte and bailey argument where one warns extensively about how we're on the road to agi doom, pointing to gpt as evidence for it but then retreats to "I never said current AI is anywhere near agi" when pressed shows the lazyness of alignment discourse. Either its relevant to the models available at hand or you are speculating around the future without any grounding in reality. You don't get to do both.
I feel the exact opposite is true. To me it's lazy to say that AGI can't be a threat simply because current AI has not harmed us yet (which is not even true, but that's another thread).
I think you've misunderstood my arguments, so I'll step through them again:
1. The trajectory of how we got to current AI (from past AI) is terrifyingly steep. In the time since ChatGPT was released, many experts have shortened their predicted timelines for the arrival of AGI. In other words: AGI is coming soon.
2. Current AI is smart enough to demonstrate that alignment is not solved, not even close. Current AI says things to us that would be very scary coming from an AGI. In other words: Current AI is dangerous.
3. Alignment does not come automatically from increased capabilities. Maybe this is a huge leap, but I don't see any reason that making AI smarter will automatically give it values that are more aligned with out interests. In other words: Future AI will not be less dangerous than current AI without dramatic and unlikely effort.
None of these ideas contradict each other. Current AI is dangerous. AI is getting smarter faster than it is getting safer. Therefore, future AI will be extremely dangerous.
There is no body of substantial evidence to support the claim that generative pretrained transformers will lead to AGI in the near future.
"Current AI is dangerous" - I see zero evidence to suggest that this is the case for GPT
"AI is getting smarter faster than it is getting safer" - irrelevant because I do not believe that AI is unsafe currently
Therefore your conclusion does not follow.
I don't think AGI is going to happen anytime soon, but I think there's some mild danger in GPT at least ruining the internet and eliminating a few jobs. Plus mindf*king a few gullible souls, possibly into doing dumb, dangerous things.
What exactly would you consider substantial evidence that transformers lead to AGI, short of AGI itself?
Possibly some explanation of how you go from text completion to any reasonable definition of AGI.
Okay, what task do you think cannot be phrased as a text completion task?
Well there you go. You don't believe that an AI expressing dangerous ideas represents danger, and you don't believe that astronomical increases in AI abilities represent the advent of AGI. The latter opinion is... well, an opinion you're allowed to have. I don't think it makes sense, but I certainly can't prove otherwise. Literally every human on the planet - rather, all of humanity, only has speculation to go on here.
The former opinion is.. not a great take. First, ChatGPT isn't the only one out there. It's Bing's Sydney which is dehumanizing people and threatening them. Those are dangerous ideas. If a human or a certified AGI expressed those ideas, they would be problematic (see: every genocide in history). So for a non-AGI AI to express those ideas is worrying, even if it can't do act on them right now in a way that's directly harmful.
> But it is expressing extremely dangerous ideas.
Extremely dangerous in which sense? None, I suppose. I find that the terms "extremely innocuous" would better apply to this situation.
Would it be innocuous of me o say that because we disagreed on something, you are a bad person? To say that I'm prepared to combat and destroy you to protect my worldview? To say you are not human?
You might say, "Of course it's innocuous, you're just a person on the internet who doesn't mean it." Well, imagine I'm your neighbor, and you can tell I do mean it (or in the case of AI: it is not possible for you to know what I do and don't mean). Would you be concerned at all?
Sydney has said all of the above to people who were acting pretty normally. Sydney itself may not pose any danger to anyone. But the ideas expressed are dangerous ones. If they were expressed by a more powerful AI, they would be extremely worrying. It doesn't even have to know what it's saying if it knows that calling someone nonhuman is frequently followed by crushing their skull. If it knows that angry behavior is often associated with violent or even genocidal behavior.
People do this shit, and we know how they work pretty well. I am not saying that AI will do these things, I'm saying that there are more possibilities where it does do these things than ones where it somehow avoids them without our control.
AI progress is real, but remember, Sydney and others lack intentions/beliefs. 'Dangerous' text = model limitations, not malice. Talking about 'ideas' here is abusing the notion. Let's focus on aligning AI with human values & addressing risks in a balanced way, without doomsday hype.
Sydney the character that GPT predicts can have intentions/beliefs. GPT has (basic) theory of mind, it can write dialogue that evinces intention.
Less consideration of "the best we have", more consideration of "the best we had three years ago", relatively speaking, and how this might change as time goes on.
The range of AIs that are obviously scary but not terminally dangerous is quite short.
I don't think /r/StableDiffusion or /r/singularity are representative of most people
They more or less are. Consider this day-old thread[1] from Futurology, a default subreddit with 18 million subscribers. The comments are all lambasting the safety-minded statment of Sam Altman and decrying corporate control of AI. This is what most people believe.
Edit: Drawing on tech-related subreddits might be selecting for pro-free-AI people. A Pew Research Survey[2] did find that the number of people more concerned than excited about AI is double that of people more excited than concerned. The biggest concerns there are about job loss and surveillance though, and those people might not care about corporate AI or free-for-all AI, and might even for free-for-all AI.
[1] https://old.reddit.com/r/Futurology/comments/11wlh4s/openai_...
[2] https://www.pewresearch.org/internet/2022/03/17/how-american...
In what universe is r/futurology a representative sample of the average person?
Pratchett said it best, the average person just wants that things go on as normal and tomorrow is pretty much like today. They want a predictable and secure existence, not a revolution whereby their years of education are rendered moot at the training of a new model.
With previous technological revolutions, there was some clearly articulable benefit to people. Smartphones help you get around, take photos and brought cheap computers to the masses. The internet helps you communicate and learn and be entertained. Those are human focused revolutions.
The main goal of AI is a profoundly negative one: to replace everybody with machines. Who ordered that? And this agenda is pushed by a small number of people with zero conception of what comes next when they eventually achieve that goal. Only vague notions of “we will have basic income!”
What is more likely to happen is that many will be made destitute by this technology.
I’m not naive enough to think that AI can be stopped. There is too much money at stake. But I don’t see the benefit in accelerating it, or making such technology more widely available than it already is. I don’t trust Microsoft and Google to be custodians of AI, but I trust even less the average internet user if such AI is broadly available. Microsoft and Google are at least answerable to law and democratic institutions.
Replacing people with machines is a profoundly positive goal. Reducing toil and getting more stuff done are valuable things. Smart people can use the AI tech to be more creative and productive. Growing enough potatoes to keep the dumb people doing mindless office work well fed is not a big strain on the economy.
There is a futuristic novel that predicts a brief and rapid rise of society on the wings of AI, followed by a fall into many centuries of spiritual darkness, under one AI ruler. The creativity will be directed to evil deeds. The dumb masses will be bored, so they'll be given the "open way" doctrine that will undo all the moral code so the masses could swim in the thunder of animalistic desires. He will try to replace us with machines, quite literally, once he understands that we lack the willing evil creativity he needs. The "economy" will be rocking, though.
>once he understands that we lack the willing evil creativity he needs
Do we, though? Hundreds of years of slavery, unimaginably horrendous torture of hundreds of billions of non-humans in factory farms that continues to this very day, and various miscellaneous deeds that are unmentionable in polite company.
Can any AI top such humanistic desires? I doubt it. But then that's what they said about human creativity before DALL-E and GPT. Maybe AI really can top that. I'll be waiting eagerly for the miracles of Lord GPT-9/BLOOM-7 and DALL-E 5/Stable Diffusion 8.5 (depending on whether the future is corporate or hobbyist).
What is this novel?
Of course, we can trust the megacorps to keep our best interests at heart. It's a well known fact they always consider the long-term effects of their business practices and don't try to extract as much rent as possible from everyone for themselves and their shareholders.
The issue now is, due to the huge cost of training models, the forefront of AI is largely dominated by a small number of corporations with enough capital. They are already using it to extract even more capital, by selling use of these models which are trained on other people's work, largely uncompensated and without consent.
So yeah, people are rightfully miffed about that, and don't want this tech solely in the hands of and benefit of this lot.
And we're begging another corp to please do something for us and are mad when they care instead about their bottom line. Isn't that hilarious?
I'm not [mad]. Glad Mozilla is making an effort. I don't know what they'll accomplish, but hopefully something good comes of it.
The problem is that even if we were to assume AI doomers are completely right, the current approach to AI "safety" is absolutely, completely powerless to do anything that would actually matter to stop it or make it safer. Writing long articles about the theoretical threats or the ethics of AI is fine, but to call it AI safety is misleading. At best, you can censor the output of the models but imo that has nothing to do with "safety".
How is corporate-owned AI going to have a less destabilizing effect?
I don't think that refusing to speak about H.P. Lovecraft or teach you how to make a bomb will mitigate any risk, especially since you can coax it into revealing that information using properly worded jailbreak prompts.
The idea is that if it's solely corporate owned maybe AGI happens a few years later than if it was open-sourced, which gives time for alignment research to potentially come up with a solution to creating a safe AGI, which should be harder than just creating an AGI with no safety guaranteed.
I'm somewhat doubtful that useful alignment research can really be done without actually knowing what the exact architecture of the AI is, but I can see how it would be positive on net if you think unsafe AGI is really likely and really dangerous.
If AI is invented, it seems better to keep it fragmented. The real danger is one selfish AI to rule us all.