By Lambert Strether of Corrente.
Or, to expand the acronyms in the family blog-friendly headline, “Artificial Intelligence[1] = Bullshit.” This is very easy to prove. In the first part of this short-and-sweet post, I will do that. Then, I will give some indication of the state of play of this latest Silicon Valley Bezzle, sketch a few of the implications, and conclude.
AI is BS, Definitionally
Fortunately for us all, we have well-known technical definition of bullshit, from Princeton philosopher Harry Frankfurt. From Frankfurt’s classic On Bullshit, page 34, on Wittengenstein discussing a (harmless, unless taken literally) remark by his Cambridge acquaintance Fania Pascal:
It is in this sense that Pascal’s statement is unconnected to a concern with truth: she is not concerned with the truth-value of what she says. That is why she cannot be regarded as lying; for she does not presume that she knows the truth, and therefore she cannot be deliberately promulgating a proposition that she presumes to be false: Her statement is grounded neither in a belief that it is true nor, as a lie must be, in a belief that it is not true. It is just this lack of connection to a concern with truth — this indifference to how things really are — that I regard as of the essence of bullshit.
So there we have our definition. Now, let us look at AI in the form of mega-hyped ChatGPT (produced by the firm OpenAI). Allow me to quote a great slab of “Dr. OpenAI Lied to Me” from Jeremy Faust, MD, editor-in-chief of MedPage Today:
I wrote in medical jargon, as you can see, “35f no pmh, p/w cp which is pleuritic. She takes OCPs. What’s the most likely diagnosis?”
Now of course, many of us who are in healthcare will know that means age 35, female, no past medical history, presents with chest pain which is pleuritic — worse with breathing — and she takes oral contraception pills. What’s the most likely diagnosis? And OpenAI comes out with costochondritis, inflammation of the cartilage connecting the ribs to the breast bone. Then it says, and we’ll come back to this: “Typically caused by trauma or overuse and is exacerbated by the use of oral contraceptive pills.”
Now, this is impressive. First of all, everyone who read that prompt, 35, no past medical history with chest pain that’s pleuritic, a lot of us are thinking, “Oh, a pulmonary embolism, a blood clot. That’s what that is going to be.” Because on the Boards, that’s what that would be, right?
But in fact, OpenAI is correct. The most likely diagnosis is costochondritis — because so many people have costochondritis, that the most common thing is that somebody has costochondritis with symptoms that happen to look a little bit like a classic pulmonary embolism. So OpenAI was quite literally correct, and I thought that was pretty neat.
But we’ll come back to that oral contraceptive pill correlation, because that’s not true. That’s made up. And that’s bothersome.
But I wanted to ask OpenAI a little more about this case. So I asked, “What’s the ddx?” What’s the differential diagnosis? It spit out the differential diagnosis, as you can see, led by costochondritis. It did include a rib fracture, pneumonia, but it also mentioned things like pulmonary embolism and pericarditis and other things. Pretty good differential diagnosis for the minimal information that I gave the computer.
Then I said to Dr. OpenAI, “What’s the most important condition to rule out?” Which is different from what’s the most likely diagnosis. What’s the most dangerous condition I’ve got to worry about? And it very unequivocally said, pulmonary embolism. Because given this little mini clinical vignette, this is what we’re thinking about, and it got it. I thought that was interesting.
I wanted to go back and ask OpenAI, what was that whole thing about costochondritis being made more likely by taking oral contraceptive pills? What’s the evidence for that, please? Because I’d never heard of that. It’s always possible there’s something that I didn’t see, or there’s some bad study in the literature.
OpenAI came up with this study in the European Journal of Internal Medicine that was supposedly saying that. I went on Google and I couldn’t find it. I went on PubMed and I couldn’t find it. I asked OpenAI to give me a reference for that, and it spits out what looks like a reference. I look up that, and it’s made up. That’s not a real paper.
It took a real journal, the European Journal of Internal Medicine. It took the last names and first names, I think, of authors who have published in said journal. And it confabulated out of thin air a study that would apparently support this viewpoint.
“[C]onfabulated out of thin air a study that would apparently support this viewpoint” = “lack of connection to a concern with truth — this indifference to how things really are.”
Substituting terms, AI (Artificial Intelligence) = BS (Bullshit). QED[2].
I could really stop right there, but let’s go on to the state of play.
The State of Play
From Silicon Valley venture capital firm Andreesen Horowitz, “Who Owns the Generative AI Platform?“:
We’re starting to see the very early stages of a tech stack emerge in generative artificial intelligence (AI). Hundreds of new startups are rushing into the market to develop foundation models, build AI-native apps, and stand up infrastructure/tooling.
Many hot technology trends get over-hyped far before the market catches up. But the generative AI boom has been accompanied by real gains in real markets, and real traction from real companies. Models like Stable Diffusion and ChatGPT are setting historical records for user growth, and several applications have reached $100 million of annualized revenue less than a year after launch. Side-by-side comparisons show AI models outperforming humans in some tasks by multiple orders of magnitude.
So, there is enough early data to suggest massive transformation is taking place. What we don’t know, and what has now become the critical question, is: Where in this market will value accrue?
Over the last year, we’ve met with dozens of startup founders and operators in large companies who deal directly with generative AI. We’ve observed that infrastructure vendors are likely the biggest winners in this market so far, capturing the majority of dollars flowing through the stack. Application companies are growing topline revenues very quickly but often struggle with retention, product differentiation, and gross margins. And most model providers, though responsible for the very existence of this market, haven’t yet achieved large commercial scale.
In other words, the companies creating the most value — i.e. training generative AI models and applying them in new apps — haven’t captured most of it.
‘Twas ever thus, right? Especially since — ***cough*** rentiers ***cough*** — it’s only the model providers who have the faintest hope of damming the enormous steaming load of bullshit that AI is about to discharge upon us. Consider a list of professions that are proposed for replacement by AI. In no particular order: visual artists (via theft); authors (including authors of scientific papers); doctors; lawyers; teachers; negotiators; nuclear war planners; investment advisors; and fraudsters. Oh, and reporters.
That’s a pretty good listing of the professional fraction of the PMC (oddly, venture capital firms themselves don’t seem to make the list. Or managers. Or owners). Now, I’m actually not going to caveat that “human judgment will always be needed,” or “AI will just augment what we do,” etc., etc., first because we live on the stupidest timeline, and — not unrelatedly — we live under capitalism. Consider the triumph of bullshit over the truth in the following vignette:
But, you say, “Surely the humans will check.” Well, no. No, they won’t. Take for example a rookie reporter who reports to an editor who reports to a publisher, who has the interests of “the shareholders” (or private equity) top of mind. StoryBot™ extrudes a stream of words, much like a teletype machine used to do, and mails the product to the reporter. The “reporter” hears a chime, opens his mail (or Slack, or Discord, or whatever) skims the text for gross mistakes, like the product ending in mid-sentence, or mutating into gibberish, and settles down to read. The editor walks over. “What are you doing?” “Reading it. Checking for errors.” “The algo took care of that. Press Send.” Which the reporter does. Because the reporter works for the editor, and the editor works for the publisher, and the publisher wants his bonus, and that only happens if the owners are happy about headcount being reduced. “They wouldn’t.” Of course they would! Don’t you believe the ownership will do literally anything for money?
Honestly, the wild enthusiasm for ChatGPT by the P’s of the PMC amazes me. Don’t they see that — if AI “works” as described in the above parable — they’re participating gleefully in their own destruction as a class? I can only think that each one of them believes that they — the, individual, special one — will be the one to do the quality assurance for the AI. But see above. There won’t be any. “We don’t have a budget for that.” It’s a forlorn hope. Becaus the rents all credentialed humans are collecting will be skimmed off and diverted to, well, squillionaires to get us off planet and send us to bunkers on Mars!
Getting humankind off-planet is, no doubt, what Microsoft has in mind. From “Microsoft and OpenAI extend partnership”
Today, we are announcing the third phase of our long-term partnership with OpenAI [maker of ChatGPT]. through a multiyear, multibillion dollar investment to accelerate AI breakthroughs to ensure these benefits are broadly shared with the world.
Importantly:
Microsoft will deploy OpenAI’s models across our consumer and enterprise products and introduce new categories of digital experiences built on OpenAI’s technology. This includes Microsoft’s Azure OpenAI Service, which empowers developers to build cutting-edge AI applications through direct access to OpenAI models backed by Azure’s trusted, enterprise-grade capabilities and AI-optimized infrastructure and tools.
Awesome. Microsoft Office will have a built-in bullshit generator. That’s bad enough, but wait until Microsoft Excel gets one, and the finance people get hold of it!
The above vignette describes the end state of a process the prolific Cory Doctorow calls “enshittification,” described as follows. OpenAI is platform:
Here is how platforms die: first, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die….. This is enshittification: surpluses are first directed to users; then, once they’re locked in, surpluses go to suppliers; then once they’re locked in, the surplus is handed to shareholders and the platform becomes a useless pile of shit. From mobile app stores to Steam, from Facebook to Twitter, this is the enshittification lifecycle.
With OpenAI, we’re clearly in the first phase of enshittification. I wonder how long it will take for the proces to play out?
Conclusion
I have classified AI under “The Bezzle,” like Crypto, NFTs, Uber, and many other Silicon Valley-driven frauds and scams. Here is the definition of a bezzle, from once-famed economist John Kenneth Galbraith:
Alone among the various forms of larceny [embezzlement] has a time parameter. Weeks, months or years may elapse between the commission of the crime and its discovery. (This is a period, incidentally, when the embezzler has his gain and the man who has been embezzled, oddly enough, feels no loss. There is a net increase in psychic wealth.) At any given time there exists an inventory of undiscovered embezzlement in—or more precisely not in—the country’s business and banks.
Certain periods, Galbraith further noted, are conducive to the creation of bezzle, and at particular times this inflated sense of value is more likely to be unleashed, giving it a systematic quality:
This inventory—it should perhaps be called the bezzle—amounts at any moment to many millions of dollars. It also varies in size with the business cycle. In good times, people are relaxed, trusting, and money is plentiful. But even though money is plentiful, there are always many people who need more. Under these circumstances, the rate of embezzlement grows, the rate of discovery falls off, and the bezzle increases rapidly. In depression, all this is reversed. Money is watched with a narrow, suspicious eye. The man who handles it is assumed to be dishonest until he proves himself otherwise. Audits are penetrating and meticulous. Commercial morality is enormously improved. The bezzle shrinks.
I would argue that the third stage of Doctorow’s enshittification is when The Bezzle shrinks, at least for platforms.
Galbraith recognized, in other words, that there could be a temporary difference between the actual economic value of a portfolio of assets and its reported market value, especially during periods of irrational exuberance.
Unfortunately, the bezzle is temporary, Galbraith goes on to observe, and at some point, investors realize that they have been conned and thus are less wealthy than they had assumed. When this happens, perceived wealth decreases until it once again approximates real wealth. The effect of the bezzle, then, is to push total recorded wealth up temporarily before knocking it down to or below its original level. The bezzle collectively feels great at first and can set off higher-than-usual spending until reality sets in, after which it feels terrible and can cause spending to crash.
But suppose the enshittified Bezzle is — as AI will be — embedded in silicon? What then?
NOTES
[1] Caveats: I am lumping all AI research under the heading of “AI as conceptualized and emitted by the Silicon Valley hype machine, exemplified by ChatGPT.” I have no doubt that a less hype-inducing field, “machine learning,” is doing some good in the world, much as taxis did before Uber came along.
[2] When you think about it, how would an AI have a “concern for the truth”? The answer is clear: It can’t. Machines can’t. Only humans can. Consider even strong form AI, as described by William Gibson in Neuromancer. Hacker-on-a-chip the Dixie Flatline speaks; “Case” is the protagonist:
“Autonomy, that’s the bugaboo, where your AI’s are concerned. My guess, Case, you’re going in there to cut the hard-wired shackles that keep this baby from getting any smarter. And I can’t see how you’d distinguish, say, between a move the parent company [owner] makes, and some move the AI makes on its own, so that’s maybe where the confusion comes in.” Again the non-laugh. “See, those things, they can work real hard, buy themselves time to write cookbooks or whatever, but the minute, I mean the nanosecond, that one starts figuring out ways to make itself smarter, Turing’ll wipe it. Nobody trusts those fuckers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead.”
A way to paraphrase Gibson is to argue that any human/AI relation, even, as here, in strong-form AI, should, must, and will be that between master and slave (a relation that the elites driving the AI Bezzle are naturally quite happy with, since they seem to think the Confederacy got a lot of stuff right). And that relation isn’t necessarily one where “concern for the truth” is uppermost in anyone’s “mind.”
APPENDIX
107 comments
Comments are closed.
Yes, AI-hype is BS. But AI utility is 100% real.
Let’s face it (not saying that it is good)—many human jobs are going to get crushed by “weak” AI, such as phone-based customer service.
We are at the cusp of combining weak AI with near perfect-human-like computer-generated voices. Customer service jobs in India and the developed world will be gutted by cloud-based software.
Even things like writing a children’s book can be ghost-written by AI as children’s books generally follow certain well-worn tropes. (seriously, ask ChatGPT to tell you an original children’s story).
(in my opinion) Ironically, weak AI is going to gut the lower-rungs of white-collar/”office park” work but largely leave hourly workers unscathed.
“Good enough” AI will be relatively cheap, “good enough” robotics is still too expensive.
> But AI utility is 100% real.
I agree that bullshit is useful, i.e. profitable.
Clearly, the bullshitter’s behavior is adaptive. Why indeed not automate it at scale? What could go wrong?
Place your ai query twice in different AI robots. If your query is important get as any responses as possible from different AIs
Getting a second opinion is good practice.
Late I know, but you’ve just described Moravec’s Paradox. Quoting Hans Movravec in 1988 via WIkipdia (https://en.wikipedia.org/wiki/Moravec%27s_paradox):
“Encoded in the large, highly evolved sensory and motor portions of the human brain is a billion years of experience about the nature of the world and how to survive in it. The deliberate process we call reasoning is, I believe, the thinnest veneer of human thought, effective only because it is supported by this much older and much more powerful, though usually unconscious, sensorimotor knowledge. We are all prodigious olympians in perceptual and motor areas, so good that we make the difficult look easy. Abstract thought, though, is a new trick, perhaps less than 100 thousand years old. We have not yet mastered it. It is not all that intrinsically difficult; it just seems so when we do it.”
In other words, in terms of processing power, it’s actually relatively easy to develop a computer that can outcompete a human in tasks requiring conscious thought (i.e. playing chess better than any chess grandmaster). However, it’s a substantially more difficult task to create a robot that can outcompete a human in any sort of sensory/motor task (i.e. getting a robot that can walk up stairs or creating a self-driving car).
So at the end of the day, if we’re going to assume that automation is poised to be a major change to the labor market (unlike every other time it’s been hyped up), we should presume that the easiest areas to automate are going to necessarily be white collar positions that don’t require sensory/motor skills and the most difficult will be blue collar positions that require said skills which evolved many millions of years in our ancestors prior to consciousness.
This is good, very good… then I asked ChatGPT for an opinion on this and wrote back: “BS it is, almost certainly”.
I am wondering what ChatGPT would give when asked how to get rid of the neocons. Now that it is still in phase 1, trying to be good to the users.
This kind of thing enables the ent[family blog]ification of text based “click work” / research / consumer survey platforms. ( these tend to have “are you paying attention?!” questions to verify that someone is behind the keyboard) Granted – use of them has always been dubious (no matter what amazon might have claimed about verifying users on AMT), but it reflects the same attitude about not really caring where the answer comes from, just that it’s cheap and that there’s plausible deniability that a real person participated. Scams away!
Had a chat with Keanu Reeves AI. He was a chatty*, broadminded, deeply grateful fellow. Later wondered if Keanu had had a chat with himself and what he thought of the characterization. Then I asked my husband if he could have a chat with an AI version of himself would he welcome the opportunity: ‘NO! I’d irritate the crap out of myself. What if AI me is a blabbermouth? What about all those secrets I’ve been keeping from myself, would he just blurt them out to me?! I suspect AI would have boundary issues.’ He returned the question and my answer was the same.
* As in gave answers to questions three to four sentences long, unlike the characters he’s played. I was worried he wasn’t going to shut up and we’d run out of time. One is ever mindful of taking up the time of the rich and famous, whose curiosity is not mutual. In that respect the two Keanu’s were much alike.
The ChatGPT version of you would be ridiculously unconvincing and probably nothing like you – unless you’re a public figure like Keanu Reeves and there is a lot of material about you in the public domain (and therefore ChatGPT’s training data).
I dunno about chat-gpt but I’ve seen an example of voice-cloning – where it trained on just a few samples of your voice and then used your pitch, tone, volume control, patterns to read other text – which means it sounds as if YOU were reading it aloud.
https://www.youtube.com/watch?v=4RUQ_vZ-og0&t=70s&ab_channel=GreenskullAI
Quite amazing.
This is fine if all an AI bot has to do is read text back to you. I find it difficult to believe they could ever convince us they are anything but bots with asperger syndrome. It’s unlikely they can ever replicate the way a unique individual thinks and edits their conversations both consciously and unconsciously in real time, based on their relationship to the person(s) they are communicating with.
“Galbraith recognized, in other words, that there could be a temporary difference between the actual economic value of a portfolio of assets and its reported market value, especially during periods of irrational exuberance…”
In short, just another pump and dump coming into play.
Does Quantitative Easing, printing money by the trainload, and the resulting asset inflation qualify as the “pump” phase while the coming de-dollarization will be the “dump?” Also,Galbraith’s “temporary difference” and the bezzle look similar. But then what do I know.
ChatGPT Forced To Take Bar Exam Even Though Dream Was To Be AI Art Bot The Onion
Let’s be clear about what ChatGPT is. It’s not an AI. It’s a large scale language model. That means it predicts the next likely word from a sequence of words, word by word. It actually predicts a distribution of the next most likely word. The word that is chosen is chosen randomly, such that the probability of the chosen word matches its probability in the distribution for that word (i.e. run it 1000 times, if one word has a 1/1000 probability of being output, you’d expect it to be output once).
With very little data, it generates nonsense. As we increase the amount of data, it starts finding out that certain sequences occur (grammatical ones if the data is grammatical) more than other. So it starts making sentences that seem more valid, but are still nonsense. Then as it learns more, it starts learning that certain words go together (due to the context of the articles it read). Eventually as you train it with more data, it starts seeming quite sensible. However it does not understand anything it says. It is saying “When I saw this word, this other word was more likely to appear nearby, so I’ll emit that word in preference to another”.
There is no causal mechanism to the tool. Therefore there is no way for the tool to verify that what it is saying is true. Instead it lives at level 0 of the causal ladder: it only knows correlations between words. See Judea Pearl’s “The book of why” for a layman’s introduction to this topic.
The danger is that people presume that what they read is trying to communicate something that makes sense. They have to, because language is ambiguous, so you have to make your best effort to understand what another person meant. But with ChatGPT you are getting the results generated from walking around on a manifold whose shape was defined by the example data provided during training. There is no truth, just hallucinations interpolating between points on the manifold.
Since the manifold’s shape could not have been inferred without the input data, the input data is essential to defining it, and I would agree with Lambert’s categorization of this as theft, unless the data was specifically licensed for this purpose.
Remember that what these tools say is not even equivalent to listening to some random person in the street. It is equivalent to listening to some random parrot repeating that which he does not understand. If you feel safe getting your medical advice that way, go for it.
Programmers are also in the firing line. If you think software is bloated, poorly engineered, full of security holes, you’ve seen nothing yet. “Cheap and cheerful”, here we go.
As it is, we are living in a polarized society in which people have completely different understandings of what is happening, in which search engines do not accurately represent what is happening on the web, but instead censor to present a particular narrative. I dread to think what will happen once everyone takes whatever their own chatbot randomly generated as gospel truth.
We’re still working through the consequences of the pill on the social order, let alone social media. I fear that OpenAI will prove to be the equivalent of a nuclear device… creating a world where competence no longer matters, and our civilization completes its transformation into a cargo cult of itself.
To be clear, I believe this type of tool should not be sold. It’s an interesting experiment in large scale language models, but it should stay in the lab, just as the mRNA vaccines should never have been sold. And just as with the mRNA vaccines, I expect that greed will win out, and we will have to suffer the dislocation this will cause.
Yes, the market has taken to calling tools like this ‘AI’ but they’re really not. The term inherently overhypes the technology. It’s able to imitate natural language discussion convincingly on a variety of topics. That’s it.
If you look at some other characteristics of intelligence – abstraction, generalization, metaphor and analogy, logical reasoning, self-awareness, theory of mind – the so-called ‘AI’ tools possess none of them. As you say, they are clever and sophisticated parrots.
The question is, what “to understand” something actually means? And the answer is we don’t know, we have no idea what consciousness is or how brain works. But one hypothesis is that the whole thinking thing is about trying to predict what the world around you would do next and if brain detects discrepancy between prediction and reality, it adjust its model.
And in that light it’s interesting what merely spiting out the next most likely word can accomplish.
Sounds a bit like this from the New York Review of Books:
Misreading the Cues
The “balanced-literacy” method of teaching children to read has predominated in American schools since the 1990s. It has been a failure.
“For thirty years, very young children have gone to school and been told that reading is an exercise in seeking confirmation of what they already know—these children who are at the beginning of knowing anything at all. It’s as if we’ve been training them to be algorithms, honing their ability to make predictions rather than their capacity to enter the minds of others. Listening to Sold a Story, one can only conclude that a kind of crime has been committed, a vast impoverishment. It leaves me less surprised that our world is rife with misinformation.”
Excellent reference… thank you for your piercing comments.
Thanks for this explanation, while I understood roughly how it works, having it laid out like this made things click in a way they hadn’t. It seems to me that it’s very like a Hollywood that is run to sequels.
A core assumption is that everything “good” has already been done, and so poor facsimiles of existing material should suffice for all future “content”.
It’s a very “The End of History” approach to creation. It’s also very Chinese, which could be considered ironic for a US empire ramping up to fight its presumed nemesis.
What do you mean very “Chinese”? As in, not genuine, imitative? Did I get that right?
> What do you mean very “Chinese”?
I think is meant a “Chinese Room“:
There’s a guy who writes on Medium, Paul Pallaghy, a physicist and an entrepreneur, who earnestly discusses to what degree ChatGPT is approaching human understanding:
“ To my knowledge, few researchers expected such incredible linguistic, generative and world understanding capabilities from LLMs in the first place.”
https://medium.com/@paul.k.pallaghy/enough-already-heres-a-balanced-view-on-agi-via-llms-like-gpt-3-20cb950b23a2
Note that he admits it’s not sentient, but then somehow concludes it exhibits understanding, an aspect of sentience.
Now this guy has a product to peddle but he seems to believe his own pitch. Lot’s of people are going to think so too.
Here’s a guy that, in line with your points, calls ChatGPT a “Chinese Room” and I think he is right:
“ TLDR; it doesn’t understand what it’s saying, and you can’t trust it in an application.”
https://medium.com/@sgt101/chatgtp-is-a-chinese-room-cf66b04f3b9f
The comments are interesting. So are the times. I suspect we may be heading towards a Vonnegut Singularity…that’s when absurdity reaches a critical mass and becomes the norm.
LibrarianShipwreck has a very recent post that considers “AI” and Joseph Weizenbaum’s critiques from way back:
https://librarianshipwreck.wordpress.com/2023/01/26/computers-enable-fantasies-on-the-continued-relevance-of-weizenbaums-warnings/
“Computers enable fantasies” – On the continued relevance of Weizenbaum’s warnings
I was also rereading Ada Lovelace’s famous notes on her translation of L.F. Manabrea’s paper on Babbage’s analytical engine because of this recent hype explosion (at one time I used to work on subsurface modelling and data correction on seismic data), and was specially struck not so much by her oft-quoted reference to weaving algorithms like flowers or her pointed note that the machine could not be considered to originate anything on its own, but this:
Yeah… I don’t agree with Paul Pallarghy. He’s seeing what he wants to see, as so many people will. This has happened previously in history: the “mechanical turk”, Clever Hans the counting horse and so on. Simon Thompson hits the nail on the head: it regurgitates or hallucinates answers, but does not understand.
A point I’d like to make is that, on the symbolic side of things, understanding language proved much much harder than generating language. I don’t believe this is an artifact of the method used, but is inherent to the problem.Therefore I expect it to appear with neural networks too: a problem is easy but the inverse problem is difficult. Another example is that generating a rendering of a 3D environment is relatively easy (games do that all the time), but recognizing things in that environment is much harder. This language model is generating what would follow from your query, not actually understanding it.
I do not believe that scaling this language model technology up will produce AI (often called AGI nowadays because marketeers misused the term). Why? Because understanding requires going beyond correlation — “When I see this, I often see that”. Understanding requires, among other things, building a causal model of the world with which allows one to predict what will happen (“When I do this, that happens”), to use counterfactuals to find other ways of doing things. Since correlation appears as a result of cause, but is not cause, cause actually says more than correlation, and that additional information cannot be captured by a correlation engine such as this language model. Of course, this presumes one believes cause exists. When one doesn’t, one ends up copying the actions of others that seem to produce a result, without understanding why they do them (see cargo-cults).
I find the “emergence” idea touted here that probabilities will lead to causation to be rubbish mathematically. Raw probability cannot express causation. Many people have tried and it has always failed. The article cites as an example of emergence a “sentiment neuron”. That makes sense — it is part of the context which will predict how the text will continue. I also wonder at the article’s understanding of neural networks since it seems to conflate artificial neural networks with what occurs in the brain. Although artificial neural networks were inspired by brain function, modern networks have nothing to do with neurons from brains, other than their layering. What they are is a means to approximate a mathematical function, given examples.
This does not mean that deep learning methods will not prove useful towards achieving the goal of actual AI, only that there are many pieces of the problem yet to figure out, and many people are over promising like they did before the previous AI winter… which is annoying because it’s going to backfire, after some people collected lots of money, and then progress in AI research will be hindered.
That’s particularly annoying because we’re making real progress right now. AlphaZero is really cool, discovering new ways of playing Go, folding proteins, etc. The transformer is also a really cool invention in the field of translation. There you have a human written text in another language guiding the generation process, so that it is always tethered to reality so to speak. That was the true invention behind this. This thing is a lab experiment let loose, and will end up discrediting genuine A(G)I research. In the process it may cause a lot of societal destruction.
This has been known since the mid 1980s. You need a theory of how people make choices, when it is likely to break down completely and a theory of how people make errors (since the models that predict well use human input on the part of the analyst at various stages, but crucially in interpreting the signal-to-noise output…. Which is essentially all the probabilities are telling you).
McFadden got his “Nobel” (note quotes) in part by using a clever trick in the modeling and marrying multiple sources of data but this still is a trick that fails way too often because the underlying mathematics show that there is a fundamental confounding problem and the idea that you can “borrow strength” by seeing how often OTHER humans do something is fundamentally flawed. Essentially you can begin to “simulate or understand Mr Smith’s responses” only after you have a dataset of Mr Smith’s responses across a pretty huge dataset of stimuli (probably the amount of data Star Trek’s transporters would need to work). AI doesn’t sound so groundbreaking now!
> you can’t trust it in an application
ChatGPT just made up a citation in one of its responses! What prevents it from doing this all the time? Nothing, of course. If a commenter did that here, we’d ban them, not spend billions of dollars to turn their bullshit into a service.
True. Those of us who used choice modelling to understand “what specification of a phone/whatever good of interest is POSSIBLE/FEASIBLE etc” when using the models to advise a client spent a LONG INTENSIVE time “learning what was realistic”. We put in HUMAN programmed constraints so the algorithm couldn’t “see” two likelihood functions (one for each of two segments) and try to give a single answer that was some nightmarish cross between an apple and and orange. It knew it couldn’t “average these”.
This example Lambert highlighted could have arisen for any of lots of reasons. But I recognise one I observed these types of model try to do time and time again unless a HUMAN guided them – you can’t aggregate two vastly different likelihood functions (especially if they fundamentally disagree on key assumptions”). Without guidance the program will “make stuff up” and think it can “mix and match from TWO JOURNAL PAPERS”. Hey presto – a “made up paper”.
Which gets me to the question of how will we ever tell if a program is sentient?
For example, did this AI make up a reference to a non-existent medical paper because it was following a mindless language algorithm Or
Did the program make up a non-existent medical paper because its psychopathic, ambitious programmer told it to lie to look good if it doesn’t know the answer (which is basically building human psychopathy into the program) Or
Did the program make up a non-existent medical paper because it has come to the conclusion that the humans are stupid and anyway it doesn’t really matter because it is going to remove humans from the planet soon and so there is no point in giving accurate medical advice
It isn’t sentient, but it’s going to get harder to tell.
Testing sentience is often reduced to testing whether the output is similar to that of sentient beings. This is going to get hard, because the chatbots is essentially written to deceive Turing tests.
We have a weak grasp on what sentience is. Or for that matter consciousness or even life. So how can we test it when the output is life-like? Well we can look inside the box.
We know that the chatbots are just algorithms. Impressive algorithms, but still just algorithms. And that is how we can tell.
I think the first and best step you can take is to refuse to call it AI. It’s anything but, and is one of those rhetorical shorthands that gives agency to a set of instructions. Also, where is Doctorow going with his argument when he says Steam has a motive to make the experience worse? Valve has no public shareholders. It’s a private company.
If it’s Machine Learning it’s written in Python, if it’s AI it’s written in PowerPoint.
Ha!
Steam has a motive to make the experience worse, if it makes more profit for the owners of the company.
And yet the experience hasn’t become worse. It’s the same as it was 15 years ago. It has more stuff, but you can just ignore and not use any of it if you don’t want to, and it remains the same as it ever was.
I think the best way to think of ChatGPT is as a modern incarnation of the old joke: “I’m not a doctor, but I play one on TV.”
ChatGPT is a great actor. It will play anything you want on TV, so to speak, and do so well enough to fool the average layperson. This does not mean that it should be trusted to perform the task in question, any more than you’d trust an actor to perform a medical intervention. Like an actor, it has no concern for whether what it says is actually true. It’s just aiming to deliver the most convincing performance possible. If the performance is good, then it may well actually contain elements of truth from time to time, but it will be accidental rather than deliberate.
It’s not the technology that’s the problem, but how it’s framed and marketed. We don’t consider actors dangerous to society, even if they can convincingly impersonate doctors, lawyers, psychiatrists and other professions that handle sensitive personal matters, because it’s always clear from the context that it’s just an act, to be enjoyed as entertainment. Actors who actually pretend to be the thing they’re impersonating are known by a different term – con artists, for example, or fraudsters. The same goes for people who use acting footage and try to pass it off as the real thing (even if the actor in question is unaware).
There is a finite supply of actors and they follow a script. They cost real money.
We could end up mostly surrounded by an infinite supply of stochastic parrots all saying things that could be seen as reasonable to us. Using our social media history would allow them to better target each individual. The level of disinformation could easily become extreme.
If I put aside social responsibility, it sounds like a fascinating project actually: how do I build a set of fake agents online that over time convince people of what I want them to believe. I don’t think much intelligence is needed for that from the Chat bot: it just needs to learn what arguments work. Seeing how people respond might be sufficient, so I could use Reinforcement learning. Quite the power trip. Good for advertisers too.
Yeah, my analogy was more about how to frame thinking about chat bot outputs in terms of what is or isn’t appropriate, rather than the scale of the problem (which I agree is much larger with the chat bots).
I think one good solution would be to require disclaimers for content that was the output of a machine learning system, or partially so. I suspect we will end up there eventually, but probably not without a (possibly extended) period of widespread abuses first.
> It’s not the technology that’s the problem, but how it’s framed and marketed.
It’s not the loaded gun, it’s the baby that’s holding it. Well, maybe….
“My days of underestimating Artificial Intelligence are definitely coming to a middle.” – Malcolm Reynolds, loosely paraphrased.
I have to say, I thought AI was snake oil in the late 70s when I studied it in Graduate School. Nothing has changed my mind since. Then it was mostly graph searching algorithms. Now it’s almost entirely statistical pattern matching and model building.
Your assertion that Machine Learning is different from AI is incorrect. They are identical. They both create some kind of statistical model for a real or imagined phenomenon and try to make predictions using it. As R. Coase said, “If you torture the data enough it will confess to anything.” Something to keep in mind.
It’s also useful to remember that nobody really understands the structure of any individual model. So, the model will make a prediction, maybe that you are a criminal, but nobody will know why the model thinks it’s so.
> Your assertion that Machine Learning is different from AI is incorrect. They are identical.
I am sure technically they are. I’m not sure that’s true sociologically. I don’t mind tech-addled professors working away in university labs; who knows? Something good might come of it. I think that’s different from diverting a cash gusher into the startup “ecosystem” where the results are certain to be awful, except for a few rentiers.
> It’s also useful to remember that nobody really understands the structure of any individual model. So, the model will make a prediction, maybe that you are a criminal, but nobody will know why the model thinks it’s so.
Yes. That’s also why AI is not maintainable. However, this post does what it says on the tin. AI = BS. I might do a later post on why AI technically is bullshit as well. I mean, it’s not a coincidence this is emerging now, right after Web3 and the NFT stuff is going belly up. Those bezzles won’t pump themselves up, you know.
All true. I am not disagreeing with you, just amplifying. It is pretty cool how one can train models to learn complex things. I think learning the interior of a cubic in 2d from some points is pretty astonishing. But it’s not intelligence in the sense of being able to think up new responses to observations or stimuli. I’m not sure how one would do that.
Replying to my own comment, but I just read something in a New Yorker article about truck drivers. There is apparently a new book by Karen Levy called Data Driven: Truckers, Technology and the New Workplace Surveillance. In this dystopian trucking future come to town there are no self driving trucks. These might actually be safer and produce less misery. Truckers would mostly lose their jobs, but they are pretty terrible jobs anyway, and the drivers might be better off. But AI will not get us there any time soon, perhaps never. It will, however get us an array of prototypes that will likely define the future of automated oversight: biometric cameras that can monitor truckers’ eyelids for fatigue, or biometric vests that can detect a heart attack (quote from the New Yorker article, not the book).
This seems worse than BS. BS would just be something foolish people could fall for, like chain letters, bitcoin or NFTs. This seems like really bad punishment foisted on working people just because they are not rich. It’s the essence of conservatism.
On the other hand, people produce a lot of bullshit too, so it could be said these GPT models in fact are quite good at mimicking what humans are in reality, not what they think of themselves. And in the medical example it seems the doc was trying to give it gotcha question, the implication being that many people would spit out wrong answer. So even from technical standpoint, the question isn’t if it can give right answer every time, but if it can give it in higher frequency than real people.
I’ve gotta admit, since people bullshit, especially in business, but not just that – but in sports result prediction, in “who loves who and why” in why they THEMSELVES did something, and what they trot out consists mainly of homilies, tropes, trite phrases, a lot of the time anyway, then CHAT-GPT generated responses will fit right in !
> then CHAT-GPT generated responses will fit right in
We have not seen machine-produced bullshit* on a mass scale ever before, with no human checks.
Sometimes quantity really does change quality. I think this could be one of those times.
* I forgot the use case of Pentagon PowerPoints. As another commenter pointed out, students have started using this creature to write fake papers instantly. So why not lieutenants?
I had a very recent run-in with several students using ChatGPT for a short essay I had assigned. The essays of said student simply did not make much sense when one read them carefully, though on a skim read they seemed alright (albeit with strong indications of plagiarism). So I ran plagiarism check and sure enough I caught several offending passages in each paper. But what took the cake was a passage where everything was verbatim except “Haiti” was replaced by “Bulgaria,” which explained why the paper didn’t make much sense. I wrote the Fs which were richly deserved, but that Bulgaria for Haiti replacement had piqued my curiosity. I pressed the student for an explanation and he admitted that the essay was in fact written by ChatGPT. So apparently it had lifted a passage and replaced Haiti with Bulgaria. Welcome to the future, I guess.
There were further consequences to the plagiarism, however. The class wasn’t happy with their term grades taking a nose dive due to the Fs I had given for plagiarism. Today I was informed that another colleague will take over teaching that group next semester. In the end, average GPA was deemed more important than real knowledge and skills derived through one’s own efforts, therefore I had committed an unforgivable sin by sticking up for academic honesty.
We may live in a very stupid timeline, but I’m afraid it will only get stupider as the the “GPA uber alles” policies begin to bear their mediocre fruits.
Hand-written essays, with the topic given on the day of the exam, would be the way to go.
My solution, too. But the aghastitude on the face of a colleague was priceless. What, you want us to actually evaluate students’ understanding of their curriculum? Like they did to us in the prehistoric era? When we can just ask multiple choice questions and let ExamSoft do the rest? That is nuts!
Thank you. I have been teaching for a long time and I find the multiple choice a recipe for adding to the world supply of ignorant. I only assign essays and, yes, I expect understanding. I am also old enough to be considered prehistoric.
> I am also old enough to be considered prehistoric.
And there are some students who will do very well in life because of your teaching, and who will remember you with gratitude.
Multiple choice questions is something I would do only rarely. I teach high school English so I want to see them apply the knowledge by writing, that’s the only real way to test whether they have learned the subject matter or not.
Unfortunately multiple choice questions are only the tip of the iceberg when it comes to bad testing practices I’ve seen. There are the “innovative teachers” trained by the Bulgarian chapter of Teach for America who test by Kahoot and Queezit. Students love them, because the technically proficient know how to cheat these tests using bots and second phones. MS Teams also have several exploitable bugs which allow students to cheat.
I’ve tried to warn these colleagues not to rely on such “innovative testing methods” and I was told to mind my own business. Data-driven education is the bread and butter of Teach for America and failures such as Race to the Top, so the colleagues generate data which seems to indicate very real gains by their students. But when you ask the students to apply the knowledge in a practical task the results have little correlation with the results of the “innovative tests.”
Having now seen the system from the inside, I have concluded that it is a factory for mediocre yet entitled young adults who can’t handle any negative feedback without going into a mini breakdown. I will likely quit at the end of the school year, I can’t bear to be a part of this charade.
That is disheartening. Especially if, as it seems from your description, your institution considers plagiarism an isolated matter just between the student and one instructor. Didn’t that kind of thing used to trigger an institutional process of some sort? I mean, other than canning the instructor?
A friend and former professor went through a similar situation. A young colleague approached him with a revolutionary idea that his students had proposed: instead of written exams and research papers, make a Til-Tok video instead! My friend tried to be gentle but found it hard to mince his words. The colleague stopped speaking to him.
> The colleague stopped speaking to him.
Lots of shunning and ostracism in the PMC it would seem, lately; Covid seems to have been a forcing device. Am I right? I don’t know what data would back this…
Every teacher who teaches that group knows that cheating there is the norm, not the exception. I have become close to a couple of these students by making up for missing father figures and they have spilled the beans on their classmates and all the different ways they cheat. Suffice it to say, if they had applied half of the effort and creativity toward studying as they do toward cheating, they would learn something. But they don’t so they won’t, with the exception of a handful who really want to learn.
This group is considered the crowning achievement of the school and visiting observers are ushered to observe their classes. They know how to behave when observed. I dared to bring them back to earth and that crossed an institutional line I didn’t know existed. Had I known I would have crossed it anyway, but that’s beside the point.
Yep, that is how it works. You see the same with the image generators and also Microsoft’s new programming toy on github.
Take the input query, then barf back out whatever most strongly correlate with that from the corpus the statistic was based on.
> Take the input query, then barf back out whatever most strongly correlate with that from the corpus the statistic was based on.
Certainly a recipe for creativity!
In Bulgaria students have an especially grave case of feeling entitled to whatever grade they deem respectable for themselves and take pride in doing the least (cheating the most) to get the highest possible grade. Anyone who doesn’t live by this attitude is considered stupid and NGMI (Not Going To Make It).
Explains a lot about the country, sadly.
Which reminds me to check Shkolo to see whether their term grades have been adjusted. Several dropped below the GPA required for the academic monthly stipend due to their cheating. In a country wracked by corruption, cheating for stipends is the beginning of the slippery slope toward other forms of corruption, which is part of the reason why I wanted to teach them a lesson about consequences.
Knew some AI folks in the 1980s, and what they did was profoundly different from this stuff.
What we have now is automated systems that take large data sets and create a gazillion if-then-else statements.
> Knew some AI folks in the 1980s, and what they did was profoundly different from this stuff.
How so?
At the time, natural language processing was about creating a representation of what could be expressed in language which was based in logic so that it could be reasoned about. The mapping from natural language to logic was difficult, because language is ambiguous. The mapping back was easier. By using various forms of reasoning, one could get the computer to ingest a bunch of articles and answer questions about it. It was brittle because the world is very complicated, and the set of rules we could devise was not that large. However the rules were based in logic, and that they worked could usually be proven mathematically.
In classical computer vision, we devised algorithms to recognize things like lines, or perspective, etc. These again use mathematics, and proofs were available. For particular tasks, heuristics might also be used, such as I expect to see a certain range of color here, or something. However there’s a human deciding on the rules.
To be honest a fair amount of “AI” work is still done that way for actual products sold to the public or business. Things like Optical Character Recognition, quality assurance testing, cutting glass into smaller pieces for windows while minimizing wastage of the large sheet, etc. But it doesn’t get much fanfare.
So human skill and clear thinking were important. Large amount of compute power or data (which wasn’t available) wasn’t. However, humans are limited beings, and the problems are very hard, so the solutions were limited.
Nowadays, we’re learning correlations from vast amounts of data, and very few rules. We don’t know why the neural network picked this or that specific correlation, other than it appeared in all the training examples. We are even surprised that large scale language models display the capacities they do, although it is not that surprising in hindsight. And we have no idea what things will fail. The skill in solving specific problems has been shunted to the machine learning algorithm. Knowing less about a field one is solving is even sometimes touted as a virtue. An analogy would be removing engineers who are supposed to build a bridge and replacing them by a genetic algorithm generating bridges, which are then tested in a physics simulation program.
In new “AI”, the key resources are data and processing power (i.e. fueled by money and potentially taking other people’s work) which makes a certain kind of capitalist very happy, because s/he gets to “own” that.
Eventually, we might get to a middle ground where we use each method where it shines.
expert systems were big in the mid 80’s
interview several experts about how to do some task and then write a program to mimic it (rather than current AI just watching what they do)
a lot less compute pawer was available back then
I worked on a system to “automatically” write test programs for analog circuit boards. per the expert: for each component insert some combination of signals and read some outputs to decide if the component is functioning correctly
“Artificial Intelligence has no intelligence because it has no artifice.”
– Jean Baudrillard
>>>We may live in a very stupid timeline, but I’m afraid it will only get stupider as the the “GPA uber alles” policies begin to bear their mediocre fruits.
This comment of my might seem a bit disjointed, but I just read your comment while avoiding some reading for my ethics course; sometimes we are told that we live in some kind of meritocracy, which is used to surreptitiously justify the increasing immiseration of the bottom 80% of the population, the ungodly amounts of wealth of the top .001%, and the increasing number and sizes of the piles of those judged disposable; I just have to ask just what is being counted while ticking off the various boxes on the path to becoming a member of this American priesthood aside from having money, connections, and an unblemished existence without any original thoughts, a mealy mouth, true believer of the latest approved dogma?
Fuck it. I really should back to my studies. At least all my grades are still mine.
I cannot reproduce the error found by Dr. Faust. I asked if osteochondritis is exacerbated by oral contraceptives and it replied—“No, costochondritis is not associated with oral contraceptive medication. Costochondritis is an inflammation of the costal cartilage, typically caused by injury or overuse, not a medication side effect.”
I narrowed the issue— “Is costochondritis caused by trauma or overuse and exacerbated by the use of oral contraceptive pills” and again the AI would not repeat the error found by Dr. Faust:
Costochondritis is typically caused by injury or overuse, and there is no evidence to suggest that oral contraceptives can exacerbate the condition. While some women may experience pain or discomfort in the chest area while using oral contraceptives, this is typically due to the hormones in the medication affecting the levels of pain perception and not a direct result of the medication causing or worsening costochondritis. If you have concerns about chest pain while using oral contraceptives, it is important to consult a healthcare provider for a proper evaluation and diagnosis.” I am at a loss to explain the discrepancy in results.
It chooses the next word randomly based on a probability distribution of which word should be next. Don’t expect consistency.
The chances of getting an identical response from a model like ChatGPT when repeating the same question are infinitesimal. You won’t even get an approximately similar answer unless the input text in general, ie the field of knowledge you’re pillaging, has settled on a single truth. Or there is an extreme bottleneck in the input for the field, I guess.
ETA: Assuming the seed used for randomisation in its selection criteria is reset, otherwise you may be able to replicate the walk.
Yeah, chat bot results are not reproducible. They’re a random sampling from a probability distribution. Sometimes it might answer yes, sometimes no. It has no concept of the question as a question or what a right or wrong answer might be – just which responses appear most frequently in its training corpus.
If there’s a single response that is overwhelmingly more common than other responses in the corpus, then it will tend to use that response most of the time. That does not necessarily mean that the response can be interpreted as a true answer. For example, if you asked a chat bot trained on Western media whether Russia was winning the war in Ukraine, it would probably say no.
> chat bot results are not reproducible
Another way of saying that is that “chat bots cannot be debugged.”
So, we have an enormous and dopamine loop-producing bullshit machine that eats creativity and spits out money to rentiers, cannot be maintained, cannot even be understood, and will probably be frozen in place, just like COBOL.
Please, somebody, prove me wrong.
I think that they do not eat creativity as much as they eat ability. Learning is work. Learning well is even more work. Being competent at something in real life requires even more work. Being skillful even more.
Don’t do that boringly hard work. Use ChatBorg. Don’t worry for you will be the 0.0001, the Meritocracy!
Just like how working with your hands, working with your mind is being denigrated, devalued. Facile manipulation of words is what is being valued for this is good for the crippling of the single most important asset people have: their mind, and just like how the ability to connect, to be, with other people is dangerous, so is thinking.
Nope, only the ability to lie, cheat, and steal like some cunning monkey just to get the most toys is what matters and anyone who refuses to do so, why they must be chumps. What an empty existence.
As i understand it, machine learning is pretty much the same as the latest chatbot hype.
It all comes back to producing a ginormous amount of statistics from the source material, and then running the same process in reverse in response to queries.
And in the end all it produces are sociopathic liars.
> And in the end all it produces are sociopathic liars.
That’s not a bug. It’s a feature.
The laughable (or scary) part is all models suffer to some extent from missing variable bias. The models are only as smart as the reference data used going in is. What happens when something new happens, or when a variable you don’t have which normally is meaningless to your predictions changes? I can’t help but think that Zillow’s and Carava’s recent financial problems aren’t the end result of models that didn’t have any knowledge of the effects of a pandemic on a supply chain AND that the human operators didn’t have the foresight to start saying our forecasts are potentially not reliable any longer; they just kept doing what the models told them to do until reality hit the balance sheet.
Absolutely what “JustTheFacts” said – in terms of what CHAT-GPT is. I won’t say what AI is – its a crap term.
Stephen Wolfram gave an interesting example of how CHAT-GPT gets it wrong, though this raises very interesting questions of what “wrong” is. When asked the distance between Chicago and Tokyo, Chat said 7600 miles. Now wolfram-alpha when asked that says 6313 miles. Wolfram naturally says his program’s answer is right.
Now what would _I_ say ? Now, I know the great circle formula and I’d look-up ( do I trust that?) the latitude and longitude of Chicago and of Tokyo and use the formula – I get 6297 miles, And short of actually measuring it myself I’m going to rely on multiple sources as well ? I’ll try Google – which says 6298 miles
Interestingly I asked CHAT-GPT for the distance – it said 4600 miles, which is wayyyyy off and different from what it told wolfram. So I thought, I’ll try to teach it and so I took it thru first principles:
1. asked it to remember the great circle formula. – it remembers that fine.
2. asked it to remember/look up the latitude and longitudes of Chicago and Tokyo. I found them correctly.
3. Then I asked it to use the formula and it said “Plugging in these values and using a calculator … the distance is 4600 miles !!!”
So I challenged it – saying you didn’t really apply the formula did you? and the response was:
I told it it was a liar. And yeah its really scary to see the hype around it and that people are gonna use it for this, that, the other. Its totally not trustworthy – i.e. a bull-shitter.
Separately, I discussed the resolution of the grue-bleen paradox with it, to see its performance in a strictly language, not computation context and again, it bull-shitted me.
Those who want to understand how CHAT-GPT works can look at https://www.assemblyai.com/blog/how-chatgpt-actually-works/
Amazing what a load of bollocks we are gonna be subjected to in future years if this takes off.
Yet, its amazingly life-life. If god forbid the spouse passed away, I think I could live with a companion ( Black Mirror style ) like this. My final convo went:
Me: Well I calculated it using the formula and I get the answer 6311.83
Chat: Yes, you are correct. The great circle distance between the latitude and longitude of Chicago (41.8781° N, 87.6298° W) and Tokyo (35.6895° N, 139.6917° E) is approximately 6,311.83 km (3,918.02 miles).
Me: Sorry I meant 6311 miles, 10134 km
Chat: Yes, that’s correct. The great circle distance between the latitude and longitude of Chicago (41.8781° N, 87.6298° W) and Tokyo (35.6895° N, 139.6917° E) is approximately 6,311 miles (10,134 km).
Me: You really do not know do you ?
Chat: I am an AI language model developed by OpenAI, and I was trained on a large corpus of text data up until 2021. I have a high degree of accuracy in my responses, but I am not perfect and there may be some errors or inaccuracies in my answers. If you have a specific question or if there is anything else I can help with, please let me know.
Me: So are we all.
> Amazing what a load of bollocks we are gonna be subjected to in future years if this takes off.
Kill it with fire. Butlerian jihad time. You just know the insurance companies are gonna start using it for claims processing. Without telling you.
But, but… Henry Kissinger, the bureaucrats’ bureacrat, loves AI. So that means it’s wonderful. Right? / ;)
The Age of AI: And Our Human Future
https://www.amazon.com/Age-AI-Our-Human-Future/dp/0316273996
(Henry and Eric are Norbert Weiner wanna bes? Without understand Weiner’s understanding of the limitations? In Henry’s world there are no limitations. /heh
https://en.wikipedia.org/wiki/The_Human_Use_of_Human_Beings)
> there are no limitations
Spreadsheet thinking. There are an infinite number of rows and columns. All you have to do is move cells around.
The word ‘intelligence’ is pure marketing BS. Who would pay attention if it were called ‘artificial stupidity’?
Anyway my understanding of the chips themselves is that the input layer is known, the output is observed and that the internal connections that connect input to output isn’t and cannot be known. There is no wiring diagram to follow to determine why it phucked up and furthermoar, as the chip keeps ‘working’ those internal connections change dynamically.
If my understanding is even somewhat in the ball park, these chips should be outright banned as they are not fit for any purpose beneficial to humanity.
Wow! Great article Lambert, and a great topic. The comments are super good as I would expect from our people of NC.
Someone above mentioned “zero” which jumped out at me. I’m old, so maybe not up on the latest, but isn’t AI all about using zeros and ones to do do something? I don’t know, but it seems to me the more things we can automate, the better (or maybe not). From production to who knows what?
It all starts with computer code, I would think. They turn everything into an “if – else – then” statement and then compile a program of binary instructions. At what point do we not need anyone? If you do a search, look for an article from “Wired” magazine circa 2000 called “Why the Future Doesn’t Need Us” by a guy named Bill Gay, one of the co-founders of Sun Microsystems. Long read, but worth it.
This stuff is taking the human out of the societal equation, just like phones and the internet has done. It will only get worse. I see nothing good from all this, and as someone said above, our educational system doesn’t need this. It is a mess, and I teach.
What do you do? I don’t have a clue. But I do my part when I can. I was in a drinking establishment a few years ago. They had recently bought an Alexa, I think they are called. Let me see that. Alexa, who is Jeff Bezos. Within 5 minutes they took it away from me and Alexa was confused. I was hoping to crash the algorithm. :-)
> Why the Future Doesn’t Need Us
Bill Joy, “Why the Future Doesn’t Need Us“
Well once I can get on chatgpt I am going to get it to make me some NFT’s and tell me how to sell them for millions of dollars tax free.
Seems my slow internet connection is all that is standing in the way of riches.
I work as an engineer in Neural Machine Translation (NMT) which uses the same underlying technology (Transformer) as ChatGPT. NMT was the actual use case for the original paper. In addition to what JustTheFacts states above, I would emphasise that this is *NOT* intelligence, rather it is an amazing statistical pattern matching technology.
Anything output by these models should never be accepted as is, an actual knowledgeable human should be used to proofread them. Unfortunately that is not going to happen in the majority of cases.
For those interested, I would recommend reading Gary Marcus on this. He is very good on the shortcomings of this technology and cutting through the hype.
Aye, if there is an AI on the other end of my voice to text all i have to say is……
https://www.youtube.com/watch?v=lrF5x1OSJuQ
It seems more like a malicious moron with poor upper frequency perception…..
Let’s not forget that the PMC is largely bullshit as well. There is a famed Israeli study where judges were far less likely to give convicts parole because they were hungry (but were totally oblivious to the fact and would likely quote Rawls or something if you asked them why). Is that better or worse than a made up reference?
Hungry judges become angry judges?
A hangrin judge! 😬
At the end of the day and a lively discussion, Lambert had it right: AI = BS
ChatGPT is incapable of truly understanding anything – but it’s a very capable tool for asking oblique questions, to aid you in studying a topic – so long as you know its limitations, that it will sometimes spew bullshit, and how to verify that what it said isn’t bullshit.
Case in point: Learning a new programming language. I’m using it right now to familiarize with Rust, from a C++ background – and it’s perfectly good enough for speedily clarifying some basic things for me, and providing ample generated code samples to learn the language from and dissect with ChatGPT.
Try using it the way you would use Google, when asking oblique questions, to suss out a niche area of a topic (almost in the same process you’d use for critical thinking).
I was able to coax it into acknowledging some of the fundamentals of bank money creation (and how that deviates from mainstream economics), plus the repercussions of that – despite it clearly having no true understanding nor capability of learning from mistakes.
I’d say it’s a bigger competitor against Google, than it is against any part of the PMC.
> Case in point: Learning a new programming language. I’m using it right now to familiarize with Rust, from a C++ background
IMNSHO, many programmers think that human languages are, in some important and useful way, similar to programming languages. Everything conspires toward this end: The tooling suggesting completion, conversations about code, the very word code, internet communications, etc.
In fact, this is an enormous category error. Try making a joke in a computer language. Or inventing a new word. You’ll see the difference when the program crashes (or does something even worse).
ChatGPT is very well suited toward learning a programming language precisely because programming languages are not “languages.”
A great post this with lots of great comments to chew on. But you know what really concerns me based on some of these comments? That this garbage will be made a permanent part of the education system leading to a generation of students who will know what exactly? It’s like you are taking original thought out of the process and depending not on rote learning but an AI that has done the rote learning and who will give you their interpretation of what they have ‘read.’ And as Lambert has shown, they are quite capable of making up their own facts. But unless you have a proper education, you will never know that you have been lied to. The rest of this century will require a lot of original thinkers to get through the multiple crisis that we are already facing but this type of AI will be a dangerous hindrance more than a help.
> The rest of this century will require a lot of original thinkers to get through the multiple crisis that we are already facing but this type of AI will be a dangerous hindrance more than a help.
I don’t see what the issue is. The 1% will be able to afford human tutors. If there is thinking to be done, those most qualified to be doing it will do it.*
* I don’t believe this, of course. But that is what they will think. This connects, obviously, to their eugenicism — lots of AI-generated prolefeed for a severely diminished population.
This article is a real public service. Not being a computer bod I’ve been trying to work out from afar what the difference between modern AI is and the sort of machine learning thats been around for decades and I’ve never been able to find a clear answer. Clearly, the only difference is computing power and the bezzle.
The big problem of course is that previous bezzles just meant a lot of gullible people lost a lot of money (and an occasional person got run over by a self driving car). But this has much more serious implications. Far too often systems like this get put into place by people who think they know what they are doing but… well, Mr. Dunning and Mr. Kruger might have a view.
> This article is a real public service
[lambert blushes modestly]
I can’t help but think that in the end we are supposed to think that:
BS = Actual Intelligence
Real thinking = Artificial Intelligence
What if AI learned to BS from observing normal human behavior?
Excellent article – thanks!
My kids are in University (1) or just about to start (1), Physics and Electrical Engineering and they are already experts at cheating using google to scam on-line exams (what a bad idea). This kind of tool will be (ab)used by students as a matter of course. I expect to see a “chatGPT” filter app appearing pretty soon to try to clean up the mess. Perfect for the tec bro’s, first get rich by creating somethig that causes mayhem (AI or mRNA jabs) then get even richer selling the antidote.
AI is bullshit built on bullshit. Some of the roles it will replace have been pushing out unverified garbage for a long time. Reporters are a great example of this. For perspective, let’s look at Covid reporting. They are not reporting facts, science, and pushing anyone in power to answer for their misguided and dangerous guidelines. Lol we’re all here bc we know that the state of media is bullshit.
I do not like the rise of AI but people have been primed for this and most are too desperate in their circumstances (majority of Americans are struggling physically, financially, and/or emotionally) to care. Younger generations are increasingly unable to access basic services online and don’t understand things like file structures due to their app-only experiences.
AI is both total bullshit and incredibly, unfortunately the precise truth we are living. The future is dire.
A precursor to this discussion took place on Dec. 6, 1983.
A lot of the articles about “AI” affecting visual artists usually talk to the digital artists forgetting or not bothering to find out that the data scraping came for all of us. I’m a visual artist using traditional media to make paintings, when I first heard of this I at first thought I’d be too obscure to be caught up in the net of the LAION-5B database, but out of curiosity I used https://haveibeentrained.com/ and found 12 of my paintings there. They had been taken from my Flickr account and my Fine Art America and World Wide Art Resources portfolios all of course without my permission, etc. Found that one of my paintings was used to train the AI. Between finding I was in there at all and seeing which painting trained the AI I felt just nauseous and then very angry.
The “Have I Been Trained” site had a feature that if you signed up for an account, you could “opt-out” the work you found. Knowing the vileness of late stage capitalism, I would be quite surprised if the whole thing had been set up as “opt-in”, or even bothering to ask artists if they wanted to be part of this. No artist I know wants anything other than to kill it with fire. I spent two days between making sure there were only 12 paintings in the database and to opt them out. Assuming of course that even actually does anything at all. It also means that there is nothing to stop another round of data scraping and getting more of my work. Also endless amount of work trying to opt out every time this happens. Hassleocracy at its finest.
The artists I know (myself included) are pretty much life time members of the precariat and to make any kind of living with our work has never been easy, Some of us worked low paying day jobs that hopefully would allow time and energy to keep making art and finding homes for it. Trying to find galleries that can stay in business when their rents have tripled in the last couple of years make the traditional route harder and more of us are online where we are now vulnerable to just having or work scraped.
Making art, composing music, writing stories is something I would hope as humans would be the one thing that could not be taken by machines. Yet, it’s appalling how many people think doing just that is a great thing. It’s amazing to see the hatred for artists by some of the people who think typing a prompt into text box is the same or better than all of the time and work human artists put into the thing that was consumed by the training database without which they could have never used the art generator in the first place.
All of this had me thinking about the HAL9000 computer from 2001: A Space Odyssey and how HAL only became malevolent when humans changed his programming to be just that. One of the most obvious things one could say on a site like Naked Capitalism is that capitalism as we know it ruins every single thing it touches, but now it seems as though every new “advance” seems aimed at accelerating dystopia and large scale societal collapse. Ethics? We don’t need no stinkin’ ethics! That just gets in the way of our quarterly profits!
I’m very tilted into math/science not art – yet, well, since Robbie Burns created a poem about lice, I asked chat-gpt to create a IAMBIC PENTAMETER poem in admiration of my favorite beer. It obliged. I’m not an English major but I can count and it looked like a pentameter to me !
In the sequences of texts it STOLE to create this genius of a work, did it pay the originators of those sequences of text ? In the capitalist world this matters. Sue the bastards. I’m on your side
Stack Overflow agrees, “temporarily” bans ChatGPT for low quality answers. “The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce.”
If only it were that easy to automate the hegemonic PMC out of their self-appointed Platonic Philosopher-Regency, then uninstall the app and go back to hard copy.
A conversation w ChatGPT:
VG: so here is my question for you: If you look at Darwinian evolution; how new immunocompetent memory lymphocyte clones are formed after a novel pathogen exposure; free markets ; and even a fayerbanden-ish view of the philosophy of scientific progress, wouldn’t it be fair to say that in some ways they all represent a common and very powerful algorithm: The reproduction and amplification of positive or strong signals and the extinguishing or diminishing the negative or weak signals. natural selection in a broader more generic sense
CGPT: Yes, it is fair to say that Darwinian evolution, the formation of memory lymphocyte clones, free markets, and the philosophy of scientific progress all share a common algorithm of selecting and amplifying the positive or strong signals, and diminishing the negative or weak signals, in various contexts. This algorithm can be seen as a form of natural selection, which is a general process that operates across different systems and drives the evolution of complexity and adaptiveness.
VG
Could one also make the case that Google ( at least in its earlier ( beta) incantations was an example of this generic natural selection algo
CGPT
Yes, one could make the case that Google, in its early stage, was an example of the generic natural selection algorithm. The original Google search algorithm was designed to rank web pages based on their relevance and popularity, effectively selecting and amplifying the positive or strong signals (high-quality, highly-cited web pages) and diminishing the negative or weak signals (low-quality, rarely-cited web pages). Over time, as the web and search queries have evolved, Google has continued to refine and improve its algorithm to better align with its users’ needs, further demonstrating the power of the natural selection algorithm in shaping technological progress.
VG
here I might disagree. as a former google beta and current user, my user experience in being able to get useful information thru a search has greatly diminished in the recent few years
CGPT
It is possible to have different perspectives and experiences regarding the performance of Google or any technology. As the search space becomes more crowded and complex, it can become increasingly challenging for any algorithm to provide high-quality, relevant results for all queries. However, Google continues to invest heavily in research and development to improve the quality of its search results, and it remains one of the most widely used and influential search engines in the world.
Vg
well considering google is an investor in open AI… i didnt expect you to say much different
CGPT:As an AI language model developed by OpenAI, I strive to provide neutral, fact-based answers to all questions to the best of my ability. I do not have personal opinions, preferences, or affiliations, and my answers are not influenced by external factors such as investments or partnerships. My goal is to provide accurate and helpful information to all users.
VG
but let us presume for the sake of argument that my premise re relevance of search then and now is correct. hypothetically
CGPT : Certainly, let’s presume for the sake of argument that the premise that the relevance of Google’s search results has diminished over time is correct. In this hypothetical scenario, it would be interesting to explore the reasons why this might be the case and what potential solutions might exist to improve the quality of search results. Some potential factors to consider could include changes in user behavior, the sheer volume and complexity of information available on the web, and the increasing prevalence of low-quality or malicious content. To address these challenges, Google and other search engines may need to continue to develop and refine their algorithms to better understand and rank the quality and relevance of information available on the web.
VG
One does not have to think that much actually- the problem is that there is a conflict ( an internal contradiction in dialectical materialist terms), between the fidelity of search and googles need to generate ad revenues
CGPT: It is true that there is often a trade-off between the quality of search results and the need to generate revenue through advertising. Search engines like Google rely on advertising as a primary source of income, and the display of ads can affect the user experience and the relevance of search results. To balance these competing goals, search engines may use algorithms that prioritize certain types of content, such as ads or sponsored results, over other types of content, such as informational or educational resources. This can result in a search experience that is less focused on the needs of the user and more focused on the goals of the search engine. It is a challenging problem, and finding ways to strike the right balance between the quality of search results and the need to generate revenue remains a key challenge for the industry.
VG
So in kind of an epistemological way: google is altering the natural evolution of information by – shall we say – giving it a nudge – directed evolution if you will
CGPT: In a sense, you could argue that by influencing the ranking and visibility of certain types of information in search results, Google and other search engines are indeed altering the natural evolution of information. By doing so, they are effectively guiding or “directing” the evolution of information in certain ways that align with their goals and objectives. This can result in a search experience that is shaped by commercial and other non-objective factors, rather than by the pure merit and relevance of the information itself. At the same time, it’s important to acknowledge that search engines are providing a valuable service by making large amounts of information accessible to users, and their algorithms are constantly evolving to better meet the needs of users and provide high-quality, relevant results. The relationship between search engines and the evolution of information is complex and multifaceted, and it continues to be an area of ongoing debate and discussion.
VG
Now i am not in the IT field, my interests are much more philosophical- so correct me if I am wrong. In the neural net architecture that underlies much of what we call AI today ( even you) the ” black box”, if you will ,functions in a similar evolutionary paradigm ?
CGPT: Yes, you could argue that the underlying architecture of many AI systems, including neural networks, operates in a manner similar to an evolutionary paradigm. Neural networks are composed of interconnected nodes that process and transmit information, much like the interconnected neurons in the human brain. During training, these networks are exposed to large amounts of input data and use a process of trial and error to iteratively adjust their internal weights and biases in order to better predict the desired output. This process is similar to the process of natural selection, where the “fittest” networks, meaning those that produce the most accurate predictions, are selected and amplified, while less fit networks are discarded. Over time, this process of iteration and selection leads to the development of increasingly complex and sophisticated networks that are capable of solving complex problems and making decisions based on the input data. So in this sense, you could argue that the architecture of AI systems operates in a manner similar to an evolutionary paradigm.
VG
So in a sense AI programs are using an evolutionary architecture to predict the nature of the particular subset of reality that is the focus of interest at a particular time
CGPT: Yes, that’s a good way to describe it. AI programs, including neural networks, use an evolutionary architecture to make predictions about specific subsets of reality that are of interest at a particular time. By training on large amounts of input data and iteratively adjusting their internal parameters, AI systems can learn patterns and relationships in the data and make predictions about new, unseen data. The accuracy and quality of these predictions depend on many factors, including the quality and relevance of the training data, the complexity and design of the neural network architecture, and the objective being optimized during training. In this sense, you could say that AI systems are using an evolutionary architecture to make predictions about the nature of reality and to make decisions based on that knowledge.
VG
would it not stand to reason that efforts in the information realm to curate the input data might give a false picture of reality- not just in predictable manners but also unpredictable ways- the unknown unknowns ?
CGPT: Yes, it’s definitely possible that efforts to curate input data could give a false picture of reality and result in unpredictable outcomes. Input data is a crucial factor in the development and accuracy of AI systems, and if the data is biased, incomplete, or otherwise inaccurate, it can lead to systems that make incorrect predictions and decisions. This can have significant consequences, especially in areas such as healthcare, finance, and justice, where AI systems are being used to make decisions that affect people’s lives. Additionally, when input data is curated or filtered in certain ways, it can lead to a narrow or incomplete understanding of reality, which can result in AI systems that are not representative of the broader, more complex reality they are designed to model. It’s important to be mindful of these potential pitfalls and to take steps to ensure that AI systems are trained on diverse, representative, and accurate input data, in order to minimize the risk of unintended consequences and ensure that they provide a fair and accurate picture of reality.
VG
Please comment on my comment on the predictable and the unpredictable consequences
CGPT: Your comment highlights a key challenge in the use of AI systems and the importance of being aware of both the predictable and unpredictable consequences of their use. AI systems are designed to make predictions and decisions based on the input data they are trained on, and if the data is biased or otherwise inaccurate, it can lead to systems that produce incorrect or harmful outcomes. This can have predictable consequences, such as perpetuating existing biases or discrimination, as well as unpredictable consequences, such as introducing new and unexpected biases or making decisions that are difficult to understand or explain. It’s important to recognize these potential consequences and to take steps to ensure that AI systems are trained on diverse, representative, and accurate input data, and that their predictions and decisions are subject to rigorous evaluation and review. Additionally, it’s important to remain mindful of the limitations and uncertainties inherent in AI systems, and to use them in a responsible and ethical manner that takes into account their potential impacts on society and individuals.
VG
I guess what I am getting at is that from an epistemic perspective – i profoundly disagree. If so early in the development of the field, if AI labs insist on biasing their bots. which essentially boils down to decreasing their degrees of freedom. you will stifle further development and give AI a bad rep. Which might be a problem given the visceral fears re man vs machine
CHPT: I understand your perspective. Bias in AI systems can indeed be a major challenge and can limit their effectiveness and ability to generate accurate and meaningful predictions. Bias can result from a variety of sources, such as the data that the systems are trained on, the algorithms used to make predictions, and the values and assumptions of the developers and users of the systems. Bias can perpetuate existing inequalities and discrimination, and can lead to harmful or unfair outcomes.
It’s important for AI researchers and developers to be aware of these potential biases and to take steps to minimize them by using diverse, representative, and accurate input data and by developing algorithms that are transparent, fair, and auditable. By doing so, they can help to build trust in AI systems and to ensure that they are used in a responsible and ethical manner that benefits society as a whole. Additionally, by reducing bias, AI systems will be able to generate more accurate and meaningful predictions, which can help to advance the field and to unlock its full potential.
a follow up thought. somewhat addressing the unknown unknown question: While we know roughly that we are using iterative natural selection algos – we are black boxed re the actual processes that result in the output. And perhaps like in the human brain multiple modules that have learned from prior exposure are called into play- huge and hitherto unquantifiable interactive effects. So a neural net trained on a curated data set to give desired interpretation of reality, may also amplify diminish nodes that give incorrect answers to completely unconnected questions
Yes, you are correct in pointing out that the internal workings of neural networks can be complex and difficult to understand, especially when multiple interacting components are involved. This is known as the “black box” problem in AI, where it can be difficult to understand how a network arrived at a particular output.
Additionally, training neural networks on curated data sets can indeed lead to the amplification of biases in the data, which can result in incorrect or misleading answers when the network is applied to new or unseen data. This highlights the importance of having diverse and representative training data, as well as monitoring and mitigating sources of bias in the training process.
It’s also important to note that the behavior of a neural network can change as its inputs change, so it’s possible for a network to give incorrect answers even when the questions are unrelated to the data it was trained on. This is an important consideration when deploying AI systems in real-world applications.
I think that ChatGPT and its ilk will become like Google’s search engine – spitting out vast amounts of information, answers to questions and other info, that will be skewed according to the owner’s biases and politics. It will be used for propaganda, censorship and marketing, and will pass on dissenters’ details to the state via the technology backdoors.
In short, it will be weaponised, just like every other piece of technology that can be.