Hopelessly optimistic techies Posted on July 22, 2010 by Richard Smith If only it was that easy. First sentence: “Pick up a piece of text and start reading and it usually becomes clear pretty quickly whether you’re reading a nonfictional news story or a fictional novel.” They lost me right there. Hat tip to Alea. Post navigation ← The bailouts continue: The Economic Populist Links 7/22/10 → Subscribe to Post Comments 11 comments JB July 22, 2010 at 10:36 am In a sort of truth recursion, we should use their fiction sorting algorithm on their own story to see if they’re telling the truth. tyaresun July 22, 2010 at 11:14 am They are using a very common technique from NLP. How do you think Google Translate works? Sufferin' Succotash July 22, 2010 at 11:44 am But how do you tell a fictional novel from a non-fictional novel? Jest askin’… Bates July 22, 2010 at 6:00 pm “But how do you tell a fictional novel from a non-fictional novel? Jest askin’…” It isn’t easy. Read any of Gore Vidal’s novels about politicians, DC, political intrigue, and it is very difficult to tell how much truth is mixed in with the fiction. Here is a bit of Vidal’s bio at Wiki…”Born in the Army Hospital at West Point where his father, one of the first Army aviators, was an aviation instructor to cadets”… …and… “Vidal was raised in Washington, D.C., where he attended Sidwell Friends School and then St. Albans School. Since Senator Gore was blind, his grandson [Gore Vidal] read aloud to him and was his guide. The senator’s isolationism contributed a major principle of his grandson’s political philosophy, which is critical of foreign and domestic policies shaped by American imperialism. In 1943, on graduating from Phillips Exeter Academy” In addition Vidal was related in one way or another to many political families. The real question is, imo, ‘does Vidal know if he is writing fact or fiction’. In any case whatever Vidal writes sells well. Here is a link to his bio at Wiki… http://en.wikipedia.org/wiki/Gore_Vidal Another interesting case is Fred Hoyle, or Sir Fred Hoyle. Hoyle was a brilliant man and over his lifetime his curosity lead him to study in many scientific fields. He began his studies in mathmatics then partical physics and later moved on to chemistry and intense study of the genome. Hoyle also wrote very complex scientific texts and a lot of science fiction. A famous quote from Hoyle, “I don’t see the logic of rejecting data just because they seem incredible.” Hoyle thought very far out of the box and I think he wrote sci-fi to ‘run his ideas up the flag-pole’. By this practice Hoyle could avoid presenting scientific theories or hypothesis in peer reviewed journals, yet he got feedback from his peers because they read his sci-fi. So, determining where fact starts and fiction ends is not easy, ever, in any discipline. It is even more difficult in the ‘soft sciences’ ie, economics, psychology, et al, because hypotheses can never be independently confirmed by another person in the discipline working in a different lab…Something routinely accomplished in say…chemistry. Here is a link to Hoyle’s bio at Wiki…http://en.wikipedia.org/wiki/Fred_Hoyle NOTaREALmerican July 22, 2010 at 11:57 am The key word is optimism. Optimism creates bullshit stories. The brain is a bullshit story generation machine. Someday, “they” will find that optimism runs at 3% per year, not coincidentally exactly what the Fed’s real inflation rate is. Bullshit generation (optimism) is what allows a sociopathic society to exist and why sociopaths thrive. Bates July 22, 2010 at 7:01 pm NOTaREALmerican… I’m throwing you a hanging curve ball and it’s right in your wheel house. If you don’t hit this one out of the park I will be surprised. Pessimists of the world unite! For every Ghandi success story millions died in similar ‘non violent protests’. ‘Psychology Optimism Positivity, Hope, Purpose’ “There may seem to be no justification for hope or optimism when faced with fanatical authoritarian leadership, environmental catastrophe, and a brainwashing corporate-controlled media. We live in a world that definitely seems to be getting worse rather than better. But there is a way by which facing these daunting realities we can reach new levels of clarity, optimism, and purpose. We know—we’ve been through it personally. By coming to understand the nature of our predicament, no matter how dire, we have taken the first step towards doing something about it.” etc, etc, etc… I wonder if this organization has taken note of what happened to those unfortunates under the thumbs of Stalin, Hitler, Mao, Ghengis, GWB, etc, that had the gall to speak truth to power? http://www.truthmove.org/content/optimism/ Peter Schaeffer July 22, 2010 at 1:45 pm Read the article. I quote ‘Measuring these two quantities alone can identify the type of story with remarkable accuracy. “Our analysis yielded a 73.8±5.15% accuracy for the correct classification of novels and 69.1 ± 1.22% for news stories,” say Stevenak and Carr.’ The authors aren’t claiming that much. Ina Deaver July 22, 2010 at 3:00 pm “Wait Wait — Don’t Tell Me” is an entire radio show premised on the fact that it is often completely impossible to tell news from fiction. This computer program of theirs is going to get pwned in record time! On a more personal level, my spouse and I love to play a little game we call “find the onion.” We take a number of real articles, and a number of articles from the Onion, and (as a kind of grown person’s drinking game akin to “bullsh*t”) try to sort the ones made up by snide hipsters and the ones that really came to pass in this Voltaire play we’re currently living. Not a game recommended for a school night. bartkid July 22, 2010 at 3:05 pm “_____ _____ felt the warm light of freedom for the first time in two years wearing a T-shirt, jogging pants, sweat socks, and deck shoes.” “_____ _____ squinted at the road ahead, which shimmered in the heat.” One quote is from this morning’s (July 22) Globe and Mail, the other is the opening sentence to Paul Western’s novel, Prison ChrisPacific July 22, 2010 at 8:04 pm The title is overblown. The article is about creating a text processing algorithm that can tell the difference between a news story and the first page of a novel. The writer of the article just chose to substitute “fact” for news story and “fiction” for novel. That’s a goal that should be quite achievable theoretically – the human brain can certainly do it with a fairly high degree of accuracy – although as bartkid’s example demonstrates, it’s not quite as simple as it sounds. Lakeesha Apperson August 18, 2010 at 7:06 pm I am as obscessed with poker as any one person can be. I have got a fan page over on Facebook. If you wanted to see: http://www.facebook.com/pages/Poker-Players-Only/136818223026737 Comments are closed. Tip Jar Please Donate or Subscribe!