Yes, dear readers, we know the site is not performing well for some of you. I’m hoping you will bear this with some patience once I give you the background.
Two weeks ago, we had two bad crashes, both after 5:00 AM when site traffic was low. This meant it almost certainly had to do with WordPress rather than our servers (which a 24/7 webhost runs) particularly since I called them, nothing had fallen over on their end and rebooting the site didn’t fix anything. The result each time was I didn’t get to bed until 8 AM or so each time, plenty upset because I could not raise my WP guy (who has staff and does work a long day but not 24/7) and as a result slept badly and got pretty sick that weekend (in bed a full 16 hours Saturday and 14 hours on Sunday, among other things leading me to turn down the opportunity to appear on Chris Hayes’ show).
He did some tweaking but that plus installing some plug-ins seems to have caused problems with comments (we also had new problems with the backstage but those seem to have been resolved). We’ve also determined we need someone who can be 24/7 but we don’t have a solution yet (or even references save WordPress VIP, which I should call but I have a prejudice against buying IBM, so I probably won’t do it until I have at least one other option for comparison purposes). We’ve had some chats with people who have sites that run traffic like ours or a low single digit multiple higher, and the word we get back is they are unhappy too. Our webhost, who runs a fair number of pretty high traffic sites, said basically that WordPress is the problem, and the only company he knew that provided really reliable service was in the $100,000 per year range.
I’m sure some of you either have blogs or know of bloggers who run sites on WP and have tech guys they are happy with. But please refer them only they have run multiple sites with 2 million page views or more per month and can provide 24/7 support. WP does not scale well, and running WP does not mean you know how to run WP on a high traffic site. I’m in the awkward size range where the site does not generate enough for me to be able to pay for full time tech employees, and third party providers don’t seem at all well equipped to handle sites in the 1 million page views a month and up traffic range.
As for current anomalies, to help us isolate problems, since most readers are able to access the site and post comments, we need you to give us sufficient detail. One thing users have difficulty understanding is that we can’t see what you see. So when you make a statement like “My comment didn’t post” all it does is produce is anxiety on our end, since it’s not detailed enough for us to figure out what is happening. Before filing a bug report, please clear your browser’s cache and cookies. Often, that will clear the problem up. Then, to do any serious debugging we need:
1. Ideally the comment text (to make sure that the comment didn’t get caught by a filter and queued). You do keep a copy of your comments before posting them, right?
2. The error message that WP gives — “Duplicate comment detected; it looks as though you’ve already said that!” for example. If there’s no message, say there’s no message
3. Your operating system with version (Windows 7, OS X 10.5.1 , Ubuntu 11.10, etc).
4. Your browser with version (Firefox 10.3.2)
5. Your platform: Computer, phone, iPad
If it’s any consolation, this is much more painful for Lambert and me than it is for you. And I’m out of town at a conference through Wednesday evening, so I can’t do much to advance the ball till later in the week. Thanks for your patience!
89 comments
Comments are closed.
WordPress is NOT the problem.
The hosting company probably is.
There are plenty of free themes that could help you organize your content in a less cluttered manner, by the way (hint, hint).
And it really boils down to your IT guy to show you three or four different layouts, clicking a couple of checkboxes and you’re golden….
Paul
By the way, FT Alphaville also runs on WP, and I’m sure they are also seeing huge amounts of traffic, since they seem to be very popular ;-)
P.
Ok. :-) Serving web-pages to clients (like my computer over here in Canada) is done by web-servers, say “servers”.
If, all combined, the servers don’t have enough “juice”: RAM memory + CPU Power, etc. ; Then, this might be a ‘hardware-cant-cope-with-traffic’ thing worth diagnosticating …
“… FT Alphaville also runs on WP…”
(to quote Vicky Pollard) Yeah, but they extensively customised it through a boutique software shop (which did such a good job Pearson bought them out and is presumably now their pet full time techie support).
They can afford such because, well, they’re part of a big media conglomerate.
Which, as Yves said, isn’t the case with NC.
PauW: Thanks, I think. I haven’t had my coffee yet, so I’ll be nice. It’s not like Yves hasn’t been doing her due diligence on this. If you can show that it’s as easy as you say given the conditions Yves laid out, we are all ears. However, other WP sites of NC’s size also seem to be having issues as well.
Wait, I thought it should be the other way around: grumpy until one’s had some coffee. Or maybe it’s a case of “life begins at caffeination” (coined by a friend of mine).
With all due respect. do you know anything about the way WP runs on high traffic sites? I have spoken to bloggers with sites with traffic levels comparable to mine and higher. They all have trouble with WP. My ad service, which pretty much deals only with high traffic sites, also confirms this.
And the suggestions you are making are not within the webhost’s purview, again pointing to being out of your depth.
WP is well known to have stability problems with high traffic sites. Asserting that this isn’t so is simply untrue.
And the template for this site is one of the simplest and has less going on that would place demands on the database caching, which is the point of weakness of WP, than most WP sites.
Please don’t come here and act as if you know better when you don’t. It’s both offensive, given how frustrating this is, and misleads readers as to the lengths we’ve gone to to try to get this resolved.
Testing 123 …
I am running
OS: Windows XP
Chrome: Version 22.0.1229.79 m
Type: Computer Desktop Tower
I have to say I have rarely ran into any problems posting comments on NC. On rare occasions I have gotten a message about “can’t connect to NC right now……..” or something to that effect, but compared to a lot of sites, I would say NC performs very well. B+.
Hope you start to feel better. I don’t know the extent of the problem, but don’t be a perfectionist and don’t take it to heart. A lot of times it is out of your control. For example, chrome runs a flash plug-in as a built in feature of the browser and many computers also have a local version on their hard drive. Javascript code that loads the local program can conflict with the built in flash component. Not a server side issue. I’m just trying to say it might not be a problem on your end, it might be the customers machine or software is not properly configured. But, I don’t know I’m not a network guy.
Anyway, you’re doing good and your time and efforts are really appreciated.
I am a reader only, never posted any comment so far. I always clear cache,
relaunch the browser,
Win7,
Firefox v13.
And, what is the probem I saw on my end, site was not getting updated for the last three days, hope it helps!
All the best!
Hold down a SHIFT key and click on the ‘Reload’ icon. This is a trick that sometimes works in these circumstances.
http://en.wikipedia.org/wiki/Wikipedia:Bypass_your_cache
Fascinating – for the last 3 days i have been unable to comment on Links, although could on other articles (figured it was my bad squirrel jokes …)
So now i posted here and at first had same problem – no new post on, AND my sign in disappeared from, the page that appeared after i hit “submit comment” …
Out of curiosity i hit the “Subscribe to: Post Comments” link below the button and Voila, my post was there …
So it appears that the system is recording the comments SOMEWHERE, wherever the “Post Comments” place resides, but not on the page that has the article one is posting on …
I came back to where I am posting now and my previous comment has since returned ….
It almost seems as though the submit comment button puts the post somewhere, but takes the person back to a previous version of the page – Like hitting an older version on Time Machine on a Mac ….
Does that make any sense?
Will be curious to see where this post goes ….
same thing – no new post and had to sign in again ….
Note to the WP working on this problem.
When I POST a comment the Request Status code is “302 Found”. Should your web server software be doing a “301” staus code for a redirect or should it be returning “200 ok”?. There is also an error message associated with the POST (domains, protocols and ports must match).
Unsafe JavaScript attempt to access frame with URL http://www.nakedcapitalism.com/2012/10/on-tech-woes.html#comment-833310 from frame with URL http://googleads.g.doubleclick.net/pagead/ads?client=ca-pub-3177324918955889&output=html&h=600&slotname=3196110895&w=160&flash=11.3.31&url=http%3A%2F%2Fwww.nakedcapitalism.com&dt=1349189239771&bpp=16&shv=r20120926&jsv=r20110914&correlator=1349189239968&frm=20&adk=1874555545&ga_vid=1138435485.1319882211&ga_sid=1349185590&ga_hid=1671375480&ga_fc=1&u_tz=-240&u_his=42&u_java=1&u_h=1024&u_w=1280&u_ah=994&u_aw=1280&u_cd=32&u_nplug=29&u_nmime=125&dff=optima&dfs=13&adx=795&ady=1618&biw=1263&bih=627&oid=3&ref=http%3A%2F%2Fwww.nakedcapitalism.com%2F2012%2F10%2Fon-tech-woes.html&loc=http%3A%2F%2Fwww.nakedcapitalism.com%2F2012%2F10%2Fon-tech-woes.html%23comment-833310&fu=0&ifi=1&dtd=371&xpc=M62Fs0SrJX&p=http%3A//www.nakedcapitalism.com. Domains, protocols and ports must match.
Resource interpreted as Script but transferred with MIME type text/x-json: “http://odb.outbrain.com/utils/get?url=http%3A%2F%2Fwww.nakedcapitalism.com%2F2012%2F10%2Fon-tech-woes.html%23comment-833310&srcUrl=http%3A%2F%2Fwww.nakedcapitalism.com%2Ffeed&settings=true&recs=true&widgetJSId=TR_1&key=AYQHSUWJ8576&idx=0&version=71141&ref=http%3A%2F%2Fwww.nakedcapitalism.com%2F2012%2F10%2Fon-tech-woes.html&apv=fals
I don’t know, just trying to tell you what is happening on my end.Hope this helps.
Good luck.
Avg Joe – I first read your comment on the “Post Comments” page that shows up when i press that link below the comment box, even though it wasn’t showing up here (on the article page). Wonder if your tech description was an explanation for what i was describing in “tech dummy” English ….
Son of a gun – it worked this time ….
I think you may be on to something there Aquifer. The script that intercepts a POST on the submit button may be different than the script that intercepts the POST on the subscribe button (I had never used it before to tell you the truth. It required me to install a google reader plugin and I like to keep plugins to a minimum).
Anyway, when you POST information back to the web server it is called sending a request. To prevent a computer hacker from hi-jacking sites, modern server software has built security that limits cross-site posts (so the response from the server is limited to the same computer, same domain (site name), and same port. There are work arounds if you configure the server software accordingly, but it is complex. Part of it is configuring the request and response headers (a block of info that servers and browsers use to gather info on the request so they can form proper communicatons with each other) properly. NC is doing a redirect (telling the software to change the location of where to find a page specified by the request). This could be the source of the problem. If this header info isn’t what a browser or server expects, the request fails.
That is, the code running from the subscribe button could be properly reading the request and configuring the response headers but the code from the submit button could be running code that is a little bit buggy.
That’s my understanding. It probably sounds very geeky and bear in mind that we are both just guessing. I’m not a network guy, although I have some experience on programming with browsers an basic info on server side software. But, you may be right about the POST script working on the link but not the button because tehy are different software routines.
Thanx for the feedback – it just seemed to me that if one wants to diagnose a problem, whether in a computer or a person – a good “history” is very useful – as in “Doc, i passed out …” Doc: “what were you doing at the time … and then what happened, etc. etc” Then maybe you can figure out what tests or studies you should go to first when there are so many possibilities ….
So I did a sort of “history” – and you did a “physical”, and maybe my “behavior” observations jelled with your “internal exam” – I think that would be pretty neat!
I have an old computer (Apple Powerbook G4) and an old version of Firefox 3.6.28 (can’t upgrade on my G4), but it was working fine on this site (occasional glitch here and there) until about 3 days ago when I couldn’t post to Links, but could to other articles, i.e treating Links posts differently from other ones, so i figured it wasn’t my set up …..
As i said and have demonstrated over and over, i am a tech dummy, but it does seem to me that behavioral observations, even if not in the “appropriate” language, using the right terminology – might be useful – then again maybe i am talking through my hat ….LOL
Using the site daily from multiple machines/different locations I did not notice problems with it. I did not post comments recently, so can’t judge that. I assume your site and servers are reliable and the problem might be corner cases in the commenting code or the network/setup of the users having issues.
Inspecting this site with the Chrome console I get a bunch of ReferenceErrors (icBrokerWidget [line 201], icBeacon [line 828]), TypeErrors (Cannot call method ‘match’ of undefined[ line 815]) and a Port error. I think all this happens in included widgets or ads. Now mind you I am running an unstable Canary version of Chrome, which might be buggy in itself, and Adblock extension that may suppress stuff (check if your users have extensions installed – they might be invasive). Nevertheless your site works fine for me and commenting is instant.
I’ve just discovered you didn’t vanish after September 26. My Chrome bookmark for you is http://www.nakedcapitalism.com, which only brings up the 9/26 page. Seeing you in EconoMonitor led me to hunt for you and to discover that IE9 and Firefox access you correctly, as does Chrome, but only if I type in a trailing /
not if the URL ends in .com
I’ve never posted before, but read you daily. Thanks for the good work.
FWIT, I’m using Vista Ultimate and Chrome version 22.0.1229.79
An easy way to check whether the problem is caused by your browser is use private browsing:
Firefox: Tools-> Start Private Browsing
Chrome: Menu -> New Incognito Window
IE8: Safety -> InPrivate Browsing
This is faster and easier than clearing the cache and cookies. If the problem disappears, then yes, the cache and cookies must be cleared, otherwise it’s probably something at the server.
What’s private browsing? I will win any tech dummy contest but now that I look it’s in an option in my Firefox Tools menu:
How is that different from just opening another window?
When you open a normal window, the browser will use its cache (could be images, code, or even entire pages) and cookies, i.e. information previously sent by the page. It will also receive cookies from the page and save page elements into cache and will use on subsequent visits.
Private browsing does not use cache or previously saved cookies, so when you visit a page you will see it as if you’ve never been on the site before. Also, while it will accept cookies (without which a page may not function properly), it will not save them for subsequent use.
So, private browsing is an easy way to check whether an issue is caused by information previously stored by your browser… Very handy for debugging Web applications, too.
I have been reading your site this week on both Windows XP, Firefox 15.0.1 and on iPad iOS6. Everytime I bring up your site, it contains the prior days data. If I then hit refresh, the new posts show.
That smacks of a page header problem within wordpress as in caching is set in a not so optimal way withing whatever new plugins were added recently (or they are interferring with the settings NC has set as default.
I’m running firefox 15.0.1 on OSX Snow Leopard (10.6.8) and I’ve seen no problems whatsoever here, as well as on Fedora 17 and wjth the same version of Firefox (15.0.1)
Your headers are correct as far as my browsers are concerned, “Pragma: NO-CACHE” and “Expires 0” (meaning now) so if the browser is working correctly it should not cache any pages. My browsers seem to be working just fine on my systems :)
Of course I’ll have a much better idea after I POST this comment.
Success, working A-OK here… as I already suspected.
Have you considered switching to Jekyll and Disqus?
Jekyll is super fast and secure. It runs your content
as static HTML so there is no database slowing things
down.
You’d need to hire someone to set it up for you
but it doesn’t require full time staff once it’s up and running.
Beware of Disqus!
I had profiles set up at other sites for years, CNN and MoJo among them, that were active and accurately tracking my activity. I joined a Disqus blog, and suddenly all those stats changed. Disqus said I needed to register with them directly and merge the other profiles with theirs. I did that and all record of my activity at those other sites disappeared, profile info disappeared, comment logs disappeared.
Logging out of Disqus changed nothing. And Disqus tells me at present there is no process that allows me to cancel my Disqus…whatever you want to call it…I got a name for it…use your imagination.
The blog I joined dumped Disqus for a straight WordPress setup.
Nothing changed. Disqus still shows me as subscribing to that blog through them. In fact, the only accurate tracking of my activity on any blog right now is on the Disqus dashboard I’m logged out of for that one blog that no longer employs Disqus.
You think WordPress support is hard to reach? Sign up for Disqus!
Agree – a lot of sites seem to be using Disqus these days and apparently “upgrading” to a new “improved” version. It works OK for me (very slow and often have to open up a separate text edit page so i can copy/paste) on several sites, but on one site, it is fubar, won’t let me in or says i am already in but won’t let me post. As this is not all the time, I don’t think it is a censor issue, but i can never tell when it will screw , very frustrating. One nice feature which, being someone who likes “response”, I really like is the option to be informed by e-mail if someone replies to one’s comment – opportunity to continue a conversation …
I agree email notifications are great…especially when the user can choose which email account to employ and not have to sign up with even more ad networks and trackers.
I used to think cut’n’paste had something to do with WP glitches, but not any more.
And Disqus? Disqus just sucks.
Disqus is the pits! I’ve been back and forth with their “support” for the past three months. All I want is a full tool bar (mostly the delete icon) on every comment in my dashboard.
I send an email with screen shots. 80 percent of the time they don’t respond at all. The 20 percent of responses are canned text: check your browser, etc. Then a week later, I find they have simply deleted my dashboard comments on their end without resolving the original concern.
What a bunch of boneheads!!!
I’ve encountered WP glitches…like disappeared comments and duplicate notifications…among other things…at “low traffic” WP blogs, too. Their frustration mirrors yours. Best of luck.
But I’m wondering…having read about the troll issues…if a comment disappears, how does one know if it’s a WP glitch or some troll scrubbing software or direct moderator intervention?
I have to say, that I post to NC from several machines and connections on two versions of OS X and two versions of FireFox — both if which have caching turned off. I have never experienced a problem. I realize that’s not helpful to readers who are, because computers.
* * *
That said, the intermittent problem of comments disappearing into the ether has been stated many times. It’s frustrating for us, because they don’t end up being tracked or logged anywhere; we will check the queues for spam, for example, and the comment will not be there. And yet, sometimes (not always) when readers resubmit, they will get the dreaded “You already said that!” message, so the comment is somewhere… Just not anywhere we can see.
What might be helpful is if some kind soul could REPRODUCE the “into the ether” problem. But I don’t even know if that’s possible; intermittent problems are the hardest to solve.
What might also be helpful is if some kind soul, who is aware of the same problem on some other WP site, could point us to a solution, ideally in the form of a link.
Lambert – could i ask, for these disappearing comments – do any of them show up on the page that shows up when you link to that “Subscribe to: Post Comments” link under the comment box? I never thought to look there before today …
Wonder if it would be useful if folks, whenever they had a comment disappear, clicked that Link to see if it was there. Wonder if that is why “you already said that” shows up, because you already did and the comment is alive and well on that “Post Comments” page … If that is, indeed, a pattern, maybe it would point to a fix ….
I am going to try that whenever i have disappearing comments to see what happens – what the heck, hey “couldn’t hoit” ….
If you lose a comment, and you go back a page, is your comment there in the box you were writing it in? Does that tell us anything? My best idea is screenshot screenshot screenshot, but I’m a dummy*.
* Lambert, caching turned off? Huh?
I’ve tried turning off caching and just when I thought that would solve the problem, it cropped up again. Really a difficult issue to get a handle on… it’s very inconsistent and I know of no site that has conquered it…high or low traffic. From a commenter’s perspective it’s frustrating, but not a deal killer…not for me anyway. I come for the content.
I also use OS X but run Safari. I’m going to upgrade my Firefox and give that a try, I think.
But…hating to be repetative…if a commenter is moderated by who or whatever, they’re informed…correct? Like if you folks think I’m a troll or violate whatever the Terms of Use are here, I’ll get an email?
I try to behave myself…but…you know…sometimes….
When I saw comments disappear (year(s) ago) it was due to included links (or maybe also other SPAM filter trigger?) Trying to submit again triggers the duplication response.
i just had a comment disappear complaining about comments disappearing…
will email details…
That’s wicked meta, RJS. Thanks.
Got it figured out?
rjs – out of curiosity, push the “Subscribe to: Post Comments” link underneath the comment box and see if you can find your comment that disappeared – if you can, then maybe there is something to this idea that, for whatever reason, the comment, though actually being posted in that “Post Comment” place, is not being routed back to the actual thread, OR the page that returns when you push Submit is a previous (before your comment) version ….
Seems to me that if this is a consistent finding for many, or several, posters, it might(?) point in the right direction toward a “diagnosis” ….
Ah, you must be the new land surveyor. Wait here, please.
Huh?
Kafka
How can you write if you don’t read?
Test
I have been writing software professionally for 25 years, the last several years I have been developing WP sites. I love WP, it has enabled me to run my small biz with a dev team of 1. However, the sites I develop are low volume small biz sites.
I know WP scales poorly, I have a neighbor who is fairly high up at Amazon Web Services running a team that is working on new DB technologies for the cloud. He says WP sites are the bane of AWS, taking up way more resources than you would expect for the traffic they handle.
I also know it is often because of bad implementations of WP plugins and templates. There are lots of ways to implement WP features and some of them are very DB intensive.
I can not help you on recommendations for the support you are asking for. It is exactly where there is a void in WP service. Most of the solutions require more money for personal and HW as well as to pay for programming to move to either another platform or start customizing you implementation (and all the large SW implementations customize. It is the only way to truly scale open source).
Have you been able to profile the site? Do you know where you spending your resources? If you knew specifically where the bottlenecks were, it would help find a specific solution.
You should be able to profile specific sql commands and get an idea how efficient they are. Table scans are deadly as your DB gets large. I have seen amazing improvements from adding a single index.
Assuming your DB is MySQL is it setup as MyISAM or Innodb? MyISAM is faster EXCEPT when you start getting lots of writes to tables. Then Innodb, which has row locking instead of table locking, is faster.
Honestly, often the best short term solution is to beef up the HW and tweak the configuration until you are able to afford better support or to move to another platform. Make sure your HW is big enough with lots of RAM. If one of my AWS sites was having this problem I would first upgrade the instance to bigger HW.
Also make sure PHP is optimized properly with compilers and caching.
I apologize if these are too simple of suggestions, I am sure you have already been through all of this.
Thanks for this. We are with a big webhost and have more than enough memory, so sadly, the easiest fix has not solved this problem.
iPhone os6
Worked fine on this thread.
>>Before filing a bug report, please clear your browser’s cache and cookies
i saw no new posts over the weekend, was actually kind of worried.
instead of clearing the cache/cookies, what worked for me was to hit Control – F5.
running firefox 15.0.1 on linux mint 13. i comment infrequently, have never had an issue with that.
Test 2
I certainly can’t help on the finer points of the tech side, but was able to assist an ex-WP blogger with an alternative when WP temporarily shut him down over his “choice of words” against power.
You can contact me off-web on how he/we handled it.
JP
Good evening,
I’ve tried “private browsing” and am still in “private browsing” mode”. As a result, I’m rather “unknown” to servers that depend on cookies.
Even here at NC, the field labelled: Name(required) had an empty fill-in Box …
For myself, I’ve resolved not to stress the NC web-site by minimizing tab-closures of NC, minimizing “refreshes” of NC pages, and so on.
There’s been debate about hardware vs. software.
If not WordPress, then what?
Drupal is a web-site content management platform that is open-source and has a strong following. I have no “stake” in WordPress, Drupal or other software to help manage web-sites with lots of traffic.
I’m obviously not advocating switching to Drupal. But, I think Drupal is worth mentioning and I’ll end with a link to a Drupal developper conference from 2011, heavily-attended. I don’t think the enthusiasm that came through in the video-taped talks at DrupalCon 2011 (Chicago?) can easily be faked.
http://chicago2011.drupal.org/
That’s the point of private browsing: no previously stored information is used, and you view the site as if you have never been there. On personal computers it’s usually not required unless you are paranoid about someone accessing your drive, or if you want e.g. to check email on somebody else’s computer and don’t want to screw up their settings. But if a page malfunctions, it may be caused by the browser’s stale cache or cookies, and then private browsing may help. If it does, cache and cookies should be deleted, which should fix the problem. But if it doesn’t help, the problem is either at the server or somewhere between you and the server (e.g. proxy, etc…)
I use Drupal at Corrente and it’s great. However, porting years of content to a new CMS is non-trivial and almost certainly requires professional services. And it’s an open question whether that’s the best use of resources.
This may be an anomaly on my end, but the past 2 days when I direct my browser to http://www.nakedcapitalism.com I have frequently been sent to a days-old version of the homepage, featuring old posts, and even clicking on various links within the site doesn’t get me back to today’s writing…
d’oh! Just figured out it was my firefox browser. IE is getting it right…
By the way, WordPress is also susceptible to viruses. Michael Pettis’s site was infected for months, which showed in Viagra and Levitra ads inserted into RSS texts and under some other conditions. We had those issues with one of your sites, too, and had to fix them manually.
I mean, one of *our* sites. I never dealt with WordPress myself though.
Attention Web Programmer (WP)
Got another error message. Http error 500.
(Are you still requesting we post error info?) Don’t want to clutter up the thread, but I don’t mind working on this problem and assisting as I can.
OS: Windows XP
Chrome: Version 22.0.1229.79 m
Type: Computer Desktop Tower
Request URL:http://ping.crowdscience.com/ping.js?url=http%3A%2F%2Fwww.nakedcapitalism.com%2F2012%2F10%2Fpricewaterhousecoopers-paid-1-billion-as-consultant-on-mortgage-settlement.html%23comment-834133&id=824d28492f&u=mozilla%2F5.0%20(windows%20nt%205.1)%20applewebkit%2F537.4%20(khtml%2C%20like%20gecko)%20chrome%2F22.0.1229.79%20safari%2F537.4&x=1349233978592&c=1&t=809fe590d4a6fb73&v=ecb8ff1107085bca&m=0&vn=2.1.2
Request Method:GET
Status Code:500 Internal Server Error
Request Headersview source
Accept:*/*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Cookie:__csmv=ecb8ff1107085bca; __csgeo=deg.17770_deg5.746_deg10.193_cc.US; __cssegs=”7b0f2f4280:0.978:4|aca6a529ce:0.183|995ecb3c9b:0.00960|b79c8841da:0.0293:158:6:gm_29|f27d415ee5:0.00530:4|e666a44190:0.00490:4|09c4fcdbd0:0.0262|f55c133937:0.0353|9f688c3c7a:0.0389″; __csv=ecb8ff1107085bca
Host:ping.crowdscience.com
Referer:http://www.nakedcapitalism.com/2012/10/pricewaterhousecoopers-paid-1-billion-as-consultant-on-mortgage-settlement.html
User-Agent:Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4
Query String Parametersview URL encoded
url:http://www.nakedcapitalism.com/2012/10/pricewaterhousecoopers-paid-1-billion-as-consultant-on-mortgage-settlement.html#comment-834133
id:824d28492f
u:mozilla/5.0 (windows nt 5.1) applewebkit/537.4 (khtml, like gecko) chrome/22.0.1229.79 safari/537.4
x:1349233978592
c:1
t:809fe590d4a6fb73
v:ecb8ff1107085bca
m:0
vn:2.1.2
Response Headersview source
Connection:keep-alive
Content-Encoding:gzip
Content-Length:404
Content-Type:text/html; charset=iso-8859-1
Date:Wed, 03 Oct 2012 03:11:34 GMT
Server:Apache/2.2.14 (Ubuntu)
Vary:Accept-Encoding
Yves, just for the record, I’ve been reading your site almost since you started and have had only a couple of difficulties commenting (I don’t comment frequently), and only a couple times couldn’t access the site.
I have Win7 Build 7601, Firefox 15.0.1 now.
A couple of ideas:
– The official WordPress hosting service: http://wordpress.com/
– WPEngine: http://wpengine.com/
If you’re not merried to wordpress:
– blogger.com
– http://www.squarespace.com/ (I think you can use the offer code TWIT here)
These are all hosted services with 24/7 support and they handle a lot of high-traffic sites.
The only important parts here are these:
Request Method: GET
Status Code: 500 Internal Server Error
1. Http error codes are all 3 digit numbers. Error codes beginning with 5 are so-called “500-errors”. These are all server errors; often caused by the failure of software actually running the website.
2. It was a GET request.
The solution to the problem is to get an IT developer in to examine your server application error logs. The specific problem(s) will be detailed there.
When an exception happens during program execution, it may bring down the server when it’s not properly handled. The 1st solution is to ensure that all errors are “handled”. The 2nd solution is to ensure that there are no errors. You can only do this by looking at the stack traces generated by the operating system etc. to discover what’s causing the failures. The host says they are software errors. I’d agree with the host on that.
Late night testing 1
Late night testing 2 with link
http://www.nakedcapitalism.com/
Late night testing 3 with more links
http://www.nakedcapitalism.com/
Read more at http://www.nakedcapitalism.com/2012/10/on-tech-woes.html#z2m07HAvCMOMWSHz.99
http://www.wikipedia.org/
http://www.bbc.co.uk/
http://www.stern.de/
Late night testing 3 with spam keywords and new email:
viagra, sex, rolex, PhD, Nigeria, IRS, Facebook, earn money
And anonymous test with new unregistered email.
Don’t know, all my comments show up. Still see no problems with the site.
Why not just shut it down and go do something else for a while?
Just shut it down for a while. It’s mostly noise anyway.
If the issue is not solved yet, and new hosting is being considered, WPEngine dot com and Gandi dot net are **quite professional**. The latter being in France.
Sounds like, since yesterday, somebody fixed something ….
Haven’t noticed any particular problems myself. I’m browsing from New Zealand where Internet tends to be on the slow side to begin with, so perhaps any slowdown is lost in the noise. I’m using Firefox 15 on WinXP (PC).
I did have a comment disappear yesterday when I tried to post a link to a story from The Onion on the Occupy the SEC article (search for “the onion lobbyist” to see it). I think that’s normal behaviour though as I have had comments with a link and not much other content trigger auto-moderation before.
Thanks to readers for the detailed comments. It’s great data and should really help the wizard ultimately chosen. Keep it coming. If there’s too much detail — hardly possible! — somebody will say….
test2
I found the problem: blogspot or maybe other non-WP adresses are being blocked somehow. Not just here but elsewhere (all WP/gravatar-managed).
However I can still post in some WP blogs (with my blogspot address). Not sure why.
I have checked elsewhere without address and now here with my backup WP adress (for my anthro blog only).
I hope that it can be solved soon because I’d like people to be able to track me back to my active blog. But meanwhile a way around may be to post without blogsopt/other-non-WP addresses associated to your name.
Maju,
I’m not hosted on WP, I have my own hosting service. And I don’t know what you mean by “non WP addresses”.
Not sure, Yves. All I know is that when I began not being able to comment here (for exactly the same problems mentioned by everyone else), I also began not being able to comment (same problems) in Kasama Project (http://kasamaproject.org/), a communist site which is hosted, at least the comments section, by WordPress (I know because it asks/allows me to login via WordPress/gravatar) and have also detected since then the same problem in other WordPress blogs, but not in all.
However yesterday I commented at an online comic I read frequently (http://legendofbill.com/) and where I had never got any problem before commenting and the same happened. After several tries I realized that I had introduced a blogspot adress in the “website URL” box, something I never do in comic sites because I do not have any artistic blog or anything similar that can be of interest, so it’s rather pointless.
So it dawned to me that the issue was being triggered by introducing a blogspot address. Tried without it and, voilá, it worked!
Then I came here and posted two tests: the first one, which did not go through said “test” and had my politico blogspot link, the second one “test 2” went with my wordpress anthropology blog backup address and it did go through as you all can see.
I had planned a third test with no URL, like this very comment (should go through) but as the second one worked, I did not feel the need. And I will try with some other links under this post to see if they go through or not.
I hope it helps to detect where the problem is but at the very least it offers a walkaround: commenting without the URL that is somehow blocking the system.
Why are blogspot address blocked? Not sure but I’m guessing (after some paranoid thoughts) that it may be a technical matter: Blogger recently (months ago) began redirecting all blogs to national subadresses, so they can comply with local orders of censorship (that was actually why I began considering moving to WP), so, outside the USA, it redirects you to the national (state) subadress: blogspot.com.ca if you link from Canada, blogspot.com.es if you link from Spain, etc. This redirecting may be triggering issues in some systems but no sure.
Blogger URL “IDs” are common enough to have caused an epidemic of sorts. My working hypothesis right now is this one. I will now posts some more test-comments to see if this is a correct hypothesis.
test 3
With the Al Jazeera URL (not my real one obviously)
test 4
With a (quite radical) counterinformation Spanish site’s address, to test the “conspiracy theory”-like paranoid fears.
I think that after the tests it’s quite clear, although we would need confirmation from others, it may be the Blogger (blogspot) addresses are what are triggering these problems. Maybe an security program is dismissing them because of the redirect, which should only happen anyhow if the system is based outside the USA (Canada is outside enough however).
test 5
Testing if the system is actually located in Canada by using my blogspot address as seen from that country i.e. ending in blogspot.com.ca
Bingo!
After the previous comment I tried again test 1 (with my USA/international blogspot.com address) and it does not go through.
So the problem is almost 100% for sure the new Blogger national subadress system, or rather how the comments’ manager program (some security subprogram, I guess) deals with its redirection.
And also now I know that your comments managing system and that of WordPress/gravatar is based in Canada (again almost 100% sure).
Hope it helps. :)
Also it has something to do with Gravatar/WP because that image that goes with my name is one I picked months ago when I created a WP/Gravatar ID (before the problems no image showed up with my name ever), incidentally a reconstruction of Combe-Capelle man.
So even if this blog is not in WP the comments section is somehow related with them via Gravatar. Check that up.
Cheers.
Hello –
Did this get resolved, or are you still looking for someone?
I can’t commit to any kind of full time availability but I could help out with some server-side optimizations or a high-level code review, to see if there are any plugins that are known to cause performance / scaling issues and help diagnose the problem. I can also put the word out in the Occupy Tech community and see who else might be available to help.
FWIW, Talking Points Memo went through a similar problem as they were scaling up their operation from “popular blog” to “full-time news publisher” a few years ago – WordPress was losing comments and pages weren’t loading. They eventually solved the problem by hiring a fulltime web operations staff and building their own CMS, but they may have come up with some good interim solutions and would be worth talking to.
I’ve built a couple of high-traffic WP sites recently – one was for a cable news personality who would get 10k pageviews per hour during prime time, and another was one of the more popular Occupy sites. In both cases, scaling issues were handled by running the site on a cluster of servers – a master database, a database slave, a separate media server, and several web frontend servers, all running behind a load-balancer. A setup like that provides redundancy and removes single points of failure. They’re fairly involved to set up though, and a 5-server set up costs 5x as much as a 1 server setup.
Here’s some general advice for speeding up WordPress on single server deployments. Some of this you may already be doing…
1) host your media on a separate service like Amazon S3 – saves on server bandwidth and load.
2) switch your webserver to Nginx. It handles high traffic better.
3) Use Nginx microcaching:
http://fennb.com/microcaching-speed-your-app-up-250x-with-no-n
On high traffic sites this can be a huge performance boost.
You can get the same effect with Varnish if you don’t want to switch off Apache.
4) Use APC caching on the server. This speeds up execution of PHP scripts. You’ll need to run in conjunction with w3totalcache – w3supercache doesn’t work with APC.
5) Install Munin on the server and start logging performance data.
use the mysql plugin here:
https://github.com/kjellm/munin-mysql
and set your database to log slow queries
http://dev.mysql.com/doc/refman/5.1/en/slow-query-log.html
Munin will help you diagnose where your bottlenecks are (CPU, Memory, Disk Access, Database Access) and give you quantitative data about how code + config changes are affecting performance.
6) Install Monit. This is a monitor program that will alert you if server load becomes critical, and in some cases can automatically restart stalled or crashed processes.
7) Use Splunk to analyze your log files – it turns gigabytes of apache access and error logs into a database that you can query and visualize. Great tool to spot server errors that might otherwise get lost in the noise. Get the most out of the free trial though, their pricing plans are kind of crazy expensive “enterprise” stuff (like minimum 5k/year. as far as I know they don’t have some $30/month consumer-level plan).
8) Max out the RAM on your server. Like at least 16GB if you can. Doing this on NYCGA.net made all of our critical performance problems go away for long enough that we could optimize code and deploy on multiple servers.
9) Use a CDN (content delivery network) service like Cloudflare. This might be your best bet for a one-shot solution. I’ve never used it but I’ve heard good things (and it’s what has kept Wikileaks online in the face of coordinated DDOS attacks). It’s a service that will set up mirrors of your site around the world to provide redundancy and speed up performance.