As lenders increasingly nudge their customers toward online services, the frequency and scale of bank outages raises serious questions about the resilience and reliability of those services.
On Oct.1, I reported on NC that banks around the world are suffering big outages, leaving millions of customers in the lurch. In the less than two months that have passed since then just about every high street lender in the UK has suffered an outage to their online banking platform and/or mobile app; Ecuador’s biggest bank has been targeted by hackers, resulting in a four-day suspension of its online services; Mexico’s largest lender has suffered its third nationwide outage of 2021, and the online banking platform of Pakistan’s third largest lender has been taken down by a cyber attack.
This week was the turn of one of Asia’s largest lenders, Singapore-based DBS, which suffered its worst system outage in 11 years. This is a bank that has won multiple “best bank” awards, from publications such as Euromoney, The Banker, and Global Finance, and is considered a pioneer in embedding digitization across the full range of banking processes and services.
DBS began suffering disruptions to its website and mobile app services on Tuesday morning. As reports came flooding in from customers that they were unable to access the bank’s mobile app or online platform, the bank released the following message on social media:
“Some of our customers are facing intermittent slowness when accessing our banking services, and we are currently working to resolve this. We apologise for the inconvenience caused during this time, and please try again later.”
Scant Reassurance
Some customers understandably took umbrage with the statement, as there was clearly nothing “intermittent” or “slow” about the disruption; the system for many customers just wasn’t working.
“Slowness implies there is some speed no matter how low,” wrote Facebook user Zhaohan Chua. “It is simply not working. Is DBS into gaslighting their customers now?”
The service outage continued through the day of Tuesday and well into the night. At 2 a.m. Wednesday morning (local time) the bank announced that all digital services had been restored, only to receive a fresh torrent of complaints from customers on social media who were still unable to access their online account or mobile app. This prompted the bank to admit that the “digital banking issue ha[d] recurred.”
An outage of length of time is extremely disruptive given the growing number of users who depend on digital banking transactions and other services. The lender has been at pains to stress that the outage was not the result of a cyber-attack and that customer accounts and data have not been compromised. On Wednesday afternoon, the bank’s country manager, Shee Tse Koon, issued a statement blaming the issue on a problem with “access control servers”, which are used to authenticate usernames, passwords and other identifiers such as biometrics and one-time passwords (OTPs).
“We identified a problem with our access control servers and this is why many of you have been unable to log in. We have since been working round the clock, together with our third-party engineering providers, to fix the problem and recover our digital banking services. In the meantime, please be assured that your deposits and monies are safe and that you can continue with your banking needs either through our branches or through phone banking.”
The last sentence may not have been overly reassuring given that DBS, like most large lenders in advanced economies, has spent the past decade closing down many of its branches, even in highly populous areas and key business districts.
The bank’s IT issues appear to have persisted into Thursday. “Although customers said they could log in to its digibank online platforms on Thursday morning, many still could not make transactions or view past ones;” reported The Strait Times. By Friday the issues appear to have been resolved.
Supervisory Action
It was the bank’s worst outage since 2010, when an internal failure knocked DBS Bank’s back-end computer systems offline, leaving customers unable to withdraw cash from ATM machines for seven hours. On Wednesday evening Singapore’s central bank and financial regulatory authority, the Monetary Authority of Singapore (MAS), said it would consider taking “supervisory action”.
“This is a serious disruption and MAS expects DBS to conduct a thorough investigation to identify the root causes and implement the necessary remedial measures,” said Mr Marcus Lim, MAS’ assistant managing director of banking and insurance. “MAS will consider appropriate supervisory actions following the investigation. MAS expects all financial institutions to have systems and processes to ensure the consistent availability of financial services to their customers.”
The central bank’s regulations stipulate that the total unscheduled downtime for critical systems affecting services for customers should not exceed four hours within any 12-month period. DBS has exceeded that limit with abandon. That said, the punishment is unlikely to be too harsh given that DBS’s controlling shareholder, with 29% of the bank’s shares, is Temasek Holdings, the country’s second-largest sovereign wealth fund. In other words, the Government of Singapore, which indirectly controls MAS, is also DBS’s most important shareholder.
Blushes All Round
But the outage is nonetheless a serious embarrassment for a bank that prides itself on its digital smarts as well as for the city of Singapore, which has topped the global Smart City Index for the past three consecutive years.
“Embracing the digital world is a strategic imperative for DBS,” wrote the bank’s Head of Global Transaction Services, John Laurens, in 2015. “We have invested in hardware – in addition to the SGD 600 million we spend every year, we will invest a further SGD 200 million over the next two years – to ensure DBS offers our customers the very latest in digital solutions. But, more importantly, we have also invested in our people. For example, over 400 of our staff have been involved in ‘hackathons’ that expose them to the digital world and give a first-hand experience of what can be achieved with new forms of rapid product development.”
Five years later, DBS’ Chief Financial Officer Chng Sok Hui, explained in a gushing interview with everyone’s favourite management consultancy firm, Mckinsey & Company, how DBS had essentially morphed from being a traditional bank to “becoming a technology company”:
I think we needed to learn how it is that we can become digital to the core, but I think that we also need to learn how to change in terms of our own mindsets and in terms of the organization culture. And we remind ourselves that it’s not only the executives at the top, that it’s not just the IT folks, that it’s not just the business folks, it’s everyone.
So, we started with a lot of change agenda items. I remember we had human-centered design thinking. In fact, almost everyone had to go through a process of coming up with an app. The other thing we did very well I thought was customer-journey thinking, and thinking about the customer.
So not so much an inward process, but how the customer would actually experience the app that we wanted to put out. So today when I look back I actually see an awesome culture change that has actually happened.
That app stopped working for many of DBS’ customers this week. “The disruption certainly had a broad and likely material impact for many customers, and we would expect to see at least a notice from the Monetary Authority of Singapore (MAS)” or a fine, Business Times reported, quoting analysts.
Roy Kee, managing director of cleaning products manufacturer JRW International: “As a new convert to e-banking, this will shake my confidence a bit. The bank could have also communicated with clients better by sending us SMS messages about the disruption.”
Global Trend?
DBS is not the only large Asian lender to have suffered outages in its digital banking services this year. Mizuho, one of Japan’s three mega-lenders, has had so many IT system glitches — eight so far this year — that they could end up triggering the resignation of the bank’s CEO Tatsufumi Sakai. Nikkei, Kyodo and NHK all reported that the CEO is likely to step down, without naming their sources.
Pakistan’s third largest lender, National Bank of Pakistan (NBP), suffered a cyber attack in early November, which brought down its IT system over an entire weekend.
Beyond Asia, banks in many other parts of the world have suffered from widespread outages and cyber attacks in recent months. Here’s a quick rundown:
- In the last five weeks, just about all of the UK’s large commercial lenders have suffered an IT outage of one form or another. Santander’s app and online services went down for a number of hours on Nov 22. A week earlier, the mobile apps and internet banking services of three large banks, Lloyds, Halifax and Bank of Scotland, which all belong to the Lloyds Banking Group, went down, leaving thousands of customers in the lurch. Two weeks earlier, another high street behemoth, Barclays, suffered a nationwide outage. A week before that, it was the turn of HSBC and NatWest.
- In Mexico the largest lender, BBVA, has so far suffered three system outages in 2021, two of which took place on a Sunday. As such, the bank’s 24 million customers were not only unable to use the bank’s ATMs, its mobile app or in-store payments; they could not even avail of its branch services.
- In October, the IT system of Ecuador’s largest lender, Banco Pichincha, was brought down by a ransomware attack for a number of days.
- In September, Venezuela’s largest lender, Banco de Venezuela, suffered a massive cyber-attack that left its 16 million customers without access to digital banking services for five days. The Venezuelan government blamed the attack on the United States government.
- Two of South Africa’s four biggest banks, Standard Bank and Absa, have suffered a number of interruptions of their online banking services.
- Kiwibank, one of New Zealand’s largest lenders and ANZ Bank, Australia’s third largest lender, both suffered distributed denial-of-service (DDoS) attacks in 2021, leading to a spate of IT system outages. In its latest Financial Stability Review, released in early October, the Reserve Bank of Australia warned that a successful cyber attack on a major financial institution is all but inevitable.
This is all happening at a time when customers and businesses are increasingly dependent on digital payment systems, for a whole host of reasons including: the recent spectacular growth in e-commerce, fuelled in part by the Covid-19 lockdowns; the greater speed, convenience and perceived lower risk of Covid-19 transmission from using contactless payments instead of cash*; and the increasing difficulty of accessing or using cash, in large part due to banks’ widespread closure of branches, ATMs and withdrawal of cash services.
At my own bank in Spain, Caixabank, most customers will now have to pay two euros for the privilege of visiting a branch for a service they could access online. The bank had already limited cash services in its branches to less than three hours a day, from 8:15 am to 11 am. Yet as lenders increasingly nudge their customers away from the branch and physical currency and toward online services, the frequency and scale of bank outages, even at highly sophisticated lenders such as DBS, raises serious questions about the resilience and reliability of those services.
* Numerous studies, including from the ECB and Bank of England, have shown that the risk of Covid-19 transmission from using cash is extremely low.
14 comments
Comments are closed.
To make things worse, most banks seem to be doubling down on making sure you have no alternatives. I’ve been trying to be a refusenik to use app based banking, but in the past month both my credit card and debit card were blocked on transactions and I could only release them by downloading the respective apps.
This of course is parallel to the shutting down of bank branches. Here in Ireland branches are being shut even in quite large towns. In the event of a big outage it will mostly I think be old people and the poor who will find themselves without any means of getting their money. I think most people will have the sense to have a small stash of cash to hand, just in case.
There is a lot of talk her about the Jackpot but personally I think that things will turn out differently. I think that things will develop like John Michael Greer talks about. That you will have a crisis and after it settles down, conditions will be a bit worse but will become the new norm. Sort of like going down a ziggurat pyramid. As an example, look at where American workers were back in the 70s and look at where they are now in the 2020s to see the effect of this process.
The relevance here? I think that rather than just a story, it is part of this process and is systematic as shown by the number of instances that Nick talks about. As things start to degrade, you would expect to see stories of how computer systems for financial institutions and banks start to fail and people can no longer get access to their money. So the take away from this for me is that going forward, it would be a good idea, if possible, to have cash on hand to deal with such outages.
So, in short: Institutional Crapification.
It’ll continue like that until it can’t, until something breaks catastrophically and there is no recovery to a reduced level.
Combining those two theories, you get the Slow Stairstep Jackpot, which I believe is what the Overclass is trying to engineer to play out over the next hundred years or so. Because if too many people get died all at once, the survivors become suspicious.
But if it plays out slowly enough, even the bereaved survivors could read my comment and dismiss it as tinfoil. And as long as that is peoples’ response, the Slow Stairstep Jackpot is successful.
The mass ubiquitous pollution of everything with endocrine disrupters, to keep reducing sperm counts to below a break-even level of fertility, will help the Slow Stairstep Jackpot by prevent theoretical hundreds of millions of future people from ever even being conceived. Which will make the slow killoff easier to achieve.
The “authorities” may try to stealth-prevent cash by limiting its availability and then zeroing it almost out by only allowing cash-withdrawals of $5.00 or $10.00 per day. They wouldn’t try doing that all at once. They would try stealthing down to it in stages.
Many people “make” so little money per unit time that they will never be able to set cash aside. Those who can, should start doing so now. They might put some of it in an “obvious” place so the break-in burglar can find it, congratulate themself on their cleverness, and go away happy. And the rest of it should be very well hidden indeed.
Any people who make enough money above survival payment needs from week to week or month to month to where they can realistically think of storing up some cash, should also stockpile longish-term supplies of whatever they will keep needing for many years or a lifetime, and store just enough that it won’t go bad before being used. One might also store up little unit-dose-packaged hard goods for bartering in agreed-upon-values with. If a certain kind or kinds of ammunition is very popular in your area, perhaps store up enough of it to use as currency. And/or store bottles of cheap alcoholic beverage and hundreds of little screw-cap mini-bottles to pour some of it into for use as “currency”.
And hope that so many millions of people get so angry when they are forced into zero cash and then have their electronic money frozen in digital bank system failures that they form mobs of hundreds of thousands or millions to tear stuff up and burn stuff down. Enough of that might get the authorities to back off if they can.
And if you have a detached house, try super-insulating it and passivizing it, setting up a multi-thousand-gallon roofwater harvesting and storage system, as much intensive yard foodgrowing as possible, a waterless composting toilet system, etc. And prepay mortgage down to zero as fast as possible so that lack of money will not necessarily translate into loss of house.
War games in the UK showed that printed porn was one of the most valuable currencies in a total breakdown.
Cash on hand is a very wise idea. I learned that lesson after the 7.2 earthquake here in the Mojave Desert. Whether it’s a natural disruption or a man-made disruption, in today’s on-line world, there are certain additions that are absolutely necessary for your “Go Bag”, and cash is most definitely one of those additions.
So if things break as you fear Rev, what will money mean? I’m thinking of the people in the failing Weimar Republic enduring hyperinflation with wheelbarrows of cash to buy bread, or the former Soviets selling their possessions during during the marketization period of the early 1990s to get cash. Even if I stuff thousands in my mattress, how much will bread cost and how long will a dollar’s value last? I’m all for grabbing the banks by the lapels and giving them a good shake, but global failure and it’s chaos contagion are something to be feared and avoided IMHO.
The banking sector has trailed almost all others in modernizing their systems, particularly the identity infrastructures that regulate who can access those systems. Ten years after MFA (Multi Factor Authentication) became a best practice, most of them are offering, at best, one time passcodes via insecure SMS text or e-mail. The leadership of these institutions simply don’t take computer security (not “cyber” security, that’s a brand for marketers) seriously, and never have.
That’s mostly a function of the delusion that industry leadership have sufficiently insulated themselves from the consequences of system failure. Coupled with a reckless lack of investment in basic operations and maintenance by most enterprises (whether customer facing or not), we’re now inexorably on our way over the proverbial cliff. Everyone should buckle up and be prepared for The Breakdown of Everything over the next decade.
Well, DBS actually was ahead of the pack because they rolled out MFA devices as early as 201x. Starting this year however, they’ve stopped supporting physical digital tokens, https://www.straitstimes.com/tech/dbs-to-fully-phase-out-use-of-physical-tokens-to-access-digital-banking-services-from-april-1.
Thank you, guys, for posting this.
I’ve never done online banking, more for security reasons than systems failure. I live in a major city in the UK where they don’t deep six the local branches because there is still actual competition.
Every time I go into my local bank, instead of being greeted with, “Hi, how can I help you?” the first words out of clerk’s mouth are, “Why don’t you get with the program and do online banking? It’s so much easier.”
Heretofore, I patiently explained security breaches – known and unknown, possibilities of TLD (“Top Level Domain”) compromise or failure, etc.
Now I will simply smile and hand them a printout of this article.
The main punishment for DBS will come from the fact that it makes people realise they need two banks not just one. Once they have two functioning relationships, switching is much easier. As the main provider of banking services in Singapore, DBS would loose from that process.
Having two functioning banking relationships is like having backup of one’s data : it is a must to counter operational risk.
This event is actually good for online banking : a good candidate for a second bank is a zero fee branch less alternative, for instance a retailer bank ( like Tesco bank in the UK ) or the online subsidiary of a banking network (like Boursorama in France). In the particular case of Singapore, one gets an extra banking relationship for free with a deposit of 1000S$ or, if one is more adventurous gets a fintech account with a company like Wise.
And no, the « good old days » of a single face to face banking relationship were not that rosy : ATM network could fail (without interoperability with other banks),check books could be not available when you were running out of checks, branches could strike or close any time (especially after a robbery). As a reminder, the Irish banking sector strike fifty years ago lasted several months, not several days…
The mega banks are like the meta verse – crappy and quite unnecessary. Physical branches are like cash on hand. Most of your problems can be solved by showing up in person in your small local institution and talking to a real human being.
For those in the US – a simple solution:
https://www.mycreditunion.gov/about-credit-unions/credit-union-locator