How financial institutions can minimise the risk of IT outages
Paul Mercina is the Director of Product Management at Park Place Technologies. Since 1991, Park Place Technologies has provided an alternative to post-warranty storage, server and networking hardware maintenance for IT data centres. As the world’s largest pure play post-warranty data centre maintenance organisation, Park Place supports tens of thousands of client organisations around the globe. Here he shares with us the measures than can be taken to prevent IT outages, and at worst, how to handle them.
How financial institutions can minimise the risk of IT outages
The cost and impact of IT system downtime has never been greater due to businesses’ increasing dependence on IT systems and infrastructure across all areas of their operations. Any system outage can have catastrophic impact on an organisation in terms of costs, lost trade and reputation. Research claims the average cost of a typical outage is over £105,000, with more catastrophic outages costing even more. With companies facing up to three outages per month, this equates to almost £4million every year for financial institutions.
Many high street banks have made the headlines in the last year after suffering system outages breaching customers’ data and affecting their access to accounts. Earlier this year, HSBC suffered technical difficulties with their personal and business online banking accounts, which caused major disruption to their customers who were not able to access their accounts for over an hour. HSBC are by no means the only bank to experience technical issues, as a survey by Which? found they are a regular occurrence in the UK. In the past year alone, the survey found one in seven consumers had experienced at least one issue using their card due to IT outages at their bank, and one in seven experienced multiple problems throughout the year. Not only does this cause significant inconvenience to customers, but 10% said they had been hit with financial penalties as a result of defaulted payments caused by an outage, and 9% said their credit score had been damaged.
Banks are, therefore, under increasing pressure from politicians and regulators to improve their response to IT problems. In November last year the Financial Conduct Authority said it was “deeply concerned” after finding that technology outages had more than doubled over the preceding 12 months, while the Treasury Select Committee launched an inquiry into the issue. The Bank of England has also threatened banks with higher capital charges if they do not do enough to deal with technical problems.
So how can financial organisations minimise the risk of IT failure causing them to become the next unwanted headline?
Prevention is better than cure
The best way to avoid losing revenue, reputation and customers is to prevent outages, especially the type of routine failures that can’t be blamed on a major disaster. Adopting best practice processes - such as running regular threat and vulnerability assessments, conducting configuration reviews and including operation process validation checkpoints - can significantly reduce your chances of suffering from a systems failure.
Testing of different systems requires time and resources that can sometimes be difficult to justify. However, it’s important to remember that thorough, targeted real-life testing can reveal incompatibilities, glitches and capacity issues unforeseen at planning stages. It was reported that one of the key causes of the Lloyds Banking Group outage which left customers unable to access their online banking services was the result of various systems not being as thoroughly tested as they should have been when accounts were migrated to the Group’s new core banking platform.
Staff engagement and training
According to report by the Ponemon Institute, human error is the second most common cause for system failure - accounting for 22% of all incidents. Employees must be regularly trained on how to avoid an outage as well as how to mitigate the damage and impact should one occur. Within financial organisations staff will be using a myriad of complex systems and technologies and it’s important to remember these technologies are only ever as good as the people using them. Clear, precise and regular usage guidance is imperative to minimise the chances of human error.
Remain vigilant at all times
Vigilance should be an essential part of any financial organisation’s IT strategy. Organisations should be working with an IT managed service provider to ensure that they are always following up to date best practice guidelines and pro-actively questioning their IT set-up and the associated risks.
Well-rehearsed recovery plan
Although an IT outage is sometimes unavoidable, prolonged downtime does not have to be. Having a well-rehearsed business continuity plan in place can help to mitigate the impact of any system failures.
Any business continuity plan needs an executive owner/sponsor who has the experience and authority to get things done in a timely and processed manner. All action plans should be regularly reviewed at board level and shared with all stakeholders across the organisation so that all the risks and organisational implications are planned for to avoid its implementation being hampered by budget or knowledge constraints.
Fastly's CDN Reportedly to Blame for Global Internet Outage
A huge outage has brought down a number of major websites around the world. Among those affected are gov.uk, Hulu, PayPal, Vimeo, and news outlets such as CNN, The Guardian, The New York Times, BBC, and Financial Times.
It is thought a glitch at Fastly ─ a popular CDN provider ─ is causing the worldwide issue. Fastly has confirmed it’s facing an outage on its status website but fails to specify a reason for the fault ─ only that the problem isn’t limited to a single data centre and, instead, is a “global CDN disruption” that is potentially affecting the company’s global network.
“We’re currently investigating potential impact to performance with our CDN services,” the firm said.
What is Fastly?
Fastly is a content delivery network (CDN) company that helps users view digital content more quickly. The company also provides security, video delivery, and so-called edge computing services. They use strategically distributed, highly performant POPs to help move data and applications closer to users and deliver up-to-date content quickly.
The firm has been proving increasingly popular among leading media websites. After going public on the New York Stock Exchange in 2019, shares rose exponentially in price, but after today’s outages, Fastly’s value has taken a sharp 5.21% fall and are currently trading at US$48.06.
What are CDNs?
Content delivery networks (CDNs) are a web of small computers, or servers, that link together to collaborate as a single computer. CDNs improve the performance of internet-connected devices by placing these servers as close as possible to the people using those devices in different locations, creating hundreds of points of presence, otherwise known as POPs.
They help minimise delays in loading web page content by reducing the physical distance between the server and the user. This helps users around the world view the same high-quality content without slow loading times.
Without a CDN, content origin servers must respond to every single end-user request. This results in significant traffic to the origin and subsequent load, thereby increasing the chances for origin failure if the traffic spikes are exceedingly high or if the load is persistent.
The Risk of CDNs
Over time, developers have attempted to protect users from the dangers of overreliance through the implementation of load balancing, DDoS (Denial of Service) protection, web application firewalls, and a myriad of other security features.
Clearly, by the state of today’s major website outage, these measures aren’t enough. Evidently, CDNs present a risk factor that is widely underestimated ─ which needs to be rectified with haste. Content delivery networks have become a key part of the global infrastructure, and so it’s imperative that organisations start to figure out risk mitigation strategies to protect companies reliant on the interconnected service from further disruption and disarray.