Big Data: The most in-demand skills right now
Businesses across every industry are facing unprecedented disruption; with tech giants such as Netflix, Amazon and Uber transforming how services are delivered and uncovering new patterns in consumer habits in the process. Business models are being rewritten, and efficiencies are being found through data-driven operations, as customers increasingly demand personalised experiences.
At the same time, the much talked about General Data Protection Regulation (GDPR) is shaking up how businesses collect, process, manage and secure their data. Designed to give EU citizens more control over their personal information and cut through the complexities that organisations face when complying with opposing local laws, the regulation is having a significant impact on businesses’ internal processes. Against this backdrop, it’s hardly surprising that Big Data professionals are highly sought after by employers. In fact, demand for Big Data expertise has skyrocketed by 78% in the last 12 months, with organisations under increasing pressure to make better use of their data to both maintain a competitive edge and remain compliant with the new regulation.
It’s the insight that can be gleaned from datasets that makes Big Data so powerful. To achieve this, employers must engage with the right professionals − those who can turn raw data into business gold. Here, I consider the reasons why employer demand for Big Data talent is on the rise, as well as the specific skillsets that are currently needed in enterprises.
Turning data into actionable insights
With the steady rise in adoption of sensor technologies, including the Internet of Things (IoT), we can expect datasets to grow and, similarly, the demand for Big Data specialists. More specifically, contractor demand has grown by 128% in the past year and permanent roles by 68% over the same period. This increase outstrips demand across other important skills markets, including Mobile (26%) and Cloud (30%).
Sought after skills
As business leaders continue to grapple with the complexity of running increasingly data-led organisations, the most in-demand skills for specific Big Data roles include AWS, Python, Hadoop, Spark, Cloudera, MongoDB, Hive, Tableau and Java. And, given the specialist nature of these skills, Big Data professionals command some of the highest salaries in the tech industry – earning an average of £67,464 – this is more than the other four tech disciplines tracked in our Tech Cities Job Watch report – Cloud (£60,494), IT Security (£59,342), Mobile (£53,552) and Web Development (£43,198).
When demand for certain skillsets grows, most people would expect the wages to be pushed up, as a result of increased competition. Whilst Big Data salaries remain the highest of the five core disciplines, remuneration has actually stalled in recent months. Permanent salaries increased by just 0.1% in the past year, and contractor day rates were down 5% over the same period.
This could be due to the fact that some Big Data projects have been put on ice in the run up to the GDPR launch, and those with data skills have been reassigned to ensuring that the necessary data processing changes are successful. Thus, those capable of working on higher value – and, by default, higher paying – Big Data projects are making do with ‘GDPR gigs’. Though one would have expected the demand to have peaked some time prior to the GDPR launch, organisations cannot risk being understaffed should their GDPR implementation not perform as intended.
It could also be because of a rebalancing in the market, and the fact that Big Data experts have seen the potential for high earnings and are upskilling themselves with the qualifications they need to secure those lucrative positions. In years gone by, Big Data skills were scarcer and employers had to put their hand in their pocket to secure the best. Now, candidate supply more closely meets demand, which could explain this pay stagnation.
Keeping top talent energised and engaged
Hiring Big Data specialists will enable organisations to compete in the digital era, and businesses should look for candidates who demonstrate an aptitude and enthusiasm to learn new skills, rather than someone who simply has the right credentials on their CV. Once these individuals are through the door, employers must also work to ensure they have processes in place to allow for career development and continuous opportunities to upskill. Team leaders should encourage staff to get involved with different projects and assignments around the business to widen their understanding and provide opportunities to develop their skillsets by completing the latest industry-certified courses.
In this data-driven world, organisations that can turn information into insight, better business decisions and personalised customer experiences are the ones that will truly thrive in the future of work. Big Data skills remain in high demand and businesses must keep a close eye on the market − to ensure they stay one step ahead of the competition and continue to position themselves as an employer of choice.
Martin Ewings, Director of Specialist Markets, Experis
SAS: Improving the British Army’s decision making with data
SAS’ long-standing relationship with the British Army is built on mutual respect and grounded by a reciprocal understanding of each others’ capabilities, strengths, and weaknesses. Roderick Crawford, VP and Country GM for SAS UKI, states that the company’s thorough grasp of the defence sector makes it an ideal partner for the Army as it undergoes its own digital transformation.
“Major General Jon Cole told us that he wanted to enable better, faster decision-making in order to improve operational efficiency,” he explains. Therefore, SAS’ task was to help the British Army realise the “significant potential” of data through the use of artificial intelligence (AI) to automate tasks and conduct complex analysis.
In 2020, the Army invested in the SAS ‘Viya platform’ as an overture to embarking on its new digital roadmap. The goal was to deliver a new way of working that enabled agility, flexibility, faster deployment, and reduced risk and cost: “SAS put a commercial framework in place to free the Army of limits in terms of their access to our tech capabilities.”
Doing so was important not just in terms of facilitating faster innovation but also, in Crawford’s words, to “connect the unconnected.” This means structuring data in a simultaneously secure and accessible manner for all skill levels, from analysts to data engineers and military commanders. The result is that analytics and decision-making that drives innovation and increases collaboration.
Crawford also highlights the importance of the SAS platform’s open nature, “General Cole was very clear that the Army wanted a way to work with other data and analytics tools such as Python. We allow them to do that, but with improved governance and faster delivery capabilities.”
SAS realises that collaboration is at the heart of a strong partnership and has been closely developing a long-term roadmap with the Army. “Although we're separate organisations, we come together to work effectively as one,” says Crawford. “Companies usually find it very easy to partner with SAS because we're a very open, honest, and people-based business by nature.”
With digital technology itself changing with great regularity, it’s safe to imagine that SAS’ own relationship with the Army will become even closer and more diverse. As SAS assists it in enhancing its operational readiness and providing its commanders with a secure view of key data points, Crawford is certain that the company will have a continually valuable role to play.
“As warfare moves into what we might call ‘the grey-zone’, the need to understand, decide, and act on complex information streams and diverse sources has never been more important. AI, computer vision and natural language processing are technologies that we hope to exploit over the next three to five years in conjunction with the Army.”
Fundamentally, data analytics is a tool for gaining valuable insights and expediting the delivery of outcomes. The goal of the two parties’ partnership, concludes Crawford, will be to reach the point where both access to data and decision-making can be performed qualitatively and in real-time.
“SAS is absolutely delighted to have this relationship with the British Army, and across the MOD. It’s a great privilege to be part of the armed forces covenant.”