AI is all the rage, with VC-funded start-ups popping up weekly offering increasingly odd services, like the ability to chat with your favorite dead scientist, game character, or a lump of cheese. But a tech skill set that remains in demand–and has less potential to be weird and creepy–is Kubernetes (K8s) and container orchestration.

It has been over ten years since Kubernetes was released, so you would expect any Kubernetes skills shortage to be resolved by now, but while demand remains high, supply is still falling short.

Currently, over 29,600 vacancies on LinkedIn mention ‘Kubernetes,’ and a search of Indeed.com for roles referencing ‘Kubernetes’ brings up over 15,000 positions that need filling. The rewards are also significant for anyone who acquires the right skill set: The average salary for a Kubernetes engineer is $146,516 a year. So, how is this shortage affecting enterprises operating and maintaining K8s?

Skeleton Crews: The impact of the Kubernetes Skill Shortage

We like our ship analogies at 360 Cloud Platforms; so operating Kubernetes with a skills shortage is like running a skeleton crew.

Most captains will tell you they prefer not to be placed in such a position and only choose it when they have experienced hands and no other option. It’s a situation that brings many negatives (as well as a few positives, e.g., reduced short-term costs):

Reduced Maneuverability: On a ship, this becomes more noticeable in adverse conditions or complex situations, potentially leading to more mistakes. For teams operating and maintaining Kubernetes, it creates a lack of agility to respond to fast-moving threats and robs them of the ability to implement platform improvements as time and resources are constrained.

Heavy Workloads: Every crew member has to take on more duties and work longer, which can lead to fatigue and extra stress and create safety issues as focus lapses and, again, mistakes happen. In tech enterprises, increased workloads lead to burnout and retention issues. According to Yerbo’s BurnoutIndex 2022, 42.1% of tech employees are working under a high risk of burnout, and 2 in 5 of those workers want to leave their company because of it.

Non-Compliance Creep: Not enough crew can mean non-compliance, as there aren’t enough people on your team or people with the right experience to follow regulations. The same applies to a platform team, which can have severe legal and financial repercussions as well as create potential security vulnerabilities.

Operational Delays: For a ship, routine checks and operational tasks, such as loading and unloading cargo, are delayed when you’re not operating at capacity, which is what happens to a platform team and can potentially lead to tests, audits and routine checks running behind schedule and impacting business profitability.

Maintenance Decline: You must remove those barnacles from the hull as they create drag and waste fuel. Without enough crew, routine maintenance and repairs can get deferred or not carried out as effectively. This decline can increase wear and tear on a ship and equipment. Falling behind with maintenance is a common issue with Kubernetes, DataDog’s 2022 Container Report found that most hosts use a version of K8s that is over 18 months old, but a bigger problem with poor maintenance, stemming from K8s skill shortage is the higher risk of service outages. Ultimately, you risk ‘a sinking ship’ when changes and upgrades aren’t carried out regularly.

Kubernetes: In Demand but Demanding

In the DevOps Institute’s Upskilling IT Report 2023, team leaders ranked container orchestration as the must-have skill (54%) for their engineers. The next in-demand skill was security and cybersecurity at 50%, which also ties heavily into K8s adoption as security concerns have become a critical issue. In fact, security has become such a shadow over K8s that this year, Red Hat’s State of Kubernetes Security, 2023, found that it was directly to blame for delayed or slowed app deployments as enterprises hesitate to trust their cluster configurations in production.

The DevOps Institute’s 2023 report also found that the top two skill gaps within a team were cloud compute platform (33%) and container orchestration (26%).

A more worrying K8s skill gap statistic is in this year’s VMware State of Kubernetes. The report found that the biggest challenge in managing Kubernetes was a lack of internal expertise at 57% of respondents, an increase of 13% since 2022.

The lack of skilled labor hasn’t dampened popularity, although you could argue it has delayed some organizations from progressing apps as quickly into production from PoCs and experimenting in test environments. Today, containers have become the preferred way to wrap applications and their dependencies as they enable consistent deployments across environments. Containers are lightweight and portable, and according to the last Cloud Native Computing Foundation (CNCF) Annual Report in 2022, they are now used by 44% of organizations for “most or all production applications and business segments.”

Meanwhile, DataDog’s 2022 Container Report suggests that half of all the containers it monitors (which for the report was roughly 1.6 billion) are using Kubernetes as the preferred container orchestration platform. Of course, that extends far beyond upstream Kubernetes into a vast landscape of many managed Kubernetes services from cloud providers, such AWS (EKS), Google Cloud Platform (GKE), and Microsoft Azure (AKS), vendor-specific K8s distributions and vendor-managed Kubernetes services, such as Red Hat Open Shift and VMware Tanzu. Regardless of what type of Kubernetes organizations are using, K8s has become the most popular framework for automating the deployment, scaling, and management of containerized applications, even if there is an ongoing Kubernetes skill shortage.

Several factors have exacerbated the K8s skill shortage. When Google open-sourced K8s, it was just the starting pistol for a platform designed to be extensible. Since 2014, we have seen an ever-growing ecosystem branching out around the platform. Even for an open-source project, K8s sees frequent feature updates, while best practice models constantly evolve along with new processes and frameworks.

Essentially, Kubernetes is a complex platform to master, which makes finding suitable candidates challenging; meanwhile, demographic pressures–affecting all skilled labor–have worsened the K8s skill shortage.

The issue is not only about a Kubernetes skill gap or shortage; talent retention is also a problem. The common consensus among engineers is that they should move companies roughly every two years or risk ‘leaving money on the table.’ This length of job tenure is effectively half the US national average of 4.2 years.

While ensuring good remuneration can help keep talent for longer, engineers are influenced by other considerations. According to Hired’s 2023 State of Software Engineers report, ‘flexible work schedules’ are still ranked the number one priority (76%), followed by co-workers they like and can learn from, the promise of great career growth and personal development, and a good boss.

Ways to Navigate the Kubernetes Skills Shortage

There’s a lot of doom and gloom around the Kubernetes skills shortage, but there are several things you can do to address the problem:

1. Invest in your team: Develop training and development programs for existing team members and potential candidates within the organization. As well as free training from vendors whose platforms and products you may choose, the CNCF has tried to fill the shortage with courses and certifications from Kubernetes Fundamentals through to accreditation as a Kubernetes Administrator (CKA), application developer (CKAD), and security specialist (CKS). For a more hands-on experience, many education providers, such as KodeKloud, supply sandboxed environments for practicing scenarios.
    
2. Encourage Mentoring: A lack of K8s skills in internal teams is a core problem, so a mentorship program is invaluable for upskilling existing employees. Bringing in an experienced Kubernetes engineer to work alongside a less experienced team member encourages knowledge transfer, which is why many organizations partner with Kubernetes support services to supplement their teams in areas such as operations, maintenance, and support. As well as supplementing a less experienced team with a pool of Kubernetes experts and benefiting from options such as on-demand support, a Kubernetes support partner will have the necessary ‘soft skills’ from their experience collaborating in open source projects and large enterprise-scale digital transformations.
    
3. Create Specialists: By defining and understanding the components that you use in your K8s ecosystem, you can create sub-teams or specialist engineers that focus on specific components, for example, deployment pipelines. You will want to consider separating platform operations and application development operations as well, but this specialization approach has the potential to motivate team members who are keen to develop expertise in a specific area. It also creates opportunities to develop deep knowledge in numerous technologies if individuals rotate their roles.
    
4. Automate Everything: As mentioned, heavy workloads lead to burnout and talent drain, so automate as much as you can. In terms of app deployments, for example, Kubernetes is concerned with reducing delivery friction and accelerating the delivery of new app features. There are many ways of automating the deployment process with a mix of tools, such as Argo CD, GitHub Actions, and cloud-provider-specific tools, e.g., Azure DevOps, but you need to unload your engineers and streamline the path to production. A simple solution is to choose the automation tool based on your teams’ most popular, efficient, and ready-adopted solution.
    
5. Collaborate with Community: Encourage a culture of learning and knowledge-sharing within your teams and with external teams at other organizations. Your engineers should be linked in with open source projects they use and their communities and developing relationships with external engineers who experience similar problems. Contributing to relevant open-source projects, even if it means posting feedback or reviewing minor tickets, is a way to find future team members, mentors, and friendly experts.
    

With Kubernetes becoming the ‘OS of the Cloud,’ the Kubernetes skill shortage is unlikely to be resolved quickly as the ecosystem continues to mature and grow, but by supporting your existing teams by supplementing them with experts, establishing training programs and developing a culture of collaboration, specialists and knowledge-sharing you can weather the storm.

Contact Us

Filling your team's Kubernetes skill shortage or skill gaps is challenging. Connect with us today to discover how we can reduce workloads and support and work with your teams.

CONNECT WITH US