Adamsquared

Contents

1. Contents
2. Prerequisites
3. Introduction
4. Technical Skills
   1. Coding
   2. Version Control
   3. DevOps
   4. Containerization
   5. Cloud Infrastructure
   6. Operating Systems
   7. Data and Analytics
   8. Security
   9. Machine Learning & AI
5. Soft Skills
   1. Communication
   2. Collaboration
   3. Creative Thinking
   4. Critical Thinking
   5. Problem Solving
6. Summary

Prerequisites

• Optional: A basic understanding of cloud computing. See this article.

Introduction

As organizations increasingly migrate their operations to the cloud, the demand for cloud engineering skills has soared to unprecedented heights. The cloud computing space is rapidly growing in size – according to a report by grand view research, the industry had a market value of around $620 billion as of the end of 2023 and is estimated to grow to over $1.5 trillion by 2030, based on current trends.

The cloud computing space covers a broad range of roles and skills. Due to the high demand for professionals in this industry, there are many high paying roles available. Some example roles in the sector include:

• Cloud Architect: Responsible for designing and implementing cloud-based solutions, including infrastructure, applications, and data storage, to meet business requirements while ensuring scalability, reliability, and security.
• Cloud Engineer: Involved in the implementation, maintenance, and optimization of cloud infrastructure and services. Tasks may include configuring virtual servers, managing databases, setting up networking, and automating deployment processes.
• DevOps Engineer: Builds and maintains CI/CD pipelines, automates infrastructure provisioning, and ensures smooth deployment of cloud hosted applications.
• Cloud Security Engineer: Specializes in securing cloud environments, including designing and implementing security controls, conducting risk assessments, monitoring for threats and vulnerabilities, and responding to security incidents.
• Cloud Administrator: Responsible for managing and maintaining cloud infrastructure, including user accounts, resource allocation, monitoring performance, and troubleshooting issues.
• Data Engineer: Works with big data technologies and cloud platforms to design, build, and maintain data pipelines, analyze data, and derive insights to drive business decision-making.
• Data Scientist: Leverages statistical analysis, machine learning, and data mining techniques to extract insights from data and solve complex business problems.

In this article we’ll look at some of the core skills that are common across many engineering roles in the cloud computing industry. We’ll begin by looking at the technical skills required to develop software for the cloud, and then we’ll explore the soft skills that will help you thrive in a professional setting. Note that this is not an exhaustive list of skills. Specific roles may require additional skills that are not listed in this article.

Technical Skills

In the context of cloud computing, technical skills are competencies that relate to the design, implementation, management and optimization of applications and services hosted in the cloud. In the sections below, we’ll explore a selection of some of the fundamental technical skills that will help you succeed in a cloud engineering role.

Coding

Familiarity with at least one programming language will go a long way to helping you succeed in the cloud. Many services offered by cloud providers require you to provide your own application code or work with an SDK or Framework that is typically provided to you as a library of code. For example, services like AWS lambda, azure functions or google cloud functions allow you to supply code that can be executed in response to some kind of event, such as a HTTP request.

Some popular languages for working with cloud services include java, python, go and javascript. The language you choose will typically depend on which category of engineering you fall into. A data scientist would likely opt for a language like python, which has a rich ecosystem of Machine Learning and AI libraries such as TensorFlow, PyTorch or Scikit-Learn, while a software engineer might opt for a complied language like java, to leverage features such as type safety and fast execution.

Version Control

Version control tools are commonly used in software engineering to manage changes to source code for applications or tools. When working in the cloud, many services will require you to provide your own source code and you will often be required to work as part of a team of engineers to develop and maintain cloud hosted applications. Version control facilitates this collaboration providing a centralized repository where everyone can access the latest version of the code and contribute changes. Some of the main functions of version control include:

1. Tracking changes: Version control systems keep track of changes made to files over time, including who made the changes and when they were made.
2. Reverting to previous versions: If you make a mistake or need to go back to an earlier version of a file, version control allows you to easily revert to a previous state.
3. Branching and merging: Some version control systems allow you to create branches, which are separate lines of development. This enables multiple developers to work on different features simultaneously without interfering with each other’s work. These branches can be merged back into the main codebase when development of a feature is completed, often subject to a successful peer review.

There are two main types of version control. Centralized version control relies on a single central repository for storing files and tracking changes. Decentralized version control gives each developer a complete copy of the repository, including its history, on their local machine. Decentralized version control typically enables more flexible workflows, better redundancy in case of server failures, and enhanced collaboration capabilities over centralized version control, as developers can work independently and synchronize changes later.

Git is the most popular version control system and has many providers such as GitHub, GitLab and Azure DevOps. Other examples of version control systems include Mercurial and Subversion.

DevOps

DevOps is a set of tools and practices that automates software development and IT operations activities in order to simplify and speed up the software development lifecycle.

One of the core principles of DevOps is Continuous Integration and Delivery (CI/CD). CI/CD involves the continuous integration, delivery and deployment of code changes. This is typically achieved by automating tasks such as building, testing and deploying code to production using a CI/CD pipeline tool. Some example CI/CD pipeline tools include Github Actions, Jenkins and CircleCI.

DevOps covers a wide range of topics. In additions to CI/CD, other aspects of DevOps include Infrastructure as Code, Site Reliability Engineering (SRE), and Lean software development.

Containerization

Containerization is a method of packaging, distributing, and running applications and their dependencies in an isolated environment called a container. A Container encapsulates everything an application needs to run, including code, runtime, system libraries and settings, ensuring consistency and portability across different computing environments.

Many fully managed cloud services are run in containers behind the scenes. Some cloud services such as google cloud run, allow you to provide your own container images, containing the application you want to run, along with all of its dependencies. Open source tools such as kubernetes allow automated deployment, scaling and management of containerized applications.

To work with containers, you need a container engine. Docker is one of the most popular container engines for building containerized applications. Docker also operates a public container repository, that provides ready to use container images for many popular applications.

Cloud Infrastructure

Not all cloud hosted applications can or should run on fully managed compute services. Often you will need to have a degree of control over the underlying infrastructure for regulatory or other concerns. As well as having a good understanding of the services offered by your chosen cloud providers, you’ll need to know how to deploy and manage a range of cloud infrastructure components including databases, VPC networks, firewall rules, identity and access management (IAM), virtual machines and more.

Infrastructure as code tools like terraform are one way to provision and manage cloud infrastructure. Instead of manually configuring infrastructure through a GUI or command-line interface, infrastructure is defined using code in a declarative format. This code describes the desired state of the infrastructure, specifying what resources should be provisioned, their configurations, dependencies, and relationships.

Operating Systems

A basic understanding of operating systems commonly used in cloud environments, in particular Linux and Windows, is an important aspect of engineering in the cloud. This includes system administration tasks and command-line interface (CLI) usage. Major cloud providers such as AWS, Azure and GCP provide a free cloud hosted terminal and CLI tools that you can use to perform administration tasks on most services offered by each platform. For example, AWS offers the AWS CLI and GCP offers the gcloud CLI.

Data and Analytics

Data and analytics knowledge is particularly important (but not exclusively so) for data engineers and scientists alike. Data can come from a wide range of sources such as application databases, IOT devices, social media feeds, third party APIs and more. In addition it can arrive in a variety of formats, be of varying quality, contain potentially sensitive information and be delivered in batches or as a continuous stream.

Organizations will often want to use this data to gain valuable insights into business operations and use this to support the decision making process. Before they can do this, the data must undergo transformations such as cleaning, validation, standardization, enrichment, and governance and security checks to ensure the data is consistent, accurate and of high quality.

Once this data is transformed, it is loaded into an analytical database for consumption by decision making processes such as machine learning models, reporting dashboards and more. This process is typically known as extract, transform and load (ETL). Variations of this process exist where the steps may occur in a different order, such as ELT.

Having an understanding of SQL will help you get the most out of your data. Most modern databases allow you to query data using SQL. Example tools in the ETL/ELT space include dbt, Apache Spark, Azure data factory, AWS Glue and Google Cloud Dataflow.

Security

Security in the cloud involves implementing and managing security measures to protect data, applications and infrastructure. This includes measures like encryption, identity and access management (IAM), network security, monitoring and compliance. Defense in depth is a common strategy for securing cloud infrastructure. This involves implementing multiple overlapping layers of security to protect systems and data such as ensuring encryption of data at rest and in transit, following principal of least privilege for access management and using firewall rules to control network traffic.

There are many types of tools that can be used to help secure a cloud environment. These include firewalls, penetration testing tools, Static application security testing (SAST) tools, intrusion detection systems (IDS), security information and event management systems (SIEM), data loss prevention tools (DLP) and antivirus software. Examples include splunk, broadcom DLP and sonarcloud.

Machine Learning & AI

Machine learning (ML) is a subset of artificial intelligence focused on developing algorithms that learn from data. ML algorithms are designed to identify patterns and relationships in data without being explicitly programmed for specific tasks. ML algorithms are used to create models to make predictions or classifications. This enables organizations to forecast future trends and helps support decision making processes.

Artificial Intelligence encompasses a broader range of techniques and methods aimed at creating systems with human-like intelligence. AI includes not only machine learning but also other branches such as natural language processing and neural networks. Increasingly, AI tools are being used to augment and support human decision-making and other activities. Tools like ChatGPT and Google Gemini use large language models (LLMs) to offer insights, suggestions and analyses that complement human expertise.

Soft Skills

Soft skills are important in any professional field and cloud computing is no exception. Soft skills are skills that focus on how you interact and collaborate with others. These skills complement your technical knowledge and allow you to work effectively in a team environment. Below, we discuss a selection of soft skills that will help you succeed in a cloud engineering role.

Communication

Communication includes the the ability to convey information effectively both verbally and through writing. As an engineer, you will often be required to explain complex technical concepts to technical and non-technical audiences, so you will need to be able to adapt your communication style accordingly. You’ll also be required to write clear and concise documentation for your code, produce technical write ups and designs, and give presentations to stakeholders and fellow engineers.

Collaboration

Collaboration involves working with others to achieve a shared goal. This requires pooling your collective resources, knowledge, and skills to create something bigger and better than you could on your own. Engineering in the cloud typically requires you to work as part of a team of people from various disciplines to deliver complex software projects. Good communication skills will help you to collaborate effectively with others. Your colleagues will often bring different strengths and perspectives to the table. Understanding how to leverage these strengths will allow you to build better solutions than you would be possible as an individual endeavor.

Creative Thinking

Cloud computing often involves finding innovative solutions to complex problems. Being able to think creatively, explore different approaches and generate new ideas can help you overcome challenges, troubleshoot complex issues and optimize existing solutions. You can use techniques like brainstorming and experimentation to come up with new ideas for solving business problems.

Critical Thinking

Critical thinking is complimentary to creative thinking. While creative thinking helps you to come up with new ideas, critical thinking helps you to refine ideas and identify the best solutions. For example, after brainstorming to come up with new ideas for tackling a technical problem, you could apply critical thinking to evaluate the feasibility, cost-effectiveness, and security implications of those ideas against certain criteria before implementing the best solution.

Problem Solving

Engineers will often face complex technical challenges. Coding skills are only part of the solution to these challenges. Engineers must also have excellent problem solving skills. Problem solving is the ability to identify, analyze, and develop solutions for these technical challenges. Typically, the problem solving process involves defining what the problem is, identifying the root cause of the problem, finding a solution to the problem, implementing the solution, and then verifying that the solution successfully solves the problem.

Effective problem solving requires you to utilize some of the other soft skills mentioned here. You’ll often need to collaborate with others to identify problems, while using your creative and critical thinking skills to find innovative solutions and identify the best one for the problem at hand. Problem solving can be a difficult skill to learn and requires practice, patience and experience to master.

Summary

In this article we explored some of the core technical and soft skills required to function effectively in cloud engineering related discipline.

In the context of cloud engineering, we defined technical skills as:

Competencies that relate to the design, implementation, management and optimization of applications and services hosted in the cloud

The technical skills we looked at were:

• Coding
• Version Control
• DevOps
• Containerization
• Cloud Infrastructure
• Operating Systems
• Data and Analytics
• Security
• Machine Learning & AI

We defined soft skills as:

Skills that focus on how you interact and collaborate with others. These skills complement your technical knowledge and allow you to work effectively in a team environment.

The soft skills we looked at were:

• Communication
• Collaboration
• Creative Thinking
• Critical Thinking
• Problem Solving

Note that this is not an exhaustive list of skills. These skills represent some of the core skills that can help you succeed in a cloud engineering role, but there are many other skills you may need for a specific role that we have not covered here.