Harnessing the cloud-native winds of change

• Cloud native applications now underpin high-performance websites
• “Internal skills shortage and a steep learning curve” among challenges

"When the winds of change blow, some people build walls and others build windmills." Thus goes an old Chinese proverb used in countless strategic presentations about how important it is to go digital. Well, if you're smart, not only will you have built windmills, you've also built the shops powered by it - and are hard at work selling even more windmills to others.

This was demonstrated in part last month during Alibaba's 11.11 Global Shopping Festival, originally called Singles Day, presumably because what else is there to do if you're alone but shop? Well, this year’s festival generated RMB498.2bil (RM308bil) in gross merchandise value (GMV) over an 11-day period. Demand was so great that after 26 seconds of shopping on 11 November, the systems were receiving 583,000 orders per second.

Many of the merchants on board had taken advantage of the Alibaba Business Operating System (ABOS), launched in January 2019, which offers customers a one-stop menu of digital services, including retail, payment, digital marketing, logistics, cloud-computing, and on-demand local services. It is also an invitation for companies to begin their journey to become web native, and follow global brands such as McDonald's and Starbucks.
 

Containers in the cloud
But what exactly does it mean to go cloud native? It is in recognition that if you have an application that will be running in the cloud, then the way you develop it should ideally take full advantage of the cloud.

For example, using cloud-native technologies enables cloud portability, giving you control of where to host your solution, thereby avoiding vendor lock-in. It also allows your cloud provider to easily scale up computing power and resources if there is greater demand.

At the moment, cloud-native development means using containers, which are fully packaged and portable computing environments. A container 'contains' everything an application needs to run, and in many ways it's similar to a virtual machine (VM), except that the container is abstracted away from a host operating system (OS). Within the containers, the applications themselves are very small microservices that start up and shut down as and when they are required.

This means a container does not need to start up a separate guest OS for each application, and the cloud provider can dynamically orchestrate the underlying cluster, for example, to cater for the CPU load and the network traffic at the time.

Billions of items per second, exabytes of data per day
For Alibaba, this meant being able to better manage the heavy loads that hosting the world’s largest shopping festival requires. They had migrated all of their core systems onto Alibaba Cloud in anticipation of last year’s sales, but this year’s event pushed the envelope even further.

The system had to process 4 billion items per second at peak periods, almost twice the 2.5 billion number last year. MaxCompute (Alibaba’s proprietary data warehousing platform) had to manage up to 1.7 exabytes (or 1.7 billion gigabytes) of data in a day. And the various databases were taxed to the limit: PolarDB set a new record with 140 million queries per second, and AnalyticDB (Alibaba Cloud’s self-developed cloud native data warehouse) processed up to 7.7 trillion lines of real-time data.

What's perhaps more remarkable is that they actually managed this with an 80% reduction of computing resources for every 10,000 transactions conducted, compared to four years ago. This is the power of cloud-native applications.

As popularity rises, skills are in demand

It is this sort of performance benefit that has in part seen the rise of cloud native development, along with the fact that containers make development more modular and easier to manage. The 2019 Cloud Native Computing Foundation (CNCF) survey found that production container usage was 23% in 2016, but had rapidly grown to 84% in 2019. Meanwhile, between 2015 and 2019, the proportion of jobs that needed Kubernetes-related skills (a popular container type) grew by 2,141%.

As a result, there is demand for the skills. The State of Containers and Kubernetes Security Report 2020 revealed that 70% of those surveyed cited "an internal skills shortage and a steep learning curve as the two most significant Kubernetes challenges impacting their companies", while the CNCF survey reported that the challenges in using containers include 38% who said "complexity" was issue, with just over 30% highlighting, "lack of training".

It is no surprise that countries like Singapore have launched initiatives like GoCloud that aims to help ICT SMEs move to a cloud native architecture by offering training and certification services over a six-month period.
Meanwhile, even private companies like Alibaba see value in offering free introductory courses on cloud computing, with additional courses that eventually lead to certification.

It doesn’t look like it will settle down soon. Gartner predicts that by 2022, more than 75% of global organizations will be running containerized applications in production (up from less than 30% today), while worldwide container management revenue will grow from US$465.8 million in 2020, to US$944 million in 2024.

Instead of following the wind, why not harness it?