Open source: turbocharging digitalisation
From operating systems to security and data-flow engines, transparency and inclusiveness are the essence of open-source. By sharing global knowledge, developers are inspired to innovate, create and, ultimately, facilitate the optimisation of organisational operations. Cloud computing echoes this spirit by providing flexible deployment and collaboration for open-source projects, making it accessible for a wider audience.
While open-source software has clear benefits, it is not easy to start without prior experience and support. Against that backdrop, cloud makes it much easier to get started. Technology providers can help developers begin their open-source journey, by providing – for example - real-world scenario experiments, free consultation and support on the integration and application lifecycle management. This all helps to inject confidence from the start of their journey.
At Alibaba Cloud, we embrace and pursue the open-source philosophy holistically and consistently, across platforms, industries and countries. We invest in both the technology and in building up the knowledge of the developer community in order to promote and accelerate digitalisation.
Driving real-time stream computing
Data processing and analytics have been growing at a rapid pace over the last decade, and the trend is rising. In the next two years alone, data volumes are expected to more than double. In order to handle the enormous amount of information and make it applicable, powerful computing functions and engines are required to enable data to be processed in the fastest possible time.
With this in mind, Apache Flink, the open-source technology for real-time stream computing, was developed. The software started as a research project at the Technical University of Berlin, which was then donated to the Apache Software Foundation. Within a few years, Flink has become one of the fastest growing communities in the Apache Software Foundation. While traditional batch computing engines like Hadoop and Spark process finite data sets, Apache Flink focuses on stream computing and is able to ingest, analyze, and process data in real time to deliver impressive results as quickly as possible.
Using Flink as the framework, Alibaba has made several significant improvements in scalability, reliability and performance as well as developing its own Flink branch, called "Blink." Shortly thereafter, Alibaba partnered with Ververica (previously known as Data Artisans), the pioneer of real-time enterprise stream processing and analytics, and contribute "blink" to the Apache Flink community. As of today, Alibaba has contributed over a million lines of code to Flink and, together with Ververica, contributed to more than 70 percent of the core improvements in the Flink community. Today, global companies such as ING and Uber have used Flink as the stream processing engine to power large-scale stateful applications, including real-time analytics, machine learning, search and content ranking and fraud detection.
Benefiting the AI development
As we look ahead, the development of AI in particular can further benefit from open source. In the open-source community, global developers can participate in the fine-tuning of open-sourced machine learning models, accelerating the pace at which tailored AI applications can be rolled out to meet the different demands of businesses across sectors.
A more dynamic and creative ecosystem where developers can collaborate and exchange information, is beneficial in generating creative AI solutions. Small companies or individual developers in particular can benefit from the open-source community as they can access a wealth of tools and resources in a cost-effective way. This drives further growth in the development and implementation of different AI applications and solutions.
In the meantime, by making source code available, developers can also work together to identify and correct flaws within the models more rapidly, and together address common concerns of AI around safety, transparency, security and privacy. As a result, this can enhance the reliability of AI tools and lead to the more responsible development of AI models.
For example, Alibaba has made the source code of its latest federated learning platform FederatedScope, a federated learning platform with easy-to-use packages, accessible to the open-source community. By sharing our self-developed federated learning technologies with the open-source community, we hope to promote the research and industrial deployment of privacy-preserving computation in different sectors. These include the healthcare and smart mobility sectors, which usually involves sensitive user data and requires strict privacy protection practices.
Contributing to the developer community
Collaboration and contribution is critical to the sustained growth of the open source community. That's why we are actively involved in open-source initiatives such as the Cloud Native Computing Foundation (CNCF), the Linux Foundation, and RISC-V International. In addition, we have contributed to more than 2,700 open-source projects on Github.
With our newly-launched Apsara global developer community, we’ve gone one step further to empowering developers: on this platform, they can find resources and information that best equip them for digitalisation trends such as AI or Web3. The community aims to bridge the gap between theory and practice, with training materials, tutorials, developer tools and projects. Meanwhile, forums, blogs, community events and hackathons give developers the opportunity to network both online and offline and compete with like-minded people from all over the world. Therefore, we hope that more developers globally can also participate and contribute to the open-source community.
Today, open source plays a vital role in driving the development of cloud computing, AI and other frontier technologies. We believe the future of open source – and all that it can help deliver – is extremely promising. Holding true to the principles of transparency and openness, the global open source community is destined to grow at a faster, turbo-charged pace, if we all continue to contribute to it by sharing our advanced technologies, expertise and capabilities.
