New Kubeflow project brings machine learning to Kubernetes
Google engineers recently announced a new project dubbed Kubeflow, which leverages machine learning to address the hurdles of launching convoluted workloads on Kubernetes.
Kubernetes is an open source platform that serves as the backbone of container orchestration management. The technology allows users to deploy multiple applications on a single, host operating system. According to a 2017 Portworx survey consisting of 491 IT pros from multiple industries, Kubernetes is the most commonly used orchestration tool. Approximately 43% of respondents said they used Kubernetes and 32% said it is their primary orchestration tool.
The purpose of Kubeflow is to use machine learning stacks anywhere Kubernetes is running. Specifically, Kubeflow includes the JupyterHub platform, which enables data science and research groups to create and manage Jupyter notebook servers. Additionally, Kubeflow includes a Tensorflow Customer Resource, which can support either CPUs or GPUs, and be tailored to manage a particular container cluster size.
“This project marks the beginning of the end of the data scientist and/or software engineer as disparate roles,” wrote Philip Winder, an engineer and consultant at Container Solutions, in a company blog post. “Like DevOps has merged operations and development, DataDevOps will consume data science.”
The company also said it believes working with multiple environments, from development to production, will become the norm for most Kubeflow users. Consequently, Kubernetes is making use of the Ksonnet project, which is intended to make it easier to transfer workloads across multiple environments.
Kubernetes is currently working to cultivate a community around the project. Among the companies collaborating on the project include: CaiCloud, Red Hat and OpenShift, Canonical, Weaveworks, Container Solutions, among others.
“The Kubeflow project was a needed advancement to make it significantly easier to set up and productionize machine learning workloads on Kubernetes, and we anticipate that it will greatly expand the opportunity for even more enterprises to embrace the platform,” said Reza Shafii, VP of product, CoreOS. “We look forward to working with the project members in providing tight integration of Kubeflow with Tectonic, the enterprise Kubernetes platform.”
