Skip to content

Home

Jump to bottom
Yi Wang edited this page Dec 24, 2019 · 2 revisions

Fluid

fluid is a transpiler that converts a Python program into a YAML file representing a workflow that can be executed by Tekton Pipeline.

fluid accepts a subset of Python syntax, namely, function definitions and function invocations. A comprehensive understandign of fluid is an Python compiler that outputs Kubernetes-native programs.

Concepts

Tekton Pipeline is a Kubernetes-native workflow engine. When we say Kubernetes-native, we mean that users could submit workflow YAML using the official Kubernetes client kubectl, and any Kubernetes cluster with Tekton Pipeline installed could understand the content of these YAML files.

Kuberenetes Objects

Kubernetes understands YAML files that define Kubernetes objects. Each Kubernetes object includes the following required fields.

  • apiVersion - Which version of the Kubernetes API you’re using to create this object.
  • kind - What kind of object you want to create.
  • metadata - Data that helps uniquely identify the object, including a name string, UID, and optional namespace.
  • spec - What state you desire for the object.

NOTE: Kubernetes objects is also known as Kubernetes resource.

Tekton Pipeline Objects

According to the official document, with Tekton Piplines installed, a Kubernetes cluster could understand the following kinds of Kubernetes objects.

The value of apiVersion of all Tekton Pipeline objects is tekton.dev/v1alpha1. The value of kind is one of the above. The value of metadata and spec depends on the kind.

Task as Function Definition

According to the definition, a Task looks like a a function definition. It consists of

  • inputs, like parameters of a function,
  • outputs, like return values of a funciton, and
  • steps, compose the function body.

The function name is specified in the the field name under metadata.

Here follows a Task example from Tekton's tutorial.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: echo-hello-world
spec:
  steps:
    - name: echo
      image: ubuntu
      command:
        - echo
      args:
        - "hello world"

We hope Fluid users could represent it with the following Python function definition.

import fluid

def echo_hello_word():
    fluid.run_container(image="ubuntu", cmd="echo", args=["hello world"])

TaskRun as Function Invocation

A TaskRun object defines a call to a Task. The following is a TaskRun example from Tekton's tutorial.

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: echo-hello-world-task-run
spec:
  taskRef:
    name: echo-hello-world

We hope Fluid users could represent it with the following function invocation.

echo_hello_word()

Please be aware that the above line of Python/Fluid code doesn't name the invocation as something like echo-hello-world-task-run the the above YAML file. To generate the above YAML file, we would need to derive a TaskRun name from the invocation, which could be the function name plus the file and line number of the invocation.

PipeLineResource as Struct/Class

A Tekton PipelineResource object is like a variable of struct type. In Tekton tutorila, the following example shows a variable of struct git, which has two fields: revision and url.

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: skaffold-git
spec:
  type: git
  params:
    - name: revision
      value: master
    - name: url
      value: https://github.com/GoogleContainerTools/skaffold

We hope Fluid users could represent it by the following line.

skaffold_git = fluid.Git(
    revision="master", 
    url="https://github.com/GoogleContainerTools/skaffold)

Please be aware that the call to fluid.Git doesn't include the name skaffold-git in the above YAML file. To generate a name, fluid.Git needs to use some heuristics. The generated name should be returned and saved in the Python variable skaffold_git, so it could be referred by some other Tekton objects.

Tekton also provides a PipelineResource of type image that refers to a Docker image.

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: skaffold-image-leeroy-web
spec:
  type: image
  params:
    - name: url
      value: gcr.io/wangkuiyi/leeroy-web

We hope Fluid users could represent the above YAML file by the following line.

skaffold_image_leeroy_web = fluid.Image(url="gcr.io/wangkuiyi/leeroy-web")

Task with Inputs and Outputs

The Tekton tutorial presents the following example Task object with inputs and outputs.

goapiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: build-docker-image-from-git-source
spec:
  inputs:
    resources:
      - name: docker-source
        type: git
    params:
      - name: pathToDockerFile
        type: string
        description: The path to the dockerfile to build
        default: /workspace/docker-source/Dockerfile
      - name: pathToContext
        type: string
        description:
          The build context used by Kaniko
          (https://github.com/GoogleContainerTools/kaniko#kaniko-build-contexts)
        default: /workspace/docker-source
  outputs:
    resources:
      - name: builtImage
        type: image
  steps:
    - name: build-and-push
      image: gcr.io/kaniko-project/executor:v0.14.0
      # specifying DOCKER_CONFIG is required to allow kaniko to detect docker credential
      env:
        - name: "DOCKER_CONFIG"
          value: "/tekton/home/.docker/"
      command:
        - /kaniko/executor
      args:
        - --dockerfile=$(inputs.params.pathToDockerFile)
        - --destination=$(outputs.resources.builtImage.url)
        - --context=$(inputs.params.pathToContext)

We hope Fluid users could represent it by the following Python/Fluid code.

def build_docker_image_from_git_source(
Clone this wiki locally