云原生Tekton CI/CD

Contents

1 Tekon的使用方式

Tekon的使用方式

下载cli文件方式：点击直达官方doc
下载最新版本的tekton配置文件：wget https://github.com/tektoncd/pipeline/releases/download/v0.17.0/release.yaml
下载配套的Dashboard配置：wget https://storage.googleapis.com/tekton-releases/dashboard/latest/tekton-dashboard-release.yaml

Task And TaskRun

Task创建简单的测试用例

创建一个简单的Task例子
- 可以使用params来得到外部传入的声明式参数
- kaniko有很多的参数详情：点击去官方看
- 第一设置好pipelineresource的gitlab地址
- 第二挂载好docker认证信息

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata: 
  name: oreo-client-git
spec:
  # 表示使用的是git
  type: git
  params:
  # 可以设置git tag
  - name: revision
    value: master
  - name: url
    value: http://gitlab4dev.5ypy.com/dongdianyu/oreo-console-client.git
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: build-oreo-client-image-from-git
spec:
  resources:
    inputs:
      - name: workspace
        type: git
  steps:
    - name: build-and-push-harbor
      # 使用kaniko代替docker
      image: harbor.996a.com/test_sync/executor:v1.3.0
      command:
        - /kaniko/executor
      # 传递参数信息
      args:
        - --dockerfile=Dockerfile
        - --destination=harbor.xxx.com/public_images/console-client:$(params.imagetag)
        - --context=/workspace/git-repo
        - --skip-tls-verify
        - --snapshotMode=redo
        - --use-new-run=true
        - --cache=true
        - --cache-dir=/kaniko-cache
      # 指定docker registy的认证信息
      # https://github.com/GoogleContainerTools/kaniko/issues/837
      volumeMounts:
        - mountPath: /kaniko/.docker/
          name: kaniko-secret
  volumes:
    - name: kaniko-secret
      secret:
        secretName: regcred
        items:
          - key: .dockerconfigjson
            path: config.json
  ---
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: test-git-build
spec:
  serviceAccountName: build-sa
  taskRef:
    name: build-oreo-client-image-from-git
  resources:
    inputs:
      - name: workspace
        resourceRef:
          name: oreo-client-git

创建Image后Push到指定Registy地址的信息（base64加密过的）

apiVersion: v1
kind: Secret
metadata:
  name: regcred
  namespace: default
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IuOTk2YS5jb20iOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2V0VvelEzRmtaV2t5
VlU4NSIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTkuMDMuMTIgKGxpbnV4KSIKCX0Kf
Q==

通过base64后解析是docker的认证加密信息
- 当使用docker login后会本地缓存登入的信息(也是base64加密)：cat /root/.docker/config.json
- 创建以上的信息快捷方式：kubectl create secret generic regcred --from-file=.dockerconfigjson=/root/.docker/config.json

# 这种方式也能
kubectl create secret docker-registry <secret名字> \
  --docker-server=<你的镜像仓库服务器> \
  --docker-username=<你的用户名> \
  --docker-password=<你的密码> \
  --docker-email=<你的邮箱地址>

执行git clone和build and Push Harbor完成
除了使用Tekton-Dashboard也可以使用cli：tkn taskrun list来查看结果

Pipeline And PipelineRun

使用Pipeline+gitlab-push的方式去实现流水线功能

使用容器化的kubectl对yaml资源列表自动化部署：点击去lachlanevenson/k8s-kubectl的GitHub地址
- 检测、调试是否能够使用：docker run --rm -v ~/.kube/config:/root/.kube/config lachlanevenson/k8s-kubectl:v1.17.12 get pods --insecure-skip-tls-verify=true查询default的namespace下的pod状态
- lachlanevenson/k8s-kubect会在当前的NS目录下创建SA测试就使用admin权限了 kubectl create clusterrolebinding pipeline-admin-binding --clusterrole=admin --serviceaccount=default:build-sa
首先接着以上的再定义个处理deployment.yaml的Task
记得事先创建好namespace否则发布会报错，也可以整合到depolyment,yaml中

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: deploy-sedconfig-kubectl
spec:
#  如果需要workspace则外部的pipeline或者taskrun也是要声明传递的
#  workspaces:
#     - name: workspace
#       description: The git repo
  resources:
    inputs:
      - name: git-repo
        type: git 
  params:
    - name: imagetag
      description: The Image tag
      default: "latest"
    - name: pathToYamlFile
      description: The path to the yaml file to deploy within the git source
      type: string
      default: "deployment.yaml"
  steps:
    # 对Git clone下来的工作目录资源清单进行修改版本号
    - name: update-yaml
      image: alpine
      command: ["sed"]
      args:
        - "-i"
        - "-e"
        - "s;imagetag;$(params.imagetag);g"
        - "git-repo/$(inputs.params.pathToYamlFile)"
    # 对更改资源清单的进行发布
    - name: apply-deploy
      image: lachlanevenson/k8s-kubectl
      command: ["kubectl"]
      args: 
        - "apply"
        - "-f"
        - "git-repo/$(inputs.params.pathToYamlFile)"

编写Pipeline整合整个Task关联信息，设置编排的过程思想和jenkins如出一则

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: build-oreo-client
spec:
  workspaces:
    - name: git-source
      optional: true
  params:
    - name: imagetag
      type: string
      description: The docker image tag
      default: "laster"
    - name: pathToYamlFile
      type: string
      default: "deployment.yaml"
  resources:
    - name: git-repo
      type: git
  tasks:
    # 执行镜像的打包和git的clone
    - name: git-and-build-pipeline
      taskRef:
        name: build-oreo-client-image-from-git
      params:
        - name: imagetag
          value: "$(params.imagetag)"
      resources:
        inputs:
          - name: git-repo
            resource: git-repo
      timeout: "0h3m30s"
    # 执行修改yaml的task任务
    - name: deploy-sedtag-config-pipeline
      taskRef: 
        name: deploy-sedconfig-kubectl
#      workspaces:
#        - name: workspace
#          workspace: git-source
      resources:
        inputs:
          - name: git-repo
            resource: git-repo
      params:
        - name: imagetag
          value: "$(params.imagetag)"
        - name: pathToYamlFile
          value: "$(params.pathToYamlFile)"
      runAfter:
        - git-and-build-pipeline
      timeout: "0h3m00s"

编写PipelineRun传递pipeline声明的参数，是一个层层递进的关系PipelineRun -> Pipeline -> Task

apiVersion: tekton.dev/v1beta1 
kind: PipelineRun
metadata:
  name: oreo-client-pipeline
spec:
  serviceAccountName: build-sa
  pipelineRef:
    name: build-oreo-client
  resources:
    - name: git-repo
      resourceRef:
        name: oreo-client-git
  params:
    - name: imagetag
      value: "20201021"
    - name: pathToYamlFile
      value: "deployment.yaml"
#  如需要绑定名称空间则实现声明PV/PVC,workspace绑定PVC提供整个pipeline共享volume
#  workspaces:
#    - name: git-source
#      persistentVolumeClaim:
#        claimName: git-source
#      subPath: git-source

创建以上所有的资源对象：kubectl create -f .
跑完后查看资源是否成功的部署：kubectl get pod -n oreo-system

Tekton And Triggers

引用官方：触发器能够让用户将事件有效负载中的字段映射到资源模板中。换句话说，允许事件既可以建模也可以实例化为Kubernetes资源。Tekton Trigger EventController接受来自Request的请求congTriggerBinding中提取事件成为模板的params,在使用tektoncd/pipeline的情况下，很容易将配置封装到PipelineRun和 PipelineResource中。以下为工作流程图：

本质上来说 Triggers 的几个资源对象都是通过了K8s CRD进行拓展实现的
- TriggerTemplate：创建资源的模板，比如用来创建 PipelineResource 和 PipelineRun
- TriggerBinding：校验事件并提取相关字段属性
- ClusterTriggerBinding：一样和 TriggerBinding 类似只是是全局的
- EventListener：连接 TriggerBinding 和 TriggerTemplate 到事件接收器，使用从各个 TriggerBinding 中提取的参数来创建 TriggerTemplate 中指定的 resources，同样通过 interceptor 字段来指定外部服务对事件属性进行预处理
插件地址：官方传送点击直达 Trigger-Github

# 以下为插件的crd安装方式，可替换镜像
kubectl apply --filename https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml

# 查询最新的Tekton状态
(mypy3) ➜  /git  kubectl get pods --namespace tekton-pipelines
NAME                                          READY   STATUS    RESTARTS   AGE
tekton-dashboard-567f7db887-vx7ls             1/1     Running   1          12d
tekton-pipelines-controller-6f6988757-txrlz   1/1     Running   1          12d
tekton-pipelines-webhook-8b4798587-8vfkj      1/1     Running   0          12d
tekton-triggers-controller-f7d94cbf8-pxp8x    1/1     Running   0          37h
tekton-triggers-webhook-9546cbcfb-g6lrg       1/1     Running   0          37h

# 列出当前Tekton的Api资源对象
(mypy3) ➜  /git kubectl api-resources |grep tekton
extensions                        ext,exts          dashboard.tekton.dev           true         Extension
clustertasks                                        tekton.dev                     false        ClusterTask
conditions                                          tekton.dev                     true         Condition
pipelineresources                                   tekton.dev                     true         PipelineResource
pipelineruns                      pr,prs            tekton.dev                     true         PipelineRun
pipelines                                           tekton.dev                     true         Pipeline
runs                                                tekton.dev                     true         Run
taskruns                          tr,trs            tekton.dev                     true         TaskRun
tasks                                               tekton.dev                     true         Task
clustertriggerbindings            ctb               triggers.tekton.dev            false        ClusterTriggerBinding
eventlisteners                    el                triggers.tekton.dev            true         EventListener
triggerbindings                   tb                triggers.tekton.dev            true         TriggerBinding
triggers                          tri               triggers.tekton.dev            true         Trigger
triggertemplates                  tt                triggers.tekton.dev            true         TriggerTemplate

实现一个完成整的自动化CI/CD过程

EventListen自动去监听来自gitlab的Push-Hooks信息触发Trigger预先定义好的Template
Trigger-bingding会绑定来自gitlab-hooks的Json信息注入到Template中
此时Trigger-CRD使用事先绑定好的RBAC权限在K8s集群中创建PipelineRun的资源
PipelineRun会传递参数到Pipeline中执行各个Task，执行失败则抛出异常中断事件，否则会执行完成所有的Task后销毁任务Pod
执行流程结束

第一步配置触发的事件模板

EventListener：该对象本质通过Nodeport、ingress在本地创建一个listen监听服务根据预先定义的监听某些特定的事件来触发TriggerTemplate和TriggerBinding联合来使用
本例字是使用Gitlab配合使用
- 需要gitlab clone的权限
- 需要配置Gitlab的钩子Hooks(其他代码仓库一样)
- 在gitlab项目-> 设置 -> 集成中可以配置
- 需要创建pipelinerun/taskrun的权限(RBAC)

# 配置Trigger-EventListener需要的Secret资源
apiVersion: v1
kind: Secret
metadata:
  name: gitlab-secret
type: Opaque
stringData:
  # 可以手动填写也可以自动生成Request的时候校验的
  secretToken: "asd465qwv6zxc"
---
# Gitlab的认证信息
apiVersion: v1
kind: Secret
metadata:
  name: gitlab-auth
  annotations:
    # 此处是Gitlab的地址：如果是在K8s中支持DNS/svc:port
    tekton.dev/git-0: http://xxx.com/
type: kubernetes.io/basic-auth
stringData:
  username: 
  password: 
--- 
# 配置ServiceAccount权限
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tekton-triggers-gitlab
secrets:
- name: gitlab-secret
- name: gitlab-auth

Trigger-EventListener需要有K8s-SA权限去创建相应的资源对象该serviceAccountName字段是必需的。EventListener接收器用来创建Tekton资源的ServiceAccount
如果使用的的EventListener正在使用ClusterTriggerBindings，将需要一个带ClusterRole的ServiceAccount去操作cluster的权限

# 对指定的NameSpace下的授权
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: tekton-triggers-minimal
  # 不指定会创建到Kube-system里面
  namespace: default
rules:
# Permissions for every EventListener deployment to function
- apiGroups: ["triggers.tekton.dev"]
  resources: ["eventlisteners", "triggerbindings", "triggertemplates", "triggers"]
  verbs: ["get"]
- apiGroups: [""]
  # secrets are only needed for Github/Gitlab interceptors
  resources: ["configmaps", "secrets"]
  verbs: ["get", "list", "watch"]
# Permissions to create resources in associated TriggerTemplates
- apiGroups: ["tekton.dev"]
  resources: ["pipelineruns", "pipelineresources", "taskruns"]
  verbs: ["create"]
- apiGroups: [""]
  resources: ["serviceaccounts"]
  verbs: ["impersonate"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tekton-triggers-binding
  namespace: default
subjects:
  - kind: ServiceAccount
    name: tekton-triggers-gitlab
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: tekton-triggers-minimal

前置条件：创建出来的EventListener会在对应的svc下创建一个监听的节点
- 需要为这个webhook构造ingress提供给外部调用如果没有则考虑走K8s的DNS或者调用在K8s中
- 点击跳转到Ingress资源列表清单地址
使用GitLab拦截器
- serviceAccountName获取之前定义的两个Gitlab的secret需要对应的SA权限
- triggers可以使用多个定义不同的拦截器
- secretRef填充：需要和Gitlab对接的验证器，K8s创建secret并且GitLab Webhook配置为使用该秘密值，和GIT的认证信息
- eventTypes字段填充类型Gitlab 支持的事件类型
- Gitlab-webhook的Request请求会带有这个字段X-Gitlab-Event: Push Hook拦截器通过这个判断
- bindings/template就是接下来需要定义的参数信息了这直接导致后续Pipeline能使用的值是什么

apiVersion: triggers.tekton.dev/v1alpha1
kind: EventListener
metadata:
  name: gitlab-listener-wenhook
spec:
  serviceAccountName: tekton-triggers-gitlab
  triggers:
    - name: trigger
      interceptors:
         - gitlab:
             # 配置gitlab的信息和Token的key
             secretRef:
               secretName: gitlab-auth
               secretKey: gitlab-secret
             eventTypes:
               - Push Hook
      bindings:
        - ref: oreo-pipeline-binding
      template:
        ref: oero-pipeline-template

部署查看情况kubectl get eventlistener关注AVAILABLE是否为True如果false则使用describe查看对应的错误
- 实际上Eventlistener是通过CRD创建了一个POD然后关联svc通过svc+ingress向集群外部暴露出端点接受请求的
- 只要能确定在集群中能够解析出Eventlistener的clusterIP就可以了，如果是ingress另外测试

配置触发的绑定

TriggerTemplate是可以模板化资源的资源。 TriggerTemplate具有可在资源模板中任何位置替换
Gitlab提供的Hooks消息众多而且每个大版本变动也是极大所以建议查看自己Gitlab的帮助信息
- 这里只使用了两个作为测试用例projectnameandtotalcount

apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
  name: pipeline-oreo-binding
spec:
  params:
    # "project":{"name":"Diaspora"}
    - name: projectname
      value: $(body.project.name)
    # "total_commits_count"
    - name: totalcount
      value: $(body.total_commits_count)

配置触发的模板

apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerTemplate
metadata:
  name: pipeline-template
spec:
  params:
  - name: projectname
    description: The git Project Name
    default: oreo-client
  - name: totalcount
    description: The Git Push Total Count
    default: "0"
  resourcetemplates:
  - apiVersion: tekton.dev/v1beta1
    kind: PipelineRun
    metadata:
      name: oreo-client-pipeline
    spec:
      serviceAccountName: build-sa
      pipelineRef:
        name: build-oreo-client
      resources:
        - name: git-repo
          resourceRef:
            name: oreo-client-git
      params:
        # 由于做测试就写死了参数值
        - name: imagetag
          value: "20201021"
        - name: pathToYamlFile
          value: "deployment.yaml"

配置Gitlab

如果在添加时候报错这个Url is blocked: Requests to the local network are not allowed：点击跳转解决问题
去到Gitlab项目的settings --> integrations然后配置相应的信息
- 请求地址就是我们之前的创建的EventListener的K8s中的DNS地址，如果是Ingress则使用域名
- 令牌：是之前SA绑定的Gitlab的Secret-gitlab-token认证信息
- 推送事件可以不写

配置好后我们触发Push操作看看Trigger EventListener做了什么事情

采坑的记录

因为Tekton及其周边插件迭代很快所以在老版本升级新版本的时候一定要去看官方文档否则会有莫名其妙的BUG出现
一定要完整的清理CRD的信息kubectl api-resources|egrep "tekton|trigger"
删除一些CRD残留kubectl get crdand kubectl delete crd xxx

董的博客

沉淀思想，跨步前行，这是生活更是理想！

Tekon的使用方式

Task And TaskRun

Pipeline And PipelineRun

Tekton And Triggers

发表评论取消回复

Tekon的使用方式

Task And TaskRun

Pipeline And PipelineRun

Tekton And Triggers

发表评论 取消回复

发表评论取消回复