kubernetes下搭建harbor企业级docker仓库

www.allocmem.com · · 557 次点击 · · 开始浏览    
这是一个创建于 的文章,其中的信息可能已经有所发展或是发生改变。

前言

harbor是由vmware开源的企业级docker repository,提供私有仓库,安全认证,权限管理,漏洞扫描及仓库复制等多种功能,因为之前搭建的harbor在线上环境存在各种各样的问题(之前搭建的方式目前发现已被官方宣布弃用https://github.com/vmware/harbor/blob/master/docs/kubernetes_deployment.md,如ui显示不正常,权限认证不正常),于是近期抽空用官方推荐的helm方式对harbor进行重新部署,并替换掉线上harbor

准备

  1. Kubernetes集群1.8+
  2. Kubernetes Ingress Controller已启用(这里我们选用的traefik-ingress)
  3. kubectl 客户端 1.8+
  4. 可选的持久化功能需要准备PV或者SC(我们这里使用nfs创建的PV和PVC,如果有条件使用SC的会更加方便)

部署过程

  1. 首先按照官方教程安装Helm,然后初始化Helm。

    注意: 初始化需要使用下面命令使用canary镜像,否则无法正常安装,会报错helm部署文件的格式不正确,目前这是一个已知issue:https://github.com/vmware/harbor/issues/4484

    helm init --canary-image
    

    如果你和我一样,在这之前已经用官方教程安装好了helm,则可使用下面命令更新helm server:

    helm init --canary-image --upgrade
    
  2. 下载helm部署代码并进入harbor helm目录。

    git clone https://github.com/vmware/harbor
    cd harbor/contrib/helm/harbor
    

    推荐使用这个连接下载指定的文件夹,否则在网络不佳的情况下拉取全部代码会比较耗时:

    https://minhaskamal.github.io/DownGit/#/home
    
  3. 更新helm dependency

    harbor的helm部署依赖了postgresql的helm,在官方的安装文档没有明确说明,因为我也是第一次使用,直接按照官方文档说明安装,就会缺失postgresql的部署,导致整个服务无法启动

    helm dependency update
    
  4. 安装harbor

    这里官方提供了两种方式,Insecure和Secure,我这里选用的是Secure安全的部署方式,让harbor自己生成CA和SSL,简单方便。执行如下命令安装harbor:

    helm install . --debug --name hub --set externalDomain=harbor.my.domain
    

    externalDomain为外部能访问到harbor的域名,到目前为止,你可以在本地/etc/hosts中添加域名解析,从本地访问进行测试,待测试完成后再加入traefik-ingress中,当然你也可以直接在traefik-ingress中添加域名解析。

  5. 添加traefik-ingress的域名解析

        apiVersion: extensions/v1beta1
        kind: Ingress
        metadata:
          name: traefik-default-ingress
          namespace: default
          annotations:
            kubernetes.io/ingress.class: "traefik"
        spec:
          rules:
          - host: harbor.my.domain
            http:
              paths:
              - path: /
                backend:
                  serviceName: my-release-harbor-ui
                  servicePort: 80
    

helm harbor自定义配置

注意: 我这里把所有自定义配置放到后面的附录中,供大家参考,同时可以点击https://github.com/Anteoy/harbor-helm/commit/226b296d130b4f956f8463eecf2aa473bc1e844c,查看我上传到github的自定义配置,从github阅读更清晰。

  1. 重要:在生产环境中需要持久化数据存储,否则pod重启或重建会造成数据丢失。如果有创建好的storageClass可以直接在values.yaml配置,如果没有或暂时不能使用SC的,比如我这里只有nfs,则需要像我一样多修改一些配置。另外,nfs其实在社区也有storageClass的驱动库,不过我看了下使用起来较为繁琐,于是这里仍然修改为使用nfs。
  2. 修改templates默认的namespace,注意需要修改依赖的chart文件夹下的postgresql-0.9-1.tgz压缩文件中的部署yaml模板,否则harbor的ui,register,mysql和postgresql不在一个namespace下,不能正常安装
  3. 注意修改postgresql的持久化数据卷,可参考https://github.com/kubernetes/charts/tree/master/stable/postgresql

附录

 templates/adminserver/adminserver-cm.yaml
@@ -1,6 +1,7 @@
 apiVersion: v1
 kind: ConfigMap
 metadata:
+  namespace: class100-ops
   name: "{{ template "harbor.fullname" . }}-adminserver"
   labels:
 {{ include "harbor.labels" . | indent 4 }}
  
1  templates/adminserver/adminserver-secrets.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: "{{ template "harbor.fullname" . }}-adminserver"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: adminserver
  
5  templates/adminserver/adminserver-ss.yaml
@@ -2,6 +2,7 @@ apiVersion: apps/v1beta2
 kind: StatefulSet
 metadata:
   name: "{{ template "harbor.fullname" . }}-adminserver"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: adminserver
@@ -42,13 +43,15 @@ spec:
         volumeMounts:
         - name: adminserver-config
           mountPath: /etc/adminserver/config
+          subPath: harbor-v1/adminserver/
         - name: adminserver-key
           mountPath: /etc/adminserver/key
           subPath: key
       volumes:
       {{- if not .Values.persistence.enabled }}
       - name: adminserver-config
-        emptyDir: {}
+        persistentVolumeClaim:
+          claimName: harbor-pvc
       {{- end }}
       - name: adminserver-key
         secret:
  
1  templates/adminserver/adminserver-svc.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Service
 metadata:
   name: "{{ template "harbor.fullname" . }}-adminserver"
+  namespace: class100-ops
 spec:
   ports:
     - port: 80
  
1  templates/clair/clair-cm.yaml
@@ -3,6 +3,7 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: {{ template "harbor.fullname" . }}-clair
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: clair
  
1  templates/clair/clair-dpl.yaml
@@ -3,6 +3,7 @@ apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
   name: {{ template "harbor.fullname" . }}-clair
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: clair
  
1  templates/clair/clair-svc.yaml
@@ -6,6 +6,7 @@ apiVersion: v1
 kind: Service
 metadata:
   name: clair
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 spec:
  
1  templates/ingress/ingress.yaml
@@ -2,6 +2,7 @@ apiVersion: extensions/v1beta1
 kind: Ingress
 metadata:
   name: "{{ template "harbor.fullname" . }}-ingress"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
   annotations:
  
1  templates/ingress/secret.yaml
@@ -5,6 +5,7 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: "{{ template "harbor.fullname" . }}-ingress"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 type: kubernetes.io/tls
  
1  templates/jobservice/jobservice-cm.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: "{{ template "harbor.fullname" . }}-jobservice"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 data:
  
1  templates/jobservice/jobservice-dpl.yaml
@@ -2,6 +2,7 @@ apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
   name: "{{ template "harbor.fullname" . }}-jobservice"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: jobservice
  
1  templates/jobservice/jobservice-secrets.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: "{{ template "harbor.fullname" . }}-jobservice"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 type: Opaque
  
1  templates/jobservice/jobservice-svc.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Service
 metadata:
   name: "{{ template "harbor.fullname" . }}-jobservice"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 spec:
  
1  templates/mysql/mysql-secret.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: "{{ template "harbor.fullname" . }}-mysql"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 type: Opaque
  
7  templates/mysql/mysql-ss.yaml
@@ -2,6 +2,7 @@ apiVersion: apps/v1beta2
 kind: StatefulSet
 metadata:
   name: "{{ template "harbor.fullname" . }}-mysql"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: mysql
@@ -33,10 +34,12 @@ spec:
         volumeMounts:
         - name: mysql-data
           mountPath: /var/lib/mysql
+          subPath: harbor-v1/mysql-data/
       {{- if not .Values.persistence.enabled }}
       volumes:
-      - name: "mysql-data"
-        emptyDir: {}
+      - name: mysql-data
+        persistentVolumeClaim:
+          claimName: harbor-pvc
       {{- end -}}
   {{- if .Values.persistence.enabled }}
   volumeClaimTemplates:
  
1  templates/mysql/mysql-svc.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Service
 metadata:
   name: "{{ template "harbor.fullname" . }}-mysql"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 spec:
  
1  templates/registry/registry-cm.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: "{{ template "harbor.fullname" . }}-registry"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 data:
  
1  templates/registry/registry-secret.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: "{{ template "harbor.fullname" . }}-registry"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 type: Opaque
  
5  templates/registry/registry-ss.yaml
@@ -2,6 +2,7 @@ apiVersion: apps/v1beta2
 kind: StatefulSet
 metadata:
   name: "{{ template "harbor.fullname" . }}-registry"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: registry
@@ -37,6 +38,7 @@ spec:
         volumeMounts:
         - name: registry-data
           mountPath: /var/lib/registry
+          subPath: harbor-v1/registry-data/
         - name: registry-root-certificate
           mountPath: /etc/registry/root.crt
           subPath: root.crt
@@ -47,7 +49,8 @@ spec:
 {{- if not .Values.registry.objectStorage }}
 {{- if not .Values.persistence.enabled }}
       - name: registry-data
-        emptyDir: {}
+        persistentVolumeClaim:
+          claimName: harbor-pvc
 {{- end }}
 {{- end }}
       - name: registry-root-certificate
  
1  templates/registry/registry-svc.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Service
 metadata:
   name: "{{ template "harbor.fullname" . }}-registry"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 spec:
  
1  templates/ui/ui-cm.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: ConfigMap
 metadata:
   name: "{{ template "harbor.fullname" . }}-ui"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 data:
  
3  templates/ui/ui-dpl.yaml
@@ -2,6 +2,7 @@ apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
   name: "{{ template "harbor.fullname" . }}-ui"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
     component: ui
@@ -54,8 +55,10 @@ spec:
           subPath: private_key.pem
         - name: ca-download
           mountPath: /etc/ui/ca
+          subPath: harbor-v1/ui-ca/
         - name: psc
           mountPath: /etc/ui/token
+          subPath: harbor-v1/ui-psc/
       volumes:
       - name: ui-config
         configMap:
  
1  templates/ui/ui-secrets.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Secret
 metadata:
   name: "{{ template "harbor.fullname" . }}-ui"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 type: Opaque
  
1  templates/ui/ui-svc.yaml
@@ -2,6 +2,7 @@ apiVersion: v1
 kind: Service
 metadata:
   name: "{{ template "harbor.fullname" . }}-ui"
+  namespace: class100-ops
   labels:
 {{ include "harbor.labels" . | indent 4 }}
 spec:
  
3  values.yaml
@@ -287,4 +287,5 @@ postgresql:
   postgresPassword: not-a-secure-password
   postgresDatabase: clair
   persistence:
-    enabled: false
+    enabled: true
+    existingClaim: harbor-pvc 

参考

  1. https://github.com/vmware/harbor/issues/4484
  2. https://github.com/vmware/harbor/tree/master/contrib/helm/harbor
  3. https://github.com/vmware/harbor/issues/4481
  4. https://kubernetes.io/docs/concepts/storage/storage-classes/#introduction

本文来自:www.allocmem.com

感谢作者:www.allocmem.com

查看原文:kubernetes下搭建harbor企业级docker仓库

557 次点击  
加入收藏 微博
暂无回复
添加一条新回复 (您需要 登录 后才能回复 没有账号 ?)
  • 请尽量让自己的回复能够对别人有帮助
  • 支持 Markdown 格式, **粗体**、~~删除线~~、`单行代码`
  • 支持 @ 本站用户;支持表情(输入 : 提示),见 Emoji cheat sheet
  • 图片支持拖拽、截图粘贴等方式上传