Last updated: 23 Sep 21 05:07:47
Helm->harbor install
튜토리얼
Helm을 설치 하다가 harbor로 설치 합니다. harbor를 설치 시 private 한 docker 레파지토리를 제공하고 있으며 helm을 포함하고 있습니다. 레파지토리의 UI를 제공하고 있어서 관리이점을 가지고 있습니다.
환경
helm version : 3.6.3 ChartMuseum version 0.12.0 (build 101e26a)
docker register 설치
helm을 사용하여 컨테이너들을 쉽게 배포 및 버전 관리 할 수 있도록 구성. . 2.X버전에서 -> 3.X로 올라가면서 변화가 생김
- helm 설치
- root 권한으로 설치
- 다운로드 helm3
> cd /root/
> curl -fsSL -o get_helm3.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
> chmod 700 get_helm.sh
> ./get_helm.sh
> helm version
version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"}
> cd /root/
> curl -fsSL -o get_helm3.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
> chmod 700 get_helm.sh
> ./get_helm.sh
> helm version
version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"}
- helm public 원격레파지토리 추가
- 3버전 부터는 기본 repo가 없다.
- 변경사항 : repo “https://kubernetes-charts.storage.googleapis.com” is no longer available; try “https://charts.helm.sh/stable” instead
- 3버전 부터는 기본 repo가 없다.
> helm repo add stable https://charts.helm.sh/stable "stable" has been added to your repositories > helm repo list NAME URL stable https://charts.helm.sh/stable
> helm repo add stable https://charts.helm.sh/stable
"stable" has been added to your repositories
> helm repo list
NAME URL
stable https://charts.helm.sh/stable- helm 로컬 레파지토리 구성
- 보안 및 외부에 연결 불가능 시에 필요.
curl -LO https://s3.amazonaws.com/chartmuseum/release/latest/bin/linux/amd64/chartmuseum openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=leesang.com/O=leesang"
curl -LO https://s3.amazonaws.com/chartmuseum/release/latest/bin/linux/amd64/chartmuseum
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=leesang.com/O=leesang"로컬을 구성하기 위해 종합적인 helm, chartmuseum 포함한 Harbor 사용.
harbor version : 2.3.2 certbot version : 1.18.0 docker version : 1.13.1-72 -> 20.10.8 Community docker compose : 1.18.0 -> 1.29.2 helm version : v3.6.3
- kernel patch
- Docker fails to start containers with cgroup memory allocation error
- github.com/docker/for-linux/issues/841
- Docker fails to start containers with cgroup memory allocation error
> vi /etc/default/grub > cat /proc/cmdline BOOT_IMAGE=/vmlinuz-3.10.0-957.1.3.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8 > vi /etc/default/grub GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet cgroup.memory=nokmem" > grub2-mkconfig -o /boot/grub2/grub.cfg Generating grub configuration file ... Found linux image: /boot/vmlinuz-3.10.0-957.1.3.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-957.1.3.el7.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-862.14.4.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-862.14.4.el7.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-693.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-693.el7.x86_64.img Found linux image: /boot/vmlinuz-0-rescue-55b8e2a5cd7a452fa963a37686914e96 Found initrd image: /boot/initramfs-0-rescue-55b8e2a5cd7a452fa963a37686914e96.img done > reboot > cat /proc/cmdline BOOT_IMAGE=/vmlinuz-3.10.0-957.1.3.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet cgroup.memory=nokmem
> vi /etc/default/grub
> cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-957.1.3.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8
> vi /etc/default/grub
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet cgroup.memory=nokmem"
> grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-957.1.3.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-957.1.3.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-862.14.4.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-862.14.4.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-693.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-693.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-55b8e2a5cd7a452fa963a37686914e96
Found initrd image: /boot/initramfs-0-rescue-55b8e2a5cd7a452fa963a37686914e96.img
done
> reboot
> cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-957.1.3.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet cgroup.memory=nokmem- harbor install
Harbor in Components
- Component Version
- Postgresql 9.6.10-1.ph2
- Redis 4.0.10-1.ph2
- Beego 1.9.0
- Chartmuseum 0.9.0
- Docker/distribution 2.7.1
- Docker/notary 0.6.1
- Helm 2.9.1
- Swagger-ui 3.22.1
-
- SSL 없이 helm 설치
-
- Let’s Encrypt SSL을 사용한 하버 설치
- certbot-auto는 공식적으로 지원하지 않음.
- snapd로 패키지 형태로 설치
- 설치 시 필요한 apache, nginx 인증 서버가 추가로 필요함.
certbot-auto certonly --standalone -d "leesang4.harbor.com" --preferred-challenges http --agree-tos -n -m "admin@leesang4.harbor.com" --keep-until-expiring #EPEL to CentOS 7 > yum install epel-release > yum install snapd > systemctl enable --now snapd.socket > snap install --classic certbot > ln -s /snap/bin/certbot /usr/bin/certbot > certbot --version certbot 1.18.0
certbot-auto certonly --standalone -d "leesang4.harbor.com" --preferred-challenges http --agree-tos -n -m "admin@leesang4.harbor.com" --keep-until-expiring
#EPEL to CentOS 7
> yum install epel-release
> yum install snapd
> systemctl enable --now snapd.socket
> snap install --classic certbot
> ln -s /snap/bin/certbot /usr/bin/certbot
> certbot --version
certbot 1.18.0-
- Local 무료 SSL 인증서 사용
> tar xvfz harbor-offline-installer-v2.3.2.tgz
> cp harbor.yml.tmpl harbor.yml
#3. 자체 서명된 SSL 인증서를 사용한 하버 설치
#3으로 자체 서명 인증서의 경우 인증서 구성 파일 생성
> mkdir -p /root/docker/harbour/certs
> vi harbor_certs.cnf
[ req ]
default_bits = 4096
default_md = sha512
default_keyfile = harbor_registry.key
prompt = no
encrypt_key = no
distinguished_name = req_distinguished_name
# distinguished_name
[ req_distinguished_name ]
countryName = "KR"
localityName = "leesang"
stateOrProvinceName = "leesang"
organizationName = "leesang"
commonName = "leesang4.harbor.com"
emailAddress = "admin@leesang4.harbor.com"
#키 및 CSR 생성
> openssl req -out harbor_registry.csr -newkey rsa:4096 -sha512 -nodes -keyout harbor_registry.key -config harbor_certs.cnf
#만료 날짜가 10년인 자체 인증 인증서를 생성합니다.
> openssl x509 -in harbor_registry.csr -out harbor_registry.crt -req -signkey harbor_registry.key -days 3650
> openssl x509 -req -extfile <(printf "subjectAltName=DNS:leesang4.harbor.com") -days 3650 -in harbor_registry.csr -CA harbor_registry.crt -CAkey harbor_registry.key -CAcreateserial -out harbor_registry.crt
openssl x509 -req -extfile <(printf "subjectAltName=DNS:leesang4.harbor.com") -days 3650 -in harbor_registry.csr -out harbor_registry.crt -signkey harbor_registry.key
# harbor에서 PEM을 요구함
# private key를 pem화
> openssl rsa -in harbor_registry.key -outform PEM -out harbor_registry.pem
# cs key를 pem화
> openssl x509 -in harbor_registry.crt -out harbor_registrycrt.pem -outform PEM
#인증서 세부 정보
> openssl x509 -text -noout -in harbor_registry.crt
# harbor https 관련 구성
> vi /root/docker/harbour/harbor/harbor.yml
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /root/docker/harbour/certs/harbor_registrycrt.pem
private_key: /root/docker/harbour/certs/harbor_registry.pem
# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: vmware1!
database:
# The password for the root user of Harbor DB. Change this before any production use.
password: vmware1!
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: leesang4.harbor.com
# The default data volume
data_volume: /root/docker/harbour/data
# docker version이 낮으면 안됨 기존 docker 삭제 및 업그레이드
> yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine
> yum install -y yum-utils
> yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
> yum install docker-ce docker-ce-cli containerd.io
> systemctl start docker
> systemctl enable docker
# 문제 : k8s가 서비스가 갑자기 안되거나 재부팅 후에 docker가 운영이 안되는 경우
> vi /var/log/messages
error: failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd"
> vi /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS=--cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1
KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1
> curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
> chmod +x /usr/local/bin/docker-compose
> ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# harbor ssl인증 및 Chartmuseum 활성화:
> ./install.sh --with-notary --with-chartmuseum
# harbor 로그
> ls -1 /var/log/harbor/
# 유지보수
# Harbour 재설정 필요 시 Notary, 차트 리포지토리 서비스와 함께 설치된 경우:
> docker-compose down -v
> vi harbor.yml
> ./prepare --with-notary --with-chartmuseum
> docker-compose up -d> tar xvfz harbor-offline-installer-v2.3.2.tgz
> cp harbor.yml.tmpl harbor.yml
#3. 자체 서명된 SSL 인증서를 사용한 하버 설치
#3으로 자체 서명 인증서의 경우 인증서 구성 파일 생성
> mkdir -p /root/docker/harbour/certs
> vi harbor_certs.cnf
[ req ]
default_bits = 4096
default_md = sha512
default_keyfile = harbor_registry.key
prompt = no
encrypt_key = no
distinguished_name = req_distinguished_name
# distinguished_name
[ req_distinguished_name ]
countryName = "KR"
localityName = "leesang"
stateOrProvinceName = "leesang"
organizationName = "leesang"
commonName = "leesang4.harbor.com"
emailAddress = "admin@leesang4.harbor.com"
#키 및 CSR 생성
> openssl req -out harbor_registry.csr -newkey rsa:4096 -sha512 -nodes -keyout harbor_registry.key -config harbor_certs.cnf
#만료 날짜가 10년인 자체 인증 인증서를 생성합니다.
> openssl x509 -in harbor_registry.csr -out harbor_registry.crt -req -signkey harbor_registry.key -days 3650
> openssl x509 -req -extfile <(printf "subjectAltName=DNS:leesang4.harbor.com") -days 3650 -in harbor_registry.csr -CA harbor_registry.crt -CAkey harbor_registry.key -CAcreateserial -out harbor_registry.crt
openssl x509 -req -extfile <(printf "subjectAltName=DNS:leesang4.harbor.com") -days 3650 -in harbor_registry.csr -out harbor_registry.crt -signkey harbor_registry.key
# harbor에서 PEM을 요구함
# private key를 pem화
> openssl rsa -in harbor_registry.key -outform PEM -out harbor_registry.pem
# cs key를 pem화
> openssl x509 -in harbor_registry.crt -out harbor_registrycrt.pem -outform PEM
#인증서 세부 정보
> openssl x509 -text -noout -in harbor_registry.crt
# harbor https 관련 구성
> vi /root/docker/harbour/harbor/harbor.yml
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /root/docker/harbour/certs/harbor_registrycrt.pem
private_key: /root/docker/harbour/certs/harbor_registry.pem
# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: vmware1!
database:
# The password for the root user of Harbor DB. Change this before any production use.
password: vmware1!
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: leesang4.harbor.com
# The default data volume
data_volume: /root/docker/harbour/data
# docker version이 낮으면 안됨 기존 docker 삭제 및 업그레이드
> yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine
> yum install -y yum-utils
> yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
> yum install docker-ce docker-ce-cli containerd.io
> systemctl start docker
> systemctl enable docker
# 문제 : k8s가 서비스가 갑자기 안되거나 재부팅 후에 docker가 운영이 안되는 경우
> vi /var/log/messages
error: failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd"
> vi /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS=--cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1
KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1
> curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
> chmod +x /usr/local/bin/docker-compose
> ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# harbor ssl인증 및 Chartmuseum 활성화:
> ./install.sh --with-notary --with-chartmuseum
# harbor 로그
> ls -1 /var/log/harbor/
# 유지보수
# Harbour 재설정 필요 시 Notary, 차트 리포지토리 서비스와 함께 설치된 경우:
> docker-compose down -v
> vi harbor.yml
> ./prepare --with-notary --with-chartmuseum
> docker-compose up -d- harbor 접속 콘설 UI
- https://leesang4.harbor.com/
- admin / vmware1!
- harbor 사용하여 docker image push
# harbor 서버측에서 클라이언트에 키전달 > scp ../certs/harbor_registrycrt.pem "root@172.16.15.140:/etc/pki/ca-trust/source/anchors/" > scp ../certs/harbor_registry.crt "root@172.16.15.140:/etc/pki/ca-trust/source/anchors/" # 클라이언트 측 host파일 등록 > vi /etc/hosts 172.16.15.241 leesang4.harbor.com > cp /etc/pki/ca-trust/source/anchors/harbor_registrycrt.pem /etc/pki/tls/certs/harbor_registrycrt.pem # 클라이언트에서 ssl등록 > update-ca-trust > systemctl restart docker.service # 클라이언트 측 확인 # SSL 확인 및 harbor API > curl --cacert /etc/pki/ca-trust/source/anchors/harbor_registrycrt.pem -u 'admin:vmware1!' https://leesang4.harbor.com/api/v2.0/users # 클라이언트 측 ssl 도커 레지스터리 harbor 접속 > docker login leesang4.harbor.com Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded #클라이언트 측 OCI 지원 활성화 #현재 OCI 지원은 실험적(experimental) 으로 간주되고 있다. #아래에서 기술된 명령들을 사용하려면 환경에서 HELM_EXPERIMENTAL_OCI 변수를 설정하자: > vi ~/.bashrc export HELM_EXPERIMENTAL_OCI=1
# harbor 서버측에서 클라이언트에 키전달
> scp ../certs/harbor_registrycrt.pem "root@172.16.15.140:/etc/pki/ca-trust/source/anchors/"
> scp ../certs/harbor_registry.crt "root@172.16.15.140:/etc/pki/ca-trust/source/anchors/"
# 클라이언트 측 host파일 등록
> vi /etc/hosts
172.16.15.241 leesang4.harbor.com
> cp /etc/pki/ca-trust/source/anchors/harbor_registrycrt.pem /etc/pki/tls/certs/harbor_registrycrt.pem
# 클라이언트에서 ssl등록
> update-ca-trust
> systemctl restart docker.service
# 클라이언트 측 확인
# SSL 확인 및 harbor API
> curl --cacert /etc/pki/ca-trust/source/anchors/harbor_registrycrt.pem -u 'admin:vmware1!' https://leesang4.harbor.com/api/v2.0/users
# 클라이언트 측 ssl 도커 레지스터리 harbor 접속
> docker login leesang4.harbor.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
#클라이언트 측 OCI 지원 활성화
#현재 OCI 지원은 실험적(experimental) 으로 간주되고 있다.
#아래에서 기술된 명령들을 사용하려면 환경에서 HELM_EXPERIMENTAL_OCI 변수를 설정하자:
> vi ~/.bashrc
export HELM_EXPERIMENTAL_OCI=1- 클라이언트 측 docker image build
> docker build --no-cache -t mariadb:1.0.0 ./
> docker build --no-cache -t mariadb:1.0.0 ./- 클라이언트 측 docker image push
> docker tag mariadb:1.0.0 leesang4.harbor.com/library/mariadb:1.0.3 > docker push leesang4.harbor.com/library/mariadb:1.0.3
> docker tag mariadb:1.0.0 leesang4.harbor.com/library/mariadb:1.0.3
> docker push leesang4.harbor.com/library/mariadb:1.0.3 - harbor를 이용한 helm chart 로 push
- 기존 docker배포 방법 보다 템플릿을 사용하여 일관적으로 인증 + 배포 + 리소스까지 컨트롤 를 쉽게 할 수 있는 방법
# 클라이언트 측 repo 등록 > helm repo add harbor https://leesang4.harbor.com/chartrepo/library "harbor" has been added to your repositories # 클라이언트 측 helm 로그인 확인 > helm registry login https://leesang4.harbor.com Username: admin Password: Login succeeded
# 클라이언트 측 repo 등록
> helm repo add harbor https://leesang4.harbor.com/chartrepo/library
"harbor" has been added to your repositories
# 클라이언트 측 helm 로그인 확인
> helm registry login https://leesang4.harbor.com
Username: admin
Password:
Login succeeded
- harbor에 helm push
# 클라이언트 측 # 기본 Helm CLI는 인증을 통한 차트 푸시를 지원하지 않으므로 helm-push플러그인 을 설치해야 합니다. > helm plugin install https://github.com/chartmuseum/helm-push > mkdir /root/helm/chart > cd /root/helm/chart # 클라이언트 측 chart 패키징 검증 된 gitea Test생성 # 신뢰할 수 있는 공급자가 패키징했다는 암호화 검증을 제공 > mkdir /root/.gnupg && chmod 0700 /root/.gnupg > cat > /root/.gnupg/my_gpg_key << EOF %echo Generating a basic OpenPGP key Key-Type: RSA Key-Length: 2048 Subkey-Type: RSA Subkey-Length: 2048 Name-Real: Helm User Name-Comment: User Name-Email: admin@leesang4.harbor.com Expire-Date: 0 %no-protection %commit EOF > gpg2 --verbose --batch --gen-key /root/.gnupg/my_gpg_key gpg: Generating a basic OpenPGP key gpg: skipping control %no-protection () gpg: writing self signature gpg: RSA/SHA1 signature from: "BCC97F6E [?]" wait.... gpg: writing key binding signature gpg: RSA/SHA1 signature from: "BCC97F6E [?]" gpg: RSA/SHA1 signature from: "84010C3B [?]" gpg: writing key binding signature gpg: RSA/SHA1 signature from: "BCC97F6E [?]" gpg: RSA/SHA1 signature from: "84010C3B [?]" gpg: writing public key to /root/.gnupg/pubring.gpg gpg: writing secret key to /root/.gnupg/secring.gpg gpg: using PGP trust model gpg: key BCC97F6E marked as ultimately trusted > gpg2 --list-secret-keys gpg2 --list-secret-keys gpg: checking the trustdb gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u /root/.gnupg/secring.gpg ------------------------ sec 2048R/BCC97F6E 2021-08-31 uid Helm User (User) <admin@leesang4.harbor.com> ssb 2048R/84010C3B 2021-08-31 > gpg2 --export-secret-keys > /root/.gnupg/private_secring.gpg # 소스 가져오기 > git clone --quiet https://github.com/jfelten/gitea-helm-chart gitea > helm package --sign --key "admin@leesang4.harbor.com" --keyring /root/.gnupg/private_secring.gpg --destination . ./gitea/ Successfully packaged chart and saved it to: /root/helm/chart/gitea-0.2.10.tgz > ls -la gitea*tgz* -rw-r--r-- 1 root root 29580 8월 31 03:37 gitea-0.2.10.tgz -rw-r--r-- 1 root root 1034 8월 31 03:37 gitea-0.2.10.tgz.prov # chart 등록 # harbor API를 이용 > curl -vv -s --cacert /etc/pki/ca-trust/source/anchors/harbor_registrycrt.pem -u 'admin:vmware1!' -X POST "https://leesang4.harbor.com/api/chartrepo/library/charts" -H "Content-Type: multipart/form-data" -F "chart=@gitea-0.2.10.tgz;type=application/x-compressed-tar" -F "prov=@gitea-0.2.10.tgz.prov" | jq "."
# 클라이언트 측
# 기본 Helm CLI는 인증을 통한 차트 푸시를 지원하지 않으므로 helm-push플러그인 을 설치해야 합니다.
> helm plugin install https://github.com/chartmuseum/helm-push
> mkdir /root/helm/chart
> cd /root/helm/chart
# 클라이언트 측 chart 패키징 검증 된 gitea Test생성
# 신뢰할 수 있는 공급자가 패키징했다는 암호화 검증을 제공
> mkdir /root/.gnupg && chmod 0700 /root/.gnupg
> cat > /root/.gnupg/my_gpg_key << EOF
%echo Generating a basic OpenPGP key
Key-Type: RSA
Key-Length: 2048
Subkey-Type: RSA
Subkey-Length: 2048
Name-Real: Helm User
Name-Comment: User
Name-Email: admin@leesang4.harbor.com
Expire-Date: 0
%no-protection
%commit
EOF
> gpg2 --verbose --batch --gen-key /root/.gnupg/my_gpg_key
gpg: Generating a basic OpenPGP key
gpg: skipping control %no-protection ()
gpg: writing self signature
gpg: RSA/SHA1 signature from: "BCC97F6E [?]"
wait....
gpg: writing key binding signature
gpg: RSA/SHA1 signature from: "BCC97F6E [?]"
gpg: RSA/SHA1 signature from: "84010C3B [?]"
gpg: writing key binding signature
gpg: RSA/SHA1 signature from: "BCC97F6E [?]"
gpg: RSA/SHA1 signature from: "84010C3B [?]"
gpg: writing public key to /root/.gnupg/pubring.gpg
gpg: writing secret key to /root/.gnupg/secring.gpg
gpg: using PGP trust model
gpg: key BCC97F6E marked as ultimately trusted
> gpg2 --list-secret-keys
gpg2 --list-secret-keys
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
/root/.gnupg/secring.gpg
------------------------
sec 2048R/BCC97F6E 2021-08-31
uid Helm User (User) <admin@leesang4.harbor.com>
ssb 2048R/84010C3B 2021-08-31
> gpg2 --export-secret-keys > /root/.gnupg/private_secring.gpg
# 소스 가져오기
> git clone --quiet https://github.com/jfelten/gitea-helm-chart gitea
> helm package --sign --key "admin@leesang4.harbor.com" --keyring /root/.gnupg/private_secring.gpg --destination . ./gitea/
Successfully packaged chart and saved it to: /root/helm/chart/gitea-0.2.10.tgz
> ls -la gitea*tgz*
-rw-r--r-- 1 root root 29580 8월 31 03:37 gitea-0.2.10.tgz
-rw-r--r-- 1 root root 1034 8월 31 03:37 gitea-0.2.10.tgz.prov
# chart 등록
# harbor API를 이용
> curl -vv -s --cacert /etc/pki/ca-trust/source/anchors/harbor_registrycrt.pem -u 'admin:vmware1!' -X POST "https://leesang4.harbor.com/api/chartrepo/library/charts" -H "Content-Type: multipart/form-data" -F "chart=@gitea-0.2.10.tgz;type=application/x-compressed-tar" -F "prov=@gitea-0.2.10.tgz.prov" | jq "."
# 클라이언트 측 chart dev-to-do-chart Test생성
> helm create dev-to-do-chart
> vi dev-to-do-chart/README.md
This chart will deploy the "To Do" application.
Set "Service" to type "LoadBalancer" in the values file to expose the application via an L4 NSX-T load balancer.
# 기본
> helm package ./dev-to-do-chart
# 기본 chart 등록
> helm push --ca-file=/etc/pki/ca-trust/source/anchors/harbor_registry.crt --username=admin --password='vmware1!' dev-to-do-chart-0.1.0.tgz harbor
Pushing dev-to-do-chart-0.1.0.tgz to harbor...
Done.
# 클라이언트 측 확인
> helm search repo dev
NAME CHART VERSION APP VERSION DESCRIPTION
harbor/dev-to-do-chart 0.1.0 1.0 A Helm chart for Kubernetes
# harbor를 통한 앱 배포
> helm -n default install gitea harbor/gitea --set gitea.admin.username=root --set gitea.admin.passowrd='vmware1!'
NAME: gitea
LAST DEPLOYED: Wed Sep 1 03:29:08 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Connect to your Gitea web URL by running:
export POD_NAME=$(kubectl get pods --namespace default -l "app=gitea-gitea" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward $POD_NAME 8080:3000
echo http://127.0.0.1:8080/
2. Connect to your Gitea ssh port:
export POD_NAME=$(kubectl get pods --namespace default -l "app=gitea-gitea" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward $POD_NAME 8022:22
echo http://127.0.0.1:8080/
kubectl port-forward $POD_NAME 8022:22
# 클라이언트 측 chart dev-to-do-chart Test생성
> helm create dev-to-do-chart
> vi dev-to-do-chart/README.md
This chart will deploy the "To Do" application.
Set "Service" to type "LoadBalancer" in the values file to expose the application via an L4 NSX-T load balancer.
# 기본
> helm package ./dev-to-do-chart
# 기본 chart 등록
> helm push --ca-file=/etc/pki/ca-trust/source/anchors/harbor_registry.crt --username=admin --password='vmware1!' dev-to-do-chart-0.1.0.tgz harbor
Pushing dev-to-do-chart-0.1.0.tgz to harbor...
Done.
# 클라이언트 측 확인
> helm search repo dev
NAME CHART VERSION APP VERSION DESCRIPTION
harbor/dev-to-do-chart 0.1.0 1.0 A Helm chart for Kubernetes
# harbor를 통한 앱 배포
> helm -n default install gitea harbor/gitea --set gitea.admin.username=root --set gitea.admin.passowrd='vmware1!'
NAME: gitea
LAST DEPLOYED: Wed Sep 1 03:29:08 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Connect to your Gitea web URL by running:
export POD_NAME=$(kubectl get pods --namespace default -l "app=gitea-gitea" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward $POD_NAME 8080:3000
echo http://127.0.0.1:8080/
2. Connect to your Gitea ssh port:
export POD_NAME=$(kubectl get pods --namespace default -l "app=gitea-gitea" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward $POD_NAME 8022:22
echo http://127.0.0.1:8080/
kubectl port-forward $POD_NAME 8022:22
'IT > kubernetes' 카테고리의 다른 글
| ISTIO 이스티오 설치 (0) | 2021.09.23 |
|---|---|
| 쿠버네티스 버전 업그레이드 (0) | 2021.09.23 |
| 쿠버네티스 설치 (0) | 2021.09.23 |