TAPにWorkloadをデプロイする際に、resourcesを指定しないとNodeのcpu, memoryを限界以上にアプリがデプロイされる可能性があるので、 resourcesの設定は推奨。これを強制するためにGatekeeperを使用する。
Gatekeeperのインストール
ドキュメントの通りにインストール
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml
Podの確認
$ kubectl get pod -n gatekeeper-system
NAME READY STATUS RESTARTS AGE
gatekeeper-audit-84b788f7c5-54m2r 1/1 Running 0 13s
gatekeeper-controller-manager-76dc9cbc64-2q26t 1/1 Running 0 13s
gatekeeper-controller-manager-76dc9cbc64-fx5m8 1/1 Running 0 13s
gatekeeper-controller-manager-76dc9cbc64-qb52w 1/1 Running 0 13s
ConstraintTemplateの作成
とりあえず spec.resources.{limits,requests}.memory
の設定を必須とするConstraintTemplate。cpuも必須にしたい場合はコメントを外す
cat <<EOF > requiredresources.yaml
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: requiredresources
spec:
crd:
spec:
names:
kind: RequiredResources
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package requiredresources
missing(obj, field) = true {
not obj[field]
}
missing(obj, field) = true {
obj[field] == ""
}
violation[{"msg": msg, "details": {}}] {
missing(input.review.object.spec, "resources")
msg := "spec.resources is missing."
}
violation[{"msg": msg, "details": {}}] {
missing(input.review.object.spec.resources, "limits")
msg := "spec.resources.limits is missing."
}
violation[{"msg": msg, "details": {}}] {
missing(input.review.object.spec.resources, "requests")
msg := "spec.resources.requests is missing."
}
# violation[{"msg": msg, "details": {}}] {
# missing(input.review.object.spec.resources.limits, "cpu")
# msg := "spec.resources.limits.cpu is missing."
# }
violation[{"msg": msg, "details": {}}] {
missing(input.review.object.spec.resources.limits, "memory")
msg := "spec.resources.limits.memory is missing."
}
# violation[{"msg": msg, "details": {}}] {
# missing(input.review.object.spec.resources.requests, "cpu")
# msg := "spec.resources.requests.cpu is missing."
# }
violation[{"msg": msg, "details": {}}] {
missing(input.review.object.spec.resources.requests, "memory")
msg := "spec.resources.requests.memory is missing."
}
EOF
kubectl apply -f requiredresources.yaml
Constraintの作成
まずはdryrun modeで試す
cat <<EOF > requiredresources-workload.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: RequiredResources
metadata:
name: requiredresources-workload
spec:
enforcementAction: dryrun
match:
kinds:
- apiGroups:
- carto.run
kinds:
- Workload
EOF
kubectl apply -f requiredresources-workload.yaml
この状態だとまだresourcesの設定なしでもWorkloadを作成できる
$ tanzu apps workload apply hello-nodejs \
--app hello-nodejs \
--git-repo https://github.com/making/hello-nodejs \
--git-branch master \
--type web \
-n demo \
-y
Create workload:
1 + |---
2 + |apiVersion: carto.run/v1alpha1
3 + |kind: Workload
4 + |metadata:
5 + | labels:
6 + | app.kubernetes.io/part-of: hello-nodejs
7 + | apps.tanzu.vmware.com/workload-type: web
8 + | name: hello-nodejs
9 + | namespace: demo
10 + |spec:
11 + | source:
12 + | git:
13 + | ref:
14 + | branch: master
15 + | url: https://github.com/making/hello-nodejs
Created workload "hello-nodejs"
To see logs: "tanzu apps workload tail hello-nodejs --namespace demo"
To get status: "tanzu apps workload get hello-nodejs --namespace demo"
制約違反があるかどうかは次のようにチェックできる。
$ kubectl get requiredresources requiredresources-workload -ojsonpath='{.status.violations}' | jq .
[
{
"enforcementAction": "dryrun",
"group": "carto.run",
"kind": "Workload",
"message": "spec.resources is missing.",
"name": "hello-nodejs",
"namespace": "demo",
"version": "v1alpha1"
}
dryrunをやめて強制モードに変更
cat <<EOF > requiredresources-workload.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: RequiredResources
metadata:
name: requiredresources-workload
spec:
enforcementAction: deny
match:
kinds:
- apiGroups:
- carto.run
kinds:
- Workload
EOF
kubectl apply -f requiredresources-workload.yaml
Workloadを一度削除する
tanzu apps workload delete -n demo hello-nodejs -y
再びWorkloadを作成しようとするとエラーになる
$ tanzu apps workload apply hello-nodejs \
--app hello-nodejs \
--git-repo https://github.com/making/hello-nodejs \
--git-branch master \
--type web \
-n demo \
-y
Create workload:
1 + |---
2 + |apiVersion: carto.run/v1alpha1
3 + |kind: Workload
4 + |metadata:
5 + | labels:
6 + | app.kubernetes.io/part-of: hello-nodejs
7 + | apps.tanzu.vmware.com/workload-type: web
8 + | name: hello-nodejs
9 + | namespace: demo
10 + |spec:
11 + | source:
12 + | git:
13 + | ref:
14 + | branch: master
15 + | url: https://github.com/making/hello-nodejs
Error: Unable to complete command, you do not have permissions for this resource: admission webhook "validation.gatekeeper.sh" denied the request: [requiredresources-workload] spec.resources is missing.
Error: exit status 1
✖ exit status 1
--request-memory
だけ指定してもエラーになる
$ tanzu apps workload apply hello-nodejs \
--app hello-nodejs \
--git-repo https://github.com/making/hello-nodejs \
--git-branch master \
--type web \
--request-memory 64Mi \
-n demo \
-y
Create workload:
1 + |---
2 + |apiVersion: carto.run/v1alpha1
3 + |kind: Workload
4 + |metadata:
5 + | labels:
6 + | app.kubernetes.io/part-of: hello-nodejs
7 + | apps.tanzu.vmware.com/workload-type: web
8 + | name: hello-nodejs
9 + | namespace: demo
10 + |spec:
11 + | resources:
12 + | requests:
13 + | memory: 64Mi
14 + | source:
15 + | git:
16 + | ref:
17 + | branch: master
18 + | url: https://github.com/making/hello-nodejs
Error: Unable to complete command, you do not have permissions for this resource: admission webhook "validation.gatekeeper.sh" denied the request: [requiredresources-workload] spec.resources.limits is missing.
Error: exit status 1
✖ exit status 1
--limit-memory
も指定するとWorkloadは作成できる
$ tanzu apps workload apply hello-nodejs \
--app hello-nodejs \
--git-repo https://github.com/making/hello-nodejs \
--git-branch master \
--type web \
--request-memory 64Mi \
--limit-memory 64Mi \
-n demo \
-y
Create workload:
1 + |---
2 + |apiVersion: carto.run/v1alpha1
3 + |kind: Workload
4 + |metadata:
5 + | labels:
6 + | app.kubernetes.io/part-of: hello-nodejs
7 + | apps.tanzu.vmware.com/workload-type: web
8 + | name: hello-nodejs
9 + | namespace: demo
10 + |spec:
11 + | resources:
12 + | limits:
13 + | memory: 64Mi
14 + | requests:
15 + | memory: 64Mi
16 + | source:
17 + | git:
18 + | ref:
19 + | branch: master
20 + | url: https://github.com/making/hello-nodejs
Created workload "hello-nodejs"
To see logs: "tanzu apps workload tail hello-nodejs --namespace demo"
To get status: "tanzu apps workload get hello-nodejs --namespace demo"