Skip to main content

Open Policy Agent - Gatekeeper

This custom chart depends upon the OPA Gatekeeper upstream Helm chart from here.

How Gatekeeper works​

To get the violations in the cluster run

kubectl describe K8sRequiredResources

Gatekeeper is project under Open Policy Agent which targets running OPA policies inside Kubernetes clusters.

Gatekeeper requires Constraint Templates and Constraints to impose policies on k8s objects. A constraint template is a custom CRD which defines the logic and input/output of the policy. The template uses Open API Schema to register the input structures with Gatekeeper. It defines the policy logic using a declarative language known as Rego.

Once the constraint template is created, we need to create a constraint to analyse k8s requests. The constraint defines the parameters for the policy, e.g. - the namespaces it needs to monitor, the apiGroups it needs to match, etc. It also provides the actual input to Gatekeeper whose structure is registered in the template. For e.g - pod labels, container memory limits, etc.

Gatekeeper can be run in Enforcement Mode or Dry run mode. The latter is better suited for testing and detecting policy violations. The Enforcement Mode is better for production as it blocks any K8s objects which do not conform to the policy.

Note : The template needs to be created before the actual constraint.

Constraint Template​

The below example defines a sample constraint template that imposes labels on a resource.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels

        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("you must provide labels: %v", [missing])
        }

Constraint​

This constraint uses the above template to enforce gatekeeper label on all namespaces.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-gk
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    labels: ["gatekeeper"]

Try creating a namespace without a label gatekeeper and k8s will throw an error with the message defined in the above template.

Policy Examples​

Adding a new Policy​

  • Write a .rego file using the Constraint Framework describing the policy. Keep the rego file under argocd-helm-charts/gatekeeper/policies.
  • Add a constraint template which imports the policy.
  • Add a constraint describing which Kubernetes objects should the policy be applied to.
  • Combine both the yaml spec into a single file and keep it under argocd-helm-charts/gatekeeper/templates.
  • Install OPA binary and test the policy using :
# navigate to the root of gatekeeper chart
cd argocd-helm-charts/gatekeeper

# test using Makefile
make test
  • Verify whether the template is correctly populated using the helm template command.

Troubleshooting​

  • While syncing Gatekeeper via ArgoCD, make sure that the Constraint Template is synced before the CRD (policy). This is because Gatekeeper creates the CRDs for each policy and it needs the Constraint Templates to create the CRD. ArgoCD does not know this as it uses helm template to generate all the YAML definitions in one go. This might be possible to solve via Helm hooks.

  • If you have enabled network policy in the values.yaml file, you might experience the Gatekeeper pods restarting and ending up in CrashloopBackOff state due to Readiness and Liveness probes failing continuously. To correct this, disable the gatekeeper-netpol network policy from the values.yaml file by specifying networkPolicy: false. This has been observed in bare metal clusters and after removing the network policy, the pods come back to healthy state after a restart.

  • If the total violation count is higher but the violations are not listed, it might be due to the constraintViolationsLimit set in the values.yaml file. This is a limit on the number of violations that can be listed in the kubectl describe command. To list all the violations, set this value to higer number.

References​