Flux GitOps with Kubernetes: Part 3 - Image Automation

Introduction

In Part 2, we explored advanced Kustomization features for managing multi-environment deployments with Flux. Now, we’ll tackle one of the most powerful automation features in Flux: Image Automation.

Image Automation allows Flux to automatically detect new container image versions in your registries and update your Git repository with the latest tags. This creates a fully automated deployment pipeline where new image builds trigger automatic updates without manual intervention.

What You’ll Learn

• How Image Automation Controllers work (Image Reflector and Image Automation)
• Scanning container registries with ImageRepository
• Creating version selection policies with ImagePolicy
• Automating Git commits with ImageUpdateAutomation
• Using image policy markers in manifests
• Authentication for different container registries (Docker Hub, GHCR, ECR, GCR, ACR)
• Best practices for production image automation

Prerequisites

• Completed Part 1 and Part 2
• Working Kubernetes cluster (kind, k3s, or production)
• Flux CLI installed (flux --version)
• Git repository for your manifests
• Access to a container registry

All examples in this post are available in the flux-demo-website repository.

Understanding Flux Image Automation

Flux Image Automation consists of two controllers that work together:

1. Image Reflector Controller

Scans container image registries at regular intervals and stores the discovered tags in an internal database. It provides:

• ImageRepository: Defines which container registry to scan
• ImagePolicy: Selects the “latest” image based on policy rules (semver, regex, alphabetical, numerical)

2. Image Automation Controller

Updates Kubernetes manifests in Git repositories with new image tags selected by ImagePolicy objects:

• ImageUpdateAutomation: Defines how and when to update Git repositories
• Commits changes automatically with customizable messages
• Supports multiple Git providers (GitHub, GitLab, Bitbucket, Azure DevOps, Gitea, Forgejo, Gerrit)

How It Works

┌────────────────┐
│ Container      │
│ Registry       │
│ (Docker Hub,   │
│  GHCR, ECR,    │
│  GCR, ACR)     │
└────────┬───────┘
         │
         │ 1. Scan for tags
         ↓
┌────────────────────┐
│ ImageRepository    │◄─── interval: 5m
│ (image-reflector)  │
└────────┬───────────┘
         │
         │ 2. Store tags
         ↓
┌────────────────────┐
│ ImagePolicy        │◄─── policy: semver
│ (image-reflector)  │     range: 1.x.x
└────────┬───────────┘
         │
         │ 3. Select latest
         ↓
┌────────────────────────┐
│ ImageUpdateAutomation  │◄─── interval: 30m
│ (image-automation)     │
└────────┬───────────────┘
         │
         │ 4. Update manifests
         │ 5. Commit & push
         ↓
┌────────────────┐
│ Git Repository │
│ (main branch)  │
└────────┬───────┘
         │
         │ 6. Flux reconciles
         ↓
┌────────────────┐
│ Kubernetes     │
│ Cluster        │
└────────────────┘

Installing Image Automation Controllers

First, verify if the controllers are already installed:

flux check

If not installed, bootstrap Flux with image automation:

flux bootstrap github \
  --owner=<your-github-username> \
  --repository=<your-repo-name> \
  --path=clusters/my-cluster \
  --personal \
  --components-extra=image-reflector-controller,image-automation-controller

Or install them separately on an existing Flux installation:

flux install \
  --components-extra=image-reflector-controller,image-automation-controller

Verify installation:

kubectl get pods -n flux-system | grep image
# image-automation-controller-xxx   1/1     Running
# image-reflector-controller-xxx    1/1     Running

ImageRepository: Scanning Container Registries

The ImageRepository resource tells Flux which container registry to scan and how often.

Basic ImageRepository (Public Registry)

For public registries like Docker Hub or GHCR without authentication:

# clusters/my-cluster/imagerepositories/flux-demo-website.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: flux-demo-website
  namespace: flux-system
spec:
  image: ghcr.io/talhajuikar/flux-demo-website
  interval: 5m
  provider: generic

Apply and verify:

kubectl apply -f clusters/my-cluster/imagerepositories/flux-demo-website.yaml

# Check status
kubectl get imagerepository -n flux-system

# NAME                 LAST SCAN              TAGS
# flux-demo-website   2025-06-25T10:30:00Z   15

# Describe for details
kubectl describe imagerepository flux-demo-website -n flux-system

The .status.lastScanResult shows discovered tags and .status.canonicalImageName shows the resolved registry URL.

ImageRepository with Authentication

For private registries, create a Secret with credentials:

Docker Registry Secret

kubectl create secret docker-registry regcred \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=<your-username> \
  --docker-password=<your-token> \
  --docker-email=<your-email> \
  -n flux-system

ImageRepository with Secret Reference

# clusters/my-cluster/imagerepositories/myapp.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: myapp
  namespace: flux-system
spec:
  image: docker.io/myorg/myapp
  interval: 10m
  provider: generic
  secretRef:
    name: regcred

Provider-Specific Authentication

Flux supports cloud provider authentication using Workload Identity:

AWS ECR with IRSA

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: myapp-ecr
  namespace: flux-system
spec:
  image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/myapp
  interval: 10m
  provider: aws
  serviceAccountName: image-reflector-controller

Patch the controller ServiceAccount with IAM role:

# clusters/my-cluster/flux-system/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - gotk-components.yaml
  - gotk-sync.yaml
patches:
  - patch: |-
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: image-reflector-controller
        annotations:
          eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/flux-image-reflector
    target:
      kind: ServiceAccount
      name: image-reflector-controller

Azure ACR with Workload Identity

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: myapp-acr
  namespace: flux-system
spec:
  image: myregistry.azurecr.io/myapp
  interval: 10m
  provider: azure
  serviceAccountName: image-reflector-controller

Patch for Azure Workload Identity:

patches:
  - patch: |-
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: image-reflector-controller
        annotations:
          azure.workload.identity/client-id: <AZURE_CLIENT_ID>
        labels:
          azure.workload.identity/use: "true"
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: image-reflector-controller
        labels:
          azure.workload.identity/use: "true"
      spec:
        template:
          metadata:
            labels:
              azure.workload.identity/use: "true"

Google GCR/Artifact Registry with Workload Identity

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: myapp-gcr
  namespace: flux-system
spec:
  image: gcr.io/my-project/myapp
  interval: 10m
  provider: gcp
  serviceAccountName: image-reflector-controller

Patch the ServiceAccount:

patches:
  - patch: |-
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: image-reflector-controller
        annotations:
          iam.gke.io/gcp-service-account: [email protected]
    target:
      kind: ServiceAccount
      name: image-reflector-controller

Advanced ImageRepository Features

Exclude Tags with Regex

Exclude signature tags or specific patterns:

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: myapp
  namespace: flux-system
spec:
  image: ghcr.io/myorg/myapp
  interval: 5m
  exclusionList:
    - "^.*\\.sig$"        # Exclude Cosign signatures
    - "^.*-debug$"        # Exclude debug builds
    - "^.*-snapshot$"     # Exclude snapshot builds

Cross-Namespace Access

Allow ImagePolicies in other namespaces to reference this ImageRepository:

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: shared-app
  namespace: flux-system
spec:
  image: ghcr.io/myorg/shared-app
  interval: 5m
  accessFrom:
    namespaceSelectors:
      - matchLabels:
          environment: production
      - matchLabels:
          team: platform

ImagePolicy: Selecting the Latest Image

The ImagePolicy resource defines rules for selecting which image tag is considered “latest” from the scanned tags.

Semantic Versioning Policy

The most common policy for production workloads:

# clusters/my-cluster/imagepolicies/flux-demo-website.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: flux-demo-website
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: flux-demo-website
  policy:
    semver:
      range: 1.0.x

This selects the latest 1.0.x version (e.g., 1.0.3).

Common SemVer Ranges

# Latest stable version (excluding pre-releases)
policy:
  semver:
    range: '>=1.0.0'

# Latest in major version 2
policy:
  semver:
    range: 2.x.x

# Latest in minor version 2.3
policy:
  semver:
    range: 2.3.x

# Latest patch in 2.3
policy:
  semver:
    range: '~2.3.0'

# Latest minor in major version 2
policy:
  semver:
    range: '^2.0.0'

# Include pre-releases (WARNING: will also select stable if available)
policy:
  semver:
    range: '^1.0.0-0'

# For pre-releases ONLY, use filterTags + alphabetical
filterTags:
  pattern: '^[0-9]+\.[0-9]+\.[0-9]+-(alpha|beta)\.[0-9]+$'
policy:
  alphabetical:
    order: asc

Important: When using semver with range: '>=1.0.0-0', it will select any matching version including stable releases. Since SemVer considers 1.0.0 > 1.0.0-beta.3, stable versions will be preferred. To select only pre-release tags, use filterTags with alphabetical policy instead.

Alphabetical Policy

For tags sorted alphabetically (useful for timestamp-based tags):

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: myapp-latest
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: myapp
  policy:
    alphabetical:
      order: asc  # or 'desc'

Numerical Policy

For numerical tags:

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: myapp-build
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: myapp
  policy:
    numerical:
      order: asc  # or 'desc'

Filter Tags with Regex

Select only specific tags before applying the policy:

# Select latest release candidate
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: flux-demo-website-rc
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: flux-demo-website
  filterTags:
    pattern: '^1\.0\..*-rc\d+$'
  policy:
    semver:
      range: 1.0.x-0

# Select latest RELEASE.timestamp tag (MinIO style)
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: minio
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: minio
  filterTags:
    pattern: '^RELEASE\.(?P<timestamp>.*)Z$'
    extract: '$timestamp'
  policy:
    alphabetical:
      order: asc

Digest Reflection

Track image digests for immutable deployments:

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: myapp-digest
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: myapp
  digestReflectionPolicy: IfNotPresent  # Never, Always, IfNotPresent
  policy:
    semver:
      range: 1.x.x

• Never: Don’t track digests (default)
• IfNotPresent: Track digest only when tag changes (recommended for immutable tags)
• Always: Always update digest, even if tag is the same (for mutable tags like latest)

When using Always, you must specify interval:

spec:
  digestReflectionPolicy: Always
  interval: 1h
  imageRepositoryRef:
    name: myapp
  policy:
    alphabetical:
      order: asc

Verify ImagePolicy

kubectl get imagepolicy -n flux-system

# NAME                 IMAGE                                TAG     READY
# flux-demo-website   ghcr.io/talhajuikar/flux-demo-website   1.0.0   True

kubectl describe imagepolicy flux-demo-website -n flux-system

# Status:
#   Latest Ref:
#     Image:   ghcr.io/talhajuikar/flux-demo-website
#     Tag:     1.0.0
#     Digest:  sha256:abc123...

ImageUpdateAutomation: Automating Git Updates

Now that we have ImageRepository scanning tags and ImagePolicy selecting the latest, we need ImageUpdateAutomation to update our Git repository.

Basic ImageUpdateAutomation

# clusters/my-cluster/imageupdateautomations/flux-system.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: [email protected]
        name: fluxcdbot
      messageTemplate: |
        Automated image update
        
        Automation name: {{ .AutomationObject }}
        
        Files:
        {{ range $filename, $_ := .Changed.FileChanges -}}
        - {{ $filename }}
        {{ end -}}
        
        Objects:
        {{ range $resource, $changes := .Changed.Objects -}}
        - {{ $resource.Kind }} {{ $resource.Name }}
          Changes:
        {{- range $_, $change := $changes }}
            - {{ $change.OldValue }} -> {{ $change.NewValue }}
        {{ end -}}
        {{ end -}}

    push:
      branch: main
  update:
    path: ./clusters/my-cluster
    strategy: Setters

How ImageUpdateAutomation Works

1. Every 30m (interval), it checks out the Git repository main branch
2. Scans all YAML files in ./clusters/my-cluster for image policy markers
3. Updates image values based on ImagePolicy .status.latestRef
4. Commits changes with the specified author and message
5. Pushes to the main branch
6. Flux detects the commit and reconciles the changes to the cluster

Marking Images for Update

To tell ImageUpdateAutomation which fields to update, add markers in comments:

Deployment with Full Image

apiVersion: apps/v1
kind: Deployment
metadata:
  name: flux-demo-website
  namespace: default
spec:
  template:
    spec:
      containers:
        - name: flux-demo-website
          image: ghcr.io/talhajuikar/flux-demo-website:1.0.0 # {"$imagepolicy": "flux-system:flux-demo-website"}

The marker format: {"$imagepolicy": "<namespace>:<policy-name>"}

HelmRelease with Separate Fields

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: flux-demo-website
  namespace: default
spec:
  values:
    image:
      repository: ghcr.io/talhajuikar/flux-demo-website # {"$imagepolicy": "flux-system:flux-demo-website:name"}
      tag: 1.0.0 # {"$imagepolicy": "flux-system:flux-demo-website:tag"}

The field marker format: {"$imagepolicy": "<namespace>:<policy-name>:<field>"}

Available fields:

• :name - Full image path without tag (e.g., ghcr.io/org/app)
• :tag - Just the tag (e.g., 1.2.3)
• :digest - Just the digest (e.g., sha256:abc123...)

With Digest Pinning

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: flux-demo-website
  namespace: default
spec:
  values:
    image:
      repository: ghcr.io/talhajuikar/flux-demo-website # {"$imagepolicy": "flux-system:flux-demo-website:name"}
      tag: 1.0.0 # {"$imagepolicy": "flux-system:flux-demo-website:tag"}
      digest: sha256:abc123... # {"$imagepolicy": "flux-system:flux-demo-website:digest"}

For digest to be available, the ImagePolicy must have digestReflectionPolicy: IfNotPresent or Always.

Advanced Git Configuration

Push to Different Branch

Useful for creating pull requests:

spec:
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: [email protected]
        name: fluxcdbot
    push:
      branch: flux-image-updates

This creates/updates the flux-image-updates branch with each update.

Using Refspec

For advanced Git workflows like Gerrit or creating PRs in Gitea/Forgejo:

spec:
  git:
    checkout:
      ref:
        branch: main
    push:
      refspec: refs/heads/main:refs/for/main

Commit Message Template with Values

spec:
  git:
    commit:
      messageTemplate: |
        [ci skip] Update images for cluster {{ .Values.clusterName }}
        
        {{ range $resource, $changes := .Changed.Objects -}}
        - {{ $resource.Kind }}/{{ $resource.Name }}: {{ range $_, $change := $changes }}{{ $change.NewValue }}{{ end }}
        {{ end -}}

      messageTemplateValues:
        clusterName: production

Template functions available include 70+ functions from Go templates and Sprig.

Signed Commits

Sign commits with GPG:

# Create GPG key secret
kubectl create secret generic git-signing-key \
  --from-file=git.asc=<path-to-gpg-key> \
  --from-literal=passphrase=<key-passphrase> \
  -n flux-system

spec:
  git:
    commit:
      author:
        email: [email protected]
        name: fluxcdbot
      signingKey:
        secretRef:
          name: git-signing-key

Push Options for GitLab MR

spec:
  git:
    push:
      branch: flux-updates
      options:
        merge_request.create: ""
        merge_request.target: main
        merge_request.title: "Automated image updates"

Push Options for Gitea/Forgejo PR

spec:
  git:
    push:
      branch: flux-updates
      refspec: refs/heads/flux-updates:refs/for/main
      options:
        topic: flux-updates
        title: "Flux Image Update"
        description: "This PR is automatically opened by flux"

Policy Selector

Limit which ImagePolicies an ImageUpdateAutomation considers:

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: production-apps
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  policySelector:
    matchLabels:
      environment: production
  git:
    # ... git config

Then label your ImagePolicies:

apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: flux-demo-website-prod
  namespace: flux-system
  labels:
    environment: production
spec:
  imageRepositoryRef:
    name: flux-demo-website
  policy:
    semver:
      range: 1.x.x

Or use matchExpressions:

spec:
  policySelector:
    matchExpressions:
      - key: app.kubernetes.io/component
        operator: In
        values:
          - frontend
          - backend

Monitoring ImageUpdateAutomation

# Check status
kubectl get imageupdateautomation -n flux-system

# NAME           LAST RUN
# flux-system    2025-06-25T14:22:34Z

# Describe for details
kubectl describe imageupdateautomation flux-system -n flux-system

# Status:
#   Last Automation Run Time:  2025-06-25T14:22:34Z
#   Last Push Commit:          abc123def456...
#   Last Push Time:            2025-06-25T14:22:34Z
#   Observed Policies:
#     Flux-Demo-Website:
#       Name:  ghcr.io/talhajuikar/flux-demo-website
#       Tag:   1.0.0

Trigger Manual Update

# Using kubectl
kubectl annotate --overwrite \
  imageupdateautomation/flux-system \
  reconcile.fluxcd.io/requestedAt="$(date +%s)" \
  -n flux-system

# Using flux CLI
flux reconcile image update flux-system -n flux-system

Complete Example: Multi-Environment Setup

Let’s create a complete example with separate ImagePolicies for dev, staging, and production:

Directory Structure

flux-demo-website/
├── .github/
│   └── workflows/
│       └── build.yml
├── apps/
│   └── demo-website/
│       ├── base/
│       │   ├── deployment.yaml
│       │   ├── service.yaml
│       │   ├── ingress.yaml
│       │   └── kustomization.yaml
│       └── overlays/
│           ├── dev/
│           ├── staging/
│           └── prod/
└── clusters/
    └── flux-demo/
        ├── flux-system/
        ├── apps.yaml
        └── image-automation/          # New directory
            ├── kustomization.yaml     # Flux will pick this up
            ├── flux-demo-website-repository.yaml
            ├── flux-demo-website-policy-dev.yaml
            ├── flux-demo-website-policy-staging.yaml
            ├── flux-demo-website-policy-prod.yaml
            └── image-update-automation.yaml

Kustomization Resource (So Flux Picks Up Image Automation)

First, create a Kustomization resource so Flux knows to reconcile these manifests:

# clusters/flux-demo/image-automation/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - flux-demo-website-repository.yaml
  - flux-demo-website-policy-dev.yaml
  - flux-demo-website-policy-staging.yaml
  - flux-demo-website-policy-prod.yaml
  - image-update-automation.yaml

Then create a Flux Kustomization to tell Flux to reconcile this directory:

# clusters/flux-demo/image-automation.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: image-automation
  namespace: flux-system
spec:
  interval: 10m
  path: ./clusters/flux-demo/image-automation
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system

ImageRepository

# clusters/flux-demo/image-automation/flux-demo-website-repository.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: flux-demo-website
  namespace: flux-system
spec:
  image: ghcr.io/talhajuikar/flux-demo-website
  interval: 5m
  provider: generic

ImagePolicies for Each Environment

# clusters/flux-demo/image-automation/flux-demo-website-policy-dev.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: flux-demo-website-dev
  namespace: flux-system
  labels:
    environment: dev
spec:
  imageRepositoryRef:
    name: flux-demo-website
  filterTags:
    pattern: '^[0-9]+\.[0-9]+\.[0-9]+-(alpha|beta)\.[0-9]+$'
    extract: '$0'
  policy:
    alphabetical:
      order: asc  # Latest alpha/beta tag alphabetically
---
# clusters/flux-demo/image-automation/flux-demo-website-policy-staging.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: flux-demo-website-staging
  namespace: flux-system
  labels:
    environment: staging
spec:
  imageRepositoryRef:
    name: flux-demo-website
  filterTags:
    pattern: '^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$'
  policy:
    semver:
      range: '>=1.0.0-0'  # Release candidates only
---
# clusters/flux-demo/image-automation/flux-demo-website-policy-prod.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: flux-demo-website-prod
  namespace: flux-system
  labels:
    environment: production
spec:
  imageRepositoryRef:
    name: flux-demo-website
  digestReflectionPolicy: IfNotPresent
  policy:
    semver:
      range: 1.x.x  # Stable releases only

Deployments with Markers

You have two options for adding image policy markers:

Option 1: Using images in kustomization.yaml (Recommended)

This is cleaner and doesn’t require separate patch files. Update your overlay kustomization.yaml files:

# apps/demo-website/overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev
resources:
  - ../../base

patchesStrategicMerge:
  - replica-patch.yaml
  - resource-patch.yaml
  - config-patch.yaml
  - ingress-patch.yaml

# Add the image override with policy marker
images:
  - name: ghcr.io/talhajuikar/flux-demo-website
    newTag: 1.0.0-alpha.1 # {"$imagepolicy": "flux-system:flux-demo-website-dev"}

commonLabels:
  environment: dev

nameSuffix: -dev

# apps/demo-website/overlays/staging/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: staging
resources:
  - ../../base

patchesStrategicMerge:
  - replica-patch.yaml
  - resource-patch.yaml
  - config-patch.yaml
  - ingress-patch.yaml

images:
  - name: ghcr.io/talhajuikar/flux-demo-website
    newTag: 1.0.0-rc.1 # {"$imagepolicy": "flux-system:flux-demo-website-staging"}

commonLabels:
  environment: staging

nameSuffix: -staging

# apps/demo-website/overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: production
resources:
  - ../../base

patchesStrategicMerge:
  - replica-patch.yaml
  - resource-patch.yaml
  - config-patch.yaml
  - ingress-patch.yaml

images:
  - name: ghcr.io/talhajuikar/flux-demo-website
    newTag: 1.0.0 # {"$imagepolicy": "flux-system:flux-demo-website-prod"}

commonLabels:
  environment: production

nameSuffix: -prod

Important Notes:

• The name field must exactly match the image name in your base deployment (without tag)
• Do NOT include newName field - only use it if you need to change the registry
• Flux will update only the newTag value based on the ImagePolicy
• The policy marker comment must be on the same line as newTag

Option 2: Using Deployment Patches (Alternative)

If you prefer separate patch files, create deployment patches:

# apps/demo-website/overlays/dev/deployment-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: flux-demo-website
spec:
  template:
    spec:
      containers:
        - name: demo-website
          image: ghcr.io/talhajuikar/flux-demo-website:1.0.0-alpha.1 # {"$imagepolicy": "flux-system:flux-demo-website-dev"}

Then reference it in your kustomization.yaml:

patchesStrategicMerge:
  - replica-patch.yaml
  - resource-patch.yaml
  - config-patch.yaml
  - ingress-patch.yaml
  - deployment-patch.yaml  # Add this

ImageUpdateAutomation

# clusters/flux-demo/image-automation/image-update-automation.yaml
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: flux-demo-website
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: [email protected]
        name: fluxcdbot
      messageTemplate: |
        [ci skip] Update flux-demo-website images
        
        {{ range $filename, $_ := .Changed.FileChanges -}}
        - {{ $filename }}
        {{ end -}}
        
        {{ range $resource, $changes := .Changed.Objects -}}
        {{ $resource.Namespace }}/{{ $resource.Kind }}/{{ $resource.Name }}
        {{- range $_, $change := $changes }}
          {{ $change.OldValue }} -> {{ $change.NewValue }}
        {{ end -}}
        {{ end -}}

    push:
      branch: main
  update:
    path: ./apps/demo-website  # Scans this directory recursively for image policy markers
    strategy: Setters

Important: The path field tells Flux where to look for image policy markers. With strategy: Setters, Flux will:

1. Recursively scan ./apps/demo-website directory and all subdirectories
2. Find all files with # {"$imagepolicy": ...} markers (including in overlays/dev/, overlays/staging/, overlays/prod/)
3. Update the image tags in those files based on the ImagePolicy status
4. Commit and push the changes

Apply Resources

# First, apply the Flux Kustomization so Flux picks up the image-automation directory
kubectl apply -f clusters/flux-demo/image-automation.yaml

# Or apply the resources directly
kubectl apply -f clusters/flux-demo/image-automation/

# Verify image automation resources are created
kubectl get imagerepository,imagepolicy,imageupdateautomation -n flux-system

# Wait for first scan and update
flux reconcile image repository flux-demo-website -n flux-system
flux reconcile image update flux-demo-website -n flux-system

# Check the status
kubectl describe imagerepository flux-demo-website -n flux-system
kubectl describe imagepolicy flux-demo-website-prod -n flux-system
kubectl describe imageupdateautomation flux-demo-website -n flux-system

Best Practices for Production

1. Use Semantic Versioning

Tag your container images with proper semantic versions:

# In your CI/CD pipeline (GitHub Actions)
# The workflow automatically builds from git tags

# Create tags for different environments:

# Stable release (for production)
git tag v1.0.0
git push origin v1.0.0

# Alpha releases (for dev environment)
git tag v1.0.1-alpha.1
git push origin v1.0.1-alpha.1

git tag v1.1.0-alpha.2
git push origin v1.1.0-alpha.2

# Release candidates (for staging environment)
git tag v1.0.1-rc.1
git push origin v1.0.1-rc.1

git tag v1.1.0-rc.2
git push origin v1.1.0-rc.2

# Beta releases
git tag v2.0.0-beta.1
git push origin v2.0.0-beta.1

How ImagePolicies Pick Them Up:

The ImagePolicy for each environment automatically selects the appropriate version:

• Dev Policy (range: '>=1.0.0-0'): Picks the latest from ALL versions including -alpha.X, -beta.X, -rc.X, and stable releases
• Staging Policy (with pattern: '^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$'): Only picks -rc.X (release candidate) tags
• Production Policy (range: 1.x.x): Only picks stable releases without pre-release identifiers

Example flow:

1. Push v1.0.1-alpha.1 → Dev environment gets updated
2. Push v1.0.1-rc.1 → Staging environment gets updated
3. Push v1.0.1 → Production environment gets updated

2. Separate Policies Per Environment

• Dev: range: '>=1.0.0-0' (all versions including pre-releases)
• Staging: range: '>=1.0.0-rc.0' (release candidates)
• Production: range: 1.x.x (stable releases only)

3. Use Digest Pinning for Production

spec:
  digestReflectionPolicy: IfNotPresent
  policy:
    semver:
      range: 1.x.x

This ensures immutable deployments even if a tag is overwritten.

4. Set Appropriate Intervals

# Image scanning (ImageRepository)
interval: 5m   # Frequent for fast feedback

# Git updates (ImageUpdateAutomation)
interval: 30m  # Less frequent to batch updates

5. Use Separate ImageUpdateAutomation for Each Environment

Instead of one automation for all environments:

# Production automation (more cautious)
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: production-updates
  namespace: flux-system
spec:
  interval: 1h
  policySelector:
    matchLabels:
      environment: production
  git:
    push:
      branch: prod-image-updates  # Create PR instead of direct push
---
# Dev automation (more aggressive)
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: dev-updates
  namespace: flux-system
spec:
  interval: 15m
  policySelector:
    matchLabels:
      environment: dev
  git:
    push:
      branch: main  # Direct push to main

6. Exclude Unwanted Tags

spec:
  exclusionList:
    - "^.*\\.sig$"        # Cosign signatures
    - "^.*-debug$"        # Debug images
    - "^.*-snapshot$"     # Snapshot builds
    - "^sha-[0-9a-f]+$"   # Git SHA tags

7. Monitor Automation Status

Set up alerts for failed image updates:

# Using Flux notification controller
apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: image-update-alerts
  namespace: flux-system
spec:
  eventSeverity: error
  eventSources:
    - kind: ImageUpdateAutomation
      name: '*'
  providerRef:
    name: slack

8. Use Commit Message Templates with Skip CI

Prevent infinite CI loops:

spec:
  git:
    commit:
      messageTemplate: |
        [ci skip] Automated image update
        
        {{ range $resource, $changes := .Changed.Objects -}}
        {{ $resource.Kind }}/{{ $resource.Name }}: {{ range $change := $changes }}{{ $change.NewValue }}{{ end }}
        {{ end -}}

9. Secure Your Automation

• Use dedicated ServiceAccounts with minimal permissions
• Rotate registry credentials regularly
• Use Workload Identity when possible (AWS IRSA, Azure Workload Identity, GCP Workload Identity)
• Audit Git commits from automation

10. Test Image Updates

Before enabling automation in production:

# Manually trigger to see what would change
flux reconcile image repository flux-demo-website -n flux-system
flux reconcile image update flux-system -n flux-system --dry-run

# Check ImagePolicy status
kubectl get imagepolicy flux-demo-website-prod -n flux-system -o jsonpath='{.status.latestRef}'

Troubleshooting

ImageRepository Not Scanning

# Check ImageRepository events
kubectl describe imagerepository flux-demo-website -n flux-system

# Common issues:
# - Authentication failure (check secret)
# - Invalid image name
# - Network issues

# Check controller logs
kubectl logs -n flux-system deploy/image-reflector-controller -f

ImagePolicy Not Selecting Latest

# Check policy status
kubectl describe imagepolicy flux-demo-website-prod -n flux-system

# Verify tags are being scanned
kubectl get imagerepository flux-demo-website -n flux-system -o jsonpath='{.status.lastScanResult.latestTags}'

# Common issues:
# - No tags match the semver range
# - Regex filter excludes all tags
# - ImageRepository not ready

ImageUpdateAutomation Not Committing

# Check automation status
kubectl describe imageupdateautomation flux-system -n flux-system

# Common issues:
# - No image policy markers in manifests
# - Git authentication failure
# - Branch protection rules prevent push
# - Path doesn't match YAML files location

# Check controller logs
kubectl logs -n flux-system deploy/image-automation-controller -f

No Updates Being Applied

# Verify marker syntax
grep -r '{"$imagepolicy"' apps/demo-website/

# Should look like:
# {"$imagepolicy": "flux-system:flux-demo-website-prod"}
# {"$imagepolicy": "flux-system:flux-demo-website-prod:tag"}

# Common issues:
# - Wrong namespace in marker
# - Typo in policy name
# - Comment syntax incorrect

Conclusion

Flux Image Automation provides a powerful, GitOps-native way to keep your Kubernetes workloads up-to-date with the latest container images. By combining:

• ImageRepository for registry scanning
• ImagePolicy for version selection
• ImageUpdateAutomation for Git updates

You can create fully automated deployment pipelines that maintain the Git repository as the single source of truth.

Key takeaways:

✅ Use semantic versioning for proper version selection
✅ Separate policies per environment (dev, staging, prod)
✅ Enable digest pinning for production immutability
✅ Set appropriate scan and update intervals
✅ Use markers to tell Flux which fields to update
✅ Leverage Workload Identity for cloud registries
✅ Monitor automation status and set up alerts
✅ Test thoroughly before enabling in production

In the next part, we’ll explore Flux notifications and integrations with monitoring systems to create a complete observability solution for your GitOps workflows.

Additional Resources

• Flux Image Automation Guide
• ImageRepository API Reference
• ImagePolicy API Reference
• ImageUpdateAutomation API Reference
• Demo Repository
• Flux Slack Channel