SyncFlows

Sync flows from Git to Kestra.

This task syncs flows from a given Git branch to a Kestra namespace. If the delete property is set to true, any flow available in kestra but not present in the gitDirectory will be deleted, considering Git as a single source of truth for your flows. Check the Version Control with Git documentation for more details.

yaml
type: "io.kestra.plugin.git.SyncFlows"

Examples

Sync all flows and scripts for selected namespaces from Git to Kestra every full hour. Note that this is a System Flow, so make sure to adjust the Scope to SYSTEM in the UI filter to see this flow or its executions.

yaml
id: git_sync
namespace: system

tasks:
  - id: sync
    type: io.kestra.plugin.core.flow.ForEach
    values: ["company", "company.team", "company.analytics"]
    tasks:
      - id: flows
        type: io.kestra.plugin.git.SyncFlows
        targetNamespace: "{{ taskrun.value }}"
        gitDirectory: "{{'flows/' ~ taskrun.value}}"
        includeChildNamespaces: false

      - id: scripts
        type: io.kestra.plugin.git.SyncNamespaceFiles
        namespace: "{{ taskrun.value }}"
        gitDirectory: "{{'scripts/' ~ taskrun.value}}"

pluginDefaults:
  - type: io.kestra.plugin.git
    values:
      username: anna-geller
      url: https://github.com/anna-geller/product
      password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
      branch: main
      dryRun: false

triggers:
  - id: every_full_hour
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 * * * *"

Sync flows from a Git repository. This flow can run either on a schedule (using the Schedule trigger) or anytime you push a change to a given Git branch (using the Webhook trigger).

yaml
id: sync_flows_from_git
namespace: system

tasks:
  - id: git
    type: io.kestra.plugin.git.SyncFlows
    gitDirectory: flows # optional; set to _flows by default
    targetNamespace: git # required
    includeChildNamespaces: true # optional; by default, it's set to false to allow explicit definition
    delete: true # optional; by default, it's set to false to avoid destructive behavior
    url: https://github.com/kestra-io/flows # required
    branch: main
    username: git_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    dryRun: true  # if true, the task will only log which flows from Git will be added/modified or deleted in kestra without making any changes in kestra backend yet

triggers:
  - id: every_full_hour
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 * * * *"

Properties

targetNamespace *string

The target namespace to which flows from the gitDirectory should be synced

If the top-level namespace specified in the flow source code is different than the targetNamespace, it will be overwritten by this target namespace. This facilitates moving between environments and projects. If includeChildNamespaces property is set to true, the top-level namespace in the source code will also be overwritten by the targetNamespace in children namespaces.

For example, if the targetNamespace is set to prod and includeChildNamespaces property is set to true, then:

namespace: dev in flow source code will be overwritten by namespace: prod,
namespace: dev.marketing.crm will be overwritten by namespace: prod.marketing.crm.

See the table below for a practical explanation:

Source namespace in the flow code	Git directory path	Synced to target namespace
namespace: dev	_flows/flow1.yml	namespace: prod
namespace: dev	_flows/flow2.yml	namespace: prod
namespace: dev.marketing	_flows/marketing/flow3.yml	namespace: prod.marketing
namespace: dev.marketing	_flows/marketing/flow4.yml	namespace: prod.marketing
namespace: dev.marketing.crm	_flows/marketing/crm/flow5.yml	namespace: prod.marketing.crm
namespace: dev.marketing.crm	_flows/marketing/crm/flow6.yml	namespace: prod.marketing.crm

branch string

Default main

The branch from which flows will be synced to Kestra

cloneSubmodules booleanstring

Whether to clone submodules

delete booleanstring

Default false

Whether you want to delete flows present in Kestra but not present in Git

It’s false by default to avoid destructive behavior. Use this property with caution because when set to true and includeChildNamespaces is also set to true, this task will delete all flows from the targetNamespace and all its child namespaces that are not present in Git rather than only overwriting the changes.

dryRun booleanstring

Default false

If true, the task will only output modifications without performing any modification to Kestra. If false (default), all listed modifications will be applied.

failOnMissingDirectory booleanstring

Default true

If true (default), the task will fail if the specified directory doesn't exist. If false, missing directories will be skipped.

gitConfig object

Git configuration to apply to the repository

Map of Git config keys and values, applied after clone few examples: - 'core.fileMode': false -> ignore file permission changes - 'core.autocrlf': false -> prevent line ending conversion

gitDirectory string

Default _flows

Directory from which flows should be synced

If not set, this task assumes your branch has a Git directory named _flows (equivalent to the default gitDirectory of the PushFlows task).

If includeChildNamespaces property is set to true, this task will push all flows from nested subdirectories into their corresponding child namespaces, e.g. if targetNamespace is set to prod, then:

flows from the _flows directory will be synced to the prod namespace,
flows from the _flows/marketing subdirectory in Git will be synced to the prod.marketing namespace,
flows from the _flows/marketing/crm subdirectory will be synced to the prod.marketing.crm namespace.

ignoreInvalidFlows booleanstring

Default false

Ignore flows when they have validation failure

Due to breaking changes, some flows may not be valid anymore by the time of the synchronisation. To avoid synchronizing flows that are no longer valid, set this property to true.

includeChildNamespaces booleanstring

Default false

Whether you want to sync flows from child namespaces as well

It’s false by default so that we sync only flows from the explicitly declared gitDirectory without traversing child directories. If set to true, flows from subdirectories in Git will be synced to child namespace in Kestra using the dot notation . for each subdirectory in the folder structure.

passphrase string

The passphrase for the privateKey

password string

The password or Personal Access Token (PAT) -- when you authenticate the task with a PAT, any flows or files pushed to Git from Kestra will be pushed from the user associated with that PAT. This way, you don't need to configure the commit author (the authorName and authorEmail properties).

privateKey string

PEM-format private key content that is paired with a public key registered on Git

To generate an ECDSA PEM format key from OpenSSH, use the following command: ssh-keygen -t ecdsa -b 256 -m PEM. You can then set this property with your private key content and put your public key on Git.

trustedCaPemPath string

Optional path to a PEM-encoded CA certificate to trust (in addition to the JVM default truststore)

Equivalent to git config http.sslCAInfo <path>. Use this for self-signed/internal CAs.

url string

The URI to clone from

username string

The username or organization

​Sync​Flows

SyncFlows