CI/CD Deployment Workflow with GitHub Actions and Terraform/Terragrunt
The deployment process for our infrastructure is automated using GitHub Actions, integrated with Terraform and Terragrunt. This setup ensures that our infrastructure as code (IaC) is consistently and reliably deployed across different environments (dev, stage, and prod).
Workflow Overview
The workflow, named “Terraform Apply,” is triggered manually through the GitHub Actions interface using the workflow_dispatch event. This allows developers to specify the branch and environment they wish to deploy to, providing flexibility and control over the deployment process.
Inputs
branch:
This input specifies the Git branch to be used for the deployment. The available options are dev, stage, and main.
environment:
This input specifies the environment to deploy to. The options are dev, stg, and prd.
Environment Variables
The workflow uses several environment variables, which are securely stored in GitHub Secrets to protect sensitive information. These variables are critical for authenticating and configuring the deployment process:
ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_TENANT_ID:
Azure service principal credentials for Terraform to authenticate and deploy resources in Azure.
DATABRICKS_ACCOUNT_ID:
Databricks account ID used during the deployment.
TF_VAR_SP_DB_SECRET, TF_VAR_SP_DB_APPLICATION_ID, TF_VAR_SP_DB_DIRECTORY_ID, TF_VAR_SP_DB_OBJECT_ID:
Variables related to service principals for Databricks.
TF_VAR_ACS_ADLS_ETL:
Access credentials for Azure Data Lake Storage (ADLS).
TF_VAR_NEOXIA_OBJECT_ID:
Object ID related to specific resources in Azure.
AGOL_USERNAME, AGOL_PASSWORD:
Credentials for accessing ArcGIS Online (AGOL) services.
tf_version:
Specifies the version of Terraform to use (e.g., 1.6.3).
tg_version:
Specifies the version of Terragrunt to use (e.g., 0.53.2).
ENVIRONMENT:
Captures the selected deployment environment from the input.
BRANCH_NAME:
Captures the Git branch name for deployment.
Job: Terraform
This job runs on an Ubuntu 22.04 environment and consists of several steps that handle the Terraform/Terragrunt deployment process.
Checkout:
The first step checks out the specified branch of the repository using the actions/checkout@v3 action.
Verify Parameters:
This step verifies the selected branch and environment. It also determines the working directory specific to the environment and branch by running a custom script (get_working_directory.sh).
Set Terragrunt Working Directory:
The working directory for Terragrunt is set based on the environment and branch, ensuring the correct configuration is applied.
Terragrunt Init:
The Terragrunt Init step initializes Terragrunt for all Terraform modules within the working directory, excluding specific directories (kubernetes_cluster_superset/ and postgresql_superset_cvat/). This initialization prepares the modules for further actions.
Terragrunt Plan:
The Terragrunt Plan step generates an execution plan, showing what actions Terraform will take to achieve the desired state without actually making any changes. It provides an opportunity to review the planned changes.
Terragrunt Apply:
Finally, the Terragrunt Apply step applies the planned changes to the infrastructure. It runs the apply command across all modules, excluding the same directories as before.
Key Points
Terragrunt:
Used as a wrapper for Terraform to manage multiple Terraform modules with consistent configurations. It simplifies and standardizes the deployment process.
Exclusions:
The workflow deliberately excludes certain directories (kubernetes_cluster_superset/ and postgresql_superset_cvat/) from the deployment process. This might be due to those components being managed separately or requiring different workflows.
This workflow is a powerful automation tool that allows developers to deploy infrastructure changes consistently across different environments while leveraging Terraform’s robust IaC capabilities and Terragrunt’s module management features.