This topic describes the process of integrating Github Actions with Terraform to automatically create resources in Amazon Web Services (AWS). The use of Github Actions allows for automation of the provisioning process and Terraform is an Infrastructure as Code (IAC) tool which provides a way to define and provision infrastructure resources.
By combining these two tools, developers can automate the creation of AWS resources such as EC2 instances, S3 buckets and Security Group, IAM roles and policies , by defining the desired state of the infrastructure in Terraform configuration files. These files can then be added to a Github repository, and Github Actions can be used to trigger Terraform to provision the resources on AWS, when certain events occur, such as a code push or a pull request.
This approach can save time and reduce errors, as well as provide a way to version-control and track changes to the infrastructure.
What is Github Actions ?
GitHub Actions is a powerful and flexible way to automate your software development workflows directly in your GitHub repository. It allows you to create custom actions, called workflows, that can be triggered by a variety of events on your repository, such as a push to a branch, a pull request, or the creation of a release.
Advantages of Github Actions
Automation: GitHub Actions allows you to automate a wide range of tasks, such as building and testing your code, deploying it to different environments, and even running automated security scans. This can save time and effort, and reduce the chances of errors.
Flexibility: GitHub Actions is highly flexible, allowing you to create custom workflows that can be triggered by a variety of events on your repository, such as a push to a branch, a pull request, or the creation of a release.
Integration: GitHub Actions can be easily integrated with other tools, such as AWS, Azure, Google Cloud, and Slack, allowing you to create powerful and streamlined workflows that can take advantage of the functionality of these tools.
Community: GitHub Actions has a large and active community, which means you can find pre-built actions for common tasks, such as running tests or deploying to different cloud platforms. Additionally, you can share your own actions with others, which allows you to collaborate and learn from others in the community.
Cost-effective: GitHub Actions is free for public repositories and provides a very affordable pricing for private repositories, which makes it a cost-effective option for automating your development workflows.
Scalability: GitHub Actions can handle an unlimited number of parallel jobs, which makes it a great option for teams of all sizes, from small open-source projects to large enterprise teams.
Step by Step Explanation of Terraform Workflow
So this is the working directory structure of the .github/workflows. There are 2 workflows – one where we have Tflint, Tfsec and terraform and installing scripts jobs, in total 4 jobs and the other which has 2 jobs – build and deploy.
Let’s check out the terraform.yml file
name: Create resources using Terraform
on:
push:
branches:
– main
jobs:
tflint-checks:
runs-on: ubuntu-latest
steps:
# Checkout Repository
– name : Check out Git Repository
uses: actions/checkout@v3
# TFLint – Terraform Check
– uses: actions/cache@v2
name: Cache plugin dir
with:
path: ~/.tflint.d/plugins
key: ${{ matrix.os }}-tflint-${{ hashFiles(‘.tflint.hcl’) }}
– uses: terraform-linters/setup-tflint@v2
name: Setup TFLint
with:
github_token: ${{ secrets.CI_GITHUB_TOKEN }}
# Print TFLint version
– name: Show version
run: tflint –version
# Install plugins
– name: Init TFLint
run: tflint –init
# Run tflint command in each directory recursively
– name: Run TFLint
run: tflint -f compact –recursive –force
tfsec-checks:
runs-on: ubuntu-latest
needs: tflint-checks
steps:
# Checkout Repository
– name : Check out Git Repository
uses: actions/checkout@v2
# Tfsec – Security scanner for your Terraform code
– name: Run Tfsec
uses: aquasecurity/tfsec-action@v1.0.0
terraform-setup:
permissions:
id-token: write # Job to connect to Identity Token to receive the token
contents: read # Read access to the repository
runs-on: ubuntu-latest
needs: tfsec-checks
steps:
# Checkout Repository
– name : Check out Git Repository
uses: actions/checkout@v3
– name: Connecting GitHub Actions To AWS Using OIDC – Roles
uses: aws-actions/configure-aws-credentials@v1
with:
aws-region: ${{ secrets.AWS_REGION }}
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
role-session-name: github-actions-session
# Terraform Installation
– name : Terraform Setup
run: |
sudo apt-get update && sudo apt-get install -y gnupg software-properties-common
wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg –dearmor | \
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
gpg –no-default-keyring \
–keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
–fingerprint
echo “deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main” | \
sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt-get install -y terraform
terraform –version
– name : Terraform Init and Validate
run: |
cd terraform_code
terraform init
terraform validate
– name: Terraform Plan
id: plan
run: |
cd terraform_code
terraform plan
continue-on-error: true
– name: Terraform Plan Status
if: ${{ steps.plan.outcome == ‘failure’ }}
run: exit 1
– name: Terraform Apply
id: apply
run: |
cd terraform_code
terraform apply -auto-approve
continue-on-error: true
– name: Terragrunt Apply Status
if: ${{ steps.apply.outcome == ‘failure’ }}
run: exit 1
install-services:
permissions:
id-token: write # Job to connect to Identity Token to receive the token
contents: read # Read access to the repository
runs-on: ubuntu-latest
needs: terraform-setup
steps:
# Checkout Repository
– name : Check out Git Repository
uses: actions/checkout@v3
– name: Connecting GitHub Actions To AWS Using OIDC – Roles
uses: aws-actions/configure-aws-credentials@v1
with:
aws-region: ${{ secrets.AWS_REGION }}
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
role-session-name: github-actions-session
# Public IP of Github Actions
– name: Public IP of Github Hosted Runner
id: ip
uses: haythem/public-ip@v1.3
# Security Group Id of EC2 Instance
– name: Get Security Group Id of EC2 Instance
id: ec2
env:
EC2_NAME: ${{ secrets.AWS_EC2_SG_NAME }}
run: |
ec2_sg_id=`aws ec2 describe-security-groups –group-names $EC2_NAME –query ‘SecurityGroups[*].[GroupId]’ –output text`
echo “::set-output name=ec2_security_group_id::$(echo $ec2_sg_id)”
– name: Add Github Runner Instance IP to Security group
run: |
aws ec2 authorize-security-group-ingress –group-id ${{ steps.ec2.outputs.ec2_security_group_id }} –protocol tcp –port 22 –cidr ${{ steps.ip.outputs.ipv4 }}/32
– name: Public IP of EC2 Instance
id: hostname
env:
EC2_NAME: ${{ secrets.AWS_EC2_NAME }}
run: |
ec2_public_ip=`aws –region ${{ secrets.AWS_REGION }} ec2 describe-instances –filters “Name= tag:Name,Values=$EC2_NAME” –query ‘Reservations[*].Instances[*].[PublicIpAddress]’ –output text`
echo “::set-output name=ec2_ip::$(echo $ec2_public_ip)”
– name: Copy Script files via ssh password
uses: appleboy/scp-action@master
with:
host: ${{ steps.hostname.outputs.ec2_ip }}
username: ${{ secrets.EC2_USER }}
key: ${{ secrets.EC2_PRIVATE_KEY }}
source: “scripts/*.sh”
target: “.”
– name: Deploy Docker Script in EC2 Instance
uses: appleboy/ssh-action@v0.1.6
with:
host: ${{ steps.hostname.outputs.ec2_ip}}
username: ${{ secrets.EC2_USER }}
key: ${{ secrets.EC2_PRIVATE_KEY }}
port: 22
script: |
whoami
ls -al
chmod +x scripts/*.sh
bash scripts/install-docker.sh
bash scripts/install-sonarqube.sh
bash scripts/install-kubectl.sh
bash scripts/install-minikube.sh
bash scripts/install-trivy.sh
bash scripts/install-terrascan.sh
rm -rf scripts
– name: Remove Github Actions IP from security group
run: |
aws ec2 revoke-security-group-ingress –group-id ${{ steps.ec2.outputs.ec2_security_group_id }} –protocol tcp –port 22 –cidr ${{ steps.ip.outputs.ipv4 }}/32
if: always()
– name: Trigger next workflow for SpringBoot
if: success()
uses: peter-evans/repository-dispatch@v1
with:
token: ${{ secrets.CI_GITHUB_TOKEN }}
repository: ${{ github.repository }}
event-type: springbootworkflow
client-payload: ‘{“ref”:”${{ github.ref }}”,”sha”:”${{github.sha}}”}’
This is a GitHub Actions workflow that automates the process of creating resources using Terraform in AWS. The pipeline is triggered when a push is made to the main branch of the repository. The pipeline is divided into four jobs, each job runs in the ubuntu-latest environment and each one depends on the previous one to finish.
Tflint-Checks JOB : The first job is tflint-checks, it runs TFLint, a linter for Terraform code, on the codebase. The job starts by checking out the repository code and then it uses actions/cache@v2 to cache the plugin directory, it then uses terraform-linters/setup-tflint@v2 to setup TFLint. TFLint version is printed and then it runs tflint –init to install plugins, after that it runs tflint -f compact –recursive –force command in each directory recursively to check the code.
Tfsec-Checks JOB : The second job is tfsec-checks it runs tfsec, a security scanner for your Terraform code on the codebases and it depends on the previous job tflint-checks to finish. It starts by checking out the repository code, then it uses aquasecurity/tfsec-action@v1.0.0 to run tfsec on the codebase.
Terraform-Setup JOB : It starts with the “terraform-setup” job that runs on an ubuntu-latest environment and needs the previous job “tfsec-checks” to complete first. This job first checks out the repository using the actions/checkout@v3 action.
Then it sets up the connection between GitHub Actions and AWS using OIDC(OpenID Connect) by using the action aws-actions/configure-aws-credentials@v1. This action is used to assume an IAM role in order to provide the necessary credentials to Terraform. The necessary secrets such as AWS_REGION, AWS_ROLE_TO_ASSUME and role-session-name are passed to this action as input.
The next step is to install Terraform using the commands provided in the action. This installs Terraform and verifies the version of it. Then it initializes and validates the Terraform code in the repository.
The next step is to create a plan of what changes will be made and run the plan with the command terraform plan. This step is followed by a check to see if the plan was successful or not and if not, it exits with a failure status and same with apply command.
If the plan is successful, the next step is to apply the changes using the command terraform apply -auto-approve. This step is also followed by a check to see if the apply was successful or not and if not, it exits with a failure status.
End Result – Resources created in AWS
Security Group for EC2 Instance
IAM and Instance Profile for EC2 Instance
EC2 Instance to deploy SonarQube, Minikube and Docker
ECR to store docker containers
Terraform Tfstate files in s3 Bucket
Install-Services JOB : The 4th job in the GitHub Actions pipeline called install-services. It runs on the ubuntu-latest environment and it depends on the previous job terraform-setup to finish.
The job starts by checking out the repository code, then it uses aws-actions/configure-aws-credentials@v1 to connect to AWS using OpenID Connect (OIDC) and assume the role that has been defined in the secrets of the repository.
Then it uses haythem/public-ip@v1.3 to get the public IP of the GitHub Actions Hosted Runner.
Next, it uses AWS CLI to get the Security Group Id of the EC2 instance where we will be in future articles will be deploying Docker, Minikube and SonarQube into it , and then add the GitHub Actions Hosted Runner IP to the security group using aws ec2 authorize-security-group-ingress. It then uses AWS CLI to get the public IP of EC2 instance and store it in an environment variable.
The next step is to copy all the script files via ssh to the EC2 instance using appleboy/scp-action@master.
The next step is to deploy the scripts on EC2 instances using appleboy/ssh-action@v0.1.6 to run the scripts of installing docker, sonarqube, kubectl and minikube. It first login into the EC2 Instance created by terraform and then installs all packages .
At the last stage it removes the GitHub Actions Hosted Runner IP from the security group using aws ec2 revoke-security-group-ingress. The if: always() statement ensures that this step runs even if the previous steps failed.
This step is used to trigger a new workflow called “springbootworkflow” in the same repository. The step uses the peter-evans/repository-dispatch action to trigger the workflow. It passes in the CI_GITHUB_TOKEN as the token, the github.repository as the repository, and sets the event type to springbootworkflow. It also passes in a client-payload which includes the ref and sha of the current commit. This allows the next workflow to access the current commit information. The if statement if: success() is used to trigger the next workflow only if this step is successful.
See, Docker is installed in EC2 Instance.
Login into SonarQube : http://<PublicIP>:9000 . Use password and username as “admin” and change the password.
See, Kubectl is installed in EC2 Instance.
See, Minikube is running on EC2 Instance
