Provision and orchestrate GCE with Terraform and Ansible

6 minute read Published:

Using Terraform and Ansible to provision Google Compute Engine servers, and orchestrate them with Ansible to install Zabbix

Links:

Google Cloud Platform setup

From the Google Cloud Platform web UI, perform all the initial setup (e.g. set up a billing account with your credit card information).

Afterwards, create a project. In my case, I named it “zabbix”. Finally, set up an API service account for it, following the steps from API Wizard:

gcp1

I named the account “zabbix-admin” and gave it all permissions for Google Compute Engine. You will be prompted to download a JSON credentials file for this account, which is the file Terraform will use to provision machines in the “zabbix” project zone.

I save the json file to ~/.gcp/account.json, to imitate the way the aws cli stores credentials in ~/.aws/config.

Now’s a good time to add your ssh key to the Compute Engine. Copy your ~/.ssh/id_rsa.pub to clipboard (e.g. by running xclip -selection c -i ~/.ssh/id_rsa.pub) and add it via the Compute Engine SSH section:

gcp2

Finally, set up the gcloud cli. Download it from the official site and install it on your box.

Setup:

$ gcloud auth login
...authentication form will open in your browser
$ gcloud config set project zabbix-140209

Some sample commands:

$ gcloud compute images list
NAME                                 PROJECT          FAMILY           DEPRECATED  STATUS
centos-6-v20160803                   centos-cloud     centos-6                     READY
centos-7-v20160803                   centos-cloud     centos-7 
...
$ gcloud compute machine-types list | grep us-east1-b
f1-micro        us-east1-b      1     0.60
g1-small        us-east1-b      1     1.70
n1-highcpu-16   us-east1-b      16    14.40
...

Now you have all the tools you need to get information about the Google Compute Engine and create your Terraform execution plan.

Terraform

First download Terraform if you don’t already have it, and put the bin somewhere in your path.

Second, have a read through the Terraform Google Cloud Platform Provider and Terraform Google Compute Engine docs.

Run Terraform

We need a handful of files in a single directory; a provider.tf, a variables.tf, and a zabbix.tf:

# provider.tf
provider "google" {
    credentials = "${file("${var.account_file}")}"
    project = "${var.google_project_id}"
    region = "${var.region}"
}

# variables.tf

variable "google_project_id" { default = "zabbix-140209" }
variable "account_file"      { default = "~/.gcp/account.json" }
variable "region" 	     { default = "us-east1" }
variable "zone" 	     { default = "us-east1-b" }
variable "tags" 	     { default = ["zabbix-server", "zabbix-agent"] }
variable "image" 	     { default = "centos-7-v20160803" }
variable "machine_type"      { default = "n1-standard-1" }

# zabbix.tf
resource "google_compute_instance" "zabbix" {
    count = "${length(var.tags)}"
    name = "zabbix-${count.index+1}"
    machine_type = "${var.machine_type}"
    zone = "${var.zone}"
    tags = ["${var.tags[count.index]}"]

    disk = {
    image = "${var.image}"
    }

    network_interface {
      network = "default"
      access_config {
	// Ephemeral IP
      }
    }
}

Run terraform plan from the dir containing the .tf files. This is a sort of preview command which tells you what Terraform thinks it will provision:

$ terraform plan

[...]

+ google_compute_instance.zabbix.0
    ...[ommitted VM metadata]

+ google_compute_instance.zabbix.1
    ...[ommitted VM metadata]

Plan: 2 to add, 0 to change, 0 to destroy.

Looks good. Now run terraform apply:

$ terraform apply
google_compute_instance.zabbix.0: Creating...
... [ommitted provisioned VM metadata]
google_compute_instance.zabbix.1: Creating...
... [ommitted provisioned VM metadata]
google_compute_instance.zabbix.0: Still creating... (10s elapsed)
google_compute_instance.zabbix.1: Still creating... (10s elapsed)
google_compute_instance.zabbix.0: Still creating... (20s elapsed)
google_compute_instance.zabbix.1: Still creating... (20s elapsed)
google_compute_instance.zabbix.1: Creation complete
google_compute_instance.zabbix.0: Creation complete

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

The state is stored in file ./terraform.tfstate. This file is important - it contains all of the information about the 2 VMs that Terraform provisioned, as well as allows you to run terraform destroy later to gracefully remove the provisioned stack.

Convert Terraform tfstate file to Ansible inventory

Now that our machines are provisioned, we’ll manually create the inventory file:

$ cat terraform.tfstate | grep '.*_nat_ip.*\|.*tags\.[0-9][0-9].*' | cut -d':' -f2 | awk '{print $1}' | cut -d'"' -f2
104.196.172.86
zabbix-server
104.196.168.252
zabbix-agent

The final inventory file looks like this:

[zabbix_server]
104.196.172.86

[zabbix_agent]
104.196.168.252

Ansible

Install Ansible on your machine. Since Ansible is agentless (only requires SSH on the target machines), that’s all you need.

Playbook & roles

An Ansible playbook is a recipe of multiple Ansible directives in a single file. We can provision both the Zabbix server and agent machines in a single playbook, by leveraging Ansible roles:

$ cat zabbix-playbook.yml 
- name: prepare zabbix servers 
  hosts: zabbix_server
  roles:
    - base 
    - mariadb
    - server

- name: prepare zabbix agent
  hosts: zabbix_agent
  roles:
    - base
    - agent

I perform the zabbix-base task on both VMs, and then I perform a per-role task - zabbix-server for the server, and zabbix-agent for the agent.

Since the goal of tools like Terraform and Ansible is infrastructure-as-code, I followed code best practises, just like with Terraform - code broken into single roles that do one thing and do it well.

$ tree
.
├── inventory
├── roles
│   ├── agent
│   │   ├── tasks
│   │   │   └── main.yml
│   │   └── vars
│   │       └── main.yml
│   ├── base
│   │   ├── files
│   │   │   └── zabbix-3.0.4.tar.gz
│   │   ├── tasks
│   │   │   └── main.yml
│   │   └── vars
│   │       └── main.yml
│   ├── mariadb
│   │   └── tasks
│   │       └── main.yml
│   └── server
│       ├── tasks
│       │   └── main.yml
│       └── vars
│           └── main.yml
└── zabbix-playbook.yml

Run the playbook

Let’s run the playbook to see if it worked:

$ ansible-playbook -i inventory zabbix-playbook.yml 

PLAY [prepare vms for zabbix] **************************************************

TASK [setup] *******************************************************************
ok: [104.196.168.252]
ok: [104.196.172.86]

[... some lines ommitted]

PLAY RECAP *********************************************************************
104.196.168.252            : ok=7    changed=6    unreachable=0    failed=0   
104.196.172.86             : ok=9    changed=8    unreachable=0    failed=0

Manual verification of server:

[sevag@zabbix-1 ~]$ cd /opt/zabbix/
[sevag@zabbix-1 zabbix]$ ls -ltrh
total 15M
-rw-r--r--.  1 root  root            15M Aug 13 14:52 zabbix.tar.gz
drwxr-xr-x. 13 sevag google-sudoers 4.0K Aug 13 14:53 zabbix-3.0.4
[sevag@zabbix-1 zabbix]$ zabbix_server 
[sevag@zabbix-1 zabbix]$ ps -ef | grep zabbix_server
sevag     9969     1  0 14:57 ?        00:00:00 zabbix_server

Manual verification of agent:

[sevag@zabbix-2 ~]$ cd /opt/zabbix/
[sevag@zabbix-2 zabbix]$ ls -ltrh
total 15M
-rw-r--r--.  1 root  root            15M Aug 13 14:52 zabbix.tar.gz
drwxr-xr-x. 13 sevag google-sudoers 4.0K Aug 13 14:53 zabbix-3.0.4
[sevag@zabbix-2 zabbix]$ zabbix_agentd
[sevag@zabbix-2 zabbix]$ ps -ef | grep zabbix
sevag     8607     1  0 14:57 ?        00:00:00 zabbix_agentd
sevag     8608  8607  0 14:57 ?        00:00:00 zabbix_agentd: collector [idle 1 sec]
sevag     8609  8607  0 14:57 ?        00:00:00 zabbix_agentd: listener #1 [waiting for connection]
sevag     8610  8607  0 14:57 ?        00:00:00 zabbix_agentd: listener #2 [waiting for connection]
sevag     8611  8607  0 14:57 ?        00:00:00 zabbix_agentd: listener #3 [waiting for connection]
sevag     8612  8607  0 14:57 ?        00:00:00 zabbix_agentd: active checks #1 [idle 1 sec]

Success.

Cleanup

To clean up, we need to go to the Terraform dir where we ran terraform apply from, which should still contain the terraform.tfstate file. From there, we can run terraform destroy to remove the 2 VMs that were created:

$ terraform destroy
Do you really want to destroy?
  Terraform will delete all your managed infrastructure.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

google_compute_instance.zabbix.0: Refreshing state... (ID: zabbix-1)
google_compute_instance.zabbix.1: Refreshing state... (ID: zabbix-2)
google_compute_instance.zabbix.0: Destroying...
google_compute_instance.zabbix.1: Destroying...
...
google_compute_instance.zabbix.0: Destruction complete
google_compute_instance.zabbix.1: Destruction complete

Apply complete! Resources: 0 added, 0 changed, 2 destroyed.