Hello folks,
‘DevOps‘ is becoming a buzzword now. Everyone wants everything to be automated right from development to production environment.
Thanks to the variety of technologies/tools available to accomplish these types of tasks.
Technologies include Chef, Puppet, Ansible, Terraform, Packer, Consul, AWS Cloud formation and so on to name a few.
Let’s get started…
Our Task : Setting up Couchbase server cluster having 3 nodes on AWS using Terraform.
Recently in one of the project on which I am working has a need of couchbase cluster having different number of nodes.
One of way to create and add nodes in to the cluster is through couchbase cluster management UI.
But one has to do all the tasks such as naming a cluster, setting up cluster ram quota, adding new node, rebalancing the cluster, creating bucket, setting per bucket ram quota, setting replication strategies manually.
So, we are going to do all such tasks through couchbase-cli command line interface on one of cluster node with the help of Terraform.
For those who are unaware about Terraform, please click here.
Prerequisites :
- You will need an AMI (Amazon Machine Image) id on which couchbase server is already installed.
- This couchbase installation on a blank AWS Ubuntu EC2 instance can also be automated by Chef and Packer.
(We will discuss in detail how to create such as AMI sometime in future.)
For those who are unaware about Packer, please click here. In short, Packer creates an EC2 instance, runs chef client on that machine, chef client then talks with chef server and downloads assigned recipes and executes them to install, configure couchbase server and then creates an AMI of that instance before terminating it.
We will be forming cluster of couchbase community edition version 4.0.0
Terraform configuration file :
Lets create your .tf file say ‘cluster_setup.tf ‘ with following content step by step.
NOTE : As par security guidelines, I am using variables in place of hard coded values in following configuration.
Do create your own variables and assign correct values to them.
You will need following variables in your ‘variables.tf’ file.
-
Create ‘variables.tf’ file with following content.
variable “access_key” {}
variable “secret_key” {}
variable “admin_password” {}
variable “region” {
default = “us-east-1”
}
variable “admin_user” {
default = “Administrator”
}
variable “cluster_ram_quota” {
default = “put_cluster_ram_quota_here”
}
variable “bucket_ram_quota” {
default = “put_bucket_ram_quota_here”
}
variable “couchbase_default_bucket_name” {
default = “put_bucket_name_here”
}
variable “couchbase_default_replicas” {
default = “put_no_of_replicas_here”
}
2. Add provider in the ‘cluster_setup.tf ‘ file.
provider “aws” {
access_key = “${var.access_key}”
secret_key = “${var.secret_key}”
region = “${var.region}”
}
3. Add resources to create three couchbase nodes in ‘cluster_setup.tf ‘ file.
resource “aws_instance” “couchbase_cluster_node1” {
ami = “put_ami_id_here”
instance_type = “t2.micro”
key_name = “put_aws_keypair_name_here”
tags {
Name = “couchbase_cluster_node1”
}
}resource “aws_instance” “couchbase_cluster_node2” {
ami = “put_ami_id_here”
instance_type = “t2.micro”
key_name = “put_aws_keypair_name_here”
tags {
Name = “couchbase_cluster_node2”
}
}resource “aws_instance” “couchbase_cluster_node3” {
ami = “put_ami_id_here”
instance_type = “t2.micro”
key_name = “put_aws_keypair_name_here”
tags {
Name = “couchbase_cluster_node3”
}
}
4. Now, we have 3 nodes of couchbase and now we have to form a cluster consisting all these 3 nodes.
To accomplish this, Terraform gives way through ‘null_resource‘ and ‘remote-exec‘ provisioners.
We will add null_resource and will connect to one of the couchbase node say node_1 through ‘remote-exec’ provisioner and then run couchbase-cli command through ‘inline’ option to form a cluster which runs following cli commands.
- Set cluster name and ram quota.
- Add nodes in cluster.
- Rebalance cluster.
- Create bucket and set ram quota.
resource “null_resource” “couchbase_cluster_formation” {
triggers {
couchbase_node1_public_ip = “${aws_instance.couchbase_cluster_node1.public_ip}”
couchbase_node1_private_ip = “${aws_instance.couchbase_cluster_node1.private_ip}”
couchbase_node2_private_ip = “${aws_instance.couchbase_cluster_node2.private_ip}”
couchbase_node3_private_ip = “${aws_instance.couchbase_cluster_node3.private_ip}”
}provisioner “remote-exec” {
connection {
type = “ssh”
user = “ubuntu”
host = “${aws_instance.couchbase_cluster_node1.public_ip}”
private_key = “${file(“put_aws_keypair_file_path_here”)}”
}inline = [
“sleep 10”,
“/opt/couchbase/bin/couchbase-cli setting-cluster –cluster-name couchbase_cluster –cluster-ramsize=${var.cluster_ram_quota} -c 127.0.0.1:8091 -u ${var.admin_user} -p ${var.admin_password}”,
“/opt/couchbase/bin/couchbase-cli cluster-edit -c ${aws_instance.couchbase_cluster_node1.private_ip}:8091 -u ${var.admin_user} -p ${var.admin_password} –services=data,index,query”,
“/opt/couchbase/bin/couchbase-cli server-add -c 127.0.0.1:8091 -u ${var.admin_user} -p ${var.admin_password} –server-add=${aws_instance.couchbase_cluster_node2.private_ip}:8091 –server-add-username=${var.admin_user} –server-add-password=${var.admin_password} –services=data,index,query”,
“/opt/couchbase/bin/couchbase-cli server-add -c 127.0.0.1:8091 -u ${var.admin_user} -p ${var.admin_password} –server-add=${aws_instance.couchbase_cluster_node3.private_ip}:8091 –server-add-username=${var.admin_user} –server-add-password=${var.admin_password} –services=data,index,query”,
“/opt/couchbase/bin/couchbase-cli rebalance -c 127.0.0.1:8091 -u ${var.admin_user} -p ${var.admin_password}”,
“/opt/couchbase/bin/couchbase-cli bucket-create -c 127.0.0.1:8091 -u ${var.admin_user} -p ${var.admin_password} –bucket=${var.couchbase_default_bucket_name} –bucket-type=couchbase –bucket-port=11211 –bucket-ramsize=${var.bucket_ram_quota} –bucket-replica=${var.couchbase_default_replicas}”
]
}
}
5. All done. Now just run ‘terraform apply‘ command, sit back and observe your resources getting created. Wait until terraform completes and says everything OK.
6. THATS IT. Congrats. You have created your couchbase cluster. Just go to web console of any node by http://node_ip:8091 and verify your cluster by your own eyes.
Happy coding…
dp's blog 