HowTo Migrate a self managed Elasticsearch cluster to graviton2 instances

4-minute read
aws • elasticsearch • graviton2 • terraform

Elasticsearch is an open-source search engine that enables you to store, search, and analyze big data in real time. It is a distributed and scalable search engine that can be used to index and search large volumes of data across multiple nodes. I’m currently managing several Elasticsearch clusters running on AWS EC2 instances. AWS offers EC2 instances powered by Graviton2 processors (their custom arm processors) that offer significant performance and cost benefits compared to traditional x86 instances (up to 40% based on AWS benchmarks, with 20% from pure cost savings and 20% from performance improvements compared to similar intel processors). In this blog post, I’ll walk you through the process of how we migrated our Elasticsearch clusters to run on Graviton EC2 instances.

The first Elasticsearch version that added support for ARM processors was Elasticsearch 7.8. This version introduced official support for ARM64 architecture and was released on May 26, 2020. Before this release, Elasticsearch was only officially supported on x86-based platforms. So in our case, this required us to migrate to a supported version first. We were running an older version in the stable branch 7.x and we upgraded to 7.17 using the standard Elasticsearch rolling upgrade docs.

Here are the steps needed for this migration:

Step 1: Create a new Graviton2-based EC2 instances

The first step in the migration process is to create new Graviton2-based EC2 instances. You can do this using the AWS Management Console or the AWS CLI, or even better use terraform as I do. Various Linux distributions run on ARM, but I have chosen to use an Amazon Linux 2 AMI because this is very well supported by AWS. We can use the AWS console and use the filter for “Architecture” to be set to “arm64” for AMI and find the latest Amazon Linux 2 AMI. Or use a simple aws cli command like:

aws ssm get-parameters --names /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-arm64-gp2 --region us-east-1

This will return the Graviton2 AMI for the specific region we are using. We would use this in our terraform code to create the new Graviton2 instances; for ex:

# Elasticsearch nodes
resource "aws_instance" "es_nodes" {
  count         = 3
  ami           = "ami-XXX" # Replace with the AMI we found above
  instance_type = "c6g.large"
  security_groups = [aws_security_group.es_node_sg.name]
  
  user_data = <<-EOF
              #!/bin/bash
              echo "cluster.name: es-cluster" >> /etc/elasticsearch/elasticsearch.yml
              echo "node.name: ${format("es-node-%02d", count.index+1)}" >> /etc/elasticsearch/elasticsearch.yml
              echo "network.host: [_ec2_:privateIpv4_, _local_]" >> /etc/elasticsearch/elasticsearch.yml
              systemctl restart elasticsearch
              EOF
  
  tags = {
    Name = "es-node-${count.index+1}"
  }
}

Step 2: Install Elasticsearch on the new instance

Normally we would install Elasticsearch on the nodes using the user_data script, but during this migration, we went with a more manual method; you can install Elasticsearch using the RPM or DEB packages provided by Elasticsearch. Here is an example command to install Elasticsearch on an Amazon Linux 2 instance:

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
sudo tee /etc/yum.repos.d/elasticsearch.repo <<EOF
[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

sudo yum install -y elasticsearch

Step 3: Configure Elasticsearch on the new instances

After installing Elasticsearch on the new Graviton2-based EC2 instance, the next step is to configure Elasticsearch to use the existing data and settings from the old Elasticsearch cluster. You can do this by copying the Elasticsearch configuration files from the old cluster to the new instance.

This might look something like this:

rsync -avz --progress --delete /path/to/old/cluster/config/ ec2-user@new-instance-ip:/etc/elasticsearch/

Step 4: Start Elasticsearch on the new instance

Finally, once the Elasticsearch configuration files are copied to the new Graviton2-based EC2 instance, the next step is to start Elasticsearch on the new instance. You can do this using the Elasticsearch service command. Here is an example command to start Elasticsearch on the new instance:

sudo service elasticsearch start

Step 5: Verify the migration

The final step in the migration process is to verify that the data and settings from the old Elasticsearch cluster have been successfully migrated to the new Graviton2-based EC2 instance. You can do this by checking the Elasticsearch logs and running some search queries on the new instance.

Here is an example command to check the Elasticsearch logs on the new instance:

sudo tail -f /var/log/elasticsearch/elasticsearch.log

This command shows the Elasticsearch logs on the new instance, and you can use it to check if any errors or warnings are reported during the migration process.

Step 6: Remove original nodes

After all the new Graviton2 instances are in sync in the cluster you can go ahead and remove the old intel instances one by one and allow the cluster to rebalance.

Conclusion

Migrating an Elasticsearch cluster to running on Graviton EC2 instances can provide significant performance and cost benefits. In this blog post, I walked you through the process of migrating an existing Elasticsearch cluster to new Graviton2-based EC2 instances. By following the steps outlined in this post, you can easily migrate your Elasticsearch cluster to Graviton2-based EC2 instances and take advantage of the cost/performance improvements they offer.

comments powered by Disqus