Migration Self-Managed Elasticsearch to Elastic Cloud

I started to migrate my Elasticsearch Self Managed from DigitalOcean Kubernetes to https://elastic.co and will tell you my experience so far.

Why?

I have been using my ElasticSearch database instance for a couple of products, since 2016 on DigitalOcean using 3 droplets. And when Kubernetes was available on Digitalocean I migrated to small cluster that I have. I haven't had any technical issues so far.

The main reason is that I am trying to reduce any extra activity at the minimum to concentrated in scaling and creating new products.

Costs of Elasticsearch

DigitalOcean

When using Digitalocean is relative simple to know how much is going to cost you at the end of the month, you just need to multiply for the number of droplets that you have or if you are using kubernetes just multiply for the numbers of nodes.

Elastic Cloud

For Elastic Cloud you have a price calculator page https://cloud.elastic.co/pricing where you can see the total per hour, however you can choose between multiple cloud providers: Google Cloud, AWS and Azure. Also you can select from 4 different subscriptions: Standard, Gold, Platinum and Enterprise.

Advantages of Self Managed Elasticsearch

• The first and main is the cost you will have a cheaper cost if you manage your own cluster.
• You can install the plugins that you want.
• Choose the version that you want.
• Integration with third party services.

Advantages of Elastic Cloud

• No management need for your infrastructure.
• Increased Security
• You receive the updates automatically
• Setup is almost instant

Elastic Cloud Free Trial:

I started with the free trial of Elastic Cloud and today 2022/07/18 (YYYY/mm/dd ) is almost expiring. The free trial is for 14 days and includes the following:

• Latest version of Elastic Stack
• Enterprise Search
• Observability
• Security
• 8GB of RAM
• 240GB of storage
• a lot of other things.

Afer the 14 days of Free Trial

I decided to test Elastic Cloud to see how it performs because i reduced from 8GB of Ram to 2GB and two availability zones (Is the tiniest configuration besides using 1 zone).

Started on the 8 day of the free trial, I change the configuration to the one above and seems to be sufficient for the application needs.

The total cost per month should be around 60 dollars per month.

I will update this article after a couple of months to see how it goes with Elastic.co

How much does the Elastic cost?

The total for August was 45.81 dollars that includes:

• Standard subscription to Elastic.co
• 2 Availability Zones
• 60GB of space
• 2GB RAM
• Kibana Instance 1GB RAM
• Integration Server 1GB RAM

This configuration is one of the most simple that you can obtain on Elastic, however also you have some extra charges for Data Transfer and Storage (These extra charges are already included on the 45.81 dollars stated before)

Data transfer and storage

The total for Data Transfer and Storage was $8.91 dollars.

And for the ElasticSearch Nodes you get and hourly rate that this is fixed in my case was $0.0496 per hour. For August 2022 they Elastic charge 743 hours. 743*$0.0496 = $36.9

The gran total was:
$36.9 + $8.91 = $45.81

So if you are looking to put your elasticsearch database on elastic.co here you have how you can calculate de price. The one thing that is very difficult to calculate is the Data Transfer and Storage because it will depend on your data, how many request do you have and the Snapshots you take.

Elasticsearch Snapshots

When you create your account on Elastic you have Snapshots available by default this is very good because you are going to have backup of your data. However it was saving an snapshot every 30 minutes that for my specific use case was to often. I changed to backup every 6 hours and you can also change the retention policy to only store specific amount of snapshots.