Add your own S3 bucket as a snapshot repository in elastic cloud.

Add your own S3 bucket as a snapshot repository in elastic cloud.

Sometimes using elastic hosted service called elastic cloud makes sense because if you need to work with some of the advanced features that are accessible via a license it might end up cheaper than hosting your own elk service and buying separate license. This was one of my projects where I needed access to searcheable snapshots which is available on the enterprise level of license. Therefore we went ahead and we have purchase a small cluster with one coordinating node, one hot data node, one warm data node and one machine learning node. Next step was to plug in our own S3 bucket so that if/when we decide to drop the project we won't loose all those ingested data and indexes therefore I requested an AWS S3 bucket. I was given a bucket name, an aws access key and a aws security key, which then I had to use to create the new repository.

First step was to create the security keys in the elastic cloud panel security section,
by clicking on the Add settings button.

A small side pannel opens up with the following options:
Screenshot-2021-04-24-at-00.26.38-1
I have choose to go with single string option and create a separate key for each of the two settings I needed to add. Therefore following the elastic documentation I have added the first key and I decided to name it in case in the future I want to create a different project in elastic cloud and I want to add another repository into that project as well so I have used s3.client.development.access_key the first one and just pasted the access key into the secret form and created a new setting called s3.client.development.secret_key and pasted it's value into the secret field as well.

Next I headed into Kibana opened up the dev tools and run the following query to make the nodes reload the settings as these are reloable settings and can be reloaded on fly. I have entered the following code into the left side and clicked the small arrow next to the command. Screenshot-2021-04-24-at-00.41.31

In a few seconds you will see on the right side the confirmation that your settings were applied. Screenshot-2021-04-24-at-00.41.46

Now you can safely register your S3 bucket as a repository by running the command from the screenshot.Screenshot-2021-04-24-at-00.36.29-1.

Note that the client value is the same as I used in the s3.client settings. If you leave that value to default you don't have to configure the client value into the command above. After running the small arrow next to the command in a few seconds you will see the acknowldgement that your repository was successfully created.