Launching Kibana

Get Elasticsearch & Kibana Running Locally in Under 20 Minutes ( Tarball, no Docker)

The good old local installation of Elasticsearch and Kibana

This is definitively not the first and only article about getting Elasticsearch up and running. Because I am planning to write a few articles about the Elasticsearch basics, I thought it would be gentle to write an article how to set it up.

Throughout the years I have been working with Elasticsearch, I have seen different ways of the installation process. During the time I gave in-class training for Elasticsearch, I included the installation as part of the training. I think it is important that you know how to set up the software and that you get to know better where it consists of. It was the longest part of the 3-day training. It took like 3 hours, including the intermediate explanation and possible issues we stumble upon during installation: Insufficient permissions on laptops, disk space issues you name it. Those were fun times.

The “Installation” chapter of the labs for the Elasticsearch Fundamentals training I used to give

I must say, the installation process of the later versions of Elasticsearch became way easier. Especially with the support and documentation of a Docker-based installation and the way the basic security is set up. There were times that we used to struggle with this as well, but during the installation of the version we are going to work with in this article, Elasticsearch 9.4.2, it was quite a breeze.

This article will focus on the local installation of elasticsearch that you can have up and running in about 20 minutes. The tar.gz version that is. In the end of this article, if followed along the instructions successfully, you have Elasticearch and Kibana, both version 9.4.2, running on your own computer.

Downloading, moving, extracting and running Elasticsearch 9.4.2

Downloading Elasticsearch 9.4.2

For many years, Elasticsearch has a great download page. It contains basically all information that you need:

• Selection of operating system
• Suggestions for the containerized installation
• Brief instructions how to get started
• Browsing trough previous releases

That is also where we start. I work on a Macbook Pro M3, so I will choose macOS aarch64.

https://www.elastic.co/downloads/elasticsearch

The official Download page for Elasticsearch

It will start downloading:

The Elasticsearch bundle is quite a large download

Moving and extracting Elasticsearch to a suitable location on your machine

When the download is finished, we will move the tar.gz file out of your Downloads folder and choose a suitable location where your Elasticsearch installation will live. I choose to put it in my Home Directory: arietwigt

The Home Directory is the directory (folder) on your computer that carries your user name. All operating systems (Windows, Mac and Linux) have a home directory. This is the directory that contains folders like Pictures, Downloads, Music etc.

Placed the elasticsearch-tar.gz file to the Home Directory and extracted it

For now, do not worry yet about the contents of the elasticsearch-9.4.2 directory. In this article we will only focus on getting it working. I will explain the exploration of the contents and options for configuration in later articles.

It is not mandatory, but you can open the elasticsearch-9.4.2 directory to see what it contains. Notice it comes shipped with its own bundled JDK, which means you do not have to install Java separately.

Running Elasticsearch

After we have downloaded, moved and extracted Elasticsearch, we can start running it. Elasticsearch is running from the Command Line, so in my case I use the default Terminal.

Navigating in the Command Line and running Elasticsearch

After opening the Terminal, we navigate to the elasticsearch directory:

Navigating to the elasticsearch-9.4.2 folder in the Terminal

The **pwd command is not mandatory, but I use it to indicate the current path/location of the command line session. I use ls to see which files and directories are at the current path, and as expected, i see the elasticsearch-9.4.2** directory. We will move to that directory:

cd elasticsearch-9–4–2

Of course, make sure that you verify the version you downloaded if that is different than 9.4.2 that we are currently working with.

In the elasticsearch-9.4.2 folder we can run the command to launch Elasticsearch:

bin/elasticsearch

Issue on MacOS

After running the command to run Elasticsearch, you might see the “jdk” Not Opened error message like in the following screenshot:

MacOS indicating that the jdk bundled with the Elasticsearch instance, it put into quarantine

For me this was new, since I managed to install Elasticsearch normally without seeing this message. I did not figure out yet if this has to do with later MacOS versions or with the newer versions of Elasticsearch. I think the first. Anyway, I did what every experienced Elasticsearch developer does when running into these kind of issues, I asked Claude by taking a screenshot of the error and pasting it into the prompt:

Asking Claude to help when running into issues installing Elasticsearch

ChatGPT or Claude are usually quite helpful giving direction to these kind of issues. Make sure that you consult the documentation of Elasticsearch to actually understand where you stumble upon.

Claude was able to give a clear explanation and solution. I was expecting this since I could not imagine I was the only one experiencing this issue.

Asking an LLM, like Claude, to explain the issue

I did some verification about the command I was about to enter to make sure I understand it and it would not harm my computer. For safety measure, MacOS put the JDK-file into quarantine. You could already tell by the icon it layed over the JDK-file in the Finder:

The command is valid. I entered the command and could continue without any issues (replace <username> with your username:

xattr -dr com.apple.quarantine /Users/<your-username>/elasticsearch-9.4.2

Running the fix for the quarantine issue and running elasticsearch afterwards.

After that, Elasticsearch started running and start showing output informing about the services that are launched and the different statuses of these services. In a few seconds, you will see a block of output that will be very important for the process of this first setup. We will discuss that in the next section.

Elasticsearch and Kibana users, keys and enrollment tokens

When Elasticsearch runs successfully for the first time, we need to pay attention to the information it returns us about:

• The pre-defined users: elastic
• The pre-defined password for the elastic user
• The enrollment token for Kibana

Again, do not worry if this does not say anything for you yet (“what is a Kibana enrollment token?”). The pieces of this two-piece jigsaw puzzle will fall in its place as we proceed further in this article.

Please note that these values are only valid for 30 minutes since the first run. It is therefore important to not let too much time between the installation and first run of Elasticsearch and installation of Kibana. There is no need to rush, but it would be great not having to stop and start Elasticsearch again to have a new Kibana enrollment token because the previous one expired.

Make sure to store this information to your password manager or another safe place. You will need this information later for running Kibana for the first time. What I usually do, and advised my students to do during the in-class training, is to paste this information in a Text Editor temporarily so you can quickly use it in the next process when installing Kibana and make sure to remove this text file later and store it into a proper Password Manager.

Opening Elasticsearch

What does “Opening” Elasticsearch actually mean. Elasticsearch by itself does not have an interface. Not to go into too much technical details, but Elasticsearch is an API layer on Apache Lucene. We talk to Elasticsearch like it is a service, so the I in API (Application Programming Interface) is the only interface we get from Elasticsearch without Kibana. Supplying a rich interface to Elasticsearch is where we will use Kibana for later.

Elasticsearch uses port 9200 by default. It is not really meant to access Elasticsearch in the browser, since that is basically useless and only enables you to get some basic information. Useful ways to interact with your running Elasticsearch instance are by using HTTP clients like Postman or **Insomnia with **localhost:9200 as the endpoint. But to show you what you can do if you really want to check it in your browser before working with Kibana, I will show you what you can do.

There are ways, end it is not even hard, to interact with the Elasticsearch instance in the browser by typing in localhost:9200. It requires some basic settings with the certificate that we have seen along with the other credentials information when running Elasticsearch for the first time, I would not dive into that for now since we would lose the focus and momentum of this setup process.

In this article we will stick by Kibana as the main way to interact with Elasticsearch. This will give you the best experience, especially in the beginning. Let’s setup Kibana in the next section.

Downloading, placing and starting Kibana

The process of downloading, moving it to the right place and launching Kibana is similar to the process as for Elasticsearch. We can download Kibana on the Kibana download page:

https://www.elastic.co/downloads/kibana

The Kibana download page looks similar to the Elasticsearch download page

After downloading Kibana, we place the kibana….tar.gz in the Home Directory (in my case arietwigt) as well. Just like we did for the elasticsearch-9.4.2 directory. We will extract the kibana tar-file as well so we have a kibana-9.4.2 directory as well.

Placing the Kibana installation in the Home Directory as well

We can open another Terminal tab and launch Kibana. It is important that Elasticsearch is running when we launch Kibana, Kibana will not work if Elasticsearch is not running. From our home directory we navigate in the Kibana directory and run it:

cd kibana-9.4.2
bin/kibana

Just like with Elasticsearch, running Kibana provides us with output containing logs about the process.

Running Kibana

Running Kibana for the first time, will tell us that Kibana is not configured yet. You can copy the link it provides in the output, and paste it into the address bar of your browser.

Running Kibana for the first time

Kibana recognizes it runs for the first time and mentions it is not configured yet. Maybe you already have an idea what is not configured yet: The enrollment token 💡! We can navigate to the URL that is provided in the terminal. Just like the following screenshot, a window is presented to you in which we can paste in the enrollment token that we got from the output in Elasticsearch earlier.

This enrollment token is the way to connect (which is not really the right terminology here, but it gives you the idea) your running Elasticsearch instance to your running Kibana instance. Kibana looks for a running Elasticsearch instance. Elasticsearch only allows Kibana if it is enrolled by the token Elasticsearch provided. We “enroll” the Kibana instance into Elasticsearch. This only has to be done once for each new installation of Elasticsearch and Kibana.

Next time, on your computer if you start Elasticsearch and Kibana again, you do not have to paste in the enrollment token again. Kibana is already enrolled. Like going to school, you enroll for a class once, the latter times you just walk in.

After pasting in and confirming the enrollment token, Kibana will display the progres of the setup. In a few minutes it will complete it.

Using Elasticsearch with Kibana

When the setup is complete, we can open Kibana to work on our Elasticsearch instance. Going to localhost:5601 will show us the login page for Kibana.

Maybe a short intermediate recap:

• Elasticsearch runs on (localhost) port 9200. We do not directly talk to Elasticsearch for now, Kibana does.
• Kibana runs on port 5601. That is how we interact with Elasticsearch for now.

When both Elasticsearch and Kibana are running, we paste localhost:5601 into our browser. We will see the login page in which we can use the elastic user and password for the elastic user.

Actually, we are logging into our Elasticsearch instance via Kibana. This basic authentication can also be used in HTTP clients like Insomnia or Postman or if you want to interact with your Elasticsearch instance. This basic authentication can also be used in your software development project or script in a programming language like Python or C# that uses the documents and indices you will be building in Elasticsearch. If you are going to build your own search engine or knowledge base with Elasticsearch and software stack of your preference, you will need these credentials as well.

The Kibana login page

In Kibana you can add, delete and manage users and passwords.

Opening Kibana for the first time, will ask you if you want to install example data. You can skip this for now, you can add example data later if you like.

The Kibana home page and main menu (on the left)

Do not feel intimidated by all the solutions the Elastic stack. After all these years I ended up using almost all of them, but I started with the Dev Tools sectioin. That is where we will start as well. Let’s finish up this article and verify if Elasticsearch is really working by adding and searching through some real data.

Running Elastic DSL queries for adding and searching documents

Using the Elasticsearch Query DSL in Dev Tools

Elasticsearch has its own Domain Specific Language (DSL) to interact with your Elasticsearch instance. It is JSON and therefore highly compatible with other services. I have used Elasticsearch in software projects in C# and Python. While C# has the Elastic.Clients.Elasticsearch (previously NEST in Elasticsearch version ≤ 7) the query DSL can be used anywhere as a JSON body that can be added to any HTTP call to Elasticsearch. Remember? Elasticsearch is actually an API.

To check if the queries you enter give you the desired result, Kibana provides us the Dev Tools window. You can open the Dev Tools window by opening the hamburger menu in the top left and scroll all the way down. (sorry for the long screenshot, but scroll down please).

The Dev Tools window is placed under the Management heading in the side nav

Again, don’t be scared by the pre-defined query DSL-you will see. My next article will be about an introduction to the query DSL in Elasticsearch. I will also explain the important terminology like indices, mappings, settings etc. For now, let’s just test if it works.

The Dev Tools window in Kibana

For now, first let’s see how many indices we have in the beginning in Elasticsearch. Since it is a fresh new instance on our computer, we expect it to be empty and to not contain any indices yet. Let’s check that with the following command:

GET _cat/indices?v

You can run the command by clicking the Play ▶️ button next to it, pressing CMD + Enter on your keyboard.

With the command we “GET a concatenated overview (_cat) of the indices we have in this Elasticsearch cluster in a nice table with verbosity(?v). We can see that we begin with two default indices. Default indices are indicated with a period (.), and are create by Elasticsearch itself. We did not create any index yet. With our next command we will create our first index.

One of the predefined query DSL commands is:

PUT /my-index

A quick and concise method of telling Elasticsearch you want to create a new index that is called “my-index”. We can run the command of listing the indices again and see that this index is available. Notice that it does not contain any documents yet: “docs.count” is 0. Which is in line of our expectation since we did not add any document yet.

Let’s add our first document, which is also one of the predefined examples (“POST /my-index/_doc …”). Run that command as well:

It will return a response, indicating the document is successfully added. The output contains additional useful information, like the version and shards. I will explain that in a later article.

Now we added a document, we can do a search query for which we know it will return this document:

As we can see, the document is returned.

These are not the most amazing operations we did in Elasticsearch and if you are new to Elasticsearch, you might see a lot of information that is still unclear. But don’t worry, we will clarify that later. The goal of this article is to install and configure Elasticearch and we reached that goal. Congratulations, you have a local instance of Elasticsearch and Kibana running!

Conclusion

The installation of Elasticsearch is a process of downloading, extracting and running it in the command line. Elasticsearch can run standalone and it is possible to interact with it from an HTTP client or programming language. For getting started, I advise to start interacting with Elasticsearch from Kibana. It provides you the Dev Tools that you can use for experimenting with the Elasticsearch Query DSL. That is just the tip of the iceberg since Kibana comes with a rich toolset for configuring your Elasticsearch instance. We did not even talk about the visualization options. Kibana deserves a chapter on it’s own, but for now, it is to my opinion the most straight forward way to interact with your local running Elasticsearch interface. Now that you have the Elasticsearch interface running, you will be able to explore Elasticsearch furter. Next time you launch you computer, just open 2 terminal sessions (one terminal session with two tabs):

• First, start Elasticsearch
• When Elasticsearch is successfully started, run Kibana

In the later article, I will show you how to setup Elasticsearch and Kibana with Docker. But for now, enjoy!