Elasticsearch – Aggregation

Elasticsearch Aggregation provides capability similar to RDBMS group by opeartor.
Facets provide a great way to aggregate data within a document set context. This context is defined by the executed query in combination with the different levels of filters that can be defined (filtered queries, top-level filters, and facet level filters). While powerful, their implementation is not designed from the ground up to support complex aggregations and is thus limited.
An aggregation can be seen as a unit-of-work that builds analytic information over a set of documents.
There are many different types of aggregation, each with it’s own purpose & output. To Better understand these type, It is often best to break down into 2 families.
1. Bucketing
– A family of aggregations that build buckets , where each bucket is associated with key and a document criterion
– When the aggregation is executed, all the buckets criteria are evaluated on every document in the context and when a criterion matches, the document is considered to “fall in” the relevant bucket
– By the end of the aggregation process, we’ll end up with a list of buckets – each one with a set of documents that “belong” to it.
2. Metric
– Aggregations that keep track and compute metrics over a set of documents.
Different kinds of aggregation is listed below:
1.Min Aggregation
2.Max Aggregation
3.Sum Aggregation
4.Avg Aggregation
5.Stats Aggregation
6.Extended Stats Aggregation
7.Value Count Aggregation
8.Percentiles Aggregation
9.Percentile Ranks Aggregation
10.Cardinality Aggregation
11.Geo Bounds Aggregation
12.Top hits Aggregation
13.Scripted Metric Aggregation
14.Global Aggregation
15.Filter Aggregation
16.Filters Aggregation
17.Missing Aggregation
18.Nested Aggregation
19.Reverse nested Aggregation
20.Children Aggregation
21.Terms Aggregation
22.Significant Terms Aggregation
23.Range Aggregation
24.Date Range Aggregation
25.IPv4 Range Aggregation
26.Histogram Aggregation
27.Date Histogram Aggregation
28.Geo Distance Aggregation
29.GeoHash grid Aggregation

Using nginx to provide authentication to Elasticsearch / Kibana

Friends authentication & authorization is always an important requirement for development of any application.

In this post i am going to show you how to provide authentication to elasticsearch / kibana using Nginx server.

Steps are given below:

1. Install the nginx server

You can follow below given link for reference.


2. Create the configuration file as kibana.conf or elasticsearch.con under /etc/nginx/conf.d (under configuration directory)

3. Add the following code to kibana.conf

server {
listen 80;
server_name yourdomain.com; ## Replace with your domain name
location / {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/conf.d/kibana.htpasswd;
proxy_pass http://yourdomain.com:5601; ## Replace with your kibana instance as kibana runs on 5601 for ES use port number 9200

4. Create kibana.htpasswd file under /etc/nginx/conf.d directory

5. Run the following command to generate the username / password for authentication

sudo htpasswd -c /etc/nginx/conf.d/kibana.htpasswd bhavesh

It will ask for password to set for username bhavesh. Enter it. It will store the generated username password in respected file colon seperated in encrypted form.

6. Restart the nginx

7. Point your browser to yourdomain.com & verify.

The above steps will provide authorization. One can provide authorization based on indices using kibana shield plugin. Otherwise you can follow below link describing tricks for MultiRole Authorization