Wednesday, August 12, 2015

Installing Kibana to work with multiple Elasticsearch clusters

Currently Kibana doesn't support querying ElasticSearch instances in different clusters. One workaround to this problem is using tribe nodes. The tribes feature allows a tribe node to act as a federated client across multiple clusters. (https://www.elastic.co/guide/en/elasticsearch/reference/1.6/modules-tribe.html)

In order to visualize data coming from different clusters in a Kibana instance we can configure it to talk to a tribe node which is connected to all the elastic search clusters. However tribe nodes can't have any data and Kibana requires certain indexes to store the information regarding dashboards & saved visualizations. This can be fixed by creating Kibana related index in any one of the cluster. 

Tribe Node configuration (With two clusters)


  • "x.x.x.a:9300","x.x.x.b:9300" and "x.x.x.c:9300" are part of cluster "x"
  • "x.x.x.d:9300" is part of cluster "y"

elasticsearch.yml for tribe node (Kibana should connect to this node)
tribe:
    t1:
        cluster.name:   x
        discovery.zen.ping.multicast.enabled: false
        discovery.zen.ping.unicast.hosts: ["x.x.x.a:9300","x.x.x.b:9300","x.x.x.c:9300"]
    t2:
        cluster.name:   y
        discovery.zen.ping.multicast.enabled: false
        discovery.zen.ping.unicast.hosts: ["x.x.x.d:9300"]

Queries to be executed in one of the nodes in any cluster (To create Kibana specific indices and mappings)


curl 'http://x.x.x.a:9200/.kibana' --data-binary '{"settings":{"number_of_shards":1,"number_of_replicas":1}}' --compressed

curl 'http://x.x.x.a:9200/.kibana/_mapping/index-pattern' -X PUT 'Connection: keep-alive' --data-binary '{"index-pattern":{"properties":{"title":{"type":"string"},"timeFieldName":{"type":"string"},"intervalName":{"type":"string"},"customFormats":{"type":"string"},"fields":{"type":"string"}}}}' --compressed

curl 'http://x.x.x.a:9200/.kibana/_mapping/search' -X PUT --data-binary '{"search":{"properties":{"title":{"type":"string"},"description":{"type":"string"},"hits":{"type":"integer"},"columns":{"type":"string"},"sort":{"type":"string"},"version":{"type":"integer"},"kibanaSavedObjectMeta":{"properties":{"searchSourceJSON":{"type":"string"}}}}}}' --compressed

curl 'http://x.x.x.a:9200/.kibana/_mapping/visualization' -X PUT --data-binary '{"visualization":{"properties":{"title":{"type":"string"},"visState":{"type":"string"},"description":{"type":"string"},"savedSearchId":{"type":"string"},"version":{"type":"integer"},"kibanaSavedObjectMeta":{"properties":{"searchSourceJSON":{"type":"string"}}}}}}' --compressed

curl 'http://x.x.x.a:9200/.kibana/_mapping/dashboard' -X PUT --data-binary '{"dashboard":{"properties":{"title":{"type":"string"},"hits":{"type":"integer"},"description":{"type":"string"},"panelsJSON":{"type":"string"},"version":{"type":"integer"},"kibanaSavedObjectMeta":{"properties":{"searchSourceJSON":{"type":"string"}}}}}}' --compressed

No comments:

Post a Comment

Google+ Followers