Zagadka's Site

I've been implementing logstash at work. Interesting stuff. The elasticsearch cluster is up and running and working great. There isn't really anything to give it a really solid workout with just yet. I have been feeding it about 3000 lines a day worth of power data which I am graphing with Kibana, but that's about it. For starters I've got 180-odd machine cluster which I am going to be reading logs from with the logstash agent and sending them over to elasticsearch. There will be a redis cluster sitting between elasticsearch and the logstash agents. I decided to implement two haproxy/keepalived setups with floating IPs for the Elasticsearch and for the redis cluster. I went with this because although Elasticsearch takes care of this by it's self normally, some code does not handle being given more than one IP or hostname to contact Elasticsearch with, the same goes for the redis. So floating IPs will solve this, it will also mean that I can add more nodes and not touch the config on either the logstash agent or on the logstash redis/elasticsearch boxes. I can't wait to try it out, the individual bits are all happy and some of the parts which can communicate are tested. Currently I'm waiting on changes and approvals and stuff to go through so I can open up the required ports and install the logstash agents.