dennisgorelik: (2009)
After several months of observations the performance of ElasticSearch instances I reported ElasticSearch memory leaks issue.

The issue was prominently closed without any resolution.

I guess now I have to just restart my ElasticSearch server every few days in order to "patch" these memory leaks.
dennisgorelik: (2009)
Couple of months ago ASmallOrange marketer contacted me and offered free 2 months trial of their Virtual Private Server (VPS).
We wanted to try hosting ElasticSearch on Linux platform.
While hosting ElasticSearch on Linux was a positive experience, hosting on ASmallOrange was so-so and ended up badly.

It went like this:
1) Got 3GB 2-cores VPS with Linux CentOs ($45/month with 2 months free trial).
2) Configured firewall.
3) Installed ElasticSearch.
4) Added another VPS (2GB 2-cores for $30/month - this time that was real money) in order to form ElasticSearch cluster.
5) Started running ElasticSearch percolation on that cluster.
6) Our VPS-es were rebooted about once per week for different type of patches/maintenance.
7) Once our VPS did not get up after such maintenance done by ASmallOrange.
After seeing crashes in our logs we had to contact ASmallOrange in order to get it up. We got about 3 hours of downtime back then.
As a "bonus", ASmallOrange tech changed our firewall settings to make it more publicly available (to the contrary of our intention to keep our VPS private as much as possible).
8) At the end of the trial period I asked ASmallOrange to convert my server that was on trial into paid account.
Time of request: 3:20 pm EDT on Friday.
ASmallOrange ignored that request and terminated my first server (that was on trial).
Termination time: 1:30 am EDT on Saturday.
9) Now ASmallOrange is not able to restore it.
Cannot find backup, cannot really do anything.
10) Fortunately, we only moved ElasticSearch percolation to ASmallOrange, so it was not that hard to move it back to our main Windows server.

Conclusions time:
1) Don't go to England Do not use ASmallOrange for anything that requires reliable work.
2) In web hosting you get what you paid for.
dennisgorelik: (2009)
We moved our ElasticSearch job percolation functionality from Windows server to ElasticSearch cluster on two Linux VPS-es (3GB RAM + 2GB RAM).
Percolation performance improved at a fraction of hosting price (relative to price of dedicated Windows server).
The most important benefit is that we can increase percolation performance just by adding more nodes to our ElasticSearch cluster.
Performance of individual percolation query on ElasticSearch cluster is about the same as on single node, but adding more nodes to ES cluster allows to execute more queries in parallel.
From our experimentation we determined that optimal number of percolation queues on 2-node cluster (2 CPU cores on each node) is ... drum-roll ... 4 (1 for each CPU core).

That configuration allows us to percolate up to 216 jobs per minute.

Q: What is ElasticSearch percolation?
You may create a job search alert.
PostJobFree will put your alert alongside with 160K+ other users' job alerts into ElasticSearch job percolation index.
Then every time when we get a new job - we percolate that job against 160K records in job percolation.
If job matches your (or anyone else's) job alert, then ElasticSearch percolator returns IDs or all these alerts, so we know to send you email about new match.

Q: Why host ElasticSearch on Linux?
Windows version of ElasticSearch does not support mlockall setting. That means there is no good way to prevent ElasticSearch from using swap-file.
dennisgorelik: (2009)
I'm playing with Linux VPS (CentOS) and want to make my VPS private (it should only respond to requests from several whitelisted IP addresses and silently ignore everything else).

That linux VPS is going to only host ElasticSearch server and I want to hide ElasticSearch from public.

It looks like configuring Advanced Policy Firewall (APF) is the right tool for that.

My first temptation was to replace couple of thousand lines of default configuration in /etc/sysconfig/iptables with just few lines, like that:
----- /etc/sysconfig/iptables -----
where and are my whitelisted IP addesses.

But support from my linux box provider ("A Small Orange") told me that we should not modify iptables directly.

What would happen if I modify iptables file directly?

If not - how else can I delete all the junk from iptables?


dennisgorelik: (Default)
Dennis Gorelik

September 2017

34567 8 9
1011 12131415 16


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Sep. 23rd, 2017 05:45 am
Powered by Dreamwidth Studios