Plugin Name: ElasticSearchOutput
Output plugin that uses HTTP or UDP to insert records into an ElasticSearch database. Note that it is up to the specified encoder to both serialize the message into a JSON structure and to prepend that with the appropriate ElasticSearch BulkAPI indexing JSON. Usually this output is used in conjunction with an ElasticSearch-specific encoder plugin, such as ElasticSearch JSON Encoder, ElasticSearch Logstash V0 Encoder, or ElasticSearch Payload Encoder.
Config:
Interval at which accumulated messages should be bulk indexed into ElasticSearch, in milliseconds. Defaults to 1000 (i.e. one second).
Number of messages that, if processed, will trigger them to be bulk indexed into ElasticSearch. Defaults to 10.
Time in milliseconds to wait for a server name resolving and connection to ES. It’s included in an overall time (see ‘http_timeout’ option), if they both are set. Default is 0 (no timeout).
Time in milliseconds to wait for a response for each http post to ES. This may drop data as there is currently no retry. Default is 0 (no timeout).
Specifies whether or not re-using of established TCP connections to ElasticSearch should be disabled. Defaults to false, that means using both HTTP keep-alive mode and TCP keep-alives. Set it to true to close each TCP connection after ‘flushing’ messages to ElasticSearch.
The username to use for HTTP authentication against the ElasticSearch host. Defaults to “” (i. e. no authentication).
The password to use for HTTP authentication against the ElasticSearch host. Defaults to “” (i. e. no authentication).
New in version 0.9.
An optional sub-section that specifies the settings to be used for any SSL/TLS encryption. This will only have any impact if URL uses the HTTPS URI scheme. See Configuring TLS.
Buffer records to a disk-backed buffer on the Heka server before writing them to ElasticSearch. Defaults to true.
Defines maximum queue buffer size, in bytes. Defaults to 0, which means no max.
Specifies how Heka should behave when the queue reaches the specified maximum capacity. There are currently three possible actions:
- shutdown - Shuts down Heka.
- drop - New messages are dropped until queue is available again. Already queued messages are unaffected.
- block - Blocks processing of messages, tries to push last message until its possible.
Defaults to shutdown.
Example:
[ElasticSearchOutput]
message_matcher = "Type == 'sync.log'"
server = "http://es-server:9200"
flush_interval = 5000
flush_count = 10
encoder = "ESJsonEncoder"