Update beats to 5.0.2.

[Filip Hajny <filip%joyent.com@localhost>]

Module Name:	pkgsrc-wip
Committed By:	Filip Hajny <filip%joyent.com@localhost>
Pushed By:	fhajny
Date:		Wed Nov 30 22:17:10 2016 +0000
Changeset:	d4629f65ff631690842b10df02be781c6256c4ea

Modified Files:
	beats/Makefile
	beats/PLIST
	beats/distinfo

Log Message:
Update beats to 5.0.2.

Beats version 5.0.2

Metricbeat
- Fix the `password` option in the MongoDB module.

Filebeat
- Fix registry cleanup issue when files falling under ignore_older after
  restart.

Beats version 5.0.1

Metricbeat
- Fix system.process.start_time on Windows.
- Fix system.process.ppid on Windows.
- Fix system process metricset for Windows XP and 2003. cmdline will be
  unavailable.
- Fix access denied issues in system process metricset by enabling
  SeDebugPrivilege on Windows.
- Fix system diskio metricset for Windows XP and 2003.
- Add username and password config options to the PostgreSQL module.
- Add username and password config options to the MongoDB module.
- Add system core metricset for Windows.

Packetbeat
- Fix 'index out of bounds' bug in Packetbeat DNS protocol plugin.
- Define client_geoip.location as geo_point in the mappings to be used
  by the GeoIP processor in the Ingest Node pipeline.

Filebeat
- Stop Filebeat on registrar loading error.

Beats version 5.0.0-GA

Affecting all Beats
- Fix kafka output re-trying batches with too large events.
- Fix kafka output protocol error if version: 0.10 is configured.
- Fix kafka output connection closed by broker on SASL/PLAIN.
- Add beat.version fields to all events.

Metricbeat
- Fix high CPU usage on macOS when encountering processes with long
  command lines.
- Fix high value of system.memory.actual.free and
  system.memory.actual.used.
- Change several OpenProcess calls on Windows to request the lowest
  possible access provilege.
- Fix system.memory.actual.free high value on Windows.

Filebeat
- Fix issue when clean_removed and clean_inactive were used together
  that states were not directly removed from the registry.
- Fix issue where upgrading a 1.x registry file resulted in duplicate
  state entries.

Beats version 5.0.0-rc1

Affecting all Beats
- A dynamic mapping rule is added to the default Elasticsearch template
  to treat strings as keywords by default.
- Make sure Beats sent always float values when they are defined as
  float by sending 5.00000 instead of 5.
- Fix ignoring all fields from drop_fields in case the first field is
  unknown.
- Fix dynamic configuration int/uint to float type conversion.
- Fix primitive types conversion if values are read from environment
  variables.

Metricbeat
- Fix default configuration file on Windows to not enabled the load
  metricset.

Packetbeat
- Fix the bpf_filter setting.

Filebeat
- Fix input buffer on encoding problem.
- Fix registry cleanup issue when files falling under ignore_older after
  restart.

Beats version 5.0.0-beta1

Affecting all Beats
- Change Elasticsearch output index configuration to be based on format
  strings. If index has been configured, no date will be appended
  anymore to the index name.
- Replace output.kafka.use_type by output.kafka.topic accepting a format
  string.
- If the path specified by the -c flag is not absolute and -path.config
  is not specified, it is considered relative to the current working
  directory.
- rename tls configurations section to ssl.
- rename certificate_key configuration to key.
- replace tls.insecure with ssl.verification_mode setting.
- replace tls.min/max_version with ssl.supported_protocols setting
  requiring full protocol name.
- Fix logstash output handles error twice when asynchronous sending
  fails.
- Fix Elasticsearch structured error response parsing error.
- Fixed the run script to allow the overriding of the configuration
  file.
- Fix logstash output crash if no hosts are configured.
- Fix array value support in -E CLI flag.
- Fix merging array values if -c CLI flag is used multiple times.
- Fix beats failing to start due to invalid duplicate key error in
  configuration file.
- Fix panic on non writable logging directory.
- Add script to generate the Kibana index-pattern from fields.yml.
- Enhance Redis output key selection based on format string.
- Configurable Redis keys using filters and format strings.
- Add format string support to output.kafka.topic.
- Add output.kafka.topics for more advanced kafka topic selection per
  event.
- Add support for Kafka 0.10.
- Add SASL/PLAIN authentication support to kafka output.
- Make Kafka metadata update configurable.
- Add Kafka version setting (optional) enabling kafka broker version
  support.
- Add Kafka message timestamp if at least version 0.10 is configured.
- Add configurable Kafka event key setting.
- Add settings for configuring the kafka partitioning strategy.
- Add partitioner settings reachable_only to ignore partitions not
  reachable by network.
- Enhance contains condition to work on fields that are arrays of
  strings.
- Lookup the configuration file relative to the -path.config CLI flag.
- Re-write import_dashboards.sh in Golang.
- Update to Go 1.7.
- Log total non-zero internal metrics on shutdown.
- Add support for encrypted private key files by introducing
  ssl.key_passphrase setting.
- Add experimental symlink support with symlinks config
- Topology map is deprecated. This applies to the settings:
  refresh_topology_freq, topology_expire, save_topology, host_topology,
  password_topology, db_topology.

Metricbeat
- Change field type system.process.cpu.start_time from keyword to date.
- redis/info metricset fields were renamed up according to the naming
  conventions.
- Fix module filters to work properly with drop_event filter.
- Use the new scaled_float Elasticsearch type for the percentage values.
- Add experimental cgroup metrics to the system/process MetricSet.
- Added a PostgreSQL module.
- Improve mapping by converting half_float to scaled_float and integers
  to long.
- Add experimental haproxy module.

Packetbeat
- Group HTTP fields under http.request and http.response
- Export http.request.body and http.response.body when configured under
  include_body_for
- Move ignore_outgoing config to packetbeat.ignore_outgoing
- Fix mapping for some Packetbeat flow metrics that were not marked as
  being longs.
- Fix handling of messages larger than the maximum message size (10MB).
- Add Cassandra protocol analyzer to Packetbeat.
- Match connections with IPv6 addresses to processes
- Add IP address to -devices command output
- Add configuration option for the maximum message size. Used to be
  hard-coded to 10 MB.

Filebeat
- Set close_inactive default to 5 minutes (was 1 hour before)
- Set clean_removed and close_removed to true by default
- Fix processor failure in Filebeat when using regex, contain, or equals
  with the message field.
- Fix async publisher sending empty events
- Fix potential issue with multiple harvester per file on large file
  numbers or slow output
- Introduce close_timeout harvester options
- Strip BOM from first message in case of BOM files
- Add harvester_limit option

Beats version 5.0.0-alpha5

Affecting all Beats
- Rename the filters section to processors.
- Introduce the condition with when in the processor configuration.
- The Elasticsearch template is now loaded by default.
- The Redis output index setting is renamed to key. index still works
  but it's deprecated.
- The undocumented file output index setting was removed. Use filename
  instead.
- Fix sync publisher PublishEvents return value if client is closed
  concurrently.
- Periodically log internal metrics.
- Add enabled setting to all output modules.
- Command line flag -c can be used multiple times.
- Add OR/AND/NOT to the condition associated with the processors.
- Add -E CLI flag for overwriting single config options via command
  line.
- Choose the mapping template file based on the Elasticsearch version.
- Check stdout being available when console output is configured.

Metricbeat
- Create a separate metricSet for load under the system module and
  remove load information from CPU stats.
- Add system.load.norm.1, system.load.norm.5 and system.load.norm.15.
- Add threads fields to mysql module.
- Do not send zero values when no value was present in the source.
- Add pgid field to process information.
  https://github.com/elastic/beats/pull/ 2021[2021]

Packetbeat
- Set enabled ` in packetbeat.protocols.icmp configuration to true by
  default.
- Add enabled setting to Packetbeat protocols.
- Add enabled setting to Packetbeat network flows configuration.

Filebeat
- Fix potential data loss between Filebeat restarts, reporting
  unpublished lines as published.
- Fix open file handler issue. 2028
- Fix filtering of JSON events when using integers in conditions.
- Introduce close_removed and close_renamed harvester options.
- Introduce close_eof harvester option.
- Add clean_removed and clean_inactive config option.
- Deprecate close_older option and replace it with close_inactive.
- Deprecate force_close_files option and replace it with close_removed
  and close_renamed.

Beats version 5.0.0-alpha4

Affecting all Beats
- The topology_expire option of the Elasticserach output was removed.
- Reset backoff factor on partial ACK.
- Fix beats load balancer deadlock if max_retries: -1 or publish_async
  is enabled in filebeat.
- Fix logstash output with pipelining mode enabled not reconnecting.
- Empty configuration sections become merge-able with variables
  containing full path.
- Fix error message about required fields missing not printing the
  missing field name.
- Improve error message if compiling regular expression from config
  files fails.
- Compression support in the Elasticsearch output.

Metricbeat
- Fix the CPU values returned for each core.
- Add MongoDB module.

Packetbeat
- Add missing nil-check to memcached GapInStream handler.
- Fix NFSv4 Operation returning the first found first-class operation
  available in compound requests.
- Fix TCP overlapping segments not being handled correctly.

Filebeat
- Stop following symlink. Symlinks are now ignored:

Beats version 5.0.0-alpha3

Affecting all Beats
- All configuration settings under shipper: are moved to be top level
  configuration settings. I.e. shipper.name: becomes name: in the
  configuration file.
- Add conditions to generic filtering.
- The support for doing GeoIP lookups is deprecated and will be removed
  in version 6.0.

Topbeat
- Topbeat is replaced by Metricbeat.

Metricbeat
- First public release, containing the following modules: apache, mysql,
  nginx, redis, system, and zookeeper.

Filebeat
- The state for files which fall under ignore_older is not stored
  anymore. This has the consequence, that if a file which fell under
  ignore_older is updated, the whole file will be crawled.
- The registry format was changed to an array instead of dict. The
  migration to the new format will happen automatically at the first
  startup.

Beats version 5.0.0-alpha2

Affecting all Beats
- On DEB/RPM installations, the binary files are now found under
  /usr/share/{{beat_name}}/bin, not in /usr/bin.
- The logs are written by default to self rotating files, instead of
  syslog.
- Remove deprecated host option from elasticsearch, logstash and redis
  outputs.
- Drain response buffers when pipelining is used by Redis output.
- Unterminated environment variable expressions in config files will now
  cause an error
- Fix issue with the automatic template loading when Elasticsearch is
  not available on Beat start.
- Fix bug affecting -cpuprofile, -memprofile, and -httpprof CLI flags
- Fix race when multiple outputs access the same event with logstash
  output manipulating event 1410
- Seed random number generator using crypto.rand package.
  https://github.com/elastic/beats/pull/1503{1503]
- Fix beats hanging in -configtest
- Fix kafka log message output
- Add support for TLS to Redis output.
- Add SOCKS5 proxy support to Redis output.
- Failover and load balancing support in redis output.
- Multiple-worker per host support for redis output.
- Added ability to escape ${x} in config files to avoid environment
  variable expansion
- Configuration options and CLI flags for setting the home, data and
  config paths.
- Configuration options and CLI flags for setting the default logs path.
- Update to Go 1.6.2
- Add Elasticsearch template files compatible with Elasticsearch 2.x.
- Add scripts for managing the dashboards of a single Beat

Packetbeat
- Configuration of redis topology support changed.
- Move all Packetbeat configuration options under the packetbeat
  namespace
- Fix compile issues for OpenBSD.

Filebeat
- Default location for the registry file was changed to be data/registry
  from the binary directory, rather than .filebeat in the current
  working directory. This affects installations for zip/tar.gz/source, the
  location for DEB and RPM packages stays the same.
- Improvements in registrar dealing with file rotation.
- Fix issue with JSON decoding where @timestamp or type keys with the
  wrong type could cause Filebeat to crash.
- Fix issue with JSON decoding where values having null as values could
  crash Filebeat.
- Multiline reader normalizing newline to use \n.

Topbeat
- Updated elastic/gosigar version so Topbeat can compile on OpenBSD.

Beats version 5.0.0-alpha1

libbeat
- Run function to start a Beat now returns an error instead of directly
  exiting.
- The method signature of HandleFlags() was changed to allow returning
  an error
- Require braces for environment variable expansion in config files

Packetbeat
- Rename output fields in the dns package. Former flag recursion_allowed
  becomes recursion_available. 803 Former SOA field ttl becomes minimum.
- The fully qualified domain names which are part of output fields
  values of the dns package now terminate with a dot.
- Remove the count field from the exported event

Topbeat
- Rename proc.cpu.user_p with proc.cpu.total_p as it includes CPU time
  spent in kernel space
- Remove count field from the exported fields
- Rename input top level config option to topbeat

Filebeat
- Scalar values in used in the fields configuration setting are no
  longer automatically converted to strings.
- Count field was removed from event as not used in filebeat

Beats version 1.3.1

Filebeat
- Fix a concurrent bug on filebeat startup with a large number of
  prospectors defined.

Packetbeat
- Fix description for the -I CLI flag.

Beats version 1.3.0

Affecting all Beats
- Fix beats load balancer deadlock if max_retries: -1 or publish_async
  is enabled in filebeat.
- Fix output modes backoff counter reset. 1803 1814
- Set logstash output default bulk_max_size to 2048.
- Seed random number generator using crypto.rand package.
- Check stdout being available when console output is configured.
- Updated to Go 1.7

Packetbeat
- Add missing nil-check to memcached GapInStream handler.
- Fix NFSv4 Operation returning the first found first-class operation
  available in compound requests.
- Fix TCP overlapping segments not being handled correctly.

Filebeat
- Undocumented support for following symlinks is deprecated. Filebeat
  will not follow symlinks in version 5.0.

Beats version 1.2.3

Topbeat
- Fix high CPU usage when using filtering under Windows.

Filebeat
- Fix rotation issue with ignore_older.
- Prevent file opening for files which reached ignore_older.

To see a diff of this commit:
https://wip.pkgsrc.org/cgi-bin/gitweb.cgi?p=pkgsrc-wip.git;a=commitdiff;h=d4629f65ff631690842b10df02be781c6256c4ea

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

diffstat:
 beats/Makefile | 16 ++++++++--------
 beats/PLIST    |  6 ++++++
 beats/distinfo |  8 ++++----
 3 files changed, 18 insertions(+), 12 deletions(-)

diffs:
diff --git a/beats/Makefile b/beats/Makefile
index d36f04c..c9c1bff 100644
--- a/beats/Makefile
+++ b/beats/Makefile
@@ -1,7 +1,7 @@
 # $NetBSD$
 #
 
-DISTNAME=		beats-1.2.2
+DISTNAME=		beats-5.0.2
 CATEGORIES=		sysutils
 MASTER_SITES=		${MASTER_SITE_GITHUB:=elastic/}
 GITHUB_PROJECT=		beats
@@ -19,9 +19,9 @@ USE_TOOLS+=		gmake
 
 WRKSRC=			${WRKDIR}/src/github.com/elastic/beats
 
-EGFILES=		filebeat.template.json filebeat.yml
-EGFILES+=		packetbeat.template.json packetbeat.yml
-#EGFILES+=		topbeat/topbeat.template.json topbeat/topbeat.yml
+EGFILES=		filebeat.template-es2x.json filebeat.template.json filebeat.yml
+EGFILES+=		metricbeat.template-es2x.json metricbeat.template.json metricbeat.yml
+EGFILES+=		packetbeat.template-es2x.json packetbeat.template.json packetbeat.yml
 
 .for file in ${EGFILES}
 CONF_FILES+=		share/examples/beats/${file} ${PKG_SYSCONFDIR}/${file}
@@ -30,9 +30,9 @@ CONF_FILES+=		share/examples/beats/${file} ${PKG_SYSCONFDIR}/${file}
 PKG_SYSCONFSUBDIR=	beats
 
 MAKE_ENV+=		GOPATH=${WRKDIR}:${PREFIX}/gopkg
-BUILD_DIRS=		libbeat packetbeat filebeat # topbeat
+BUILD_DIRS=		libbeat filebeat metricbeat packetbeat
 
-BUILD_TARGET=		build
+BUILD_TARGET=		#
 
 INSTALLATION_DIRS+=	bin share/examples/beats
 
@@ -43,10 +43,10 @@ post-extract:
 do-install:
 	${INSTALL_PROGRAM} ${WRKSRC}/filebeat/filebeat ${DESTDIR}${PREFIX}/bin
 	${INSTALL_PROGRAM} ${WRKSRC}/libbeat/libbeat ${DESTDIR}${PREFIX}/bin
+	${INSTALL_PROGRAM} ${WRKSRC}/metricbeat/metricbeat ${DESTDIR}${PREFIX}/bin
 	${INSTALL_PROGRAM} ${WRKSRC}/packetbeat/packetbeat ${DESTDIR}${PREFIX}/bin
-#	${INSTALL_PROGRAM} ${WRKSRC}/topbeat ${DESTDIR}${PREFIX}/bin
 .for file in ${EGFILES}
-	${INSTALL_DATA} ${WRKSRC}/*/etc/${file} \
+	${INSTALL_DATA} ${WRKSRC}/*/${file} \
 		${DESTDIR}${PREFIX}/share/examples/beats
 .endfor
 
diff --git a/beats/PLIST b/beats/PLIST
index 98f84e1..bf32af3 100644
--- a/beats/PLIST
+++ b/beats/PLIST
@@ -1,8 +1,14 @@
 @comment $NetBSD$
 bin/filebeat
 bin/libbeat
+bin/metricbeat
 bin/packetbeat
+share/examples/beats/filebeat.template-es2x.json
 share/examples/beats/filebeat.template.json
 share/examples/beats/filebeat.yml
+share/examples/beats/metricbeat.template-es2x.json
+share/examples/beats/metricbeat.template.json
+share/examples/beats/metricbeat.yml
+share/examples/beats/packetbeat.template-es2x.json
 share/examples/beats/packetbeat.template.json
 share/examples/beats/packetbeat.yml
diff --git a/beats/distinfo b/beats/distinfo
index 80f47f4..8ecb2f1 100644
--- a/beats/distinfo
+++ b/beats/distinfo
@@ -1,6 +1,6 @@
 $NetBSD$
 
-SHA1 (beats-1.2.2.tar.gz) = e74fac4538350da9f380e42c00fc7034c1cb8291
-RMD160 (beats-1.2.2.tar.gz) = 1484fefd8418f458b74b685d68a0b574f9dbc1ac
-SHA512 (beats-1.2.2.tar.gz) = 44682089b0f06c171672dbe3f5d41b8f5c3071dfc34a5a40fd0e329882b06d7c493b6fa99a1f88a220602bd794120a457fdd5139b95f67bc4f64704da4b6d347
-Size (beats-1.2.2.tar.gz) = 9918529 bytes
+SHA1 (beats-5.0.2.tar.gz) = aa815924cf212aa080eb984e1d7bd660a4d09eef
+RMD160 (beats-5.0.2.tar.gz) = b7f77ac0a8b71fa154a1180a88c04af6a33aafbe
+SHA512 (beats-5.0.2.tar.gz) = 2481ed584efda67051e59a27b45ba52039d960d6dc335d9225e2370bc71e5167f5e4e9e7cc0cb7e689c9a73d5edf27655c4764b0acb6f1422f18bfc10fc44fe9
+Size (beats-5.0.2.tar.gz) = 15774694 bytes