pkgsrc-Changes-HG archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

[pkgsrc/trunk]: pkgsrc/sysutils/beats Update beats to version 6.6.2.



details:   https://anonhg.NetBSD.org/pkgsrc/rev/5eddbc6a6c84
branches:  trunk
changeset: 331267:5eddbc6a6c84
user:      he <he%pkgsrc.org@localhost>
date:      Fri Mar 15 15:03:02 2019 +0000

description:
Update beats to version 6.6.2.

Pkgsrc changes:
 * Add patches to work around runtime.support_avx and runtime.support_avx2
   being undefined when building with go 1.11.
 * Add patch to cludge around build issue by commenting out BindToInterface
   invocations, which is only implemented for Linux and Darwin.

Upstream changes:

6.6.2:
======
Auditbeat
 * System module: Fix and unify bucket closing logic. 10897
Filebeat
 * Fix a bug when converting NetFlow fields to snake_case. 10950
Metricbeat
 * Fix issue in kubernetes module preventing usage percentages to
   be properly calculated. 10946
Packetbeat
 * Avoid reporting unknown MongoDB opcodes more than once. 10878
Winlogbeat
 * Prevent Winlogbeat from dropping events with invalid XML. 11006

6.6.1:
======
Breaking changes

Affecting all Beats
 * Fix stopping of modules started by kubernetes autodiscover. 10476
Auditbeat
 * Enable System module config on Windows. 10237
Filebeat
 * Fix bad bytes count in docker input when filtering by stream. 10211
 * Add convert_timezone option to Logstash module to convert dates
   to UTC. 9756 9797
 * Add convert_timezone option to Elasticsearch module to convert
   dates to UTC. 9756 9761
 * Make elasticsearch/audit fileset be more lenient in parsing node
   name. 10035 10135
Journalbeat
 * Fix fields.yml indentation of audit group which had the effect
   of creating an incomplete Elasticsearch index template. 10556
Metricbeat
 * Fix issue with elasticsearch/node_stats metricset (x-pack) not
   indexing source_node field. 10639
Packetbeat
 * Fixed a crash when using af_packet capture 10477
Functionbeat
 * Ensure that functionbeat is logging at info level not debug. 10262

Added
Filebeat
 * Teach elasticsearch/audit fileset to parse out some more fields.
   10134 10137
Journalbeat
 * Migrate registry from previously incorrect path. 10486

6.6.0:
======
Breaking changes

Affecting all Beats
 * Dissect syntax change, use * instead of ? when working with field
   reference. 8054
Filebeat
 * Allow beats to blacklist certain part of the configuration while
   using Central Management. 9099
Metricbeat
 * Allow beats to blacklist certain part of the configuration while
   using Central Management. 9099
Functionbeat
 * The CLI will now log CloudFormation Stack events. 8912
 * Correctly normalize Cloudformation resource name. 10087

Bugfixes

Affecting all Beats
 * Fix autodiscover configurations stopping when metadata is missing. 8851
 * Refresh host metadata in add_host_metadata. 9359
 * When collecting swap metrics for beats telemetry or system
   metricbeat module handle cases of free swap being bigger than
   total swap by assuming no swap is being used. 6271 9383
 * Ignore non index fields in default_field for Elasticsearch. 9549
 * Update Golang to 1.10.6. 9563
 * Update Kibana index pattern attributes for objects that are disabled. 9644
 * Enforce validation for the Central Management access token. 9621
 * Fix registry handle leak on Windows
   (https://github.com/elastic/go-sysinfo/pull/33). 9920
 * Gracefully handle TLS options when enrolling a Beat. 9129
 * Allow to unenroll a Beat from the UI. 9452
 * The backing off now implements jitter to better distribute the load. 10172
 * Fix config appender registration. 9873
 * Fix TLS certificate DoS vulnerability. 10304

Filebeat
 * Fix improperly set config for CRI Flag in Docker Input 8899
 * Just enabling the elasticsearch fileset and starting Filebeat
   no longer causes an error. 8891
 * Support IPv6 addresses with zone id in IIS ingest pipeline. 9836
   error log: 9869 access log: 10030
 * Support haproxy log lines without captured headers. 9463 9958

Heartbeat
 * Heartbeat now always downloads the entire body of HTTP endpoints,
   even if no checks against the body content are declared. This
   fixes an issue where timing metrics would be incorrect in
   scenarios where the body wasn't used since the connection would
   be closed soon after the headers were sent, but before the entire
   body was. 8894

Metricbeat
 * Add missing namespace field in http server metricset 7890
 * Fix issue with not collecting Elasticsearch cross-cluster
   replication stats correctly. 9179
 * The node.name field in the elasticsearch/node metricset now
   correctly reports the Elasticsarch node name. Previously this
   field was incorrectly reporting the node ID instead. 9209
 * Fix panics in vsphere module when certain values where not
   returned by the API. 9784
 * Fix pod UID metadata enrichment in Kubernetes module. 10081

Packetbeat
 * Fix issue with process monitor associating traffic to the wrong
   process. 9151 9443
 * Fix DHCPv4 dashboard that wouldn't load in Kibana. 9850

Added

Affecting all Beats
 * Unify dashboard exporter tools. 9097
 * Dissect will now flag event on parsing error. 8751
 * Added the redirect_stderr option that allows panics to be logged
   to log files. 8430
 * Add cache.ttl to add_host_metadata. 9359
 * Add support for index lifecycle management (beta). 7963
 * Always include Pod UID as part of Pod metadata. {pull]9517[9517]
 * Release Jolokia autodiscover as GA. 9706

Auditbeat
 * Add system module. 9546

Filebeat
 * Added detect_null_bytes selector to detect null bytes from a io.reader. 9210
 * Added syslog_host variable to HAProxy module to allow syslog
   listener to bind to configured host. 9366
 * Allow to force CRI format parsing for better performance 8424
 * Add event.dataset to module events. 9457
 * Add field log.source.address and log.file.path to replace source. 9435
 * Add support for multi-core thread_id in postgresql module 9156 9482
 * Added netflow input type that supports NetFlow v1, v5, v6, v7,
   v8, v9 and IPFIX. 9399

Journalbeat
 * Add the ability to check against JSON HTTP bodies with conditions. 8667
 * Add cursor_seek_fallback option. 9234

Metricbeat
 * Collect custom cluster display_name in elasticsearch/cluster_stats
   metricset. 8445
 * Test etcd module with etcd 3.3. 9068
 * All elasticsearch metricsets now have module-level cluster.id
   and cluster.name fields. 8770 8771 9164 9165 9166 9168
 * All elasticsearch node-level metricsets now have node.id and
   node.name fields. 9168 9209
 * Add settings to disable docker and cgroup cpu metrics per core.
   9187 9194 9589
 * The elasticsearch/node metricset now reports the Elasticsearch
   cluster UUID. 8771
 * Support GET requests in Jolokia module. 8566 9226
 * Add freebsd support for the uptime metricset. 9413
 * Add host.os.name field to add_host_metadata processor. 8948 9405
 * Add field event.dataset which is {module}.{metricset). 9393

Deprecated

Filebeat
 * Deprecate field source. Will be replaced by log.source.address
   and log.file.path in 7.0. 9435

Metricbeat
 * Deprecate field metricset.rtt. Replaced by event.duration which
   is in nano instead of micro seconds. 9393

Packetbeat
 * Support new TLS version negotiation introduced in TLS 1.3. 8647.

6.5.4:
======

Bugfixes

Affecting all Beats
 * Update Golang to 1.10.6. This fixes an issue in remote certificate
   validation CVE-2018-16875. 9563

Filebeat
 * Fix saved objects in filebeat haproxy dashboard. 9417
 * Fixed a memory leak when harvesters are closed. 7820

Added

Filebeat
 * Added support on Traefik for Common Log Format and Combined Log
   Format mixed which is the default Traefik format 8015 6111 8768.

6.5.3:
======

Bugfixes

Affecting all Beats
 * Log events at the debug level when dropped by encoding problems. 9251

Filebeat
 * Correctly parse December or Dec in the Syslog input. 9349
 * Don't generate incomplete configurations when logs collection
   is disabled by hints. 9305
 * Stop runners disabled by hints after previously being started. 9305
 * Fix installation of haproxy dashboard. 9307 9313

6.5.2:
======

Bugfixes

Affecting all Beats
 * Propagate Sync error when running SafeFileRotate. 9069

Metricbeat
 * Fix panic on docker healthcheck collection on dockers without
   healthchecks. 9171
 * Fix issue preventing diskio metrics collection for idle disks.
   9124 9125

6.5.1:
======

Bugfixes

Affecting all Beats
 * Fix windows binaries not having an enroll command. 9096 8836
Journalbeat
 * Fix journalbeat sometimes hanging if output is unavailable. 9106
Metricbeat
 * Fix race condition when enriching events with kubernetes metadata. 9055 9067

Added
Journalbeat
 * Add minimal kibana dashboard. 9106

6.5.0:
======

Bugfixes

Affecting all Beats

 * Fixed add_host_metadata not initializing correctly on Windows.
   7715
 * Fixed missing file unlock in spool file on Windows, so file can
   be reopened and locked. 7859
 * Fix spool file opening/creation failing due to file locking on
   Windows. 7859
 * Fix size of maximum mmaped read area in spool file on Windows.
   7859
 * Fix potential data loss on OS X in spool file by using fcntl
   with F_FULLFSYNC. 7859
 * Improve fsync on linux, by assuming the kernel resets error
   flags of failed writes. 7859
 * Remove unix-like permission checks on Windows, so files can be
   opened. 7849
 * Replace index patterns in TSVB visualizations. 7929
 * Deregister pipeline loader callback when inputsRunner is stopped.
   7893[7893]
 * Add backoff support to x-pack monitoring outputs. 7966
 * Removed execute permissions systemd unit file. 7873
 * Fix a race condition with the add_host_metadata and the event
   serialization. 8223 8653
 * Enforce that data used by k8s or docker doesn't use any reference. 8240
 * Switch to different UUID lib due to to non-random generated UUIDs. 8485
 * Fix race condition when publishing monitoring data. 8646
 * Fix bug in loading dashboards from zip file. 8051
 * Fix in-cluster kubernetes configuration on IPv6. 8754
 * The export config subcommand should not display real value for
   field reference. 8769
 * The setup command will not fail if no dashboard is available to
   import. 8977
 * Fix central management configurations reload when a configuration
   is removed in Kibana. 9010

Auditbeat
 * Fixed a crash in the file_integrity module under Linux. 7753
 * Fixed the RPM by designating the config file as configuration
   data in the RPM spec. 8075
 * Fixed a concurrent map write panic in the auditd module. 8158
 * Fixed a data race in the file_integrity module. 8009
 * Fixed a deadlock in the file_integrity module. 8027

Filebeat

 * Fix date format in Mongodb Ingest pipeline. 7974
 * Fixed a docker input error due to the offset update bug in
   partial log join.8177
 * Update CRI format to support partial/full tags. 8265
 * Fix some errors happening when stopping syslog input. 8347
 * Fix RFC3339 timezone and nanoseconds parsing with the syslog input. 8346
 * Mark the TCP and UDP input as GA. 8125
 * Support multiline logs in logstash/log fileset of Filebeat. 8562

Heartbeat
 * Fixed bug where HTTP responses with larger bodies would
   incorrectly report connection errors. 8660

Metricbeat
 * Fix golang.heap.gc.cpu_fraction type from long to float in Golang
   module. 7789
 * Fixed the RPM by designating the modules.d config files as
   configuration data in the RPM spec. 8075
 * Fixed the location of the modules.d dir in Deb and RPM packages. 8104
 * Add docker diskio stats on Windows. 6815 8126
 * Fix incorrect type conversion of average response time in Haproxy
   dashboards 8404
 * Added io disk read and write times to system module 8473 8508
 * Avoid mapping issues in kubernetes module. 8487
 * Recover metrics for old apache versions removed by mistake on #6450. 7871
 * Fix dropwizard module parsing of metric names. 8365 8385
 * Fix issue that would prevent kafka module to find a proper broker
   when port is not set 8613
 * Fix range colors in multiple visualizations. 8633 8634
 * Fix incorrect header parsing on http metricbeat module 8564 8585
 * Fixed a panic when the kvm module cannot establish a connection
   to libvirtd. 7792.
 * Add missing namespace field in http server metricset 7890

Packetbeat
 * Fixed a seccomp related error where the fcntl64 syscall was
   not permitted on 32-bit Linux and the sniffer failed to start.
   7839
 * Added missing cmdline and client_cmdline fields to index template. 8258

Added

Affecting all Beats
 * Added time-based log rotation. 8349
 * Add backoff on error support to redis output. 7781
 * Allow for cloud-id to specify a custom port. This makes cloud-id
   work in ECE contexts. 7887
 * Add support to grow or shrink an existing spool file between
   restarts. 7859
 * Make kubernetes autodiscover ignore events with empty container
   IDs 7971
 * Implement CheckConfig in RunnerFactory to make autodiscover
   check configs 7961
 * Add DNS processor with support for performing reverse lookups
   on IP addresses. 7770
 * Support for Kafka 2.0.0 in kafka output 8399
 * Add setting setup.kibana.space.id to support Kibana Spaces 7942
 * Better tracking of number of open file descriptors. 7986
 * Report number of open file handles on Windows. 8329
 * Added the add_process_metadata processor to enrich events with
   process information. 6789
 * Add Beats Central Management 8559
 * Report configured queue type. 8091
 * Enable host and cloud metadata processors by default. 8596

Filebeat
 * Add tag "truncated" to "log.flags" if incoming line is longer
   than configured limit. 7991
 * Add haproxy module. 8014
 * Add tag "multiline" to "log.flags" if event consists of multiple
   lines. 7997
 * Release docker input as GA. 8328
 * Keep unparsed user agent information in user_agent.original. 7832
 * Added default and TCP parsing formats to HAproxy module 8311 8637
 * Add Suricata IDS/IDP/NSM module. 8153 8693
 * Support for Kafka 2.0.0 8853

Heartbeat
 * Heartbeat is marked as GA.
 * Add automatic config file reloading. 8023
 * Added autodiscovery support 8415
 * Added support for extra TLS/x509 metadata. 7944
 * Added stats and state metrics for number of monitors and endpoints
   started. 8621

Journalbeat
 * Add journalbeat. 8703

Metricbeat
 * Add replstatus metricset to MongoDB module 7604
 * Add experimental socket summary metricset to system module 6782
 * Move common kafka fields (broker, topic and partition.id) to
   the module level to facilitate events correlation 7767
 * Add fields for memory fragmentation, memory allocator stats,
   copy on write, master-slave status, and active defragmentation
   to info metricset of Redis module. 7695
 * Increase ignore_above for system.process.cmdline to 2048. 8100
 * Add support to renamed fields planned for redis 5.0. 8167
 * Allow TCP helper to support delimiters and graphite module to
   accept multiple metrics in a single payload. 8278
 * Added died PID state to process_system metricset on system module 8275
 * Add metrics metricset to MongoDB module. 7611
 * Added ccr metricset to Elasticsearch module. 8335
 * Support for Kafka 2.0.0 8399
 * Added support for query params in configuration 8286 8292
 * Add container image for docker metricsets. 8214 8438
 * Precalculate composed id fields for kafka dashboards. 8504
 * Add support for full status page output for php-fpm module as
   a separate metricset called process. 8394
 * Add Kafka dashboard. 8457
 * Release Kafka module as GA. 8854

Packetbeat
 * Added DHCP protocol support. 7647

Functionbeat
 * Initial version of Functionbeat. 8678

Deprecated

Heartbeat
 * watch.poll_file is now deprecated and superceded by automatic
   config file reloading.

Metricbeat
 * Redis info replication.master_offset has been deprecated in
   favor of replication.master.offset.7695
 * Redis info clients fields longest_output_list and biggest_input_buf
   have been renamed to max_output_buffer and max_input_buffer
   based on the names they will have in Redis 5.0, both fields will
   coexist during a time with the same value 8167.
 * Move common kafka fields (broker, topic and partition.id) to
   the module level 7767.

6.4.3:
======

Bugfixes

Affecting all Beats
 * Fix a race condition with the add_host_metadata and the event
   serialization. 8223 8653
 * Fix race condition when publishing monitoring data. 8646
 * Fix bug in loading dashboards from zip file. 8051
 * The export config subcommand should not display real value for
   field reference. 8769

Filebeat
 * Fix typo in Filebeat IIS Kibana visualization. 8604

Metricbeat
 * Recover metrics for old Apache versions removed by mistake on
   #6450. 7871
 * Avoid mapping issues in Kubernetes module. 8487
 * Fixed a panic when the KVM module cannot establish a connection
   to libvirtd. 7792

6.4.2:
======

Bugfixes

Filebeat
 * Fix some errors happening when stopping syslog input. 8347
 * Fix RFC3339 timezone and nanoseconds parsing with the syslog input. 8346

Metricbeat
 * Fix incorrect type conversion of average response time in Haproxy
   dashboards 8404
 * Fix dropwizard module parsing of metric names. 8365 8385

6.4.1:
======

Bugfixes

Affecting all Beats
 * Add backoff support to x-pack monitoring outputs. 7966
 * Removed execute permissions systemd unit file. 7873
 * Fix a race condition with the add_host_metadata and the event
   serialization. 8223
 * Enforce that data used by k8s or docker doesn't use any reference.
   8240
 * Implement CheckConfig in RunnerFactory to make autodiscover
   check configs 7961
 * Make kubernetes autodiscover ignore events with empty container
   IDs 7971

Auditbeat
 * Fixed a concurrent map write panic in the auditd module. 8158
 * Fixed the RPM by designating the config file as configuration
   data in the RPM spec. 8075

Filebeat
 * Fixed a docker input error due to the offset update bug in
   partial log join.8177
 * Update CRI format to support partial/full tags. 8265

Metricbeat
 * Fixed the location of the modules.d dir in Deb and RPM packages.
   8104
 * Fixed the RPM by designating the modules.d config files as
   configuration data in the RPM spec. 8075
 * Fix golang.heap.gc.cpu_fraction type from long to float in Golang
   module. 7789

Packetbeat
 * Added missing cmdline and client_cmdline fields to index template. 8258

6.4.0:
======

Known issue

Due to a packaging mistake, the modules.d configuration directory
is installed in the wrong path in the Metricbeat DEB and RPM
packages. This issue results in an empty list when you run metricbeat
modules list and failures when you try to enable or disable modules.
To work around this issue, run the following command:

sudo cp -r /usr/share/metricbeat/modules.d /etc/metricbeat/

This issue affects all new installations on DEB and RPM. Upgrades
will run, but use old configurations defined in the modules.d
directory from the previous installation.

The issue will be fixed in the 6.4.1 release.

Breaking changes

Affecting all Beats
 * Set default kafka version to 1.0.0 in kafka output. Older versions
   are still supported by configuring the version setting. Minimally
   supported version is 0.11 (older versions might work, but are
   untested). 7025

Heartbeat
 * Rename http.response.status to http.response.status_code to
   align with ECS. 7274
 * Remove type field as not needed. 7307

Metricbeat
 * Fixed typo in values for state_container status.phase, from
   terminate to terminated. 6916
 * RabbitMQ management plugin path is now configured at the module
   level instead of having to do it in each of the metricsets. New
   management_path_prefix option should be used now 7074
 * RabbitMQ node metricset only collects metrics of the instance
   it connects to, node.collect: cluster can be used to collect
   all nodes as before. 6556 6971
 * Change http/server metricset to put events by default under
   http.server and prefix config options with server.. 7100
 * Disable dedotting in docker module configuration. This will
   change the out-of-the-box behaviour, but not the one of already
   configured instances. 7485
 * Fix typo in etcd/self metricset fields from *.bandwithrate to
   *.bandwidthrate. 7456
 * Changed the definition of the system.cpu.total.pct and
   system.cpu.total.norm.cou fields to exclude the IOWait time.
   7691

Bugfixes

Affecting all Beats
 * Error out on invalid Autodiscover template conditions settings. 7200
 * Allow to override the ignore_above option when defining new
   field with the type keyword. 7238
 * Fix a panic on the Dissect processor when we have data remaining
   after the last delimiter. 7449
 * When we fail to build a Kubernetes' indexer or matcher we produce
   a warning but we don't add them to the execution. 7466
 * Fix default value for logging.files.keepfiles. It was being set
   to 0 and now it's set to the documented value of 7. 7494
 * Retain compatibility with older Docker server versions. 7542
 * Fix errors unpacking configs modified via CLI by ignoring -E
   key=value pairs with missing value. 7599

Auditbeat
 * Allow auditbeat setup to run without requiring elevated privileges
   for the audit client. 7111
 * Fix goroutine leak that occurred when the auditd module was stopped. 7163

Filebeat
 * Fix a data race between stopping and starting of the harvesters. 6879
 * Fix an issue when parsing ISO8601 dates with timezone definition 7367
 * Fix Grok pattern of MongoDB module. 7568
 * Fix registry duplicates and log resending on upgrade. 7634

Metricbeat
 * Fix Windows service metricset when using a 32-bit binary on a
   64-bit OS. 7294
 * Do not report Metricbeat container host as hostname in Kubernetes
   deployment. 7199
 * Ensure metadata updates don't replace existing pod metrics. 7573
 * Fix kubernetes pct fields reporting. 7677
 * Add support for new kube_node_status_condition in Kubernetes
   state_node. 7699

Added

Affecting all Beats
 * Add dissect processor. 6925
 * Add IP-addresses and MAC-addresses to add_host_metadata. 6878
 * Added a seccomp (secure computing) filter on Linux that whitelists
   the necessary system calls used by each Beat. 5213
 * Ship fields.yml as part of the binary 4834
 * Added options to dev-tools/cmd/dashboards/export_dashboard.go:
   -indexPattern to include index-pattern in output, -quiet to be
   quiet. 7101
 * Add Indexer indexing by pod uid. Enable pod uid metadata gathering
   in add_kubernetes_metadata. Extended Matcher log_path matching
   to support volume mounts 7072
 * Add default_fields to Elasticsearch template when connecting to
   Elasticsearch >= 7.0. 7015
 * Add support for loading a template.json file directly instead
   of using fields.yml. 7039
 * Add support for keyword multifields in field.yml. 7131
 * Add experimental Jolokia Discovery autodiscover provider. 7141
 * Add owner object info to Kubernetes metadata. 7231
 * Add Beat export dashboard command. 7239
 * Add support for docker autodiscover to monitor containers on
   host network 6708
 * Add ability to define input configuration as stringified JSON
   for autodiscover. 7372
 * Add processor definition support for hints builder 7386
 * Add support to disable html escaping in outputs. 7445
 * Refactor error handing in schema.Apply(). 7335
 * Add additional types to Kubernetes metadata 7457
 * Add module state reporting for Beats Monitoring. 7075
 * Release the rename processor as GA. 7656
 * Add support for Openstack Nova in add_cloud_metadata processor.
   7663

Auditbeat
 * Added XXH64 hash option for file integrity checks. 7311
 * Added the show auditd-rules and show auditd-status commands to
   show kernel rules and status. 7114
 * Add Kubernetes specs for auditbeat file integrity monitoring 7642

Filebeat
 * Add Kibana module with log fileset. 7052
 * Support MySQL 5.7.19 by mysql/slowlog 6969
 * Correctly join partial log lines when using docker input. 6967
 * Add support for TLS with client authentication to the TCP input 7056
 * Converted part of pipeline from treafik/access metricSet to
   dissect to improve efficiency. 7209
 * Add GC fileset to the Elasticsearch module. 7305
 * Add Audit log fileset to the Elasticsearch module. 7365
 * Add Slow log fileset to the Elasticsearch module. 7473
 * Add deprecation fileset to the Elasticsearch module. 7474
 * Add convert_timezone option to Kafka module to convert dates to
   UTC. 7546 7578
 * Add patterns for kafka 1.1 logs. 7608
 * Move debug messages in tcp input source 7712

Metricbeat
 * Add experimental Elasticsearch index metricset. 6881
 * Add dashboards and visualizations for haproxy metrics. 6934
 * Add Jolokia agent in proxy mode. 6475
 * Add message rates to the RabbitMQ queue metricset 6442 6606
 * Add exchanges metricset to the RabbitMQ module 6442 6607
 * Add Elasticsearch index_summary metricset. 6918
 * Add shard metricset to Elasticsearch module. 7006
 * Add apiserver metricset to Kubernetes module. 7059
 * Add maxmemory to redis info metricset. 7127
 * Set guest as default user in RabbitMQ module. 7107
 * Add postgresql statement metricset. 7048 7060
 * Update state_container metricset to support latest kube-state-metrics
   version. 7216
 * Add TLS support to MongoDB module. 7401
 * Added Traefik module with health metricset. 7413
 * Add Elasticsearch ml_job metricsets. 7196
 * Add support for bearer token files to HTTP helper. 7527
 * Add Elasticsearch index recovery metricset. 7225
 * Add locks, global_locks, oplatencies and process fields to status
   metricset of MongoDB module. 7613
 * Run Kafka integration tests on version 1.1.0 7616
 * Release raid and socket metricset from system module as GA. 7658
 * Release elasticsearch module and all its metricsets as beta. 7662
 * Release munin and traefik module as beta. 7660
 * Add envoyproxy module. 7569
 * Release prometheus collector metricset as GA. 7660
 * Add Elasticsearch cluster_stats metricset. 7638
 * Added basepath setting for HTTP-based metricsets 7700

Packetbeat
 * The process monitor now reports the command-line for all processes,
   under Linux and Windows. 7135
 * Updated the TLS protocol parser with new cipher suites added to
   TLS 1.3. 7455
 * Flows are enriched with process information using the process
   monitor. 7507
 * Added UDP support to process monitor. 7571

Deprecated

Metricbeat
 * Kubernetes state_container cpu.limit.nanocores and cpu.request.nanocores
   have been deprecated in favor of cpu.*.cores. 6916

6.3.2:
======

Bugfixes

Affecting all Beats
 * Fix registry duplicates and log resending on upgrade. 7634
 * Fix default value for logging.files.keepfiles. It was being set
   to 0 and now it's set to the documented value of 7. 7494
 * Retain compatibility with older Docker server versions. 7542

Metricbeat
 * Fix missing hosts config option in Ceph module. 7596
 * Ensure metadata updates don't replace existing pod metrics. 7573

Added

Metricbeat
 * Add support for bearer token files to HTTP helper. 7527

Packetbeat
 * Updated the TLS protocol parser with new cipher suites added
   to TLS 1.3. 7455

6.3.1:
======

Bugfixes

Affecting all Beats
 * Allow index-pattern only setup when setup.dashboards.only_index=true.
   7285
 * Preserve the event when source matching fails in add_docker_metadata.
   7133
 * Negotiate Docker API version from our client instead of using
   a hardcoded one. 7165
 * Fix duplicating dynamic_fields in template when overwriting the
   template. 7352

Auditbeat
 * Fixed parsing of AppArmor audit messages. 6978

Filebeat
 * Comply with PostgreSQL database name format 7198
 * Optimize PostgreSQL ingest pipeline to use anchored regexp and
   merge multiple regexp into a single expression. 7269
 * Keep different registry entry per container stream to avoid
   wrong offsets. 7281
 * Fix offset field pointing at end of a line. 6514
 * Commit registry writes to stable storage to avoid corrupt registry
   files. 6792

Metricbeat
 * Fix field mapping for the system process CPU ticks fields. 7230
 * Ensure canonical naming for JMX beans is disabled in Jolokia module. 7047
 * Fix Jolokia attribute mapping when using wildcards and MBean
   names with multiple properties. 7321

Packetbeat
 * Fix an out of bounds access in HTTP parser caused by malformed
   request. 6997
 * Fix missing type for http.response.body field. 7169

Added

Auditbeat
 * Added caching of UID and GID values to auditd module. 6978
 * Updated syscall tables for Linux 4.16. 6978
 * Added better error messages for when the auditd module fails
   due to the Linux kernel not supporting auditing (CONFIG_AUDIT=n).
   7012

Metricbeat
 * Collect accumulated docker network metrics and mark old ones
   as deprecated. 7253

6.3.0:
======

Breaking changes

Affecting all Beats
 * De dot keys of labels and annotations in kubernetes meta processors
   to prevent collisions. 6203
 * Rename beat.cpu.*.time metrics to beat.cpu.*.time.ms. 6449
 * Add host.name field to all events, to avoid mapping conflicts.
   This could be breaking Logstash configs if you rely on the host
   field being a string. 7051

Filebeat
 * Add validation for Stdin, when Filebeat is configured with Stdin
   and any other inputs, Filebeat will now refuse to start. 6463
 * Mark system.syslog.message and system.auth.message as text
   instead of keyword. 6589

Metricbeat
 * De dot keys in kubernetes/event metricset to prevent collisions.
   6203
 * Add config option for windows/perfmon metricset to ignore non
   existent counters. 6432
 * Refactor docker CPU calculations to be more consistent with
   docker stats. 6608
 * Update logstash.node_stats metricset to write data under
   logstash.node.stats.*. 6714

Bugfixes

Affecting all Beats
 * Fix panic when Events containing a float32 value are normalized. 6129
 * Fix setup.dashboards.always_kibana when using Kibana 5.6. 6090
 * Fix for Kafka logger. 6430
 * Remove double slashes in Windows service script. 6491
 * Ensure Kubernetes labels/annotations don't break mapping 6490
 * Ensure that the dashboard zip files can't contain files outside
   of the kibana directory. 6921
 * Fix map overwrite panics by cloning shared structs before doing
   the update. 6947
 * Fix delays on autodiscovery events handling caused by blocking
   runner stops. 7170
 * Do not emit Kubernetes autodiscover events for Pods without IP
   address. 7235
 * Fix self metrics when containerized 6641

Auditbeat
 * Add hex decoding for the name field in audit path records. 6687
 * Fixed a deadlock in the file_integrity module under Windows. 6864

Filebeat
 * Fix panic when log prospector configuration fails to load. 6800
 * Fix memory leak in log prospector when files cannot be read. 6797
 * Add raw JSON to message field when JSON parsing fails. 6516
   Commit registry writes to stable storage to avoid corrupt registry
   files. 6877
 * Fix a parsing issue in the syslog input for RFC3339 timestamp
   and time with nanoseconds. 7046
 * Fix an issue with an overflowing wait group when using the TCP
   input. 7202
 * Fix an issue when parsing ISO8601 dates with timezone definition 7367

Heartbeat
 * Fix race due to updates of shared a map, that was not supposed
   to be shared between multiple go-routines. 6616

Metricbeat
 * Fix the default configuration for Logstash to include the default
   port. 6279
 * Fix dealing with new process status codes in Linux kernel 4.14+. 6306
 * Add filtering option by exact device names in system.diskio.
   diskio.include_devices. 6085
 * Add connections metricset to RabbitMQ module 6548
 * Fix panic in http dependent modules when invalid config was used. 6205
 * Fix system.filesystem.used.pct value to match what df reports. 5494
 * Fix namespace disambiguation in Kubernetes state_* metricsets. 6281
 * Fix Windows perfmon metricset so that it sends metrics when an
   error occurs. 6542
 * Exclude bind mounts in fsstat and filesystem metricsets. 6819
 * Don't stop Metricbeat if aerospike server is down. 6874
 * Disk reads and write count metrics in RabbitMQ queue metricset
   made optional. 6876
 * Add mapping for docker metrics per cpu. 6843

Winlogbeat
 * Fixed a crash under Windows 2003 and XP when an event had less
   insert strings than required by its format string. 6247

Added

Affecting all Beats
 * Update Golang 1.9.4 6326
 * Add the ability to log to the Windows Event Log. 5913
 * The node name can be discovered automatically by machine-id
   matching when beat deployed outside Kubernetes cluster. 6146
 * Panics will be written to the logger before exiting. 6199
 * Add builder support for autodiscover and annotations builder 6408
 * Add plugin support for autodiscover builders, providers 6457
 * Preserve runtime from container statuses in Kubernetes autodiscover 6456
 * Experimental feature setup.template.append_fields added. 6024
 * Add appender support to autodiscover 6469
 * Add add_host_metadata processor 5968
 * Retry configuration to load dashboards if Kibana is not reachable
   when the beat starts. 6560
 * Add has_fields conditional to filter events based on the existence
   of all the given fields. 6285 6653
 * Add support for spooling to disk to the beats event publishing pipeline. 6581
 * Added logging of system info at Beat startup. 5946
 * Do not log errors if X-Pack Monitoring is enabled but Elastisearch
   X-Pack is not. 6627
 * Add rename processor. 6292
 * Allow override of dynamic template match_mapping_type for fields
   with object_type. 6691

Filebeat
 * Add IIS module to parse access log and error log. 6127
 * Renaming of the prospector type to the input type and all
   prospectors are now moved to the input folder, to maintain
   backward compatibility type aliasing was used to map the old
   type to the new one. This change also affect YAML configuration.
   6078
 * Addition of the TCP input 6700
 * Add option to convert the timestamps to UTC in the system module. 5647
 * Add Logstash module support for main log and the slow log,
   support the plain text or structured JSON format 5481
 * Add stream filtering when using docker prospector. 6057
 * Add support for CRI logs format. 5630
 * Add json.ignore_decoding_error config to not log json decoding
   erors. 6547
 * Make registry file permission configurable. 6455
 * Add MongoDB module. 6238
 * Add Ingest pipeline loading to setup. 6814
 * Add support of log_format combined to NGINX access logs. 6858
 * Release config reloading feature as GA.
 * Add support human friendly size for the UDP input. 6886
 * Add Syslog input to ingest RFC3164 Events via TCP and UDP 6842
 * Remove the undefined username option from the Redis input and
   clarify the documentation. 6662

Heartbeat

 * Made the URL field of Heartbeat aggregateable. 6263
 * Use match.Matcher for checking Heartbeat response bodies with
   regular expressions. 6539

Metricbeat
 * Support apache status pages for versions older than 2.4.16. 6450
 * Add support for huge pages on Linux. 6436
 * Support to optionally de dot keys in http/json metricset to
   prevent collisions. 5970
 * Add graphite protocol metricbeat module. 4734
 * Add http server metricset to support push metrics via http. 4770
 * Make config object public for graphite and http server 4820
 * Add system uptime metricset. 4848
 * Add experimental queue metricset to RabbitMQ module. 4788
 * Add additional php-fpm pool status kpis for Metricbeat module 5287
 * Add etcd module. 4970
 * Add ip address of docker containers to event. 5379
 * Add ceph osd tree information to metricbeat 5498
 * Add ceph osd_df to metricbeat 5606
 * Add basic Logstash module. 5540
 * Add dashboard for Windows service metricset. 5603
 * Add pct calculated fields for Pod and container CPU and memory usages. 6158
 * Add statefulset support to Kubernetes module. 6236
 * Refactor prometheus endpoint parsing to look similar to upstream
   prometheus 6332
 * Making the http/json metricset GA. 6471
 * Add support for array in http/json metricset. 6480
 * Making the jolokia/jmx module GA. 6143
 * Making the MongoDB module GA. 6554
 * Allow to disable labels dedot in Docker module, in favor of a
   safe way to keep dots. 6490
 * Add experimental module to collect metrics from munin nodes. 6517
 * Add support for wildcards and explicit metrics grouping in
   jolokia/jmx. 6462
 * Set collector as default metricset in Prometheus module. 6636 6747
 * Set mntr as default metricset in Zookeeper module. 6674
 * Set default metricsets in vSphere module. 6676
 * Set status as default metricset in Apache module. 6673
 * Set namespace as default metricset in Aerospike module. 6669
 * Set service as default metricset in Windows module. 6675
 * Set all metricsets as default metricsets in uwsgi module. 6688
 * Allow autodiscover to monitor unexposed ports 6727
 * Mark kubernetes.event metricset as beta. 6715
 * Set all metricsets as default metricsets in couchbase module. 6683
 * Mark uwsgi module and metricset as beta. 6717
 * Mark Golang module and metricsets as beta. 6711
 * Mark system.raid metricset as beta. 6710
 * Mark http.server metricset as beta. 6712
 * Mark metricbeat logstash module and metricsets as beta. 6713
 * Set all metricsets as default metricsets in Ceph module. 6676
 * Set container, cpu, diskio, healthcheck, info, memory and network
   in docker module as default. 6718
 * Set cpu, load, memory, network, process and process_summary as
   default metricsets in system module. 6689
 * Set collector as default metricset in Dropwizard module. 6669
 * Set info and keyspace as default metricsets in redis module. 6742
 * Set connection as default metricset in rabbitmq module. 6743
 * Set all metricsets as default metricsets in Elasticsearch module. 6755
 * Set all metricsets as default metricsets in Etcd module. 6756
 * Set server metricsets as default in Graphite module. 6757
 * Set all metricsets as default metricsets in HAProxy module. 6758
 * Set all metricsets as default metricsets in Kafka module. 6759
 * Set all metricsets as default metricsets in postgresql module. 6761
 * Set status metricsets as default in Kibana module. 6762
 * Set all metricsets as default metricsets in Logstash module. 6763
 * Set container, node, pod, system, volume as default in Kubernetes
   module. https://github.com/elastic/beats/pull/ 6764[6764]
 * Set stats as default in memcached module. 6765
 * Set all metricsets as default metricsets in Mongodb module. 6766
 * Set pool as default metricset for php_fpm module. 6768
 * Set status as default metricset for mysql module.
   https://github.com/elastic/beats/pull/ 6769[6769]
 * Set stubstatus as default metricset for nginx module. 6770
 * Added support for haproxy 1.7 and 1.8. 6793
 * Add accumulated I/O stats to diskio in the line of docker stats. 6701
 * Ignore virtual filesystem types by default in system module. 6819
 * Release config reloading feature as GA. 6891
 * Kubernetes deployment: Add ServiceAccount config to system metricbeat. 6824
 * Kubernetes deployment: Add DNS Policy to system metricbeat. 6656

Packetbeat
 * Add support for condition on bool type 5659 5954
 * Fix high memory usage on HTTP body if body is not published. 6680
 * Allow to capture the HTTP request or response bodies independently. 6784
 * HTTP publishes an Error event for unmatched requests or responses. 6794

Winlogbeat
 * Use bookmarks to persist the last published event. 6150

diffstat:

 sysutils/beats/Makefile                                                               |   6 +-
 sysutils/beats/distinfo                                                               |  13 +-
 sysutils/beats/patches/patch-vendor_github.com_insomniacslk_dhcp_dhcpv4_client.go     |  38 +++++++
 sysutils/beats/patches/patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.go |  48 ++++++++++
 sysutils/beats/patches/patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.s  |  30 ++++++
 5 files changed, 127 insertions(+), 8 deletions(-)

diffs (161 lines):

diff -r e3b8e0d78e29 -r 5eddbc6a6c84 sysutils/beats/Makefile
--- a/sysutils/beats/Makefile   Fri Mar 15 14:42:54 2019 +0000
+++ b/sysutils/beats/Makefile   Fri Mar 15 15:03:02 2019 +0000
@@ -1,7 +1,7 @@
-# $NetBSD: Makefile,v 1.37 2019/03/09 10:05:11 bsiegert Exp $
+# $NetBSD: Makefile,v 1.38 2019/03/15 15:03:02 he Exp $
 
-DISTNAME=              beats-6.2.4
-PKGREVISION=           5
+DISTNAME=              beats-6.6.2
+#PKGREVISION=          5
 CATEGORIES=            sysutils
 MASTER_SITES=          ${MASTER_SITE_GITHUB:=elastic/}
 GITHUB_PROJECT=                beats
diff -r e3b8e0d78e29 -r 5eddbc6a6c84 sysutils/beats/distinfo
--- a/sysutils/beats/distinfo   Fri Mar 15 14:42:54 2019 +0000
+++ b/sysutils/beats/distinfo   Fri Mar 15 15:03:02 2019 +0000
@@ -1,7 +1,10 @@
-$NetBSD: distinfo,v 1.24 2018/04/18 12:18:49 fhajny Exp $
+$NetBSD: distinfo,v 1.25 2019/03/15 15:03:02 he Exp $
 
-SHA1 (beats-6.2.4.tar.gz) = 4bec24f50654e0712bce08f3835609ea450caf95
-RMD160 (beats-6.2.4.tar.gz) = aa5ccfcb92b4d61daa3d0a615050296540e16115
-SHA512 (beats-6.2.4.tar.gz) = 49293e28e4ca769a88f7b889ea888a559ae7e142698714e24bf5519d00c2030237cac591590aa284f17792906a6137333173716ddf5db4496217f8b5a054b19c
-Size (beats-6.2.4.tar.gz) = 19466271 bytes
+SHA1 (beats-6.6.2.tar.gz) = ca8b05fa23d383e90b5d8357224491b8d3f38c10
+RMD160 (beats-6.6.2.tar.gz) = cdb95337644c09a40a2eb556b76e9cff6e93bde7
+SHA512 (beats-6.6.2.tar.gz) = 1bc16fdd4525a197e4deeedb29fbc77ec44418c5eb371be95fa09555c1344a2e423aab83207c8313ac6cd33ce7fe91fe7ca06837302917465b5e1803dbf7f66d
+Size (beats-6.6.2.tar.gz) = 29119533 bytes
+SHA1 (patch-vendor_github.com_insomniacslk_dhcp_dhcpv4_client.go) = bad4afc5985be96705cc12defb26349646b92a6a
 SHA1 (patch-vendor_github.com_tsg_gopacket_pcap_pcap.go) = 2ef13f84cbf2e1da428e7b08625ffa6111ebcdf5
+SHA1 (patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.go) = 0c4a21e7d73c4e4a91965af6d87e70ed164a6d32
+SHA1 (patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.s) = e9cbc76f1ac5f27666a54b4541bab8f15665fc7d
diff -r e3b8e0d78e29 -r 5eddbc6a6c84 sysutils/beats/patches/patch-vendor_github.com_insomniacslk_dhcp_dhcpv4_client.go
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/sysutils/beats/patches/patch-vendor_github.com_insomniacslk_dhcp_dhcpv4_client.go Fri Mar 15 15:03:02 2019 +0000
@@ -0,0 +1,38 @@
+$NetBSD: patch-vendor_github.com_insomniacslk_dhcp_dhcpv4_client.go,v 1.1 2019/03/15 15:03:02 he Exp $
+
+BindToInterface is apparently only implemented for Linux and Darwin.
+So much for portable software.  Since I fail to grasp what dhcp has
+to do with shipping logs, just comment out the code to make it build.
+
+--- vendor/github.com/insomniacslk/dhcp/dhcpv4/client.go.orig  2019-03-06 10:13:10.000000000 +0000
++++ vendor/github.com/insomniacslk/dhcp/dhcpv4/client.go
+@@ -90,10 +90,10 @@ func MakeBroadcastSocket(ifname string) 
+       if err != nil {
+               return fd, err
+       }
+-      err = BindToInterface(fd, ifname)
+-      if err != nil {
+-              return fd, err
+-      }
++//    err = BindToInterface(fd, ifname)
++//    if err != nil {
++//            return fd, err
++//    }
+       return fd, nil
+ }
+ 
+@@ -113,10 +113,10 @@ func MakeListeningSocket(ifname string) 
+       if err = syscall.Bind(fd, &syscall.SockaddrInet4{Port: ClientPort, Addr: addr}); err != nil {
+               return fd, err
+       }
+-      err = BindToInterface(fd, ifname)
+-      if err != nil {
+-              return fd, err
+-      }
++//    err = BindToInterface(fd, ifname)
++//    if err != nil {
++//            return fd, err
++//    }
+       return fd, nil
+ }
+ 
diff -r e3b8e0d78e29 -r 5eddbc6a6c84 sysutils/beats/patches/patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.go
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/sysutils/beats/patches/patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.go     Fri Mar 15 15:03:02 2019 +0000
@@ -0,0 +1,48 @@
+$NetBSD: patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.go,v 1.1 2019/03/15 15:03:02 he Exp $
+
+We get link errors about runtime.support_avx and runtime.support_avx2, so
+revert the offending commit from
+https://github.com/golang/crypto/commit/b0697eccbea9adec5b7ba8008f4c33d98d733388
+I've not been able to track which version of golang.org/x/crypto is
+embedded in the beats package, and frankly am befuddled that this sort
+of thing makes it into released code, in this day and age with widespread
+use of CI etc.  This coupled with the apparent widespread practice of
+"vendoring" code (copy a vendor's code and ship it as part of your source
+package) makes it difficult to stamp out this sort of bugs.  Sigh!
+
+--- vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.go.orig       2019-03-06 10:13:10.000000000 +0000
++++ vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.go
+@@ -6,20 +6,22 @@
+ 
+ package blake2b
+ 
+-func init() {
+-      useAVX2 = supportsAVX2()
+-      useAVX = supportsAVX()
+-      useSSE4 = supportsSSE4()
+-}
++import _ "unsafe"
+ 
+-//go:noescape
+-func supportsSSE4() bool
++//go:linkname x86_HasAVX internal/cpu.X86.HasAVX
++var x86_HasAVX bool
+ 
+-//go:noescape
+-func supportsAVX() bool
++//go:linkname x86_HasAVX2 internal/cpu.X86.HasAVX2
++var x86_HasAVX2 bool
+ 
+-//go:noescape
+-func supportsAVX2() bool
++//go:linkname x86_HasAVX internal/cpu.X86.HasSSE4
++var x86_HasSSE4 bool
++
++func init() {
++      useAVX2 = x86_HasAVX2
++      useAVX = x86_HasAVX
++      useSSE4 = x86_HasSSE4
++}
+ 
+ //go:noescape
+ func hashBlocksAVX2(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
diff -r e3b8e0d78e29 -r 5eddbc6a6c84 sysutils/beats/patches/patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.s
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/sysutils/beats/patches/patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.s      Fri Mar 15 15:03:02 2019 +0000
@@ -0,0 +1,30 @@
+$NetBSD: patch-vendor_golang.org_x_crypto_blake2b_blake2bAVX2__amd64.s,v 1.1 2019/03/15 15:03:03 he Exp $
+
+We get link errors about runtime.support_avx and runtime.support_avx2, so
+revert the offending commit from
+https://github.com/golang/crypto/commit/b0697eccbea9adec5b7ba8008f4c33d98d733388
+I've not been able to track which version of golang.org/x/crypto is
+embedded in the beats package, and frankly am befuddled that this sort
+of thing makes it into released code, in this day and age with widespread
+use of CI etc.  This coupled with the apparent widespread practice of
+"vendoring" code (copy a vendor's code and ship it as part of your source
+package) makes it difficult to stamp out this sort of bugs.  Sigh!
+
+--- vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.s.orig        2019-03-06 10:13:10.000000000 +0000
++++ vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.s
+@@ -748,15 +748,3 @@ noinc:
+ 
+       MOVQ BP, SP
+       RET
+-
+-// func supportsAVX2() bool
+-TEXT ·supportsAVX2(SB), 4, $0-1
+-      MOVQ runtime·support_avx2(SB), AX
+-      MOVB AX, ret+0(FP)
+-      RET
+-
+-// func supportsAVX() bool
+-TEXT ·supportsAVX(SB), 4, $0-1
+-      MOVQ runtime·support_avx(SB), AX
+-      MOVB AX, ret+0(FP)
+-      RET




Home | Main Index | Thread Index | Old Index