-
Notifications
You must be signed in to change notification settings - Fork 4.9k
/
CHANGELOG.asciidoc
11243 lines (7722 loc) · 511 KB
/
CHANGELOG.asciidoc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
// Use these for links to issue and pulls. Note issues and pulls redirect one to
// each other on Github, so don't worry too much on using the right prefix.
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/
[[release-notes-8.16.1]]
=== Beats version 8.16.1
https://github.com/elastic/beats/compare/v8.16.0\...v8.16.1[View commits]
==== Breaking changes
*Packetbeat*
- Expire source port mappings. {pull}41581[41581]
==== Bugfixes
*Filebeat*
- Fix AWS region in aws-s3 input S3 polling mode. {pull}41572[41572]
[[release-notes-8.16.0]]
=== Beats version 8.16.0
https://github.com/elastic/beats/compare/v8.15.4\...v8.16.0[View commits]
==== Known issues
*Filebeat*
- The AWS S3 input polling mode is not working when the S3 bucket is not in the `us-east-1` default region. This also impacts all AWS integrations and any custom AWS log integration which uses the `aws-s3` input polling mode. When using Filebeat, please add a `default_region` configuration with the region of the S3 bucket. For example:
+
["source","yaml"]
----
filebeat.inputs:
- type: aws-s3
enabled: true
credential_profile_name: elastic-observability
default_region: us-east-2
number_of_workers: 5
bucket_arn: 'arn:aws:s3:::test1'
----
==== Breaking changes
*Affecting all Beats*
- Fix FQDN being lowercased when used as `host.hostname`. {issue}39993[39993]
- Beats won't log start up information when running under the Elastic Agent. {pull}40390[40390]
- Filebeat now needs `dup3`, `faccessat2`, `prctl` and `setrlimit` syscalls to run the journald input. If this input is not being used, the syscalls are not needed. All Beats have those syscalls allowed now because the default seccomp policy is global to all Beats. {pull}40061[40061]
- Beats will rate limit the logs about errors when indexing events on Elasticsearch, logging a summary every 10s. The logs sent to the event log is unchanged. {issue}40157[40157]
*Filebeat*
- Filebeat, when running with Elastic-Agent, reports status for Filestream input. {pull}40121[40121]
- Added support for hyphens in extension keys in `decode_cef` Filebeat processor. {pull}40427[40427]
- Journald: removed configuration options `include_matches.or`, `include_matches.and`, `backoff`, `max_backoff`, `cursor_seek_fallback`. {pull}40061[40061]
- Journald: `include_matches.match` now behaves in the same way as matchers in `journalctl`. Users should carefully update their input configuration. {pull}40061[40061]
- Journald: `seek` and `since` behaviour have been simplified, if there is a cursor (state) `seek` and `since` are ignored and the cursor is used. {pull}40061[40061]
- Redis: Added replication role as a field to submitted slowlogs.
- Added `container.image.name` to `journald` Filebeat input's Docker-specific translated fields. {pull}40450[40450]
- Remove deprecated awscloudwatch field from Filebeat. {pull}41089[41089]
- The performance of ingesting SQS data with the S3 input has improved by up to 60x for queues with many small events. `max_number_of_messages` config for SQS mode is now ignored, as the new design no longer needs a manual cap on messages. Instead, use `number_of_workers` to scale ingestion rate in both S3 and SQS modes. The increased efficiency may increase network bandwidth consumption, which can be throttled by lowering `number_of_workers`. It may also increase number of events stored in memory, which can be throttled by lowering the configured size of the internal queue. {pull}40699[40699]
*Metricbeat*
- Add support for specifying a custom endpoint for GCP service clients. {issue}40848[40848] {pull}40918[40918]
==== Bugfixes
*Auditbeat*
- Request status from a separate socket to avoid data congestion. {pull}41207[41207]
*Filebeat*
- Fix crashes in the journald input. {pull}40061[40061]
- Fix long filepaths in diagnostics exceeding max path limits on Windows. {pull}40909[40909]
- Fix a bug in Salesforce input to only handle responses with 200 status code. {pull}41015[41015]
- Fixed failed job handling and removed false-positive error logs in the GCS input. {pull}41142[41142]
- Bump github.com/elastic/go-sfdc dependency used by x-pack/filebeat/input/salesforce. {pull}41192[41192]
- Journald input now can read events from all boots {issue}41083[41083] {pull}41244[41244]
- Fix errors in SQS host resolution in the `aws-s3` input when using custom (non-AWS) endpoints. {pull}41504[41504]
*Metricbeat*
- Add GCP 'instance_id' resource label in ECS cloud fields. {issue}40033[40033] {pull}40062[40062]
- Remove excessive info-level logs in cgroups setup. {pull}40491[40491]
- Fix http server helper SSL config. {pull}39405[39405]
==== Added
*Filebeat*
- Implement Elastic Agent status and health reporting for Netflow Filebeat input. {pull}40080[40080]
- Add SSL and username support for Redis input, now the input includes support for Redis 6.0+. {pull}40111[40111]
- Add scaling up support for Netflow input. {issue}37761[37761] {pull}40122[40122]
- Update CEL mito extensions to v1.15.0. {pull}40294[40294]
- Improve logging in Okta Entity Analytics provider. {issue}40106[40106] {pull}40347[40347]
- Document `winlog` input. {issue}40074[40074] {pull}40462[40462]
- Added retry logic to websocket connections in the streaming input. {issue}40271[40271] {pull}40601[40601]
- Disable event normalization for netflow input. {pull}40635[40635]
- Allow attribute selection in the Active Directory entity analytics provider. {issue}40482[40482] {pull}40662[40662]
- Improve error quality when CEL program does not correctly return an events array. {pull}40580[40580]
- Added support for Microsoft Entra ID RBAC authentication. {issue}40434[40434] {pull}40879[40879]
- Add `use_kubeadm` config option for filebeat (both filbeat.input and autodiscovery) in order to toggle kubeadm-config api requests. {pull}40301[40301]
- Make HTTP library function inclusion non-conditional in CEL input. {pull}40912[40912]
- Add support for Crowdstrike streaming API to the streaming input. {issue}40264[40264] {pull}40838[40838]
- Add support to CEL for reading host environment variables. {issue}40762[40762] {pull}40779[40779]
- Add CSV decoder to awss3 input. {pull}40896[40896]
- Change request trace logging to include headers instead of complete request. {pull}41072[41072]
- Improved GCS input documentation. {pull}41143[41143]
- Add CSV decoding capacity to azureblobstorage input. {pull}40978[40978]
- Add CSV decoding capacity to gcs input. {pull}40979[40979]
- Add support to source AWS cloudwatch logs from linked accounts. {pull}41188[41188]
- Jounrald input now supports filtering by facilities. {pull}41061[41061]
- Add support to include AWS cloudwatch linked accounts when using log_group_name_prefix to define log group names. {pull}41206[41206]
*Heartbeat*
- Add journey duration to synthetics browser events. {pull}40230[40230]
*Metricbeat*
- Add new metrics fot datastore and minor changes to overall vSphere metrics. {pull}40766[40766]
- Add new metricset datastorecluster for vSphere module. {pull}40634[40634] {pull}40694[40694]
- Add AWS Cloudwatch capability to retrieve tags from AWS/ApiGateway resources. {pull}40755[40755]
- Add new metrics for the vSphere Virtualmachine metricset. {pull}40485[40485]
- Add `metrics_count` to Prometheus module if `metrics_count: true` is set. {pull}40411[40411]
[[release-notes-8.15.5]]
=== Beats version 8.15.5
https://github.com/elastic/beats/compare/v8.15.4\...v8.15.5[View commits]
==== Breaking changes
*Packetbeat*
- Expire source port mappings. {pull}41581[41581]
==== Bugfixes
*Affecting all Beats*
- Fix metrics not being ingested, due to "Limit of total fields [10000] has been exceeded while adding new fields [...]". The total fields limit has been increased to 12500. No significant performance impact on Elasticsearch is anticipated. {pull}41640[41640]
*Filebeat*
- Fix AWS region in `aws-s3` input S3 polling mode. {pull}41572[41572]
[[release-notes-8.15.4]]
=== Beats version 8.15.4
https://github.com/elastic/beats/compare/v8.15.3\...v8.15.4[View commits]
==== Breaking changes
*Osquerybeat*
- Disable `allow_unsafe` osquery configuration. {pull}40130[40130]
==== Bugfixes
*Affecting all Beats*
- Fix issue where old data could be saved in the memory queue after acknowledgment, increasing memory use. {pull}41356[41356]
*Filebeat*
- Log bad handshake details when websocket connection fails. {pull}41300[41300]
- Improve modification time handling for entities and entity deletion logic in the Active Directory entityanalytics input. {pull}41179[41179]
- Fix double encoding of `client_secret` in the Entity Analytics input's Azure Active Directory provider. {pull}41393[41393]
- The azure-eventhub input now correctly reports its status to the Elastic Agent on fatal errors. {pull}41469[41469]
*Metricbeat*
- Fix Kubernetes metadata sometimes not being present after startup. {pull}41216[41216]
*Winlogbeat*
- Fix truncated windows event log message. {pull}41327[41327]
==== Added
*Affecting all Beats*
- Replace Ubuntu 20.04 with 24.04 for Docker base images. {issue}40743[40743] {pull}40942[40942]
- Reduce memory consumption of k8s autodiscovery and the `add_kubernetes_metadata` processor when Deployment metadata is enabled.
*Heartbeat*
- Add monitor status reporter under managed mode. {pull}41077[41077]
*Metricbeat*
- Only watch metadata for ReplicaSets in metricbeat k8s module. {pull}41289[41289]
[[release-notes-8.15.3]]
=== Beats version 8.15.3
https://github.com/elastic/beats/compare/v8.15.2\...v8.15.3[View commits]
==== Known issues
*Affecting all Beats*
- Memory usage is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. {issue}41355[41355]
==== Breaking changes
*Filebeat*
- Change `log.file.path` field in awscloudwatch input to nested object. {pull}41099[41099]
==== Bugfixes
*Affecting all Beats*
- Allow port number 0 in the community ID flowhash processor. {pull}40259[40259]
- The journald input now restarts if there is an error/crash. {issue}32782[32782] {pull}40558[40558]
*Filebeat*
- Fix replace processor handling of zero string replacement validation. {pull}40751[40751]
- Add backup and delete for AWS S3 polling mode feature back. {pull}41071[41071]
*Metricbeat*
- Use namespace for GetListMetrics when it exists in AWS. {pull}41022[41022]
*Packetbeat*
- Fix upload of bundled ingest pipelines on Windows. {pull}41110[41110]
==== Added
*Affecting all Beats*
- Update Go version to 1.22.8. {pull}41139[41139]
- Add kafka compression support for ZSTD.
*Metricbeat*
- Restore `docker.network.in.*` and `docker.network.out.*` fields in docker module. {pull}40968[40968]
[[release-notes-8.15.2]]
=== Beats version 8.15.2
https://github.com/elastic/beats/compare/v8.15.0\...v8.15.2[View commits]
==== Known issues
*Affecting all Beats*
- Beats Docker images do not log to stderr by default. The workaround is to pass the CLI flag `-e` or to set `logging.to_stderr: true` in the configuration file. {issue}41118[41118]
==== Known issues
*Affecting all Beats*
- Memory usage is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. {issue}41355[41355]
==== Bugfixes
*Affecting all Beats*
- Fix bug that prevented the Elasticsearch output from recovering from an interrupted connection. {issue}40705[40705] {pull}40769[40796]
*Metricbeat*
- Add GCP organization and project details to ECS cloud fields. {pull}40461[40461]
[[release-notes-8.15.1]]
=== Beats version 8.15.1
https://github.com/elastic/beats/compare/v8.15.0\...v8.15.1[View commits]
==== Known issues
*Affecting all Beats*
- Beats Docker images do not log to stderr by default. The workaround is to pass the CLI flag `-e` or to set `logging.to_stderr: true` in the configuration file.
- Beats stop publishing data after a network error unless restarted. Avoid upgrading to 8.15.1. Affected Beats log `Get \"https://${ELASTICSEARCH_HOST}:443\": context canceled` repeatedly. {issue}40705{40705}
- Memory usage is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. {issue}41355[41355]
==== Bugfixes
*Affecting all Beats*
- Aborts all active connections for Elasticsearch output. {pull}40572[40572]
- Closes beat Publisher on beat stop and by the Agent manager. {pull}40572[40572]
- Fix handling of escaped brackets in syslog structured data. {issue}40445[40445] {pull}40446[40446]
*Auditbeat*
- Fix segfaults that may happen if user runs multiple instances of the package metricset {pull}40525[40525]
- Fix incorrect definition of struct utmp for arm64 {pull}40541[40541]
*Filebeat*
- Relax requirements in Okta entity analytics provider user and device profile data shape. {pull}40359[40359]
- Fix bug in Okta entity analytics rate limit logic. {issue}40106[40106] {pull}40267[40267]
- Fix order of configuration for EntraID entity analytics provider. {pull}40487[40487]
- Ensure Entra ID request bodies are not truncated and trace logs are rotated before 100MB. {pull}40494[40494]
- The Elasticsearch output now correctly logs the event fields to the event log file {issue}40509[40509] {pull}40512[40512]
- Fix the "No such input type exist: 'azure-eventhub'" error on the Windows platform {issue}40608[40608] {pull}40609[40609]
- awss3 input: Fix handling of SQS notifications that don't contain a region. {pull}40628[40628]
- Fix credential handling when workload identity is being used in GCS input. {issue}39977[39977] {pull}40663[40663]
- Fix high IO and handling of a corrupted registry log file. {pull}35893[35893]
- Fix filestream's registry GC: registry entries will never be removed if clean_inactive is set to "-1". {pull}40258[40258]
*Metricbeat*
- Fix first HTTP 401 error when fetching metrics from the Kubelet API caused by a token update {pull}40636[40636]
- Fix needlessly verbose logging in cgroups setup {issue}40620[40620]
==== Added
*Filebeat*
- Enable file ingestion to report detailed status to Elastic Agent {pull}40075[40075]
- Added `ignore_empty_values` flag in `decode_cef` Filebeat processor. {pull}40268[40268]
*Metricbeat*
- Added back `elasticsearch.node.stats.jvm.mem.pools.*` to the `node_stats` metricset {pull}40571[40571]
[[release-notes-8.15.0]]
=== Beats version 8.15.0
https://github.com/elastic/beats/compare/v8.14.3\...v8.15.0[View commits]
==== Known issues
*Affecting all Beats*
- Beats Docker images do not log to stderr by default. The workaround is to pass the CLI flag `-e` or to set `logging.to_stderr: true` in the configuration file. {issue}41118[41118]
*Filebeat*
- The Azure EventHub input in Filebeat is not found when running on Windows. Please refrain from upgrading to 8.15. See {issue}40608[40608] for details.
- Memory usage is not correctly limited by the number of events actively in the memory queue, but rather the maximum size of the memory queue regardless of usage. {issue}41355[41355]
- The AWS S3 input polling mode is not working when the S3 bucket is not in the `us-east-1` default region. This also impacts all AWS integrations and any custom AWS log integration which uses the `aws-s3` input polling mode. When using Filebeat, please add a `default_region` configuration with the region of the S3 bucket. For example:
+
["source","yaml"]
----
filebeat.inputs:
- type: aws-s3
enabled: true
credential_profile_name: elastic-observability
default_region: us-east-2
number_of_workers: 5
bucket_arn: 'arn:aws:s3:::test1'
----
==== Breaking changes
*Filebeat*
- Tag events that come from a filestream in "take over" mode. {pull}39828[39828]
- Fix filestream's registry garbage collection: registry entries will never be removed if `clean_inactive` is set to "-1". {pull}40258[40258]
*Metricbeat*
- Remove fallback to the node limit for the `kubernetes.pod.cpu.usage.limit.pct` and `kubernetes.pod.memory.usage.limit.pct` metrics calculation.
- Add support for Kibana status metricset in v8 format. {pull}40275[40275]
*Osquerybeat*
- Add action responses data stream, allowing Osquerybeat to post action results directly to Elasticsearch. {pull}39143[39143]
==== Bugfixes
*Affecting all Beats*
- Rename the field "apache2.module.error" to "apache.module.error" in Apache error visualization. {issue}39480[39480] {pull}39481[39481]
- Validate config of the `replace` processor. {pull}40047[40047]
*Filebeat*
- Fix for Google Workspace duplicate events issue by adding canonical sorting over fingerprint keys array to maintain key order. {pull}40055[40055] {issue}39859[39859]
- Prevent panic in CEL and salesforce inputs when `github.com/hashicorp/go-retryablehttp` exceeds maximum retries. {pull}40144[40144]
- Update CEL mito extensions to v1.13.1. {pull}40307[40307]
- Fix bug in CEL input rate limit logic. {issue}40106[40106] {pull}40270[40270]
*Metricbeat*
- Set GCP metrics config period to the default (60s) when the value is below the minimum allowed period. {issue}30434[30434] {pull}40020[40020]
- Fix statistic methods for metrics collected for SQS. {pull}40207[40207]
- Update beat module with apm-server monitoring metrics fields. {pull}40127[40127]
- Fix Azure Monitor metric timespan to restore Storage Account PT1H metrics. {issue}40376[40376] {pull}40367[40367]
==== Added
*Affecting all Beats*
- Update Go version to 1.22.5. {pull}40082[40082]
- Introduce log message for not supported annotations for Hints based autodiscover. {pull}38213[38213]
- Add persistent volume claim name to volume if available. {pull}38839[38839]
- Raw events are now logged to a different file, this prevents potentially sensitive information from leaking into log files. {pull}38767[38767]
- Websocket input: Added runtime URL modification support based on state and cursor values. {issue}39858[39858] {pull}39997[39997]
*Auditbeat*
- Reduce data size for `add_session_metadata` processor by removing unneeded fields. {pull}39500[39500]
- Enrich process events with user and group names, with `add_session_metadata` processor. {pull}39537[39537]
*Filebeat*
- Ensure all responses sent by HTTP Endpoint are HTML-escaped. {pull}39329[39329]
- Improve logging of request and response with request trace logging in error conditions. {pull}39455[39455]
- Implement Elastic Agent status and health reporting for CEL Filebeat input. {pull}39209[39209]
- Add HTTP metrics to CEL input. {issue}39501[39501] {pull}39503[39503]
- Add default user-agent to CEL HTTP requests. {issue}39502[39502] {pull}39587[39587]
- Improve reindexing support in security module pipelines. {issue}38224[38224] {pull}39588[39588]
- Make HTTP Endpoint input GA. {issue}38979[38979] {pull}39410[39410]
- Add support for base64-encoded HMAC headers to HTTP Endpoint. {pull}39655[39655]
- Add user group membership support to Okta entity analytics provider. {issue}39814[39814] {pull}39815[39815]
- Add request trace support for Okta and EntraID entity analytics providers. {pull}39821[39821]
- Allow elision of set and append failure logging. {issue}34544[34544] {pull}39929[39929]
- Add ability to remove request trace logs from CEL input. {pull}39969[39969]
- Add ability to remove request trace logs from HTTPJSON input. {pull}40003[40003]
- Update CEL mito extensions version to v1.13.0 {pull}40035[40035]
- Add Jamf entity analytics provider. {pull}39996[39996]
- Add ability to remove request trace logs from `http_endpoint` input. {pull}40005[40005]
- Add ability to remove request trace logs from `entityanalytics` input. {pull}40004[40004]
- Relax constraint on Base DN in entity analytics Active Directory provider. {pull}40054[40054]
- Enhance input state reporting for CEL evaluations that return a single error object in events. {pull}40083[40083]
- Allow absent credentials when using GCS with Application Default Credentials. {issue}39977[39977] {pull}40072[40072]
- Allow cross-region bucket configuration in S3 input. {issue}22161[22161] {pull}40309[40309]
*Metricbeat*
- Support `schema_name` for MySQL performance metricset. {pull}38363[38363]
- Add `last_terminated_timestamp` metric in Kubernetes module. {pull}39200[39200] {issue}3802[3802]
- Add `pod.status.ready_time` and `pod.status.reason` metrics in Kubernetes module. {pull}39316[39316]
- Add "Buffer cache hit ratio base" to calculate "Buffer cache hit ratio" for performance metrics. {pull}40022[40022]
- Add support of Graphite series 1.1.0+ tagging extension for statsd module. {pull}39619[39619]
[[release-notes-8.14.3]]
=== Beats version 8.14.3
https://github.com/elastic/beats/compare/v8.14.2\...v8.14.3[View commits]
==== Bugfixes
*Filebeat*
- Fix handling of deeply nested numeric values in HTTP Endpoint CEL programs. {pull}40115[40115]
*Metricbeat*
- Fix `namespace` filter option on metricset `state_namespace` enricher. {pull}39934[39934]
- Fix `namespace` filter option at Kubernetes provider level. {pull}39881[39881]
- Fix handling of access errors when reading process metrics. {pull}39627[39627]
- Fix behavior of cgroups path discovery when monitoring the host system from within a container. {pull}39627[39627]
- Fix issue where Beats may report incorrect metrics for its own process when running inside a container. {pull}39627[39627]
- Fix for MySQL/Performance - Query failure for MySQL versions below v8.0.1, for performance metric `quantile_95`. {pull}38710[38710]
- Fix Prometheus helper text parser to store each metric family type. {pull}39743[39743]
- Normalize AWS RDS CPU utilization values before making the metadata API call. {pull}39664[39664]
- Fix behavior of `pagetypeinfo` metrics. {pull}39985[39985]
- Fix query logic for temp and non-temp tablespaces in Oracle module. {issue}38051[38051] {pull}39787[39787]
- Fix missing metrics from CloudWatch when include_linked_accounts set to false. {issue}40071[40071] {pull}40135[40135]
==== Added
*Affecting all Beats*
- Update Go version to 1.21.12. {pull}40114[40114]
[[release-notes-8.14.2]]
=== Beats version 8.14.2
https://github.com/elastic/beats/compare/v8.14.1\...v8.14.2[View commits]
==== Breaking changes
*Filebeat*
- Fix high I/O and handling of a corrupted registry log file. {pull}35893[35893]
==== Bugfixes
*Filebeat*
- Fix request trace filename handling in `http_endpoint` input. {pull}39410[39410]
- Fix filestream not correctly tracking the offset of a file when using the `include_message` parser. {pull}39873[39873] {issue}39653[39653]
- Upgrade `github.com/hashicorp/go-retryablehttp` to mitigate CVE-2024-6104. {pull}40036[40036]
==== Added
*Affecting all Beats*
- Update Go version to 1.21.11. {pull}39851[39851]
*Filebeat*
- Fix handling of infinite rate values in CEL rate limit handling logic. {pull}39940[39940]
*Heartbeat*
- Upgrade Node version to the latest LTS v18.20.3. {pull}40038[40038]
*Winlogbeat*
- Add ERROR_INVALID_PARAMETER to the list of recoverable errors. {pull}39781[39781]
[[release-notes-8.14.1]]
=== Beats version 8.14.1
https://github.com/elastic/beats/compare/v8.14.0\...v8.14.1[View commits]
==== Bugfixes
*Heartbeat*
- Fix import of browser plugin for Agentbeat. {pull}39818[39818]
[[release-notes-8.14.0]]
=== Beats version 8.14.0
https://github.com/elastic/beats/compare/v8.13.4\...v8.14.0[View commits]
==== Breaking changes
*Filebeat*
- Removed deprecated ZScaler from Beats. Use the https://docs.elastic.co/integrations/zscaler_zia[Zscaler Internet Access] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Tomcat from Beats. Use the https://docs.elastic.co/integrations/apache_tomcat[Apache Tomcat] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Squid from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated SonicWall from Beats. Use the https://docs.elastic.co/integrations/sonicwall[SonicWall Firewall] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Snort from Beats. Use the https://docs.elastic.co/integrations/snort[Snort] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Radware from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Proofpoint from Beats. Use the https://docs.elastic.co/integrations/proofpoint_tap[Proofpoint TAP] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Netscout from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Microsoft DHCP from Beats. Use the https://docs.elastic.co/integrations/microsoft_dhcp[Microsoft DHCP] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Juniper Junos from Beats. Use the https://docs.elastic.co/integrations/juniper_srx[Juniper SRX] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Juniper Netscreen from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Infoblox from Beats. Use the https://docs.elastic.co/integrations/infoblox_nios[Infoblox NIOS] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Impreva from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Fortinet Client Endpoint from Beats. Use the https://docs.elastic.co/integrations/fortinet_forticlient[Fortinet FortiClient Logs] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Fortinet Fortimail from Beats. Use the https://docs.elastic.co/integrations/fortinet_fortimail[Fortinet FortiMail] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Fortinet Fortimanager from Beats. Use the https://docs.elastic.co/integrations/fortinet_fortimanager[Fortinet FortiManager Logs] Elastic integration instead. {pull}38037[38037]
- Removed deprecated F5 from Beats. Use the https://docs.elastic.co/integrations/f5_bigip[F5 BIG-IP] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Cylance from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Cisco Meraki from Beats. Use the https://docs.elastic.co/integrations/cisco_meraki[Cisco Meraki] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Cisco Nexus from Beats. Use the https://docs.elastic.co/integrations/cisco_nexus[Cisco Nexus] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Bluecoat from Beats. See {filebeat-ref}/migrate-from-deprecated-module.html[Migrate from a deprecated module] for migration options. {pull}38037[38037]
- Removed deprecated Barracuda from Beats. Use the https://docs.elastic.co/integrations/barracuda[Barracuda Web Application Firewall] Elastic integration instead. {pull}38037[38037]
- Removed deprecated Sophos UTM from Beats. Use the https://docs.elastic.co/integrations/sophos[Sophos] Elastic integration instead. {pull}38037[38037]
- Introduce input/netmetrics and refactor netflow input metrics. {pull}38055[38055]
- Update Salesforce module to use new Salesforce input. {pull}37509[37509]
*Heartbeat*
- Fix monitor state loader to not wait extra seconds for the last attempt. {pull}39621[39621]
==== Bugfixes
*Auditbeat*
- Set field types to correctly match ECS in sessionmd processor. {issue}38955[38955] {pull}38994[38994]
- Fix failing to enrich process events in sessionmd processor. {issue}38955[38955] {pull}39173[39173] {pull}39243[39243]
- Fix seccomp policy of FIM kprobes backend on arm64. {pull}39759[39759]
*Filebeat*
- Fix handling of endpoint for custom domains and ensure region, default_region, and region parsed from queue_url are applied in the order specified in the documentation for the awss3 input. {pull}39709[39709]
- Prevent HTTPJSON holding response bodies between executions. {issue}35219[35219] {pull}38116[38116]
- Fix the incorrect values generated by the uri_parts processor. {pull}38216[38216]
- Rename `activity_guid` to `activity_id` in ETW input events to suit other Windows inputs. {pull}38530[38530]
- Add missing provider registration and fix published entity for Active Directory entityanalytics provider. {pull}38645[38645]
- Fix handling of un-parsed JSON in O365 module. {issue}37800[37800] {pull}38709[38709]
- Fix filestream's registry GC: registry entries are now removed from the in-memory and disk store when they're older than the set TTL. {issue}36761[36761] {pull}38488[38488]
- Fix handling of truncated files in Filestream {issue}38070[38070] {pull}38416[38416]
- Fix panic when more than 32767 pipeline clients are active. {issue}38197[38197] {pull}38556[38556]
- Fix a bug in CloudWatch task allocation that could skip some logs. {issue}38918[38918] {pull}38953[38953]
- Prevent GCP Pub/Sub input blockage by increasing default value of `max_outstanding_messages`. {issue}35029[35029] {pull}38985[38985]
- entity-analytics input: Improve structured logging. {pull}38990[38990]
- Upgrade `azure-event-hubs-go` and `azure-storage-blob-go` dependencies. {pull}38861[38861]
- Fix concurrency/error handling bugs in the AWS S3 input that could drop data and prevent ingestion of large buckets. {pull}39131[39131]
- Fix EntraID query handling. {issue}39419[39419] {pull}39420[39420]
- Expand ID patterns in request trace logger for HTTP Endpoint. {pull}39656[39656]
- Fix awscloudwarch input: set startTime to `0` for the first iteration of retrieving log events from CloudWatch. {pull}40079[40079]
*Heartbeat*
- Redact synthexec cmd output. {pull}39535[39535]
*Metricbeat*
- RabbitMQ/queue - Change the mapping type of `rabbitmq.queue.consumers.utilisation.pct` to `scaled_float` from `long` because the values fall within the range of `[0.0, 1.0]`. Previously, conversion to integer resulted in reporting either `0` or `1`.
- Fix timeout caused by the retrival of which indices are hidden. {pull}39165[39165]
*Winlogbeat*
- Fix error handling in perfmon metrics. {issue}38140[38140] {pull}39404[39404]
==== Added
*Affecting all Beats*
- Update Go version to 1.21.10. {pull}39467[39467]
- Enable early event encoding in the Elasticsearch output, improving CPU and memory use. {pull}38572[38572]
*Auditbeat*
- Add `add_session_metadata` processor, which enables session viewer on Auditbeat data. {pull}37640[37640]
- Add procfs backend to the `add_session_metadata` processor. {pull}38799[38799]
- Add `process.entity_id`, `process.group.name` and `process.group.id` in `add_process_metadata` processor. Make FIM module with Kprobes backend to always add an appropriately configured `add_process_metadata` processor to enrich file events. {pull}38776[38776]
*Filebeat*
- Add Saved Object name field to Kibana audit logs. {pull}38307[38307]
- Add Salesforce input. {pull}37331[37331]
- Add logging for cache processor file reads and writes. {pull}38052[38052]
- Support VPC endpoint for aws-s3 input SQS queue url. {pull}38189[38189]
- Add support for complex event objects in the HTTP Endpoint input. {issue}37910[37910] {pull}38193[38193]
- Parse more fields from Elasticsearch slowlogs. {pull}38295[38295]
- Update CEL mito extensions to v1.10.0 to add keys/values helper. {pull}38504[38504]
- Add support for Active Directory an entity analytics provider. {pull}37919[37919]
- Add AWS AWSHealth metricset. {pull}38370[38370]
- Add debugging breadcrumb to logs when writing request trace log. {pull}38636[38636]
- Add benchmark input and discard output. {pull}37437[37437]
*Libbeat*
- Add support for Linux capabilities in `add_process_metadata`. {pull}38252[38252]
*Metricbeat*
- Add support for `shards_stats.total_count` in Elasticsearch Monitoring data. {pull}38891[38891]
- Add SSL support to MySQL module. {pull}37997[37997]
- Add SSL support for Aerospike module. {pull}38126[38126]
*Winlogbeat*
- Use fixed size buffer at first pass for event parsing, improving throughput. {issue}39530[39530] {pull}39544[39544]
==== Deprecated
*Filebeat*
- Deprecate `syslog` input in favor of `syslog` processor. {issue}37555[37555] {pull}38277[38277]
- Deprecate `o365audit` input in favor of `CEL` input. {issue}37719[37719] {pull}38922[38922]
[[release-notes-8.13.4]]
=== Beats version 8.13.4
https://github.com/elastic/beats/compare/v8.13.3\...v8.13.4[View commits]
==== Bugfixes
*Auditbeat*
- Prevent scenario of losing children-related file events in a directory for recursive fsnotify backend of auditbeat file integrity module. {pull}39133[39133]
- Allow extra syscalls by auditbeat required in FIM with kprobes back-end. {pull}39361[39361]
- Fix losing events in FIM for MacOS X by allowing always to walk an added directory to monitor. {pull}39362[39362]
*Metricbeat*
- Fix Azure Monitor support for multiple aggregation types. {issue}39192[39192] {pull}39204[39204]
[[release-notes-8.13.3]]
=== Beats version 8.13.3
https://github.com/elastic/beats/compare/v8.13.2\...v8.13.3[View commits]
==== Breaking changes
*Metricbeat*
- Setting period for counter cache for Prometheus `remote_write` to at least to 60 seconds. {pull}38553[38553]
==== Bugfixes
*Affecting all Beats*
- Change cache processor documentation from `write_period` to `write_interval`. {pull}38561[38561]
- Fix cache processor expiries heap cleanup on partial file writes. {pull}38561[38561]
- Fix cache processor expiries infinite growth when large a large TTL is used and recurring keys are cached. {pull}38561[38561]
- Fix parsing of RFC 3164 process IDs in syslog processor. {issue}38947[38947] {pull}38982[38982]
*Filebeat*
- Fix indexing failures by re-enabling event normalisation in netflow input. {issue}38703[38703] {pull}38780[38780]
- Fix config validation for CEL and HTTPJSON inputs when using password grant authentication and `client.id` or `client.secret` are not present. {pull}38962[38962]
- Updated Websocket input title to align with existing inputs. {pull}39006[39006]
- [threatintel] MISP splitting fix for empty responses. {issue}38739[38739] {pull}38917[38917]
- Restore netflow input on Windows. {pull}39024[39024]
==== Added
*Affecting all Beats*
- Update Go version to 1.21.9. {pull}38727[38727]
- The environment variable `BEATS_ADD_CLOUD_METADATA_PROVIDERS` overrides configured/default `add_cloud_metadata` providers. {pull}38669[38669]
*Auditbeat*
- Add process data to file events (Linux only, eBPF backend). {pull}38199[38199]
- Add container id to file events (Linux only, eBPF backend). {pull}38328[38328]
*Metricbeat*
- Add new fields to configure the lease duration, retry and renew when using leader elector with Kubernetes autodiscover. {pull}38471[38471]
[[release-notes-8.13.2]]
=== Beats version 8.13.2
https://github.com/elastic/beats/compare/v8.13.1\...v8.13.2[View commits]
==== Bugfixes
*Heartbeat*
- Fix reference yaml format. {pull}38584[38584]
[[release-notes-8.13.1]]
=== Beats version 8.13.1
https://github.com/elastic/beats/compare/v8.13.0\...v8.13.1[View commits]
==== Bugfixes
*Filebeat*
- Fix Filebeat GCS input panic. {pull}38407[38407]
*Heartbeat*
- Reset prctl dumpable flag after cap drop. {pull}38269[38269]
[[release-notes-8.13.0]]
=== Beats version 8.13.0
https://github.com/elastic/beats/compare/v8.12.2\...v8.13.0[View commits]
==== Breaking changes
*Affecting all Beats*
- Upgrade Go version to 1.21.8. Removes support for Windows 8.1. See https://tip.golang.org/doc/go1.21#windows. {pull}38209[38209]
- The behavior of `queue.mem.flush.min_events` has been simplified. It now serves as a simple maximum on the size of all event batches. There are no longer performance implications in its relationship to `bulk_max_size`. {pull}37795[37795]
*Auditbeat*
- Add opt-in `KProbes` backend for file_integrity module. {pull}37796[37796]
*Filebeat*
- Convert netflow input to API v2 and disable event normalisation. {pull}37901[37901]
*Winlogbeat*
- Add "keystore.path" configuration settings to $workdir\data\{{.BeatName}}.keystore. {issue}12315[12315] {pull}37237[37237]
==== Bugfixes
*Affecting all Beats*
- Support Elastic Agent control protocol chunking support. {pull}37343[37343]
- Upgrade elastic-agent-libs to v0.7.5. Removes obsolete "Treating the CommonName field on X.509 certificates as a host name..." deprecation warning for 8.0. {pull}37755[37755]
- Fix the paths in the .cmd script added to the path by the Windows MSI to point to the new C:\Program Files installation location. https://github.com/elastic/elastic-stack-installers/pull/238
- Upgrade elastic-agent-system-metrics to v0.9.2. Skips permissions errors when reading /proc/pid/io. {pull}38234[38234].
*Filebeat*
- Fix a race condition that could crash Filebeat with a "negative WaitGroup counter" error. {pull}38094[38094]
- Fix "failed processing S3 event for object key" error on aws-s3 input when key contains the "+" character. {issue}38012[38012] {pull}38125[38125]
- Fix duplicated addition of regexp extension in CEL input. {pull}38181[38181]
- Fix HTTPJSON handling of empty object bodies in POST requests. {issue}33961[33961] {pull}38290[38290]
- Fix PEM key validation for CEL and HTTPJSON inputs. {pull}38405[38405]
*Heartbeat*
- Adjust State loader to only retry when response code status is 5xx. {pull}37981[37981]
*Metricbeat*
- Fix Azure Monitor 429 error by causing Metricbeat to retry the request again. {pull}38294[38294]
- Fix fields not being parsed correctly in postgresql/database. {issue}25301[25301] {pull}37720[37720]
==== Added
*Affecting all Beats*
- Ignore Kubernetes node and namespace update events that do not change pod metadata. {issue}37338[37338] {pull}37431[37431]
- Enhance add_cloud_metadata processor with `orchestrator.cluster.name`, `orchestrator.cluster.id` and `azure.resourcegroup.name` when running inside an AKS cluster. {issue}33081[33081] {pull}37685[37685]
- Upgrade go-sysinfo from 1.12.0 to 1.13.1. {pull}37996[37996]
- Make `range` condition work with numeric values as strings. {pull}38080[38080]
- Allow users to configure number of output workers (for outputs that support workers) with either `worker` or `workers`. {pull}38257[38257]
- Kafka output now validates the `topics` and `topic` configuration values. {pull}38058[38058]
*Auditbeat*
- Add Linux capabilities to processes in the system/process. {pull}37453[37453]
- Add opt-in eBPF backend for file_integrity module. {pull}37223[37223]
*Filebeat*
- Update SQL input documentation regarding Oracle DSNs {pull}37590[37590]
- Add support for complete URL replacement in HTTPJSON chain steps. {pull}37486[37486]
- Add support for user-defined query selection in EntraID entity analytics provider. {pull}37653[37653]
- Update CEL extensions library to v1.8.0 to provide runtime error location reporting. {issue}37304[37304] {pull}37718[37718]
- Add request trace logging for chained API requests. {issue}37551[36551] {pull}37682[37682]
- Add support for PEM-based Okta auth in HTTPJSON. {pull}37772[37772]
- Prevent complete loss of long request trace data. {issue}37826[37826] {pull}37836[37836]
- Added experimental version of the Websocket Input. {pull}37774[37774]
- Add support for PEM-based Okta auth in CEL. {pull}37813[37813]
- Add ETW input. {pull}36915[36915]
- Update CEL mito extensions to v1.9.0 to add keys/values helper. {pull}37971[37971]
- Add parseDateInTZ value template for the HTTPJSON input. {pull}37738[37738]
- Improve rate limit handling by HTTPJSON. {issue}36207[36207] {pull}38161[38161] {pull}38237[38237]
*Libbeat*
- Add watcher that can be used to monitor Linux kernel events. {pull}37833[37833]
- Added support for ETW reader. {pull}36914[36914]
*Heartbeat*
- Upgrade github.com/elastic/go-elasticsearch/v8 to v8.12.0. {pull}37673[37673]
*Metricbeat*
- Fix containerd metrics grouping for TSDB. {pull}37537[37537]
*Packetbeat*
- Bump Windows Npcap version to v1.79. {pull}37733[37733]
- Add support for pipeline loading. {pull}37291[37291]
[[release-notes-8.12.2]]
=== Beats version 8.12.2
https://github.com/elastic/beats/compare/v8.12.1\...v8.12.2[View commits]
==== Bugfixes
*Filebeat*
- [threatintel] MISP pagination fixes. {pull}37898[37898]
- Fix file handle leak when handling errors in filestream. {pull}37973[37973]
*Packetbeat*
- Fix interface device parsing for packetbeat protocols. {pull}37946[37946]
==== Added
*Metricbeat*
- Update `getOpTimestamp` in `replstatus` to fix sort and temp files generation issue in MongoDB. {pull}37688[37688]
[[release-notes-8.12.1]]
=== Beats version 8.12.1
https://github.com/elastic/beats/compare/v8.12.0\...v8.12.1[View commits]
==== Known Issues
*Affecting all Beats*
Performance regression in AWS S3 inputs using SQS notification.
In 8.12 the default memory queue flush interval was raised from 1 second to 10 seconds. In many configurations this improves performance because it allows the output to batch more events per round trip, which improves efficiency. However, the SQS input has an extra bottleneck that interacts badly with the new value. For more details see {issue}37754[37754].
If you are using the Elasticsearch output, and your output configuration uses a performance preset, switch it to `preset: latency`. If you use no preset or use `preset: custom`, then set `queue.mem.flush.timeout: 1s` in your queue or output configuration.
==== Breaking changes
*Affecting all Beats*
- add_cloud_metadata processor: `huawei` provider is now treated as `openstack`. Huawei cloud runs on OpenStack
platform, and when viewed from a metadata API standpoint, it is impossible to differentiate it from OpenStack. If you
know that your deployments run on Huawei Cloud exclusively, and you wish to have `cloud.provider` value as `huawei`,
you can achieve this by overwriting the value using an `add_fields` processor. {pull}35184[35184]
==== Bugfixes
*Affecting all Beats*
- aws: Add credential caching for `AssumeRole` session tokens. {issue}37787[37787]
- Lower logging level to debug when attempting to configure beats with unknown fields from autodiscovered events/environments. {pull}[37816][37816]
*Filebeat*
- Fix nil pointer dereference in the httpjson input. {pull}37591[37591]
- Fix TCP/UDP metric queue length parsing base. {pull}37714[37714]
- Fix m365_defender cursor value and query building. {pull}37116[37116]
- Update github.com/lestrrat-go/jwx dependency. {pull}37799[37799]
*Heartbeat*
- Fix setuid root when running under cgroups v2. {pull}37794[37794]
*Metricbeat*
- Fix Azure Resource Metrics missing metrics (min and max aggregations) after upgrade to 8.11.3. {issue}37642[37642] {pull}37643[37643]
==== Added
*Filebeat*
- Relax TCP/UDP metric polling expectations to improve metric collection. {pull}37714[37714]
[[release-notes-8.12.0]]
=== Beats version 8.12.0
https://github.com/elastic/beats/compare/v8.11.4\...v8.12.0[View commits]
==== Known Issues
*Affecting all Beats*
Performance regression in AWS S3 inputs using SQS notification.
In 8.12 the default memory queue flush interval was raised from 1 second to 10 seconds. In many configurations this improves performance because it allows the output to batch more events per round trip, which improves efficiency. However, the SQS input has an extra bottleneck that interacts badly with the new value. For more details see {issue}37754[37754].
If you are using the Elasticsearch output, and your output configuration uses a performance preset, switch it to `preset: latency`. If you use no preset or use `preset: custom`, then set `queue.mem.flush.timeout: 1s` in your queue or output configuration.
If you are not using the Elasticsearch output, set `queue.mem.flush.timeout: 1s` in your queue or output configuration.
==== Breaking changes
*Affecting all Beats*
- Windows MSI installers now store configuration in C:\Program Files instead of C:\ProgramData. https://github.com/elastic/elastic-stack-installers/pull/209
*Heartbeat*
- Decrease the ES default timeout to 10 for the load monitor state requests.
- Windows MSI installers now store configuration in C:\Program Files instead of C:\ProgramData. https://github.com/elastic/elastic-stack-installers/pull/209
*Osquerybeat*
- Upgrade to osquery 5.10.2. {pull}37115[37115]
==== Bugfixes
*Filebeat*
- Add validation to the `http_endpoint` config for empty URL. {pull}36816[36816] {issue}36772[36772]
- Fix merging of array fields (processors, paths, parsers) in configurations generated from hints and default config. {issue}36838[36838] {pull}36857[36857]
==== Added
*Affecting all Beats*
- Allow `queue` configuration settings to be set under the output. {issue}35615[35615] {pull}36788[36788]
- Raise up logging level to warning when attempting to configure {beats} with unknown fields from autodiscovered events/environments.
- Elasticsearch output now supports `idle_connection_timeout`. {issue}35616[35615] {pull}36843[36843]
- Upgrade to Go 1.20.12. {pull}37350[37350]
- The Elasticsearch output can now configure performance presets with the `preset` configuration field. {pull}37259[37259]
- Upgrade `elastic-agent-system-metrics` to v0.9.1. See https://github.com/elastic/elastic-agent-system-metrics/releases/tag/v0.9.1. {pull}37353[37353]
- Upgrade to elastic-agent-libs v0.7.3 and golang.org/x/crypto v0.17.0. {pull}37544[37544]
*Auditbeat*
- Add `ignore_errors` option to audit module. {issue}15768[15768] {pull}36851[36851]
- Fix copy arguments for strict aligned architectures. {pull}36976[36976]
*Filebeat*
- Allow http_endpoint input to receive PUT and PATCH requests. {pull}36734[36734]
- Avoid unwanted publication of Azure entity records. {pull}36753[36753]
- Avoid unwanted publication of Okta entity records. {pull}36770[36770]
- Add support for Digest Authentication to CEL input. {issue}35514[35514] {pull}36932[36932]
- Use filestream input with `file_identity.fingerprint` as default for hints autodiscover. {issue}35984[35984] {pull}36950[36950]
- Add network processor in addition to interface based direction resolution. {pull}37023[37023]
- Make CEL input log current transaction ID when request tracing is turned on. {pull}37065[37065]
- Make Azure Blob Storage input GA and update docs accordingly. {pull}37128[37128]
- Add request trace logging to http_endpoint input. {issue}36951[36951] {pull}36957[36957]
- Make GCS input GA and update docs accordingly. {pull}37127[37127]
- Suppress and log max HTTP request retry errors in CEL input. {pull}37160[37160]
- Prevent CEL input from re-entering the eval loop when an evaluation failed. {pull}37161[37161]
- Update CEL extensions library to v1.7.0. {pull}37172[37172]
*Auditbeat*
- Upgrade go-libaudit to v2.4.0. {issue}36776[36776] {pull}36964[36964]
- Add a `/inputs/` route to the HTTP monitoring endpoint that exposes metrics for each dataset instance. {pull}36971[36971]
*Heartbeat*
- Capture and log the individual connection metrics for all the lightweight monitors.
*Metricbeat*
- Add metrics grouping by dimensions and time to Azure app insights. {pull}36634[36634]
- Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms. {pull}36647[36647]
- Enhance GCP billing with detailed tables identification, additional fields, and optimized data handling. {pull}36902[36902]
- Add a `/inputs/` route to the HTTP monitoring endpoint that exposes metrics for each metricset instance. {pull}36971[36971]
- Add Linux IO metrics to system/process. {pull}37213[37213]
- Add new memory/cgroup metrics to Kibana module. {pull}37232[37232]
*Packetbeat*
- Add metrics for TCP flags. {issue}36992[36992] {pull}36975[36975]
*Winlogbeat*
- Make ingest pipeline routing robust to letter case of channel names for forwarded events. {issue}36670[36670] {pull}36899[36899]
- Document minimum permissions required for local user account. {issue}15773[15773] {pull}37176[37176]
==== Deprecated
*Filebeat*
- Deprecate rsa2elk Filebeat modules. {issue}36125[36125] {pull}36887[36887]