forked from zeek/zeek
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathNEWS
More file actions
7471 lines (5656 loc) · 332 KB
/
NEWS
File metadata and controls
7471 lines (5656 loc) · 332 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
This document summarizes the most important changes in the current Zeek
release. For an exhaustive list of changes, see the ``CHANGES`` file
(note that submodules, such as Broker, come with their own ``CHANGES``.)
Zeek 8.2.0
==========
We'd like to thank ... for their contributions to this release.
Breaking Changes
----------------
- The minimum version requirement for ZeroMQ is now 4.3.0. This version has
been released at the end of 2018 - a good 7 years ago.
New Functionality
-----------------
- Well-known ports of built-in analyzers can now be configured via redef. For example:
redef HTTP::ports += { 8181/tcp };
For protocol analyzers, the variable name is generally ``Analyzer_Module::ports``,
but in certain cases (SSL, QUIC, RDP, ...) the variable name includes the protocol
name to distinguish different internal analyzers. ``SSL::ssl_ports`` and ``SSL::dtls_ports``
as a concrete example. Check the analyzer module's ``export`` section or online documentation.
For packet analyzers, the analyzer name is included in the variable name for historical
reasons. Example:
redef PacketAnalyzer::VXLAN::vxlan_ports += { 4711/udp };
- The number of messages and bytes processed by the central ZeroMQ XPUB/XSUB
cluster component is now exposed as metrics of the form:
zeek_cluster_zeromq_proxy_{frontend,backend}_{bytes,messages}_{sent,received}
by the process running this component in a cluster. Note that Zeek uses multipart
messages for events: Publishing a single event currently counts as four messages
as we're simply exposing ZeroMQ's low-level ``zmq::proxy_steerable()`` statistics.
Changed Functionality
---------------------
- The ``to_subnet()`` built-in function's parsing of IPv6 subnets was changed
to align with how IPv6 subnet literals are parsed in Zeek scripts. Previously,
any subnet containing the ``::ffff:0:0/96`` IPv4-mapped prefix would produce
an IPv4 subnet, even if provided prefix was shorter than the required 96 bits.
Internally, the behavior of the ``IPPrefix::ConvertString()`` API was changed
to accommodate for this.
Deprecated Functionality
------------------------
Zeek 8.1.0
==========
We'd like to thank Andrew Raffman (@andyraf), @arm7mm, Artyom Kalabukhov (@predator89090),
Benjamin Grap (@blightzero), @biswajitutil, Jean-Samuel Marier (@cccs-jsjm), @chrisjlly,
Craig P (@multicast-bytes), @DigiAngel, Mike Dopheide (@dopheide-esnet), Edoardo Mich
(@edoardomich), @franky-m, Fupeng Zhao (@AmazingPP), Jan Grashöfer (@J-Gras), Klemens
Nanni (@klemens-ya), Klemens Nanni (@klemensn), @kshitiz56, Mark Overholser
(@markoverholser), Mohan Dhawan (@Mohan-Dhawan), Peter Cullen (@pbcullen), RyugaXhypeR
(@RyugaXhypeR), Seth Hall (@sethhall), Yacin Nadji (@ynadji), Zhang Liang (@JosenLiang),
Zach Robinette (@zrobinette12) and @zzzdong for their contributions to this release.
Breaking Changes
----------------
- A standalone ``zeek`` invocation on the command line will now default to running a No-op
Cluster backend (``Cluster::CLUSTER_BACKEND_NONE``). Previously this defaulted to Broker
without having any real effect. For a multi-process / cluster deployment of Zeek, a
cluster backend policy script needs to be loaded explicitly.
Zeekctl has been updated to use the ZeroMQ cluster backend by default and should require
no other action. If you encounter issues with ZeroMQ, it is possible to re-enable the
Broker cluster backend by adding the following entry into ``zeekctl.cfg`` and running
`zeekctl deploy``:
ClusterBackend = Broker
Note that ZeroMQ cluster communication is unencrypted and in the clear by default.
If you're running Zeek on a single host, communication happens via localhost only.
When running a multi host Zeek setup, Zeekctl will output the following warning,
informing you about this behavior:
Warning: ZeroMQ cluster backend enabled and multi-node cluster detected (10.0.0.1, 10.0.0.2).
Communication between Zeek nodes using ZeroMQ is currently unencrypted. Use Broker with TLS if this
is concerning to you. ZeroMQ encryption is tracked at https://github.com/zeek/zeek/issues/4432
You may disable this warning by setting the following option in zeekctl.cfg:
cluster_backend_zeromq.disable_unencrypted_warning = 1
Support for opt-in ZeroMQ encryption is targeted for Zeek 8.2. If this is a
requirement for your environment, please report and for the time being continue
using the Broker backend.
If you manage a non-Zeekctl Zeek setup, load
``policy/frameworks/cluster/backend/zeromq`` or
``policy/frameworks/cluster/backend/broker``. If you forget to do so, Zeek will
recognize that ``Cluster::node`` is set (e.g. the ``CLUSTER_NODE`` environment variable
or ``Cluster::node=...`` is set) but no cluster backend was chosen and will terminate
with a fatal error:
fatal error in <...>/main.zeek, line 425: Cluster::node set to 'worker-1', but
Cluster::backend is Cluster::CLUSTER_BACKEND_NONE - please select a cluster backend to
use.
If you used Zeekctl on a single physical system or virtual machine previously, this
should all be transparent to you.
- The Broker-specific ``&backend`` and ``&broker_store`` attributes will result in startup
errors when a non-Broker cluster backend is selected. See the entry about their
deprecation below for more details.
- Python 3.10 is now required for Zeek and all of its associated subprojects.
- The ``&optional`` script attribute will now error when applied to anything that's not a
record field. Previously, this would have surprising behavior.
- The BinPAC, Bifcl, and Gen-ZAM tools have all moved directly into the Zeek repo, which
should ease maintenance on them a bit. They were moved from the ``auxil`` directory to
the ``tools`` directory. Along with this, the ``--gen-zam`` argument for ``configure``
was removed and the internal version will always be used.
- The zeek-af_packet-plugin git submodule was moved directly into the Zeek repo. This used
to live in the ``auxil`` directory, after having moved there from an external plugin.
It is now automatically built as part of the main Zeek build whenever building on Linux.
- The global ``g_frame_stack`` C++ variable was removed. This variable was used to track
the current stack of script-function frames, mostly for the built-in script debugger. We
realized that we were also tracking this information in the global ``call_stack``
variable, used by the script profiler. These two have now been combined together into
``call_stack``. Plugins using ``g_frame_stack`` will need to be adapted to use the other
variable. We opted to not use a deprecation cycle for this due to the complication with
managing the state across both variables.
- The `unique_id` and `unique_id_from` BIFs now return 96-bit UIDs, which makes them
consistent with other UIDs created by Zeek such as connection IDs. They were previously
returning shorter 64-bit UIDs.
- The metrics ``zeek_msgthread_pending_messages_in_buckets`` and
``zeek_msgthread_pending_messages_out_buckets`` had labels that did not conform with
Prometheus' checks via ``promtool``. The labels have been renamed from ``le`` to
``leq``, given that they technically are less-than-or-equal values.
- The ``iosource/Packet.h`` header no longer depends on libpcap being installed and no
longer directly includes the ``pcap.h`` header. This may cause some knock-on effects in
building plugins that were depending on this header and others that it included
indirectly.
- External plugins do not implicitly require OpenSSL and Broker anymore. Starting with
Zeek 8.1, plugins that depend on these libraries are now required to explicitly call
`find_package(OpenSSL REQUIRED)` and pass `OPENSSL_LIBRARIES` as dependencies to the
`zeek_add_plugin()` call:
find_package(OpenSSL REQUIRED)
zeek_add_plugin(
...
INCLUDE_DIRS
${OPENSSL_INCLUDE_DIR}
DEPENDENCIES
${OPENSSL_LIBRARIES}
)
- We are revisiting removing support for MD5 hashing across the Zeek code base. In support
of this, the following changes were made:
- The ``host_key`` field in ssh.log was removed and replaced with a new
``host_key_fingerprint`` field. The old field held an MD5 hash of the host key. The
new field holds a base64-encoded version of the hash of the host key, in the same
format that the ``ssh-keygen -l`` command would return. It currently uses SHA256 for
the hashing, so the output is of the format ``SHA256:<fingerprint>``. A new
``policy/protocols/ssh/md5-host-key-logging.zeek`` script was added to restore the old
field if desired.
- The ``auth_ticket`` and ``new_ticket`` fields in krb.log were removed and replaced
with new ``auth_ticket_sha256`` and ``new_ticket_sha256`` fields. The old fields held
MD5 hashes of the tickets and the new fields hold SHA256 hashes of the same. A new
``policy/protocols/krb/md5-ticket-logging.zeek`` script was added to restore the old
fields if desired.
- A new ``mime_content_hash_sha256`` event was added to send a SHA256 version of the
content in MIME messages. This is similar to the existing ``mime_content_hash`` event
that sends MD5 versions.
- The MD5 IP anonymizers (``RANDOM_MD5`` and ``PREFIX_PRESERVING_MD5``) have been
deprecated in favor of new SHA256 anonymizers.
- The ``md5_hmac`` BIF has been deprecated in favor of a new ``sha256_hmac`` BIF.
New Functionality
-----------------
- After recently-merged fixes to the ``node`` formula for Homebrew, JavaScript support via
ZeekJS can now be built on macOS using that installation of Node.
- A new ``TapAnalyzer`` class was added allowing to tap into all packets delivered to
child analyzers attached to session adapters.
- Two new hooks, ``Cluster::on_subscribe()`` and ``Cluster::on_unsubscribe()`` have been
added to allow observing ``Subscribe()`` and ``Unsubscribe()`` calls on backends by Zeek
scripts.
- The ability to control the length of strings and containers in log output was added. The
maximum length of individual log fields can be set, as well as the total length of all
string or container fields in a single log record. This feature is controlled via four
new script-level variables:
Log::default_max_field_string_bytes
Log::default_max_total_string_bytes
Log::default_max_field_container_elements
Log::default_max_total_container_elements
When one of the ``field`` limits is reached, the individual field is truncated. When one
of the ``total`` limits is reached, all further strings will returned as empty and all
further container elements will not be output. See the documentation for those variables
for more detail.
The above variables control the truncation globally, but they can also be set for log
streams individually. This is controlled by variables with the same names that can be
set when the log stream is created.
Two new weirds were added to report the truncation: ``log_string_field_truncated`` and
``log_container_field_truncated``. New metrics were added to track how many truncations
have occurred: ``zeek_log_writer_truncated_string_fields_total`` and
``zeek_log_writer_truncated_containers_total``. The metrics are reported for each log
stream.
- There exists a new configuration file ``<PREFIX>/etc/zeek/zeek.conf`` and a small
executable called ``zeek-systemd-generator`` that acts as a systemd unit file
generator. These two pieces allow for quickly running an opinionated single node cluster
deployment using systemd. See the ``tools/systemd-generator/README.md`` and
``tools/systemd-generator/etc/zeek/zeek.conf`` files for more information.
If you've previously built Zeek process management by hand or with tools like
supervisord, runit, s6, etc on Linux, this might be a viable option to explore.
This is all Linux specific right now. However, the ``zeek.conf`` file is OS-agnostic to
keep options open for re-use it elsewhere.
- The DNS analyzer now returns the set of parameters for SVCB data. It previously handled
SVCB packets, but omitted the parameters while parsing.
- The QUIC analyzer now raises a ``QUIC::discarded_packet`` event when a packet with
``fixed_bit`` set to 0 is encountered. Such an occurrence is included in the QUIC
history as ``X``. This functionality can be controlled with
``QUIC::max_discarded_packet_events``. Setting this variable to -1 disables the
``QUIC::discarded_packet`` event.
- Added SHA224, SHA384, and SHA512 calculation BiFs: ``sha224_hash``, ``sha224_hash_init`,
``sha224_hash_update``, ``sha224_hash_finish``. ``sha384_hash``, ``sha384_hash_init`,
``sha384_hash_update``, ``sha384_hash_finish``. ``sha512_hash``, ``sha512_hash_init`,
``sha512_hash_update``, and ``sha512_hash_finish``.
- Added SHA224, SHA384 and SHA512 file hash analyzers. The analyzers are not enabled by
default.
- Added the ``TCP::raw_options`` BiF, which returns the list of TCP options in the current
packet, including their raw data.
- For SNMPv3, Zeek now parses the Security Parameters for the User-Based Security
model. If present, the data is added to the new ``user_security_parameters`` field of
the ``SNMP::HeaderV3`` record.
- The QUIC analyzer now raises ``QUIC::short_header_packet_threshold_crossed()`` for every
binary logarithmic threshold for QUIC packets with short headers. Each threshold is
added to the history as ``O`` or ``o``. E.g., a history containing ``oOoO`` indicates
that originator and responder each send at least 3 short packets, while ``oOOoooooo``
would indicate the originator sent at least 3 short packets early in the connection,
while the responder sent at least 127 packets.
- Added SHA256 calculation BiFs: ``sha512_hash``, ``sha512_hash_init`,
``sha512_hash_update``, and ``sha512_hash_finish``.
- Added VLAN PCP and DEI bits when parsing packets. This is exposed alongside ``vlan`` and
``inner_vlan`` in a packet's ``l2_hdr`` when scripting, if a VLAN header was present.
- The DNS analyzer now supports parsing Dynamic Update messages (RFC 2136) and reporting
information about them into ``dns.log``. The ``answer`` field in the log will contain
the prerequisites and updates for the dynamic update requests, whereas the ``query``
field logs the zone name for the update request.
This also adds new ``opcode`` and ``opcode_name`` fields to the DNS log. Previously Zeek
was only handling opcode 0 (normal requests) so that information wasn't pertinent, but
with the addition of dynamic updates we're now handling a second opcode. The new policy
script ``policy/protocols/dns/disable-opcode-log-fields.zeek`` can be loaded to remove
these fields from the log if they are not desired.
As part of these additions, an extra field ``is_netbios`` was added to the the
``dns_msg`` script-layer record. This boolean flag is ``T`` if the DNS message is from
NetBIOS, and ``F`` if the message is standard DNS.
- The `policy/misc/dump-events.zeek` script now features a ``DumpEvents::use_json``
boolean toggle, false by default, to report Zeek's events in JSON format.
- The list of HTTP methods accepted by dynamic protocol detection was updated to add some
missing methods: BIND, LINK, MKCALENDAR, MKREDIRECTREF, REBIND, UNBIND, UNLINK.
- The Docker images generated by Zeek's CI process now contain OCI-approved annotation
labels containing various metadata about the image. See
https://github.com/opencontainers/image-spec/blob/main/annotations.md for more details
about the labels.
- A new packet source was added that supports reading pcapng files. This supports packet
captures with multiple interfaces, packet metadata, and capture file metadata. It will
be automatically used when the magic number in the file header is detected, or if
``pcapng::`` is passed as a prefix to the filename with Zeek's ``-r`` argument. It does
not currently support BPF filtering via Zeek's ``-f`` argument.
Changed Functionality
---------------------
- A Zeekctl-managed Zeek cluster now uses the ZeroMQ cluster backend by default.
- The manager node of a Zeekctl-managed Zeek cluster now listens on 127.0.0.1:27759 for
WebSocket connections by default. This can be disabled with the ``UseWebSocket`` setting
in ``zeekctl.cfg``, or adapted using the ``WebSocketHost`` and ``WebSocketPort``
settings.
- The var-extraction-uri.zeek policy does not include the path in the ``uri_vars`` field
anymore.
- The ``get_current_packet_header()`` BIF now populates the returned record also for
fragmented IP datagrams.
- The QUIC parser discards packets with the ``fixed_bit`` field set to 0, rather than
continuing to parse and potentially running into analyzer violations.
- The Notice framework's suppression implementation was changed to batch suppression
information for up up to 10msec before distributing it to other nodes in a Zeek cluster.
This reduces cluster communication overhead when there's a storm of notices
generated. Two new options, ``Notice::suppression_batch_period`` and
``Notice::suppression_batch_max_size`` can be used to tune this behavior. See their
documentation for details. This most notably reduces CPU usage and cluster event traffic
during cluster startup.
- Non-printable ASCII control characters now always result in ``\uXXXX`` characters when
logging them as JSON. This follows the JSON standard more closely. Previously, they
would have been logged in the ``\xXX`` format that Zeek typically uses. For example, an
HTTP request with a non-printable character in the URI would have a ``uri`` field that
changes from:
"uri":"/non_printable_\\x07"
to
"uri":"/non_printable_\u0007"
- For SNMPv3, the ``community`` field of ``snmp.log`` now contains the SNMPv3 username (if
present). Previously, the field was empty for all SNMPv3 connections.
- The ``x509.log`` deduplication logic has been reimplemented with explicit calls to
``Cluster::publish()``, rather than relying ``&backend`` tables using Broker.
- Sanitizer flags passed to a Zeek build with ``configure``'s ``--sanitizers`` argument
are now automatically passed to plugins built against that Zeek source tree.
- The ``hash-all-files`` policy script now enables SHA256 logging, in addition to MD5 and
SHA1.
- The LDAP analyzer was fixed to correctly escape non-ASCII characters when logging filter
strings.
- The AF_Packet packet source now reports a warning if you attempt to use it on a loopback
interface. This is because AF_PACKET will receive duplicates of every packet when run on
a loopback interface and this can be confusing.
- The GSSAPI analyzer gained some fixes to reliably process Kerberos data when that data
isn't preceded by an OID. This is valid a ASN.1 structure that wasn't being handled
correctly.
- SSL certificates are now correctly logged even when protocol violations occur, assuming
the certificate was encountered prior to the violation.
Deprecated Functionality
------------------------
- The ``&backend`` and ``&broker_store`` attributes for script-layer tables have been
deprecated. These are only functional with the Broker cluster backend and their
semantics and guarantees considered too loose. There is ongoing discussion and
experiments how to provide state propagation and synchronization across cluster
processes. Explicit topic-based state propagation using ``Cluster::publish()``, or
switching to the storage framework for persistence are available today.
If you've used ``&backend`` and ``&broker_store`` in your scripts and find it hard or
inconvenient to replace them, please reach out with your use-cases, on GitHub or
Slack. We're happy to support.
Some references to GitHub discussion and prototypical ideas:
https://github.com/zeek/zeek/discussions/5020
https://github.com/zeek/zeek/pull/5040
- In Zeek 9.1, returning a value from a hook will become an error. Return values from a
hooks were previously silently discarded. This behavior was found surprising in relation
with the ``break`` statement available in hooks.
- The ``zeek::util::json_escape_utf8`` methods are being replaced by a new ``escape_utf8``
method in order to fix a bug with how the ``escape_printable_controls`` argument was
implemented. Previously, it was implemented in the reverse of what was intuitive given
the name. It would escape the control characters when false, and not escape them when
true. The new method has the correct semantics, as well as a new argument for
controlling whether non-printable control characters are escaped.
- The ``zeromq/connect`` script has been deprecated. Load the
``frameworks/cluster/backend/zeromq`` package directly instead.
- The ``rfb_auth_result`` event from the RFB analyzer was deprecated and replaced with a
new ``rfb_authentication_result`` event. This new event sets the correct value in the
``rfb$auth`` field in the connection.
Zeek 8.0.0
==========
We would like to thank @aidans111, Anthony Verez (@netantho), Baa (@Baa14453),
Bhaskar Bhar (@bhaskarbhar), @dwhitemv25, EdKo (@ephikos), @edoardomich, Fupeng
Zhao (@AmazingPP), hendrik.schwartke@os-s.de (@hendrikschwartke), @i2z1, Jan
Grashöfer (@J-Gras) Jean-Samuel Marier, Justin Azoff (@JustinAzoff), Mario D
(@mari0d), Markus Elfring (@elfring), Peter Cullen (@pbcullen), Sean Donaghy,
Simeon Miteff (@simeonmiteff), Steve Smoot (@stevesmoot), @timo-mue,
@wojciech-graj, and Xiaochuan Ye (@XueSongTap) for their contributions to this
release.
Breaking Changes
----------------
- Zeek by default now depends on the availability of the ZeroMQ library for building
and running. This is in preparation of switching to the ZeroMQ-based cluster backend
by default in future Zeek versions. On an Ubuntu based system, the required system
packages are ``libzmq5``, ``libzmq3-dev`` and ``cppzmq-dev``. See the Dockerfiles
in the ``ci/`` directory for other supported platforms.
- Zeek and all of its associated submodules now require C++20-capable compilers to
build. This will let us move forward in using more modern C++ features and replace some
workarounds that we have been carrying. Minimum recommended versions of compilers are
GCC 10, Clang 8, and Visual Studio 2022.
- The ``zeek::Span`` class has been deprecated and the APIs in the telemetry subsystem
switched to use ``std::span`` instead of ``zeek::Span``. If your plugin instantiates
counter or gauge instances using the telemetry subsystem and you've previously used
``zeek::Span`` explicitly, updates may be needed.
- The code base underwent a big cleanup of #include usage, across almost all of the
files. We tested builds of all of the existing third-party packages and only noticed one
or two failures, but there is a possibility for breakage related to this cleanup.
- The ``lookup_connection()`` and ``connection_exists()`` builtin functions
now require ``conn_id`` instances as argument, rather than internally supporting
duck type matching ``conn_id``-like records.
- Network timestamps are not added to events by default anymore. Use the following
redef line to enable them:
redef EventMetadata::add_network_timestamp = T;
The background is that event metadata has become more generic and may incur
a small overhead when enabled. There's not enough users of network timestamp
metadata to justify the complexity of treating it separate.
- The ASCII writer's ``JSON::TS_MILLIS`` timestamp format was changed to produce
signed integers. This matters for the representation for timestamps that are
before the UNIX epoch. These are now written as negative values, while previously
the negative value was interpreted as an unsigned integer, resulting in very large
timestamps, potentially causing issues for downstream consumers.
If you prefer to always have unsigned values, it's possible to revert to the previous
behavior by setting:
redef LogAscii::json_timestamps = JSON::TS_MILLIS_UNSIGNED;
- The "endpoint" label of metrics exposed via Prometheus or the ``telemetry.log``
was renamed to "node". This is done for consistency with cluster terminology:
The label values have always been the value of ``Cluster::node`, so it's more intuitive
to call it. The "endpoint" name originated from a time when the telemetry framework
was implemented in Broker.
To revert to the "endpoint" label, you can do the following, but we strongly
suggest to migrate to the new default "node" instead:
redef Telemetry::metrics_endpoint_label = "endpoint";
- The ``current_event_time()`` builtin function as well as ``Event::Time()``
and ``EventMgr::CurrentEventTime()`` now return ``-1.0`` if no timestamp
metadata is available for the current event, or if no event is being
dispatched. Previously this would've been 0.0, or the timestamp of the previously
dispatched event.
- Missing network timestamp metadata on remote events is not set to the local
network time anymore by default. This potentially hid useful debugging information
about another node not sending timestamp metadata. The old behavior can be
re-enabled as follows:
redef EventMetadata::add_missing_remote_network_timestamp = T;
- The ``IsPacketSource()`` method on ``IOSource`` was removed. It was unused
and incorrectly returned ``false`` on all packet sources.
- The ``--with-binpac`` and ``--with-bifcl`` arguments for ``configure`` are now
deprecated. Both arguments have for a long time just used the internal version of the
tooling even if something was passed, so they were mostly useless. This may cause
breakage of cross-compiling, where the ``binpac`` and ``bifcl`` tooling needs to be run
on the host machine. We haven't heard from anyone that this is the case with the
arguments in their currently-broken state.
- The parsing of data for the ``ssl_session_ticket_handshake`` event was fixed.
In the past, the data contained two extra bytes before the session ticket
data. The event now contains only the session ticket data. You might have to
adjust your scripts if you manually worked around this bug in the past.
New Functionality
-----------------
- Zeek now supports pluggable and customizable connection tracking. The default
behavior remains unchanged and uses a connection's five tuple based on the
IP/port pairs and proto field. Zeek 8 ships with one additional implementation,
to factor VLAN tags into the connection tracking. To switch to VLAN-aware
connection tracking:
@load frameworks/conn_key/vlan_fivetuple
By convention, additional fields used by alternative ConnKey implementations are
added into the new ``ctx`` field of ``conn_id``. The type of ``ctx`` is ``conn_id_ctx``.
The ``vlan_fivetuple`` script adds two additional fields to the ``conn_id_ctx``
record type, representing any VLAN tags involved. Accordingly, every log
using ``conn_id`` reflects the change as well as ``ctx`` and the VLAN fields have
the ``&log`` attribute. The columns used for logging will be named ``id.ctx.vlan``
and ``id.ctx.inner_vlan``.
This feature does not automatically provide a notion of endpoint that
corresponds with the effective connection tuple. For example, applications tracking
endpoints by IP address do not somehow become VLAN-aware when enabling
VLAN-aware tracking.
Users may experiment with their own notion of endpoint by combining the ``orig_h``
or ``resp_h`` field of ``conn_id`` with the new ``ctx`` field. For example, tracking
the number of connections from a given host in a VLAN-aware fashion can be done
as follows:
global connection_counts: table[conn_id_ctx, addr] of count &default=0;
event new_connection(c: connection) {
++connection_counts[c$id$ctx, c$id$orig_h];
}
Note that this script snippet isn't VLAN-specific, yet it is VLAN-aware if the
``vlan_fivetuple`` script is loaded. In future Zeek versions, this pattern is
likely to be used to adapt base and policy scripts for more "context awareness".
Users may add their own plugins (for example via a zkg package) to provide
alternative implementations. This involves implementing a factory for
connection "keys" that factor in additional flow information. See the VLAN
implementation in the ``src/packet_analysis/protocol/ip/conn_key/vlan_fivetuple``
directory for an example.
- Added support to ZeekControl for seamlessly switching to ZeroMQ as cluster
backend by adding the following settings to zeekctl.cfg:
ClusterBackend = ZeroMQ
UseWebSocket = 1
With the ZeroMQ cluster backend, Zeekctl requires to use Zeek's WebSocket API
to communicate with individual nodes for the ``print`` and ``netstats`` commands.
Setting the ``UseWebSocket`` option enables a WebSocket server on the manager
node, listening on 127.0.0.1:27759 by default (this is configurable with using
the newly introduced ``WebSocketHost`` and ``WebSocketPort`` options).
The ``UseWebSocket`` option can also be used when ``ClusterBackend`` is set
to ``Broker``, but isn't strictly required.
For ZeroMQ (or other future cluster backends), setting ``UseWebSocket`` is a
requirement as Zeekctl does not speak the native ZeroMQ protocol to communicate
with cluster nodes for executing commands. This functionality requires the
``websockets`` Python package with version 11.0 or higher.
- Cluster telemetry improvements. Zeek now exposes a configurable number of
metrics regarding outgoing and incoming cluster events. By default, the number
of events sent and received by a Zeek cluster node and any attached WebSocket
clients is tracked as four individual counters. It's possible to gather more
detailed information by adding ``Cluster::Telemetry::VERBOSE`` and
``Cluster::Telemetry::DEBUG`` to the variables ``Cluster::core_metrics`` and
``Cluster::webscoket_metrics``:
redef Cluster::core_metrics += { Cluster::Telemetry::VERBOSE };
redef Cluster::websocket_metrics += { Cluster::Telemetry::DEBUG };
Configuring verbose, adds metrics that are labeled with the event handler
and topic name. Configuring debug, uses histogram metrics to additionally track
the distribution of the serialized event size. Additionally, when debug is selected,
outgoing events are labeled with the script location from where they were published.
- Support for the X-Application-Name HTTP header was added to the WebSocket API at
``v1/messages/json``. A WebSocket application connecting to Zeek may set the
X-Application-Name header to a descriptive identifier. The value of this header
will be added to the cluster metrics as ``app`` label. This allows to gather
incoming and outgoing event metrics of a specific WebSocket application, simply
by setting the X-Application-Name header.
- The SMTP analyzer can now optionally forward the top-level RFC 822 message individual
SMTP transactions to the file analysis framework. This can be leveraged to extract
emails in form of ``.eml`` files from SMTP traffic to disk.
To enable this feature, set the ``SMTP::enable_rfc822_msg_file_analysis`` option
and implement an appropriate ``file_new()`` or ``file_over_new_connection()`` handler:
redef SMTP::enable_rfc822_msg_file_analysis = T;
event file_over_new_connection(f: fa_file, c: connection, is_orig: bool) {
if ( f$id == c$smtp$rfc822_msg_fuid )
Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename="email"]);
}
- Generic event metadata support. A new ``EventMetadata`` module was added allowing
to register generic event metadata types and accessing the current event's metadata
using the functions ``current()`` and ``current_all()`` of this module.
- A new plugin hook, ``HookPublishEvent()``, has been added for intercepting
publishing of Zeek events. This hook may be used for monitoring purposes,
modifying or rerouting remote events.
Plugins can implement and enable this hook by calling the following method
within their Configure() implementation.
EnableHook(HOOK_PUBLISH_EVENT)
The signature of ``HookPublishEvent()`` is as follows.
bool HookPublishEvent(zeek::cluster::Backend& backend,
const std::string& topic,
zeek::cluster::detail::Event& event);
- Zeek now includes the Redis protocol analyzer from the evantypanski/spicy-redis
project (https://github.com/evantypanski/spicy-redis). This analyzer is enabled
by default. This analyzer logs Redis commands and their associated replies in
``redis.log``.
To disable the analyzer in case of issues, use the following snippet:
redef Analyzer::disabled_analyzers += {
Analyzer::ANALYZER_REDIS,
};
- The FTP analyzer now supports explicit TLS via AUTH TLS.
- Two new script-level hooks in the Intel framework have been added.
hook indicator_inserted(indicator_value: string, indicator_type: Intel::Type)
hook indicator_removed(indicator_value: string, indicator_type: Intel::Type)
These are reliably invoked on worker and manager nodes the first time an
indicator value is inserted into the store and once it has been completely
removed from the store.
- The ``frameworks/intel/seen`` scripts have been annotated with event groups
and a new ``frameworks/intel/seen/manage-event-groups`` policy script added.
The motivation is to allow Zeek distributors to load the ``intel/seen`` scripts
by default without incurring their event overhead when no Intel indicators are
loaded. Corresponding event handlers are enabled once the first Intel indicator
of a given ``Intel::Type`` is added. Event handlers are disabled when the last
indicator is removed, again.
Note that the ``manage-event-groups`` script interacts with the ``Intel::seen_policy``
hook: If no indicators for a given ``Intel::Type`` are loaded, the ``Intel::seen_policy``
will not be invoked as the event handlers extracting indicators aren't executed.
If you rely on the ``Intel::seen_policy`` hook to be invoked regardless of the
contents of the Intel store, do not load the ``manage-event-groups`` or set:
redef Intel::manage_seen_event_groups = F;
- The DNS analyzer was extended to support NAPTR RRs (RFC 2915, RFC 3403).
A corresponding ``dns_NAPTR_reply`` event was added.
- A new ``get_tags_by_category`` BIF method was added that returns a list of tags for a
specified plugin category. This can be used in lieu of calling ``zeek -NN`` and
parsing the output. For example, this will return the list of all analyzer plugins
currently loaded:
get_tags_by_category("ANALYZER");
- A new ``conn_generic_packet_threshold_crossed`` event was introduced. The event triggers
for any IP-based session that reaches a given threshold. Multiple packet thresholds can
be defined in ``ConnThreshold::generic_packet_thresholds``. The generic thresholds refer
to the total number of packets on a connection without taking direction into account
(i.e. the event also triggers on one-sided connections).
The event is intended as an alternative to the ``new_connection`` event that allows for
ignoring short-lived connections like DNS or scans. For example, it can be used to set
up traditional connection monitoring without introducing overhead for connections that
would never reach a larger threshold anyway.
- Zeek now supports extracting the PPPoE session ID. The ``PacketAnalyzer::PPPoE::session_id``
BiF can be used to get the session ID of the current packet.
The ``conn/pppoe-session-id-logging.zeek`` policy script adds pppoe session IDs to the
connection log.
The ``get_conn_stats()`` function's return value now includes the number of packets
that have not been processed by any analyzer. Using data from ``get_conn_stats()`` and
``get_net_stats()``, it's possible to determine the number of packets that have
been received and accepted by Zeek, but eventually discarded without processing.
Changed Functionality
---------------------
- The `Conn::set_conn` function is now always run in `new_connection`, instead of only
being run in `connection_state_remove`.
- Logging of failed analyzers has been overhauled. `dpd.log` was replaced
by a new `analyzer.log` that presents a more unified and consistent view
of failed analyzers. The previous `analyzer.log` was renamed to `analyzer-debug.log`;
see below for more details.
For protocol analyzers, `analyzer.log` now reports initially confirmed analyzers that
Zeek subsequently removed from the connection due to a protocol violation.
For file and packet analyzers, all errors will be logged to `analyzer.log`.
As part of this work, a new `analyzer_failed` event has been introduced. This event
is raised when an analyzer is removed because of raising a violation.
- `analyzer.log` was renamed to `analyzer_debug.log`, and is no longer created
by default. The log file will be created if the `frameworks/analyzer/debug-logging.zeek`
policy script is loaded.
Note that the namespace for options in the script changed to
`Analyzer::DebugLogging`. Furthermore the default options changed to enable
more detailed output by default.
- Record fields with a ``&default`` attribute are now consistently re-initialized
after deleting such fields. Previously, this would only work for constant
expressions, but has been extended to apply to arbitrary expressions.
- Publishing remote events with vector arguments that contain holes is now
rejected. The receiver side never had a chance to figure out where these
holes would have been. There's a chance this breaks scripts that accidentally
published vectors with holes. A reporter error is produced at runtime when
serialization of vectors with holes is attempted.
- Kerberos support on macOS has been enabled. Due to incompatibilities, the system
provided libkrb5 is ignored, however. Only versions from homebrew are supported and
found/picked-up by default. Use --with-krb5 for pointing at a custom librkb5
installation.
- The ``$listen_host`` configuration for ``Cluster::listen_websocket()``'s
``WebSocketServerOptions`` was deprecated. Use the new ``$listen_addr`` field
instead.
- The `service_violation` field of the connection record was marked as deprecated.
Consider using the new `failed_analyzers` field of the connection record instead.
- `detect-protocol.zeek was the last non-deprecated policy script left in
`frameworks/dpd`. It was moved to `frameworks/analyzer/detect-protocol.zeek`.
- Running Zeek with Zeekygen for documentation extraction (-X|--zeekygen
<cfgfile>) now implies -a, i.e., parse-only mode.
- The `not_valid_before` and `not_valid_after` times of X509 certificates are
now logged as GMT timestamps. Before, they were logged as local times; thus
the output was dependent on the timezone that your system is set to.
Similarly, the related events and the Zeek data structures all interpreted
times in X509 certificates as local times.
- The PPPoE parser now respects the size value given in the PPPoE header. Data
beyond the size given in the header will be truncated.
- Record fields with ``&default`` attributes initializing empty ``vector``, ``table``
or ``set`` instances are now deferred until they are accessed, potentially
improving memory usage when such fields are never accessed.
- The handling of truncated 802.3 Ethernet frames changed. Before, truncated 802.3 frames
were always discarded. Now, truncated 802.3 Ethernet frame data is forwarded to downstream
analyzers. This aligns the handling of 802.3 Ethernet frames with the (much more common)
Ethernet 2 frames which are forwarded when truncated.
Due to this change, the ``truncated_vlan_frame`` was removed. 802.3 frames with a truncated
header now raise the ``truncated_VLAN_header`` weird, which is similarly raised by Ethernet 2
frames with a truncated VLAN header.
- Data beyond the length indicated in the header of an 802.3 Ethernet frame is no longer
forwarded to downstream analyzers. This aligns our handling of 802.3 Ethernet frames with
other protocols that have length headers.
Removed Functionality
---------------------
- The ``--with-bind`` argument for ``configure`` was removed. We removed the need for the
BIND library from our CMake setup in the v7.2 release, but this non-functional argument
was left behind.
- The ``--disable-archiver`` argument for ``configure`` was removed. This was deprecated
and scheduled to be removed in v7.1, but we apparently missed it during the cleanup for
that release.
Deprecated Functionality
------------------------
- The `dpd.log` is now deprecated and replaced by `analyzer.log` (see above).
`dpd.log` is no longer created by default, but can be loaded using the
`frameworks/analyzer/deprecated-dpd-log.zeek` policy script.
Relatedly, the `service_violation` field of the connection record is
deprecated and will only be present if the
`frameworks/analyzer/deprecated-dpd-log.zeek` policy script is loaded.
- The ``protocols/http/detect-sqli.zeek`` script has been deprecated in favor of a
new ``protocols/http/detect-sql-injection.zeek`` script to switch from the victim
host being placed into the ``src`` field of a notice to instead use ``dst``.
The attacker host is now placed into ``src``. Further, notices hold the first
sampled connection uid.
Note that the ``Notice::Type`` enumeration names remain the same. You can determine
which script was used by the presence of populated ``uid`` and ``dst`` fields in the
``notice.log`` entries.
The replacement script doesn't populate the ``email_body_sections`` anymore either.
- Using ``&default`` and ``&optional`` together on a record field has been deprecated
as it would only result in ``&default`` behavior. This will become an error starting
with Zeek 8.1.
- The ``zeek::Event()`` constructor was deprecated. Use ``event_mgr::Enqueue()``
or ``event_mgr::Dispatch()`` instead.
- Passing ``ts`` as the last argument to ``EventMgr::Enqueue()`` has been deprecated
and will lead to compile time warnings. Use ``EventMgr::Enqueue(detail::MetadataVectorPtr meta, ...)``
for populating ``meta`` accordingly.
- For plugin authors: in the core, the constructor for Connection instances has
been deprecated in favor of a new one to support pluggable connection
tuples. The ConnTuple struct, used by this deprecated Connection constructor,
is now deprecated as well.
- The ``zeek::filesystem`` namespace alias is deprecated in favor of using
``std::filesystem`` directly. Similarly, the ``ghc::filesystem`` submodule stored in
``auxil/filessytem`` has been removed and the files included from it in the Zeek
installation will no longer be installed. Builds won't warn about the deprecation of
``zeek::filesystem`` due to limitations of how we can mark deprecations in C++.
- The ``zeek::util::starts_with`` and ``zeek::util::ends_with`` functions are deprecated.
``std::string`` and ``std::string_view`` added ``begins_with`` and ``ends_with`` methods
in C++ 20, and those should be used instead.
- The ``record_type_to_vector`` BIF is deprecated in favor of using the newly ordered
``record_fields`` BIF.
Zeek 7.2.0
==========
We would like to thank Aashish Sharma (@initconf), Anthony Verez (@netantho), Anthony
Kasza (@anthonykasza), @biswajitutil, Brendan Kapp (@BrendanKapp), Carlos Lopez, Chris
Hinshaw (@MMChrisHinshaw), Faan Rossouw (@faanross), @FishyFluffer, Fupeng Zhao
(@AmazingPP), Herbert (@Herbert-Karl), @jbaggs, Jan Grashöfer (@J-Gras), Julian Krieger
(@juliankrieger), Justin Azoff (@JustinAzoff), Kshitiz Bartariya (@kshitiz56), @Laotree,
Mark Overholser (@markoverholser), Mike Dopheide (@dopheide-esnet), @mnhsrj, Mohan Dhawan
(@Mohan-Dhawan), @philipp-tg, Seth Hall (@sethhall), and @timo-mue for their contributions
to this release.
Breaking Changes
----------------
- The ``is_remote_event()``, ``current_analyzer()`` and ``current_event_time()`` builtin
functions do not return the previous event's values anymore when event draining has
completed. The same applies to the corresponding C++ accessors on the ``EventMgr``
class. The functions now return false, 0 or the zero time instead.
- The ``to_int()`` built-in function was changed to match the return behavior of
``to_count()``. Previously, ``to_int()`` would silently ignore invalid inputs and return a
``0``. It now returns an error instead.
New Functionality
-----------------
- The following dependencies have had updates:
- The bundled version of c-ares has been updated to v1.34.5.
- The bundled version of ZeekJS has been updated to v0.17.0.
- Some DNS events are not raised when ``dns_skip_all_addl`` is set to true. Zeek now
raises a warning when a script declares these events while this option is set to true.
- Types can now be used as constants in Zeek script. This allows types to be directly
passed into BIFs without aliasing.
- A new ``enc_part`` field was added to the Kerberos ``KRB_Response`` record passed as
part of the ``krb_as_response`` event. This field contains the encrypted session
information from a Kerberos response, including the cipher and encrypted data.
- Geneve tunnel options of the current packet can be extracted from scripts using the new
``PacketAnalyzer::Geneve::get_options()`` builtin function.
- The new ``is_valid_subnet()`` function mirrors ``is_valid_ip()``, for subnets.
- A new Storage framework was merged into the Zeek tree. The intention with this framework
is to eventually replace the storage functionality that Broker provides, including
direct storage via calls such as ``Cluster::create_store`` and ``Broker::put_unique`` as
well as storage-backed tables via the ``&backend`` attribute. This is an initial version
for testing, and will be expanded upon in the future. The current state of the framework
is as follows:
- A new API was added for storage backend plugins.
- Script-level functions for opening and closing backends, and insertion, retrieval, and
erasure of elements are available.
- Backends can support both asynchronous mode (using ``when`` statements) and
synchronous mode (blocking until the operation completes). BIF methods were added
under new ``Storage::Async`` and ``Storage::Sync`` modules for these two modes. The
modes can be used interchangeably with the same backend handle.
- SQLite and Redis backends exist in the Zeek tree by default. We are working on a
backend for NATS that will be available as an external plugin, but it is not quite
ready yet. Both of the existing backends support usage in a cluster environment.
- Improved alternative cluster backend support.
The ZeroMQ cluster backend added in Zeek 7.1 has received various correctness,
performance and robustness fixes, particularly concerning shutdown and high-load
scenarios.
Initial performance testing indicates less CPU time used on a large single node
instance with high logging and eventing rates.
We're evaluating switching the default cluster backend from Broker to ZeroMQ With
Zeek 8.1. Therefore, we welcome early adopters and testers to validate ZeroMQ as an
alternative to Broker. If you're not using Broker specific integrations (e.g. Broker's
Python or C++ bindings) and run a single-node Zeek cluster, switching to ZeroMQ
should be as simple as loading the following script on each of cluster node.
@load frameworks/cluster/backend/zeromq/connect
A proof-of-concept plugin for the open-source NATS messaging system is available at
https://github.com/zeek/zeek-cluster-backend-nats for testing and experimentation.
- Broker now exposes more information through ``broker.log``. Broker generated log
messages are now propagated as events to Zeek. This allows exposing more information for
debugging and operational behavior of Broker via Zeek logs. Two new script-level
options ``Broker::log_severity_level`` and ``Broker::log_stderr_severity_level`` have
been introduced to control the which events to expose by default.
- Broker's new per-peer send buffer backpressure handling, introduced in 7.1,
has received several updates. We've increased the default buffer sizes to 8192
messages for both peers and websockets, and switched the default overflow
handling policy to "drop_oldest", meaning that in a full buffer the oldest
message enqueued gets dropped to allow enqueuing a new one. Three additional
metrics are available to understand the health of each peering's buffer,
regardless of the overflow policy active. These are:
- zeek_broker_peer_buffer_messages: a gauge of the current buffer fill level,
- zeek_broker_peer_buffer_recent_max_messages: a gauge that tracks the maximum
buffer fill level seen over the last ``Broker::buffer_stats_reset_interval`.
- zeek_broker_peer_buffer_overflows_total: a counter that tracks the number
of times a given peering's send buffer has overflowed. For the "drop_oldest"
and "drop_newest" policies, this is the count of messages dropped.
Each of these is labeled with the current endpoint and the peer's, as provided
by the cluster topology.
- New WebSocket functionality was added to Zeek's cluster component.
Users of Broker's WebSocket interface should replace their ``Broker::listen_websocket()``
usage with ``Cluster::listen_websocket()``. The latter will support any cluster
backends, while ``Broker::listen_websocket()`` is specific to Broker.
A crucial difference between these two methods is that ``Cluster::listen_websocket()``
will have TLS disabled by default, while Broker usually defaults to TLS enabled.
If you require a TLS configuration for your WebSocket deployment, pass an appropriate
``WebSocketTLSOptions`` record or setup a TLS reverse proxy in front of Zeek.
For WebSocket clients, there should not be an observable difference in behavior
regarding the handshake and publishing of events. However, the implementation uses
a different code path and supporting library (machinezone/IXWebSocket). If you
observe any differences in behavior, please report these as regressions.
Note that the new ``Cluster::listen_websocket()`` API will only become stable
with Zeek 8.0.
Two new events, ``Cluster::websocket_client_added()`` and ``Cluster::websocket_client_lost()``,
have been added for WebSocket clients connecting and disconnecting. Note that
currently, even after ``Cluster::websocket_client_lost()`` ran, events sent from
that client may still be in transit and later executed, even on the node running
the WebSocket server.
- Vectors containing ``pattern`` values can now be compared using ``==`` and ``!=`` in
scripts. This previously resulted in a fatal error.
- The set of non-routable subnets defined in ``Site::private_address_space`` was expanded
to include ``239.0.0.0/8``, ``224.0.0.0/24`, ``[2002:e000::]/40``, ``[2002:ef00::]/24``,
and ``[fec0::]/10`. These addresses come from RFCs 2365, 3058, 3879, and 5771. This may
result in traffic being considered as local traffic that wasn't previously.