[pkgsrc/trunk]: pkgsrc/net/ntopng ntopng: updated to 4.2

[adam <adam%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/17ea8f314dc2
branches:  trunk
changeset: 454869:17ea8f314dc2
user:      adam <adam%pkgsrc.org@localhost>
date:      Wed Jun 23 19:34:15 2021 +0000

description:
ntopng: updated to 4.2

4.2 Stable

Breakthroughs

Flexible Alert Handling
Added recipients and endpoints to send alerts to different recipients on different channels, including email, Discord, Slack and Elasticsearch
Initial SCADA protocol support
Many internal components of ntopng have been rewritten in order to improve the overall ntopng performance, reduce system load, and capable of processing more data while reducing memory usage with 
respect to 4.0.
Cybersecurity extensions have been greatly enhanced by leveraging on the latest nDPI enhancements that enabled the creation of several user scripts able to supervise many security aspects of modern 
systems.
Behavioral traffic analysis and lateral traffic movement detection for finding cybersecurity threats in traffic noise.
Initial Scada support with native IEC 60870-5-104 support. We acknowledge switch.ch for having supported this development.
Consolidation of Suricata and external alerts integration to further open ntopng to the integration of commercial security devices.
SNMP support has been enhanced in terms of speed, SNMPv3 protocol support, and variety of supported devices.
New REST API that enabled the integration of ntopng with third party applications such as CheckMK.

New features

Traffic Behavioral Analysis
Periodic Traffic
Lateral Movements
TLS with self-signed certificates, issuerDN, subjectDN
Support for Industrial IOT and Scada with modbus, DNP3 and IEC60870
Support for attack mitigation via SNMP
Active monitoring
Support for ICMP v4/v6, HTTP, HTTPS and Speedtest
Ability to generate alerts upon unreachable or slow hosts or services
Detection of unexpected servers
DHCP, NTP, SMTP, DNS
Services map
nIndex direct to maximixe flows dump performance
MacOS package

Improvements

Implements per-category indicator of compromise score
Flexible configuration import/export/reset
Ability to import/export/reset all the ntopng configurations or parts of it
Increased nIndex dump throughput by a factor 10
Increased user scripts execution throughput
Massive cleanup/simplifications of plugins to ease community contributions
Improved cardinality estimation (e.g., number of contacted hosts, number of contacted ports) using Hyper-Log-Log
Added DSCP information
Reworked handling of dissected virtual hosts to improve speed and reduce memory

nEdge

Support for hardware bypass

Fixes

Fixed race conditions in view interfaces
Fixed crash when restoring serialized hosts in memory
Fixed conditions causing high CPU load
Fixes CSRF vulnerabilities when POSTing JSON
Fixes heap-use-after-free on HTTP dissected last_url

diffstat:

 net/ntopng/Makefile                                       |    9 +-
 net/ntopng/PLIST                                          |  812 ++++++++++---
 net/ntopng/distinfo                                       |   16 +-
 net/ntopng/patches/patch-configure.seed                   |   32 +-
 net/ntopng/patches/patch-src_Flow.cpp                     |   15 -
 net/ntopng/patches/patch-src_SyslogCollectorInterface.cpp |   15 -
 net/ntopng/patches/patch-src_Utils.cpp                    |   17 +-
 7 files changed, 623 insertions(+), 293 deletions(-)

diffs (truncated from 1378 to 300 lines):

diff -r 5c1219efd9bb -r 17ea8f314dc2 net/ntopng/Makefile
--- a/net/ntopng/Makefile       Wed Jun 23 19:31:49 2021 +0000
+++ b/net/ntopng/Makefile       Wed Jun 23 19:34:15 2021 +0000
@@ -1,7 +1,6 @@
-# $NetBSD: Makefile,v 1.51 2021/05/14 12:57:27 nia Exp $
+# $NetBSD: Makefile,v 1.52 2021/06/23 19:34:15 adam Exp $
 
-DISTNAME=      ntopng-4.0
-PKGREVISION=   6
+DISTNAME=      ntopng-4.2
 CATEGORIES=    net
 MASTER_SITES=  ${MASTER_SITE_GITHUB:=ntop/}
 
@@ -44,10 +43,10 @@
 CHECK_PORTABILITY_SKIP=        third-party/zeromq-4.1.7/configure
 
 pre-configure:
-       cd ${WRKSRC} && cp configure.seed configure.ac && autoreconf -fiv
+       cd ${WRKSRC} && ${CP} configure.seed configure.ac && autoreconf -fiv
 
 post-build:
-       find ${WRKSRC}/httpdocs -type f -print0 | xargs -0 chmod a-x
+       ${FIND} ${WRKSRC}/httpdocs -type f -print0 | xargs -0 chmod a-x
 
 .include "../../databases/hiredis/buildlink3.mk"
 .include "../../databases/rrdtool/buildlink3.mk"
diff -r 5c1219efd9bb -r 17ea8f314dc2 net/ntopng/PLIST
--- a/net/ntopng/PLIST  Wed Jun 23 19:31:49 2021 +0000
+++ b/net/ntopng/PLIST  Wed Jun 23 19:34:15 2021 +0000
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.5 2020/05/25 20:26:51 adam Exp $
+@comment $NetBSD: PLIST,v 1.6 2021/06/23 19:34:15 adam Exp $
 bin/ntopng
 man/man8/ntopng.8
 share/ntopng/httpdocs/bootstrap-4.4.0-dist/css/bootstrap-grid.css
@@ -21,6 +21,9 @@
 share/ntopng/httpdocs/bootstrap-4.4.0-dist/js/bootstrap.js.map
 share/ntopng/httpdocs/bootstrap-4.4.0-dist/js/bootstrap.min.js
 share/ntopng/httpdocs/bootstrap-4.4.0-dist/js/bootstrap.min.js.map
+share/ntopng/httpdocs/bootstrap-custom/ntopng-theme.css
+share/ntopng/httpdocs/css/Makefile
+share/ntopng/httpdocs/css/README.md
 share/ntopng/httpdocs/css/Rickshaw/detail.css
 share/ntopng/httpdocs/css/Rickshaw/graph.css
 share/ntopng/httpdocs/css/Rickshaw/legend.css
@@ -29,7 +32,6 @@
 share/ntopng/httpdocs/css/bootstrap3-datetimepicker.css
 share/ntopng/httpdocs/css/bubble-chart.css
 share/ntopng/httpdocs/css/cal-heatmap.css
-share/ntopng/httpdocs/css/clean-css.md
 share/ntopng/httpdocs/css/cubism.css
 share/ntopng/httpdocs/css/custom_theme.css
 share/ntopng/httpdocs/css/dark-mode.css
@@ -52,6 +54,8 @@
 share/ntopng/httpdocs/css/images/ui-icons_ffd27a_256x240.png
 share/ntopng/httpdocs/css/images/ui-icons_ffffff_256x240.png
 share/ntopng/httpdocs/css/jquery-ui.css
+share/ntopng/httpdocs/css/minified/bootstrap-orange.min.css
+share/ntopng/httpdocs/css/minified/dark-mode.min.css
 share/ntopng/httpdocs/css/minified/fontawesome-custom.min.css
 share/ntopng/httpdocs/css/minified/heatmap.min.css
 share/ntopng/httpdocs/css/minified/ntopng.min.css
@@ -63,6 +67,7 @@
 share/ntopng/httpdocs/css/pie-chart.css
 share/ntopng/httpdocs/css/rickshaw.css
 share/ntopng/httpdocs/css/sequence_sunburst.css
+share/ntopng/httpdocs/css/sidebar.css
 share/ntopng/httpdocs/css/tablesorted.css
 share/ntopng/httpdocs/css/timeline.css
 share/ntopng/httpdocs/css/timepicker.css
@@ -1756,10 +1761,10 @@
 share/ntopng/httpdocs/img/network_icon.png
 share/ntopng/httpdocs/img/os_icon.png
 share/ntopng/httpdocs/img/throbber.gif
+share/ntopng/httpdocs/img/virustotal.svg
 share/ntopng/httpdocs/img/warning.png
 share/ntopng/httpdocs/inc/as_stats_id.inc
 share/ntopng/httpdocs/inc/country_stats_id.inc
-share/ntopng/httpdocs/inc/flows_stats_id.inc
 share/ntopng/httpdocs/inc/geolocation_disclaimer.inc
 share/ntopng/httpdocs/inc/host_to_server_id.inc
 share/ntopng/httpdocs/inc/host_to_server_top.inc
@@ -1823,34 +1828,66 @@
 share/ntopng/httpdocs/js/notification-daemon.js
 share/ntopng/httpdocs/js/ntop.min.js
 share/ntopng/httpdocs/js/ntop.min.js.map
-share/ntopng/httpdocs/js/ntopng_utils.js
+share/ntopng/httpdocs/js/ntop_utils.js
 share/ntopng/httpdocs/js/ntopng_validators.js
 share/ntopng/httpdocs/js/nv.d3.js
 share/ntopng/httpdocs/js/osm-maps.js
+share/ntopng/httpdocs/js/pages/datasource_list.js
+share/ntopng/httpdocs/js/pages/endpoint-notifications-list.js
+share/ntopng/httpdocs/js/pages/flowdevice_details.js
+share/ntopng/httpdocs/js/pages/flowdevices_stats.js
+share/ntopng/httpdocs/js/pages/import-export-config.js
+share/ntopng/httpdocs/js/pages/manage-configurations.js
+share/ntopng/httpdocs/js/pages/pools/host-members.js
+share/ntopng/httpdocs/js/pages/pools/pools.js
+share/ntopng/httpdocs/js/pages/recipients-endpoints.js
+share/ntopng/httpdocs/js/pages/sflowdevice_details.js
+share/ntopng/httpdocs/js/pages/sflowdevices_stats.js
+share/ntopng/httpdocs/js/pages/snmpdevices_stats/overview.js
+share/ntopng/httpdocs/js/pages/system_alerts_stats.js
+share/ntopng/httpdocs/js/pages/widget_list.js
 share/ntopng/httpdocs/js/pie-chart.js
-share/ntopng/httpdocs/js/plugin-script-datatable.js
 share/ntopng/httpdocs/js/push.min.js
 share/ntopng/httpdocs/js/push.min.js.map
 share/ntopng/httpdocs/js/quotas_utils.js
 share/ntopng/httpdocs/js/rickshaw.js
 share/ntopng/httpdocs/js/ripe_widget_api.js
-share/ntopng/httpdocs/js/rtt/rtt-utils.js
 share/ntopng/httpdocs/js/sankey.js
 share/ntopng/httpdocs/js/sequence_sunburst.js
 share/ntopng/httpdocs/js/sidebar.js
+share/ntopng/httpdocs/js/string_utils.js
+share/ntopng/httpdocs/js/syslog/syslog-producers-utils.js
 share/ntopng/httpdocs/js/tempusdominus.min.js
 share/ntopng/httpdocs/js/timeline.js
+share/ntopng/httpdocs/js/utils/alert-notifications-utils.js
+share/ntopng/httpdocs/js/utils/blog-notification-utils.js
+share/ntopng/httpdocs/js/utils/datatable-ipsorting.js
+share/ntopng/httpdocs/js/utils/datatable-macsorting.js
+share/ntopng/httpdocs/js/utils/datatable-utils.js
+share/ntopng/httpdocs/js/utils/modal-utils.js
 share/ntopng/httpdocs/js/validator.js
 share/ntopng/httpdocs/js/vis-network.js
 share/ntopng/httpdocs/js/vis-network.min.js
+share/ntopng/httpdocs/js/widgets/ntop-widget-utils.js
+share/ntopng/httpdocs/js/widgets/ntop-widgets.js
+share/ntopng/httpdocs/js/widgets/templates/default-template.js
+share/ntopng/httpdocs/js/widgets/templates/donut-chart-template.js
+share/ntopng/httpdocs/js/widgets/templates/multibar-chart-template.js
+share/ntopng/httpdocs/js/widgets/templates/percent-top-template.js
+share/ntopng/httpdocs/js/widgets/templates/pie-chart-template.js
+share/ntopng/httpdocs/js/widgets/templates/stacked-chart-template.js
+share/ntopng/httpdocs/js/widgets/templates/table-template.js
 share/ntopng/httpdocs/leaflet/MarkerCluster.Default.css
 share/ntopng/httpdocs/leaflet/MarkerCluster.css
 share/ntopng/httpdocs/leaflet/images/layers-2x.png
 share/ntopng/httpdocs/leaflet/images/layers.png
 share/ntopng/httpdocs/leaflet/images/marker-icon-2x.png
+share/ntopng/httpdocs/leaflet/images/marker-icon-red-2x.png
+share/ntopng/httpdocs/leaflet/images/marker-icon-red.png
 share/ntopng/httpdocs/leaflet/images/marker-icon.png
 share/ntopng/httpdocs/leaflet/images/marker-shadow.png
 share/ntopng/httpdocs/leaflet/leaflet.css
+share/ntopng/httpdocs/leaflet/leaflet.curve.js
 share/ntopng/httpdocs/leaflet/leaflet.js
 share/ntopng/httpdocs/leaflet/leaflet.js.map
 share/ntopng/httpdocs/leaflet/leaflet.markercluster.js
@@ -1859,6 +1896,7 @@
 share/ntopng/httpdocs/misc/ntopng-utils-manage-config
 share/ntopng/httpdocs/misc/ntopng-utils-manage-config.in
 share/ntopng/httpdocs/misc/ntopng-utils-manage-updates
+share/ntopng/httpdocs/misc/ntopng-utils-manage-updates.in
 share/ntopng/httpdocs/misc/ntopng_template_elk.json
 share/ntopng/httpdocs/misc/ntopng_template_elk6.json
 share/ntopng/httpdocs/misc/ntopng_template_elk7.json
@@ -1876,19 +1914,67 @@
 share/ntopng/httpdocs/other/trackers.txt
 share/ntopng/httpdocs/popper-1.12.9/js/popper.js
 share/ntopng/httpdocs/popper-1.12.9/js/popper.min.js
+share/ntopng/httpdocs/selectpicker/css/bootstrap-select.css.map
+share/ntopng/httpdocs/selectpicker/css/bootstrap-select.min.css
+share/ntopng/httpdocs/selectpicker/js/bootstrap-select.min.js
+share/ntopng/httpdocs/selectpicker/js/bootstrap-select.min.js.map
 share/ntopng/httpdocs/ssl/README
 share/ntopng/httpdocs/ssl/ntopng-cert.pem.dummy
 share/ntopng/httpdocs/sw.js
-share/ntopng/httpdocs/templates/config_list.html
-share/ntopng/httpdocs/templates/config_list_components/apply_modal.html
-share/ntopng/httpdocs/templates/config_list_components/clone_modal.html
-share/ntopng/httpdocs/templates/config_list_components/delete_modal.html
-share/ntopng/httpdocs/templates/config_list_components/import_modal.html
-share/ntopng/httpdocs/templates/config_list_components/rename_modal.html
+share/ntopng/httpdocs/templates/defs_list.html
 share/ntopng/httpdocs/templates/import_modal.html
 share/ntopng/httpdocs/templates/modal_confirm_dialog.html
-share/ntopng/httpdocs/templates/script_list.html
-share/ntopng/httpdocs/templates/script_list_components/edit_config_modal.html
+share/ntopng/httpdocs/templates/on_off_switch.html
+share/ntopng/httpdocs/templates/pages/components/ifaces-select.template
+share/ntopng/httpdocs/templates/pages/components/manage-configuration-link.template
+share/ntopng/httpdocs/templates/pages/components/monaco-editor.template
+share/ntopng/httpdocs/templates/pages/components/nav_tabs.template
+share/ntopng/httpdocs/templates/pages/components/notes.template
+share/ntopng/httpdocs/templates/pages/components/notification.template
+share/ntopng/httpdocs/templates/pages/components/notification_container.template
+share/ntopng/httpdocs/templates/pages/components/pool-select.template
+share/ntopng/httpdocs/templates/pages/components/reset-config-link.template
+share/ntopng/httpdocs/templates/pages/datasource_list.template
+share/ntopng/httpdocs/templates/pages/edit_configset.html
+share/ntopng/httpdocs/templates/pages/endpoint_notifications_list.template
+share/ntopng/httpdocs/templates/pages/manage_configurations.template
+share/ntopng/httpdocs/templates/pages/manage_data.template
+share/ntopng/httpdocs/templates/pages/manage_host_members.template
+share/ntopng/httpdocs/templates/pages/modals/datasources/add.template
+share/ntopng/httpdocs/templates/pages/modals/datasources/edit.template
+share/ntopng/httpdocs/templates/pages/modals/datasources/remove.template
+share/ntopng/httpdocs/templates/pages/modals/edit_configset/edit_config_modal.html
+share/ntopng/httpdocs/templates/pages/modals/endpoint_notifications/add.template
+share/ntopng/httpdocs/templates/pages/modals/endpoint_notifications/edit.template
+share/ntopng/httpdocs/templates/pages/modals/endpoint_notifications/remove.template
+share/ntopng/httpdocs/templates/pages/modals/factory-reset.template
+share/ntopng/httpdocs/templates/pages/modals/import_export_config/import.template
+share/ntopng/httpdocs/templates/pages/modals/pools/host_pool_member.template
+share/ntopng/httpdocs/templates/pages/modals/pools/pool_modal.template
+share/ntopng/httpdocs/templates/pages/modals/pools/remove_host_pool_member.template
+share/ntopng/httpdocs/templates/pages/modals/pools/remove_pool.template
+share/ntopng/httpdocs/templates/pages/modals/recipients_endpoint/add.template
+share/ntopng/httpdocs/templates/pages/modals/recipients_endpoint/edit.template
+share/ntopng/httpdocs/templates/pages/modals/recipients_endpoint/remove.template
+share/ntopng/httpdocs/templates/pages/modals/recipients_endpoint/users.template
+share/ntopng/httpdocs/templates/pages/modals/scripts_config/clone_modal.html
+share/ntopng/httpdocs/templates/pages/modals/scripts_config/delete_modal.html
+share/ntopng/httpdocs/templates/pages/modals/scripts_config/import_modal.html
+share/ntopng/httpdocs/templates/pages/modals/scripts_config/rename_modal.html
+share/ntopng/httpdocs/templates/pages/modals/widgets/add.template
+share/ntopng/httpdocs/templates/pages/modals/widgets/edit.template
+share/ntopng/httpdocs/templates/pages/modals/widgets/embed.template
+share/ntopng/httpdocs/templates/pages/modals/widgets/remove.template
+share/ntopng/httpdocs/templates/pages/recipients_list.template
+share/ntopng/httpdocs/templates/pages/scripts_config.template
+share/ntopng/httpdocs/templates/pages/simple_widgets_list.template
+share/ntopng/httpdocs/templates/pages/system_alerts_stats.template
+share/ntopng/httpdocs/templates/pages/table_pools.template
+share/ntopng/httpdocs/templates/pages/tabs/manage_data/delete.template
+share/ntopng/httpdocs/templates/pages/tabs/manage_data/export.template
+share/ntopng/httpdocs/templates/pages/tabs/manage_data/inactive_interfaces_data.template
+share/ntopng/httpdocs/templates/pages/tabs/manage_data/system_interface_data.template
+share/ntopng/httpdocs/templates/pages/widgets_list.template
 share/ntopng/httpdocs/templates/traffic_extraction_dialog.html
 share/ntopng/httpdocs/templates/traffic_report.html
 share/ntopng/httpdocs/templates/typeahead_input.html
@@ -1900,11 +1986,10 @@
 share/ntopng/scripts/callbacks/interface/flow.lua
 share/ntopng/scripts/callbacks/interface/host.lua
 share/ntopng/scripts/callbacks/interface/hourly.lua
-share/ntopng/scripts/callbacks/interface/ht_state_update.lua
 share/ntopng/scripts/callbacks/interface/interface.lua
 share/ntopng/scripts/callbacks/interface/minute.lua
 share/ntopng/scripts/callbacks/interface/network.lua
-share/ntopng/scripts/callbacks/interface/stats_update.lua
+share/ntopng/scripts/callbacks/interface/periodic_user_scripts.lua
 share/ntopng/scripts/callbacks/system/5min.lua
 share/ntopng/scripts/callbacks/system/boot.lua
 share/ntopng/scripts/callbacks/system/daily.lua
@@ -1912,6 +1997,7 @@
 share/ntopng/scripts/callbacks/system/hourly.lua
 share/ntopng/scripts/callbacks/system/housekeeping.lua
 share/ntopng/scripts/callbacks/system/minute.lua
+share/ntopng/scripts/callbacks/system/notifications.lua
 share/ntopng/scripts/callbacks/system/pinger.lua
 share/ntopng/scripts/callbacks/system/recovery.lua
 share/ntopng/scripts/callbacks/system/second.lua
@@ -1931,15 +2017,13 @@
 share/ntopng/scripts/locales/pt.lua
 share/ntopng/scripts/lua/API.lua
 share/ntopng/scripts/lua/about.lua
-share/ntopng/scripts/lua/admin/add_user.lua
 share/ntopng/scripts/lua/admin/change_user_prefs.lua
-share/ntopng/scripts/lua/admin/delete_user.lua
 share/ntopng/scripts/lua/admin/dhcp.lua
 share/ntopng/scripts/lua/admin/edit_categories.lua
 share/ntopng/scripts/lua/admin/edit_category_lists.lua
 share/ntopng/scripts/lua/admin/edit_configset.lua
 share/ntopng/scripts/lua/admin/edit_device_protocols.lua
-share/ntopng/scripts/lua/admin/edit_profiles.lua
+share/ntopng/scripts/lua/admin/endpoint_notifications_list.lua
 share/ntopng/scripts/lua/admin/get_category_lists.lua
 share/ntopng/scripts/lua/admin/get_custom_categories_hosts.lua
 share/ntopng/scripts/lua/admin/get_device_protocols.lua
@@ -1947,9 +2031,13 @@
 share/ntopng/scripts/lua/admin/get_user_info.lua
 share/ntopng/scripts/lua/admin/get_users.lua
 share/ntopng/scripts/lua/admin/host_pools.lua
+share/ntopng/scripts/lua/admin/manage_configurations.lua
+share/ntopng/scripts/lua/admin/manage_host_members.lua
 share/ntopng/scripts/lua/admin/manage_pool_members.lua
+share/ntopng/scripts/lua/admin/manage_pools.lua
 share/ntopng/scripts/lua/admin/password_reset.lua
 share/ntopng/scripts/lua/admin/prefs.lua
+share/ntopng/scripts/lua/admin/recipients_list.lua
 share/ntopng/scripts/lua/admin/remote_assistance.lua
 share/ntopng/scripts/lua/admin/scripts_config.lua
 share/ntopng/scripts/lua/admin/service_restart.lua
@@ -1970,12 +2058,26 @@
 share/ntopng/scripts/lua/containers_stats.lua
 share/ntopng/scripts/lua/country_details.lua
 share/ntopng/scripts/lua/country_stats.lua
+share/ntopng/scripts/lua/datasources/iface_local_stats.lua
+share/ntopng/scripts/lua/datasources/iface_pkt_distro.lua
+share/ntopng/scripts/lua/datasources/iface_pkt_version.lua
+share/ntopng/scripts/lua/datasources/iface_tcpflags_pkt_distro.lua
+share/ntopng/scripts/lua/datasources/interface.lua
+share/ntopng/scripts/lua/datasources/sample_table.lua
+share/ntopng/scripts/lua/datasources/timeseries.lua
+share/ntopng/scripts/lua/datasources_list.lua