CVS commit: pkgsrc/geography/osm2pgsql

["Greg Troxel" <gdt%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   gdt
Date:           Mon Jan  6 19:18:00 UTC 2025

Modified Files:
        pkgsrc/geography/osm2pgsql: Makefile PLIST distinfo
Added Files:
        pkgsrc/geography/osm2pgsql/patches: patch-docs_CMakeLists.txt
Removed Files:
        pkgsrc/geography/osm2pgsql/patches: patch-CMakeLists.txt

Log Message:
geography/osm2pgsql: Update to 1.9.2

The old version was quite behind and stopped building.  This version
packages and the main program runs; therefore it is better, even if
not tested.

Packaging changes:
  Take MAINTAINERship (ok wiedi@).
  Adjust dependencies to match upstream docs.
  Hack in finding boost headers (should not be necessary).

Upstream NEWS content: (Much has changed and this summary omits much,
including bug-fix releases entirely.  See
https://github.com/osm2pgsql-dev/osm2pgsql/releases for more details
-- there is no NEWS.)

Release 1.9.0

This release brings three new major features:

    a new osm2pgsql_properties table that saves command line options and reuses them on updates
    a new database middle saves raw OSM data in JSONB format and is explicitly designed to be queried by the user
    the new (and still experimental) osm2pgsql-gen adds geometry generalization to osm2pgsql (thanks @joto)

Release 1.8.0

The largest change is the addition of much more flexible index support in the flex output. The table definitions have a new (optional) field called indexes now which takes a list of index 
definitions. If the field is not there, we fall back to what we did before and create a GIST index on the only/first geometry column of a table. But you can also define any kind of index you want: 
define which index method (BTREE, GIST, ...) to use on which columns, define WHERE clauses and expression indexes and much more. See the flex-config/indexes.lua Lua config for some usage examples and 
the manual for all the details. You can also force osm2pgsql to always build the id indexes which are normally only built in slim mode.

The gazetteer output and the command line option --with-forward-dependencies are deprecated in this release and will be removed soon. They were only needed for Nominatim which switched to using the 
flex output recently.

Here are the other changes:

    Fix a problem when using osm2pgsql with a projection other than WGS84 (EPSG:4326) or Web Mercator (EPSG:3857) which made the program really slow.
    New pole_of_inaccessibility() Lua function to generate reasonably good label points from polygons. (This function is currently marked as experimental, which means it can change without notice at 
any time.)

Release 1.7.0

For this version we rebuilt a lot of the code around geometry processing and around expire. The different parts -- creation of geometries from OSM data, transforming geometries (like merging and 
splitting linestrings) and finally writing them out in WKB format for import into the database -- are now well separated and tested on their own. And we added some functions for geometry processing, 
too. osm2pgsql can now calculate the centroid of a polygon and simplify linestrings using the Douglas-Peucker algorithm.

But the best part is that all of that new geometry goodness is now available from the Lua config files when using the flex output. There are many new ways of processing geometries from Lua:

    The get_bbox() is now available for relations, too.
    There are new functions as_point(), as_linestring(), as_polygon(), as_multilinestring(), as_multipolygon(), and as_geometrycollection() to create geometries from OSM objects.
    Geometries can be manipulated in Lua with several functions modeled after the same functions in PostGIS: area(), centroid(), geometry_type(), line_merge(), num_geometries(), segmentize(), 
simplify(), srid(), and transform(). We expect more to come in the future. This way you can do more geometry processing on import removing the need for some post-processing in SQL.
    We used to have the somewhat magic handling of geometries with the add_row() function which only allowed a limited set of operations. This function is still available for backwards compatibility, 
but there is a new function insert() now which doesn't have this magic. Instead geometries are treated like any other data type giving you a lot more flexiblity. Check out the example config files 
addresses.lua, generic.lua, simple.lua and geometries-using-insert.lua in the flex-config directory for some ideas on what can be done.

In this version we enabled the bucket index for way nodes by default. This had been around for a while but you needed a command line option to enable it. After some positive feedback from the 
community we decided to make this the new default. It will be used on new imports (existing databases will keep using the old index). The new bucket index is much smaller and can save you hundreds of 
gigabytes of disk space. See
https://osm2pgsql.org/doc/manual.html#bucket-index-for-slim-mode for the details.

Release 1.6.0

    The osm2pgsql-replication script which has been included in the osm2pgsql repository for a while will now be installed together with its man page on "make install". To use it you need Python3, 
psycopg2 (or psycopg3), and PyOsmium installed. See the manual for details.

Release 1.5.0

This release brings quite a lot of improvements. We removed the "experimental" label from the flex output which we introduced in version 1.3.0. There are some small changes you might have to make to 
your flex configurations, see the Upgrading chapter of the manual for details.

This release also contains a rewrite of the code used to temporarily store OSM data in memory while processing the data in non-slim mode, i.e. when you import data without --slim. It now uses much 
less memory.

Release 1.4.0

The project has a new website at https://osm2pgsql.org now with extensive documentation and examples, and with sections on support, contributing, news, etc. Most of the documentation from the 
repository and the OSM wiki was moved there. We still have a man page, it is now maintained in markdown format. All the documentation, man page, help texts etc. have been cleaned up, made more 
consistent and brought up to date.

The program has a much improved log output now. Each line is prefixed with a date/timestamp and by default osm2pgsql isn't as verbose any more. You can change the verbosity using several options. You 
can even have super-verbose logging of all SQL commands issued and all data written to the database. Warnings and errors now appear in red color if your console supports it. Progress output can be 
disabled, for instance, when the output is redirected to a file. When printing how low something took, osm2pgsql will now not only print the seconds but also a more human readable format with hours, 
minutes, and seconds.

In the last release (version 1.3.0) we have already added a warning when you used input files with negative OSM object ids or input files which are not ordered correctly. These are now not allowed 
any more and osm2pgsql will stop with an error if it detects these. See the manual for how to work around this. This allowed us to improve the handling of multiple input files. Osm2pgsql now reads 
multiple input files at the same time merging the contents. This means that you can now import several extracts in one go. Note that the extracts still have to come from the same point in time!

Changes in the flex output (which is still marked experimental). Not that some of these are breaking changes compared to the behaviour in version 1.3.0:

    Fix: Flex output sometimes created two id indexes on the same table.
    Set projection for geometry columns in the table configuration. The command line options --latlong, -m, --merc, -E, and --proj are not used by the flex output any more.
    Flex mode setting type_column fixed. Now also supports id columns compatible with Imposm.
    Optionally wrap polygon geometries in multipolygons if the geometry column of the target table is of type MultiPolygon.
    Switch multipolygon generation from default off to default on. The multi option on the area geometry transformation has been removed and there is a new option split_at.
    Add several Lua helper functions for flex config files.


To generate a diff of this commit:
cvs rdiff -u -r1.55 -r1.56 pkgsrc/geography/osm2pgsql/Makefile
cvs rdiff -u -r1.2 -r1.3 pkgsrc/geography/osm2pgsql/PLIST
cvs rdiff -u -r1.10 -r1.11 pkgsrc/geography/osm2pgsql/distinfo
cvs rdiff -u -r1.1 -r0 \
    pkgsrc/geography/osm2pgsql/patches/patch-CMakeLists.txt
cvs rdiff -u -r0 -r1.1 \
    pkgsrc/geography/osm2pgsql/patches/patch-docs_CMakeLists.txt

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/geography/osm2pgsql/Makefile
diff -u pkgsrc/geography/osm2pgsql/Makefile:1.55 pkgsrc/geography/osm2pgsql/Makefile:1.56
--- pkgsrc/geography/osm2pgsql/Makefile:1.55    Sun Jan  5 14:14:13 2025
+++ pkgsrc/geography/osm2pgsql/Makefile Mon Jan  6 19:18:00 2025
@@ -1,33 +1,50 @@
-# $NetBSD: Makefile,v 1.55 2025/01/05 14:14:13 gdt Exp $
+# $NetBSD: Makefile,v 1.56 2025/01/06 19:18:00 gdt Exp $
 
-DISTNAME=      osm2pgsql-1.3.0
-PKGREVISION=   24
+DISTNAME=      osm2pgsql-1.9.2
 CATEGORIES=    geography
 MASTER_SITES=  ${MASTER_SITE_GITHUB:=openstreetmap/}
 GITHUB_TAG=    ${PKGVERSION_NOREV}
 
-MAINTAINER=    wiedi%frubar.net@localhost
+MAINTAINER=    gdt%NetBSD.org@localhost
 HOMEPAGE=      https://github.com/openstreetmap/osm2pgsql/
 COMMENT=       OpenStreetMap data to PostgreSQL converter
 LICENSE=       gnu-gpl-v2
 
-USE_LANGUAGES= c c++
+USE_LANGUAGES=         c c++
+USE_CXX_FEATURES+=     c++17
 
 .include "../../devel/cmake/build.mk"
 
+# Added without understanding, after a complaint of undefined.
+CMAKE_CONFIGURE_ARGS+= -DBoost_INCLUDE_DIR=${BUILDLINK_PREFIX.boost-headers}/include
+
+REPLACE_PYTHON+=       scripts/osm2pgsql-replication
+
+# To rebuid man page (which we don't), pandoc is required.
+
 # Test dependencies.
-USE_TOOLS+=    python
 CMAKE_CONFIGURE_ARGS+= -DBUILD_TESTS=ON
 TEST_TARGET=   test
+# python, psycopg2
 
-.include "../../geography/proj/buildlink3.mk"
 .include "../../archivers/bzip2/buildlink3.mk"
+.include "../../devel/boost-headers/buildlink3.mk"
 .include "../../devel/boost-libs/buildlink3.mk"
 .include "../../devel/zlib/buildlink3.mk"
+.include "../../geography/proj/buildlink3.mk"
 .include "../../lang/lua/buildlink3.mk"
-# Per README, not needed, as python is for running tests, but
-# without it cmake finds 2.7 and errors out.
 .include "../../lang/python/application.mk"
 .include "../../mk/pgsql.buildlink3.mk"
 .include "../../textproc/expat/buildlink3.mk"
+.include "../../textproc/nlohmann-json/buildlink3.mk"
+
+# vendored libs which could be provided:
+#   fmt
+#   libosmium
+#   protozero
+
+# optional for generalization:
+#   cimg
+#   potrace
+
 .include "../../mk/bsd.pkg.mk"

Index: pkgsrc/geography/osm2pgsql/PLIST
diff -u pkgsrc/geography/osm2pgsql/PLIST:1.2 pkgsrc/geography/osm2pgsql/PLIST:1.3
--- pkgsrc/geography/osm2pgsql/PLIST:1.2        Sat Jun 10 20:00:41 2017
+++ pkgsrc/geography/osm2pgsql/PLIST    Mon Jan  6 19:18:00 2025
@@ -1,5 +1,7 @@
-@comment $NetBSD: PLIST,v 1.2 2017/06/10 20:00:41 wiedi Exp $
+@comment $NetBSD: PLIST,v 1.3 2025/01/06 19:18:00 gdt Exp $
 bin/osm2pgsql
+bin/osm2pgsql-replication
+man/man1/osm2pgsql-replication.1
 man/man1/osm2pgsql.1
 share/osm2pgsql/default.style
 share/osm2pgsql/empty.style

Index: pkgsrc/geography/osm2pgsql/distinfo
diff -u pkgsrc/geography/osm2pgsql/distinfo:1.10 pkgsrc/geography/osm2pgsql/distinfo:1.11
--- pkgsrc/geography/osm2pgsql/distinfo:1.10    Tue Oct 26 10:45:13 2021
+++ pkgsrc/geography/osm2pgsql/distinfo Mon Jan  6 19:18:00 2025
@@ -1,6 +1,6 @@
-$NetBSD: distinfo,v 1.10 2021/10/26 10:45:13 nia Exp $
+$NetBSD: distinfo,v 1.11 2025/01/06 19:18:00 gdt Exp $
 
-BLAKE2s (osm2pgsql-1.3.0.tar.gz) = cd5553b762a7ae15e03102e73473f5e38e6043b154e67c87a76a38b00638f421
-SHA512 (osm2pgsql-1.3.0.tar.gz) = e643f3a1987ead36763c96d34b5dd7e170273b77a521106c21311dc7b01d2b34470b2e00e3214050f2cbdd3d89703a1827ee6c35928ed77cc7f35fc4e20331f3
-Size (osm2pgsql-1.3.0.tar.gz) = 2460451 bytes
-SHA1 (patch-CMakeLists.txt) = a33fcf5b935c515a82817fe233a9d9bb6b9e7372
+BLAKE2s (osm2pgsql-1.9.2.tar.gz) = 3ff365887a93410bc40d2a6c46037a505e5934e0ff1d4f591132f4d11c27ad82
+SHA512 (osm2pgsql-1.9.2.tar.gz) = b8685ad564be43cf7c4ea9e1bacb6cb00c88fb894f13888e802d86bf7912ade582b30eb05c53bb66119b7f14e4db0c8acced55e74ee4dc6c7c35fc96b8effc9f
+Size (osm2pgsql-1.9.2.tar.gz) = 2598971 bytes
+SHA1 (patch-docs_CMakeLists.txt) = 9b1d0e80cdaa2174328081a1d5e076e5d5d9e007

Added files:

Index: pkgsrc/geography/osm2pgsql/patches/patch-docs_CMakeLists.txt
diff -u /dev/null pkgsrc/geography/osm2pgsql/patches/patch-docs_CMakeLists.txt:1.1
--- /dev/null   Mon Jan  6 19:18:00 2025
+++ pkgsrc/geography/osm2pgsql/patches/patch-docs_CMakeLists.txt        Mon Jan  6 19:18:00 2025
@@ -0,0 +1,17 @@
+$NetBSD: patch-docs_CMakeLists.txt,v 1.1 2025/01/06 19:18:00 gdt Exp $
+
+--- docs/CMakeLists.txt.orig   2025-01-06 18:19:00.158960664 +0000
++++ docs/CMakeLists.txt
+@@ -62,9 +62,9 @@ if(MANPAGE_TARGETS)
+ endif()
+ 
+ if(ENABLE_INSTALL)
+-    install(FILES osm2pgsql.1 DESTINATION share/man/man1)
+-    install(FILES osm2pgsql-replication.1 DESTINATION share/man/man1)
++    install(FILES osm2pgsql.1 DESTINATION ${CMAKE_INSTALL_MANDIR}/man1)
++    install(FILES osm2pgsql-replication.1 DESTINATION ${CMAKE_INSTALL_MANDIR}/man1)
+     if (BUILD_GEN)
+-        install(FILES osm2pgsql-gen.1 COMPONENT gen EXCLUDE_FROM_ALL DESTINATION share/man/man1)
++        install(FILES osm2pgsql-gen.1 COMPONENT gen EXCLUDE_FROM_ALL DESTINATION ${CMAKE_INSTALL_MANDIR}/man1)
+     endif()
+ endif()