pkgsrc-Changes-HG archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

[pkgsrc/trunk]: pkgsrc/databases/elasticsearch Update databases/elasticsearch...



details:   https://anonhg.NetBSD.org/pkgsrc/rev/0ed45b42351f
branches:  trunk
changeset: 372827:0ed45b42351f
user:      fhajny <fhajny%pkgsrc.org@localhost>
date:      Mon Dec 18 14:27:37 2017 +0000

description:
Update databases/elasticsearch to 6.1.0.


== 6.1.0 Release Notes

=== Breaking changes

Network::
- Allow only a fixed-size receive predictor

REST::
- Standardize underscore requirements in parameters

Scroll::
- Fail queries with scroll that explicitely set request_cache

Search::
- Add a limit to from + size in top_hits and inner hits.



=== Breaking Java changes

Aggregations::
- Moves deferring code into its own subclass

Core::
- Unify Settings xcontent reading and writing

Settings::
- Return List instead of an array from settings
- Remove `Settings,put(Map<String,String>)`



=== Deprecations

Aggregations::
- Deprecate global_ordinals_hash and global_ordinals_low_cardinality

Allocation::
- Add deprecation warning for negative
  index.unassigned.node_left.delayed_timeout

Analysis::
- Add limits for ngram and shingle settings

Geo::
- [GEO] 6x Deprecate ShapeBuilders and decouple geojson parse logic

Mapping::
- Deprecate the `index_options` parameter for numeric fields

Plugin Repository Azure::
- Azure repository: Move to named configurations as we do for S3
  repository and secure settings

Search::
- doc: deprecate _primary and _replica shard option



=== New features

Aggregations::
- Aggregations: bucket_sort pipeline aggregation
- Add composite aggregator

Analysis::
- Added Bengali Analyzer to Elasticsearch with respect to the lucene
  update

Ingest::
- add URL-Decode Processor to Ingest

Java High Level REST Client::
- Added Delete Index support to high-level REST client

Nested Docs::
- Multi-level Nested Sort with Filters

Query DSL::
- Add terms_set query
- Introduce sorted_after query for sorted index
- Add support for auto_generate_synonyms_phrase_query in match_query,
  multi_match_query, query_string and simple_query_string

Search::
- Expose `fuzzy_transpositions` parameter in fuzzy queries
- Add upper limit for scroll expiry
- Implement adaptive replica selection
- configure distance limit

Similarities::
- Add a scripted similarity.

Suggesters::
- Expose duplicate removal in the completion suggester
- Support must and should for context query in context suggester



=== Enhancements

Aggregations::
- Allow aggregation sorting via nested aggregation

Allocation::
- Tie-break shard path decision based on total number of shards on
  path
- Balance shards for an index more evenly across multiple data paths
- Expand "NO" decision message in NodeVersionAllocationDecider
- _reroute's retry_failed flag should reset failure counter

Analysis::
- Add configurable `max_token_length` parameter to whitespace
  tokenizer

CRUD::
- Add wait_for_active_shards parameter to index open command

Core::
- Fix classes that can exit
- Replace empty index block checks with global block checks in
  template delete/put actions
- Allow Uid#decodeId to decode from a byte array slice
- Use separate searchers for "search visibility" vs "move indexing
  buffer to disk
- Add ability to split shards
- Make circuit breaker mutations debuggable

Dates::
- DateProcessor Locale

Discovery::
- Stop responding to ping requests before master abdication

Engine::
- Ensure external refreshes will also refresh internal searcher to
  minimize segment creation
- Move IndexShard#getWritingBytes() under InternalEngine
- Refactor internal engine

Geo::
- Add ignore_malformed to geo_shape fields

Ingest::
- add json-processor support for non-map json types
- Introduce templating support to timezone/locale in DateProcessor
- Add support for parsing inline script (#23824)
- Consolidate locale parsing.
- Accept ingest simulate params as ints or strings

Internal::
- Avoid uid creation in ParsedDocument
- Upgrade to Lucene 7.1.0 snapshot version
- Remove `_index` fielddata hack if cluster alias is present

Java High Level REST Client::
- Adjust RestHighLevelClient method modifiers
- Decouple BulkProcessor from ThreadPool

Logging::
- Add more information on _failed_to_convert_ exception (#21946)
- Improve shard-failed log messages.

Mapping::
- Allow ip_range to accept CIDR notation
- Deduplicate `_field_names`.
- Throw a better error message for empty field names
- Stricter validation for min/max values for whole numbers
- Make FieldMapper.copyTo() always non-null.

Nested Docs::
- Use the primary_term field to identify parent documents
- Prohibit using `nested_filter`, `nested_path` and new `nested`
  Option at the same time in FieldSortBuilder

Network::
- Remove manual tracking of registered channels
- Remove tcp profile from low level nio channel
- Decouple `ChannelFactory` from Tcp classes

Percolator::
- Use Lucene's CoveringQuery to select percolate candidate matches
- Add support to percolate query to percolate multiple documents
  simultaneously
- Hint what clauses are important in a conjunction query based on
  fields
- Add support for selecting percolator query candidate matches
  containing range queries

Plugin Discovery EC2::
- update AWS SDK for ECS Task IAM support in discovery-ec2

Plugin Lang Painless::
- Painless: Only allow Painless type names to be the same as the
  equivalent Java class.
- Allow for the Painless Definition to have multiple instances for
  white-listing
- Separate Painless Whitelist Loading from the Painless Definition
- Remove Sort enum from Painless Definition

Plugin Repository Azure::
- Add azure storage endpoint suffix #26432
- Support for accessing Azure repositories through a proxy

Plugin Repository S3::
- Remove S3 output stream
- Update to AWS SDK 1.11.223

Plugins::
- Plugins: Add versionless alias to all security policy codebase
  properties
- Allow plugins to plug rescore implementations

Query DSL::
- Add support for wildcard on `_index`

Reindex API::
- Update by Query is modified to accept short `script` parameter.
- reindex: automatically choose the number of slices

Rollover::
- Add size-based condition to the index rollover API
- Add size-based condition to the index rollover API

Scripting::
- Script: Convert script query to a dedicated script context

Search::
- Make fields optional in multi_match query and rely on
  index.query.default_field by default
- fix unnecessary logger creation
- `ObjectParser` : replace `IllegalStateException` with
  `ParsingException`
- Uses norms for exists query if enabled
- Cross Cluster Search: make remote clusters optional
- Enhances exists queries to reduce need for `_field_names`
- Change ParentFieldSubFetchPhase to create doc values iterator once
  per segment
- Change VersionFetchSubPhase to create doc values iterator once per
  segment
- Change ScriptFieldsFetchSubPhase to create search scripts once per
  segment
- Make sure SortBuilders rewrite inner nested sorts
- Extend testing of build method in ScriptSortBuilder
- Accept an array of field names and boosts in the
  index.query.default_field setting
- Reject IPv6-mapped IPv4 addresses when using the CIDR notation.
- Rewrite range queries with open bounds to exists query

Sequence IDs::
- Only fsync global checkpoint if needed
- Log primary-replica resync failures
- Lazy initialize checkpoint tracker bit sets
- Returns the current primary_term for Get/MultiGet requests

Settings::
- Allow affix settings to specify dependencies
- Represent lists as actual lists inside Settings
- Remove Settings#getAsMap()
- Replace group map settings with affix setting
- Throw exception if setting isn't recognized
- Settings: Move keystore creation to plugin installation

Snapshot/Restore::
- Remove XContentType auto detection in BlobStoreRepository
- Snapshot: Migrate TransportRequestHandler to
  TransportMasterNodeAction
- Fix toString of class SnapshotStatus (#26851)

Stats::
- Adds average document size to DocsStats
- Stats to record how often the ClusterState diff mechanism is used
  successfully
- Expose adaptive replica selection stats in /_nodes/stats API
- Add cgroup memory usage/limit to OS stats on Linux
- Add segment attributes to the `_segments` API.

Suggesters::
- Improve error message for parse failures of completion fields
- Support 'AND' operation for context query in context suggester



=== Bug fixes

Aggregations::
- Disable the "low cardinality" optimization of terms aggregations.
- scripted_metric _agg parameter disappears if params are provided

Cluster::
- Properly format IndexGraveyard deletion date as date
-  Remove optimisations to reuse objects when applying a new
   `ClusterState`

Core::
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Ensure logging is configured for CLI commands
- Protect shard splitting from illegal target shards
- Avoid NPE when getting build information
- Fix `ShardSplittingQuery` to respect nested documents.
- When building Settings do not set SecureSettings if empty

Engine::
- Reset LiveVersionMap on sync commit
- Carry over version map size to prevent excessive resizing

Geo::
- Correct two equality checks on incomparable types
- [GEO] fix pointsOnly bug for MULTIPOINT

Index Templates::
- Prevent constructing an index template without index patterns

Ingest::
- Add pipeline support for REST API bulk upsert
- Fixing Grok pattern for Apache 2.4

Inner Hits::
- Return an empty _source for nested inner hit when filtering on a
  field that doesn't exist

Internal::
- When checking if key exists in ThreadContextStruct:putHeaders()
  method, should put requestHeaders in map first
- Adding a refresh listener to a recovering shard should be a noop

Java High Level REST Client::
- Register ip_range aggregation with the high level client
- add top hits as a parsed aggregation to the rest high level client

Mapping::
- Fix dynamic mapping update generation.
- Fix merging of _meta field
- Fixed rounding of bounds in scaled float comparison

Nested Docs::
- Ensure nested documents have consistent version and seq_ids
- Prevent duplicate fields when mixing parent and root nested includes

Network::
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams
- Do not set SO_LINGER on server channels
- Do not set SO_LINGER to 0 when not shutting down
- Close TcpTransport on RST in some Spots to Prevent Leaking TIME_WAIT
  Sockets

Packaging::
- Removes minimum master nodes default number
- setgid on /etc/elasticearch on package install

Percolator::
- Avoid TooManyClauses exception if number of terms / ranges is
  exactly equal to 1024

Plugin Analysis ICU::
- Catch InvalidPathException in IcuCollationTokenFilterFactory

Plugin Lang Painless::
- Painless: Fix variable scoping issue in lambdas
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository GCS::
- Create new handlers for every new request in
  GoogleCloudStorageService

Recovery::
- Flush old indices on primary promotion and relocation

Reindex API::
- Reindex: Fix headers in reindex action

Scroll::
- Fix scroll query with a sort that is a prefix of the index sort

Search::
- Fix profiling naming issues
- Fix max score tracking with field collapsing
- Apply missing request options to the expand phase
- Calculate and cache result when advanceExact is called
- Filter unsupported relation for RangeQueryBuilder
- Handle leniency for phrase query on a field indexed without
  positions

Sequence IDs::
- Obey translog durability in global checkpoint sync
- Fix resync request serialization

Settings::
- Allow index settings to be reset by wildcards

Snapshot/Restore::
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Delete shard store files before restoring a snapshot
- Fix snapshot getting stuck in INIT state
- Fix default value of ignore_unavailable for snapshot REST API
- Do not create directory on readonly repository (#21495)

Stats::
- Include internal refreshes in refresh stats
- Make Segment statistics aware of segments hold by internal readers
- Ensure `doc_stats` are changing even if refresh is disabled



=== Upgrades

Core::
- Upgrade to Jackson 2.8.10
- Upgrade to Lucene 7.1

Plugin Discovery EC2::
- Upgrade AWS SDK Jackson Databind to 2.6.7.1

Plugin Discovery GCE::
- Update Google SDK to version 1.23.0

Plugin Lang Painless::
- Upgrade Painless from ANTLR 4.5.1-1 to  ANTLR 4.5.3.




== 6.0.1 Release Notes

=== Breaking changes

Scroll::
- Fail queries with scroll that explicitely set request_cache



=== Enhancements

Core::
- Fix classes that can exit

Discovery::
- Stop responding to ping requests before master abdication

Plugin Repository S3::
- Remove S3 output stream
- Update to AWS SDK 1.11.223

Search::
- fix unnecessary logger creation

Sequence IDs::
- Log primary-replica resync failures

Snapshot/Restore::
- Remove XContentType auto detection in BlobStoreRepository



=== Bug fixes

Cluster::
- Properly format IndexGraveyard deletion date as date

Core::
- Do not set data paths on no local storage required
- Ensure threadcontext is preserved when refresh listeners are invoked
- Avoid NPE when getting build information
- When building Settings do not set SecureSettings if empty

Engine::
- Reset LiveVersionMap on sync commit
- Carry over version map size to prevent excessive resizing

Inner Hits::
- Return an empty _source for nested inner hit when filtering on a
  field that doesn't exist

Mapping::
- Fix dynamic mapping update generation.
- Fixed rounding of bounds in scaled float comparison

Nested Docs::
- Ensure nested documents have consistent version and seq_ids

Network::
- Throw UOE from compressible bytes stream reset
- Bubble exceptions when closing compressible streams

Plugin Lang Painless::
- Painless: Fix errors allowing void to be assigned to def.

Plugin Repository GCS::
- Create new handlers for every new request in
  GoogleCloudStorageService

Recovery::
- Flush old indices on primary promotion and relocation

Reindex API::
- Reindex: Fix headers in reindex action

Search::
- Fix profiling naming issues

Sequence IDs::
- Fix resync request serialization

Snapshot/Restore::
- Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic()
- Delete shard store files before restoring a snapshot
- Fix snapshot getting stuck in INIT state
- Fix default value of ignore_unavailable for snapshot REST API
  (#25359)
- Do not create directory on readonly repository (#21495)



=== Upgrades

Plugin Discovery EC2::
- Upgrade AWS SDK Jackson Databind to 2.6.7.1

Plugin Discovery GCE::
- Update Google SDK to version 1.23.0

diffstat:

 databases/elasticsearch/Makefile |   4 +-
 databases/elasticsearch/PLIST    |  50 +++++++++++++++++++++------------------
 databases/elasticsearch/distinfo |  10 ++++----
 3 files changed, 34 insertions(+), 30 deletions(-)

diffs (122 lines):

diff -r dbdb6a8bbfbc -r 0ed45b42351f databases/elasticsearch/Makefile
--- a/databases/elasticsearch/Makefile  Mon Dec 18 14:12:40 2017 +0000
+++ b/databases/elasticsearch/Makefile  Mon Dec 18 14:27:37 2017 +0000
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.37 2017/12/13 12:53:45 fhajny Exp $
+# $NetBSD: Makefile,v 1.38 2017/12/18 14:27:37 fhajny Exp $
 
-DISTNAME=      elasticsearch-6.0.1
+DISTNAME=      elasticsearch-6.1.0
 CATEGORIES=    textproc
 MASTER_SITES=  https://artifacts.elastic.co/downloads/elasticsearch/
 
diff -r dbdb6a8bbfbc -r 0ed45b42351f databases/elasticsearch/PLIST
--- a/databases/elasticsearch/PLIST     Mon Dec 18 14:12:40 2017 +0000
+++ b/databases/elasticsearch/PLIST     Mon Dec 18 14:27:37 2017 +0000
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.24 2017/12/13 12:53:45 fhajny Exp $
+@comment $NetBSD: PLIST,v 1.25 2017/12/18 14:27:37 fhajny Exp $
 bin/elasticsearch
 bin/elasticsearch-plugin
 lib/elasticsearch/bin/elasticsearch-env
@@ -6,11 +6,12 @@
 lib/elasticsearch/bin/elasticsearch-translog
 lib/elasticsearch/lib/HdrHistogram-2.1.9.jar
 lib/elasticsearch/lib/${PKGNAME}.jar
+lib/elasticsearch/lib/elasticsearch-cli-${PKGVERSION}.jar
 lib/elasticsearch/lib/hppc-0.7.1.jar
-lib/elasticsearch/lib/jackson-core-2.8.6.jar
-lib/elasticsearch/lib/jackson-dataformat-cbor-2.8.6.jar
-lib/elasticsearch/lib/jackson-dataformat-smile-2.8.6.jar
-lib/elasticsearch/lib/jackson-dataformat-yaml-2.8.6.jar
+lib/elasticsearch/lib/jackson-core-2.8.10.jar
+lib/elasticsearch/lib/jackson-dataformat-cbor-2.8.10.jar
+lib/elasticsearch/lib/jackson-dataformat-smile-2.8.10.jar
+lib/elasticsearch/lib/jackson-dataformat-yaml-2.8.10.jar
 lib/elasticsearch/lib/java-version-checker-${PKGVERSION}.jar
 lib/elasticsearch/lib/jna-4.4.0-1.jar
 lib/elasticsearch/lib/joda-time-2.9.5.jar
@@ -19,24 +20,24 @@
 lib/elasticsearch/lib/log4j-1.2-api-2.9.1.jar
 lib/elasticsearch/lib/log4j-api-2.9.1.jar
 lib/elasticsearch/lib/log4j-core-2.9.1.jar
-lib/elasticsearch/lib/lucene-analyzers-common-7.0.1.jar
-lib/elasticsearch/lib/lucene-backward-codecs-7.0.1.jar
-lib/elasticsearch/lib/lucene-core-7.0.1.jar
-lib/elasticsearch/lib/lucene-grouping-7.0.1.jar
-lib/elasticsearch/lib/lucene-highlighter-7.0.1.jar
-lib/elasticsearch/lib/lucene-join-7.0.1.jar
-lib/elasticsearch/lib/lucene-memory-7.0.1.jar
-lib/elasticsearch/lib/lucene-misc-7.0.1.jar
-lib/elasticsearch/lib/lucene-queries-7.0.1.jar
-lib/elasticsearch/lib/lucene-queryparser-7.0.1.jar
-lib/elasticsearch/lib/lucene-sandbox-7.0.1.jar
-lib/elasticsearch/lib/lucene-spatial-7.0.1.jar
-lib/elasticsearch/lib/lucene-spatial-extras-7.0.1.jar
-lib/elasticsearch/lib/lucene-spatial3d-7.0.1.jar
-lib/elasticsearch/lib/lucene-suggest-7.0.1.jar
+lib/elasticsearch/lib/lucene-analyzers-common-7.1.0.jar
+lib/elasticsearch/lib/lucene-backward-codecs-7.1.0.jar
+lib/elasticsearch/lib/lucene-core-7.1.0.jar
+lib/elasticsearch/lib/lucene-grouping-7.1.0.jar
+lib/elasticsearch/lib/lucene-highlighter-7.1.0.jar
+lib/elasticsearch/lib/lucene-join-7.1.0.jar
+lib/elasticsearch/lib/lucene-memory-7.1.0.jar
+lib/elasticsearch/lib/lucene-misc-7.1.0.jar
+lib/elasticsearch/lib/lucene-queries-7.1.0.jar
+lib/elasticsearch/lib/lucene-queryparser-7.1.0.jar
+lib/elasticsearch/lib/lucene-sandbox-7.1.0.jar
+lib/elasticsearch/lib/lucene-spatial-7.1.0.jar
+lib/elasticsearch/lib/lucene-spatial-extras-7.1.0.jar
+lib/elasticsearch/lib/lucene-spatial3d-7.1.0.jar
+lib/elasticsearch/lib/lucene-suggest-7.1.0.jar
 lib/elasticsearch/lib/plugin-cli-${PKGVERSION}.jar
 lib/elasticsearch/lib/securesm-1.2.jar
-lib/elasticsearch/lib/snakeyaml-1.15.jar
+lib/elasticsearch/lib/snakeyaml-1.17.jar
 lib/elasticsearch/lib/spatial4j-0.6.jar
 lib/elasticsearch/lib/t-digest-3.0.jar
 lib/elasticsearch/modules/aggs-matrix-stats/aggs-matrix-stats-${PKGVERSION}.jar
@@ -52,20 +53,23 @@
 lib/elasticsearch/modules/lang-expression/asm-commons-5.0.4.jar
 lib/elasticsearch/modules/lang-expression/asm-tree-5.0.4.jar
 lib/elasticsearch/modules/lang-expression/lang-expression-${PKGVERSION}.jar
-lib/elasticsearch/modules/lang-expression/lucene-expressions-7.0.1.jar
+lib/elasticsearch/modules/lang-expression/lucene-expressions-7.1.0.jar
 lib/elasticsearch/modules/lang-expression/plugin-descriptor.properties
 lib/elasticsearch/modules/lang-expression/plugin-security.policy
 lib/elasticsearch/modules/lang-mustache/compiler-0.9.3.jar
 lib/elasticsearch/modules/lang-mustache/lang-mustache-${PKGVERSION}.jar
 lib/elasticsearch/modules/lang-mustache/plugin-descriptor.properties
 lib/elasticsearch/modules/lang-mustache/plugin-security.policy
-lib/elasticsearch/modules/lang-painless/antlr4-runtime-4.5.1-1.jar
+lib/elasticsearch/modules/lang-painless/antlr4-runtime-4.5.3.jar
 lib/elasticsearch/modules/lang-painless/asm-debug-all-5.1.jar
 lib/elasticsearch/modules/lang-painless/lang-painless-${PKGVERSION}.jar
 lib/elasticsearch/modules/lang-painless/plugin-descriptor.properties
 lib/elasticsearch/modules/lang-painless/plugin-security.policy
+lib/elasticsearch/modules/mapper-extras/mapper-extras-${PKGVERSION}.jar
+lib/elasticsearch/modules/mapper-extras/plugin-descriptor.properties
 lib/elasticsearch/modules/parent-join/parent-join-${PKGVERSION}.jar
 lib/elasticsearch/modules/parent-join/plugin-descriptor.properties
+lib/elasticsearch/modules/percolator/mapper-extras-${PKGVERSION}.jar
 lib/elasticsearch/modules/percolator/percolator-${PKGVERSION}.jar
 lib/elasticsearch/modules/percolator/plugin-descriptor.properties
 lib/elasticsearch/modules/reindex/commons-codec-1.10.jar
diff -r dbdb6a8bbfbc -r 0ed45b42351f databases/elasticsearch/distinfo
--- a/databases/elasticsearch/distinfo  Mon Dec 18 14:12:40 2017 +0000
+++ b/databases/elasticsearch/distinfo  Mon Dec 18 14:27:37 2017 +0000
@@ -1,8 +1,8 @@
-$NetBSD: distinfo,v 1.38 2017/12/13 12:53:45 fhajny Exp $
+$NetBSD: distinfo,v 1.39 2017/12/18 14:27:37 fhajny Exp $
 
-SHA1 (elasticsearch-6.0.1.tar.gz) = fb8400dac5e8853d8f040aaa398ae5e623a2907f
-RMD160 (elasticsearch-6.0.1.tar.gz) = fb36b0e848ad24998105e3788b204faa56a957fc
-SHA512 (elasticsearch-6.0.1.tar.gz) = b86a04acd194e7e96e3a32de6ab4983d6569ffb1714f2af9e2b49623004987e13e57c5db055153a188f5d2d7eea63d649fa87769f7625f3fc4923e0cd5b8f3ee
-Size (elasticsearch-6.0.1.tar.gz) = 28026460 bytes
+SHA1 (elasticsearch-6.1.0.tar.gz) = 20856cf7b920a8e9bbf16646bbf1b938b0ea9758
+RMD160 (elasticsearch-6.1.0.tar.gz) = 9685aae4d561d3e681aa12a4e7728a32b5ad0d37
+SHA512 (elasticsearch-6.1.0.tar.gz) = ba4829c7bb422b42bdc480797521399d9b39c6fd9bb2597585f8fdb0a261e8e11bd8e98b3df562b7320b2775643be9918b043d818caa79b0a6403c9c0657774e
+Size (elasticsearch-6.1.0.tar.gz) = 28535876 bytes
 SHA1 (patch-bin_elasticsearch-env) = 7d8f71c6c1a99168cc4a7322e054ddc8fe94b68e
 SHA1 (patch-config_elasticsearch.yml) = f66dad65dec48a45a4d52b685d6e3ab29de62cf2



Home | Main Index | Thread Index | Old Index