CVS commit: pkgsrc/devel/MoarVM

["Benny Siegert" <bsiegert%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   bsiegert
Date:           Sun Feb  5 12:36:59 UTC 2017

Modified Files:
        pkgsrc/devel/MoarVM: Makefile PLIST distinfo
        pkgsrc/devel/MoarVM/patches: patch-build_probe.pm

Log Message:
Update MoarVM to 2017.01.

2016.04
- Implement heap snapshots support, exposed through the profiling API
- Don't mark tc->cur_frame twice during GC
- Move thread object mark into thread root mark, removing a special case in the
  main collector
- Add API for adding a permanent root together with a description
- Mark thread blocked for GC when doing synchronous reads
- Make MVM_bigint_from_num jit-friendly
- JIT coerce_nI as a call to bigint_from_num
- Set debug_name on a few more built-in types
- Add REPR API for getting unmanaged size, implement it for many REPRs
- Fix compile on FreeBSD 9
- Handle single-char build directory names by changing capture to lookahead
- Fix performance of reading very long lines
- Fix utf8-c8 decode crash caused by off-by-1
- Fix a full vs. partial collection detection bug
- Factor unmanaged size into promoted bytes
- Enhance scheme for deciding when to do a full collect
- Bail out if HLL name string index is invalid in bytecode unpack
- Don't try to set flags beyond num_lexicals when reading in call frames
- Adjust byte count when UTF-16 BOM marker consumed
- Fix buffer overrun in utf8-c8 decoding
- Initialize object registers with VMNull when allocating frames, to avoid a
  number of possible segfaults
- Fix incorrect boolification of "" under mode MODE_UNBOX_STR_NOT_EMPTY_OR_ZERO
- Utilize debug_name in nativecall error messages

2016.05
- Use MSVC-provided stdint.h and inttypes.h when possible
- Improved various error messages to include the debug type name
- Eliminate call frame reference counts, instead using a linear call stack for
  frames that never become heap-referenced and using the garbage collector to
  manage those that do
- Tune number of gen2 sized bins
- Improve error reporting in P6opaque's compose function
- Improved GC debugging support to catch more problems
- Fix a race between serialization context resolution and garbage collection
- Add a missing MVMROOT in shell/spawn
- Fix segfault when trying to serialize an uncomposed P6opaque type
- Update the build system to autodetect system provided libs
- Add missing rooting, since the instrumentation level barrier may allocate
- Fix cleanup of temp roots at thread exit
- Add missing rooting of exception message in `die`
- Fix CUnion layout computation GC issues
- Fix data race in clearing of "in inter-gen set" flag
- Various improvements and fixes to the spesh_graph tool
- Add missing optimization of decont_s and decont_u ops
- Eliminate duplicate MVMContext creation code
- Fix full cleanup crash: the NFG uses the FSA, so have to destroy NFG first
- Close dynvar log filehandle in instance destroy
- Clean up cross-thread write logging mutex
- Free debug names in STables
- Free the string_heap_fast_table in CompUnit bodies
- Free permroot descriptions
- Free thread's finalization queue in tc destruction
- Add docs about MVMROOT
- Fix crash when GCing an ended thread
- Implement loadbytecodebuffer and loadbytecodefh ops
- Speed up initialization of non-specialized frames
- Use varints to make serialization of various integers more compact, reducing
  size of compiled output
- Ensure we always produce at least one snapshot when heap profiling
- Remove debug output when taking heap snapshots

2016.06
- Serialize the HLL role field of STables
- Avoid VMNull setup memcpy/loop in specialized frame initialization
- Stop caching MVMContext on a frame
- Use frames directly when serializing closures, rather than having to create
  an MVMContext wrapper for every one
- Write SC index when deserializing, repossessing, and preparing to serialize,
  saving a huge amount of linear scanning
- Only used cached SC index if SC itself matches
- Remove an unrequired memset to clear the args buffer
- Inline args preparation into interpreter, and JIT it directly instead of
  making a call to a tiny function
- Mark getlexcaller as :noinline
- Don't lose handlers during multi-level inlines
- Eliminate loop in P6opaque's get_boxed_ref function, decreasing cost
- Implement payload throw/handler support
- Implement callerlex throwing mode
- Make moar-gdb.py source-able, instead of only autoloadable
- Fix inlining causing wrong lexical handler lookup
- Make build in dir containing space work
- Allow HLL handler for unhandled lexical exception
- Add debug_name to "cannot iterate X with Y REPR" message
- Give "this is not a X iterator" more info, too

2016.07
- Implement a new multi-dispatch cache, structured as a tree, able to hold more
  entries more memory-compactly, and able to cache dispatches involving named
  parameters
- JIT read_fhs op
- Implement elems REPR function for MVMContext
- Implement a BB-splitting manipulation function in the optimizer
- Fix sha1 op to work with null bytes
- Fix bad interaction between profiling and dead allocation elimination; the
  presence of allocation logging would prevent the optimization, leading to
  misleading profiler output results

2016.08
- Don't crash in P6opaque on NULL name_to_index_mapping
- Fix off-by-one in grapheme iterator
- Block/unblock thread for GC on semaphore wait
- Mark blocked around some more lock acquisitions
- Fix various cases of out-dated pointer reads on concurrency control constructs
- Hold uv_sem_t at a level of indirection, so it won't move in memory
- Add a flag for PIPE_MERGED_OUT_ERR; fix stdio setup for merge
- Fix EOF detection when reading files from /proc and similar
- Fix lost socket listen errors
- Implement async cancellation completion notifications
- Add some missing fact dependencies in spesh, fixing some wrong guard
  eliminations
- Avoid use of possibly-invalidated decont facts in spesh
- Remove keep_caller from MVMFrame
- Correctly NULL-terminate the buffer in MVM_vm_dump_file

2016.09
- Do not crash when the container configuration can't be read
- Correctly calculate the work_size when inlining between compilation units
- Validate indices of param_ ops and require checkarity before param_*
- Fix tautological comprison; snprintf returns an int, not a size_t
- Fix tell for files that haven't been read from
- Don't allow zero alignment in P6opaque storage spec
- Index check lexicals when reading static flags
- Add Decoder REPR and a number of ops that provide access to VM-backed
  streaming decoding
- Don't segfault when serializing an uncomposed p6opaque
- Don't read_int into a size_t which is unsigned
- Never leak sym_name from nativecall_build
- Fix a couple of potential leaks in heap snapshots
- Kick arg_ and argconst_ ops from the correct basic block when inlining
- Use HASH_FIND_CACHE instead of HASH_FIND in MVMHash
- Correct wording of named arguments error
- Bounds check for hints in get_attribute and bind_attribute
- Use hints for is_attribute_initialized, and null-check repr_data
- Clean up various unused variables
- Mark thread GC blocked while accepting a socket to prevent deadlocks
- Configure.pl now has an explicit use lib "."
- Fix missing finalization queue cleanup at thread exit
- Mark SC used in bytecode loading as claimed to avoid keeping them around for
  too long
- Fix an error in multi caching of named arguments that could cause lookups to
  fail and the cache to keep growing

2016.10
- Fix build without libtommath source
- Make MVM_file_open_fh() throw if the file we opened was a directory
- Fix bug and memory leaks in MVM_file_open_fh()
- Add error message for likely MacOS build failure
- Don't set inheriting process on inherited pipe, fixing a segfault
- Update to the Unicode 9 database
- Die on CStruct without any fields
- Constant-fold unipropcode and unipvalcode
- Use PRId64 instead of %d for 64 bit int format string args
- "Cannot unbox type object" error gets debugname and native type info
- Fix uninitialized arg_names of an MVMCallsite
- Handle C++ constructors on libffi
- Fix CUnion get_attribute treating inlined attrs as pointers
- Fix 32 bit issue with rw args in NativeCall callbacks
- Use better throw-away type for void nativecalls
- Gracefully handle a 0 RSS reported by the Linux kernel
- Fix "Invalid free()" in empty repossessed arrays
- Make sure we mean "signed char" when we say "char"
- Fix calculating structure sizes for arm64 and others
- Disable JIT on x32, since it has a different calling convention
- Fix pointer size unit in configure message
- Use set_uint64 in from_num, otherwise we overflow on x32
- Rewrite mkdir_p api, no function changes
- Improve nativecall attribute error messages
- Include debug_name in crossthreadwritelog and serialization errors
- Deprecate async string I/O ops
- Deprecate flattenropes op
- Implement indexingoptimized op (replaces flattenropes, but is not in-place)
- Fix memory leadk in ord_basechar_at
- Ensure errno is grabbed before MVM_free is called
- Make extra sure unlock only happens if lock happened

2016.11
- Workaround tommath issue #56 which affects random bigint numbers > 32 bits
- Fix memory leaks in nqp_nfa_run
- Fix a sizeof arg that allocated a much-too-big buffer for callsite arg names
- Add missing breaks in MVMString's copy_to
- Prevent null deref when calling MVM_string_utf16_encode
- Corrections to mkdir on Windows
- Introduce MVM_SPESH_LIMIT, which limits how many specializations will be
  performed; this is useful for debugging which specialized block is to blame
  for a bug
- Make sure we never box a NULL filename when creating backtraces
- Fix the nativecall attribute error messages
- Remove (non-existent) num16 from error message
- Make the fixed size allocator provide useful information to Valgrind
- Implement serialization of SCRef
- Include file/line of unserializable closure, to aid debugging
- Add type info when failing to bind an attribute
- Use ffi_arg type for libffi nativecall return types
- Handle libffi return type more correctly
- Work around missing libatomic_ops prototype on s390x
- Panic when trying to GC a locked mutex
- Make VM panic output state that a panic occurred
- Fix JIT code generation bug in nqp::exception
- Add missing rooting of value pushed to concurrent queue
- Add src/gc/debug.h dependency to Makefile
- Mark a thread GC-blocked while it is in native code
- Have "Cannot * a type object" also outputs the type's debug name
- Don't reveal partially deserialized method cache
- Give a bunch of exception ops REPR and debug name output
- Prevent segfault when null string used as hash key
- Request POSIX.1.c compliance on solaris
- Change to use readdir insteal of readdir_r
- Use GCC on Solaris by default
- Give diagnostic output when compiling fails
- Handle current Solaris compiler not understanding -mt
- Fixed readdir so it won't check old errno
- Work around clock_gettime issue on OSX
- Refactor hashes to no longer need to flatten ropes in string keys
- Save 8 (64-bit) or 4 (32-bit) bytes per entry in the MVMHash REPR
- Remove the deprecated flattenropes op
- Remove now-unused MVM_string_flatten function
- Don't treat an nread of 0 in libuv read callbacks as an error
- Bump to latest libatomic_ops

2016.12
- Decode Latin-1 and UTF-8 strings to 8-bit width when possible
- Teach a few string functions to compress results into 8-bit storage
- Fix for AIX's INFINITY not being a constant
- Unlink libmoar before (re)installing it
- Add configuration for AIX
- Support Perl older than 5.10 for Configure.pl
- Use "pkgconfig --libs libffi" additionally to --cflags
- Fix powerpc detection on AIX (gcc)
- Make multidim error messages say they are multidim
- Add comment about where to find UNIDATA
- Implement captureinnerlex op, for fixing QUIT/LAST phaser scoping in Perl 6
- Fix mis-sized free in the NFG trie that only showed up in 32-bit
- Add GC debug helper to "find" a pointer in nurseries/gen2 bins
- Make ASSERT_NOT_FROMSPACE check fromspaces of all threads
- Fix missing MVMROOT around an allocation
- Fix typo in nfg.h comment
- Fix native callback and GC interaction when embedded
- Add GC block management functions to the public API
- Mark throwpayload* as :throwish in oplist
- Optimize the check for negative bignums
- Remove useless mp_neg calls
- Fix premature free of UV socket handles
- Fix premature handle free in async UDP sockets
- Add a "check every register access" GC debug mode
- Provide a #define to disable dynlex caching
- Provide a #define for deopt logging
- Invalidate dynlex caches during deopt
- Fix pow_I when it takes an exponent larger than 2**32
- Only do MVM_ASSERT_NOT_FROMSPACE in GC debug mode
- Add a fromspace assertion in finalize
- Avoid a number of spesh GC invariant violations
- Panic if we try to GC when speshing/JITing
- Ensure we don't leak partially deserialized objects
- Fix unrooted frame around SC object lookup
- Remove some GC debug code
- Avoid reading nativerefs in spesh, since it can cause boxing and thus GC
- Cope with push being used on concurrent queues
- MVMROOT around putting work on concurrent queue
- MVMROOT eventloop queue when polling it
- Do MVM_ASSIGN_REF after block/unblock in concurrent queue
- Fix more unrooted frame around SC object lookup

2017.01
- Extract spesh block allocator from spesh, for wider use
- Add a number of missing MVMROOTs, which could lead to outdated pointers; also
  removed some unrequired MVMROOTs
- Fix arg_flags allocation sizes
- Remove an obsolete path from ldrpath
- Check lexical accesses in GC debug mode 2 also
- Add a #define to turn on inline logging
- Fix callstack reset bug, which could corrupt deeply recursing callstacks
- Don't allow re-compose of a P6opaque
- Add a new unicmp_s op, which compares using the Unicode Collation Algorithm
- Make sure we generate all values of the Line_Break property
- Re-implement utf8-c8 encoding, fixing bugs and ensuring that non-NFC input
  will round-trip properly also
- Implement Bidi_Mirroring_Glyph as an integer property
- Implement Emoji grapheme breaking and other combined codes
- Add support for Grapheme_Cluster_Break=Prepend from Unicode 9.0
- Make sure we break after Prepend if it's a control character
- Add a script to download the latest version of all of the Unicode data
- Missing rooting GC in rare exit handler case
- Implement new setdispatcherfor op and add JIT for it
- Use much faster atoi function in normalizer; 14% less CPU use when slurping a
  file in UTF-8 encoding
- For Decompose_Type, use int lookup instead of str for better performance
- Fix heap snapshot crash on eventloop thread
- Use /usr/bin/env perl for ./Configure.pl
- Don't break after ZWJ or for MALE SIGN and FEMALE SIGN
- Take into account actual allocated size of I/O buffers in gen2 promotion
  statistics
- Tweak full collection criteria in heap profiling
- Free up spesh log slots after specialization, avoiding some leaks
- Removed the getregref_ ops (unused by NQP or Perl 6)
- Removed the continuationclone op (unused by Perl 6)
- Enforce one-shot invocation of continuations
- Greatly simplify handling of call frame working register lifetimes, leading
  to consistently shorter lifetimes, less memory pressure, and faster calling
- Eliminate now-unused `tc` field in MVMFrame, saving a pointer per callframe
- Simplify args cleanup functions
- Reduce number of checks in call frame marking, making it faster
- Implement getstrfromname op, to get named Unicode sequences
- Fix GC in spesh triggered by managed mutex use
- Fix data race in inlining extop fixup
- Fix data race in callsite fixup during inlining
- Fix data race in string fixup during inlining
- Fix charname lookups of LINE FEED and CARRIAGE RETURN
- Remove Unicode 1 names and add Unicode Name Aliases
- Convert MVM_malloc+memset to MVM_calloc
- Clean out ancient "remove after rebootstrap" line
- Fix typo in MVM_CALLSTACK_REGIONS_SIZE's name
- Fix missing cleanup of managed DecodeStream
- Have DecodeStrem clean leftover char buffers
- Fix a typo in decode stream destroy function


To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.3 pkgsrc/devel/MoarVM/Makefile
cvs rdiff -u -r1.1 -r1.2 pkgsrc/devel/MoarVM/PLIST \
    pkgsrc/devel/MoarVM/distinfo
cvs rdiff -u -r1.1 -r1.2 pkgsrc/devel/MoarVM/patches/patch-build_probe.pm

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/devel/MoarVM/Makefile
diff -u pkgsrc/devel/MoarVM/Makefile:1.2 pkgsrc/devel/MoarVM/Makefile:1.3
--- pkgsrc/devel/MoarVM/Makefile:1.2    Sat Jul  9 06:38:04 2016
+++ pkgsrc/devel/MoarVM/Makefile        Sun Feb  5 12:36:59 2017
@@ -1,7 +1,6 @@
-# $NetBSD: Makefile,v 1.2 2016/07/09 06:38:04 wiz Exp $
+# $NetBSD: Makefile,v 1.3 2017/02/05 12:36:59 bsiegert Exp $
 
-DISTNAME=      MoarVM-2016.03
-PKGREVISION=   1
+DISTNAME=      MoarVM-2017.01
 CATEGORIES=    devel perl6
 MASTER_SITES=  http://www.moarvm.org/releases/
 

Index: pkgsrc/devel/MoarVM/PLIST
diff -u pkgsrc/devel/MoarVM/PLIST:1.1 pkgsrc/devel/MoarVM/PLIST:1.2
--- pkgsrc/devel/MoarVM/PLIST:1.1       Fri Apr  8 20:26:46 2016
+++ pkgsrc/devel/MoarVM/PLIST   Sun Feb  5 12:36:59 2017
@@ -1,20 +1,5 @@
-@comment $NetBSD: PLIST,v 1.1 2016/04/08 20:26:46 bsiegert Exp $
+@comment $NetBSD: PLIST,v 1.2 2017/02/05 12:36:59 bsiegert Exp $
 bin/moar
-include/libuv/android-ifaddrs.h
-include/libuv/pthread-fixes.h
-include/libuv/stdint-msvc2008.h
-include/libuv/tree.h
-include/libuv/uv-aix.h
-include/libuv/uv-bsd.h
-include/libuv/uv-darwin.h
-include/libuv/uv-errno.h
-include/libuv/uv-linux.h
-include/libuv/uv-sunos.h
-include/libuv/uv-threadpool.h
-include/libuv/uv-unix.h
-include/libuv/uv-version.h
-include/libuv/uv-win.h
-include/libuv/uv.h
 include/moar/6model/6model.h
 include/moar/6model/bootstrap.h
 include/moar/6model/containers.h
@@ -29,6 +14,7 @@ include/moar/6model/reprs/CStruct.h
 include/moar/6model/reprs/CUnion.h
 include/moar/6model/reprs/ConcBlockingQueue.h
 include/moar/6model/reprs/ConditionVariable.h
+include/moar/6model/reprs/Decoder.h
 include/moar/6model/reprs/HashAttrStore.h
 include/moar/6model/reprs/KnowHOWAttributeREPR.h
 include/moar/6model/reprs/KnowHOWREPR.h
@@ -73,6 +59,7 @@ include/moar/core/args.h
 include/moar/core/bytecode.h
 include/moar/core/bytecodedump.h
 include/moar/core/callsite.h
+include/moar/core/callstack.h
 include/moar/core/coerce.h
 include/moar/core/compunit.h
 include/moar/core/continuation.h
@@ -91,6 +78,7 @@ include/moar/core/nativecall_dyncall.h
 include/moar/core/nativecall_libffi.h
 include/moar/core/oplabels.h
 include/moar/core/ops.h
+include/moar/core/regionalloc.h
 include/moar/core/threadcontext.h
 include/moar/core/threads.h
 include/moar/core/validation.h
@@ -130,6 +118,7 @@ include/moar/mast/driver.h
 include/moar/mast/nodes.h
 include/moar/math/bigintops.h
 include/moar/math/num.h
+include/moar/memdebug.h
 include/moar/moar.h
 include/moar/platform/inttypes.h
 include/moar/platform/io.h
@@ -139,6 +128,7 @@ include/moar/platform/stdint.h
 include/moar/platform/sys.h
 include/moar/platform/threads.h
 include/moar/platform/time.h
+include/moar/profiler/heapsnapshot.h
 include/moar/profiler/instrument.h
 include/moar/profiler/log.h
 include/moar/profiler/profile.h
@@ -151,6 +141,7 @@ include/moar/spesh/facts.h
 include/moar/spesh/graph.h
 include/moar/spesh/inline.h
 include/moar/spesh/log.h
+include/moar/spesh/lookup.h
 include/moar/spesh/manipulate.h
 include/moar/spesh/optimize.h
 include/moar/spesh/osr.h
@@ -164,6 +155,7 @@ include/moar/strings/normalize.h
 include/moar/strings/ops.h
 include/moar/strings/unicode.h
 include/moar/strings/unicode_gen.h
+include/moar/strings/unicode_ops.h
 include/moar/strings/utf16.h
 include/moar/strings/utf8.h
 include/moar/strings/utf8_c8.h
Index: pkgsrc/devel/MoarVM/distinfo
diff -u pkgsrc/devel/MoarVM/distinfo:1.1 pkgsrc/devel/MoarVM/distinfo:1.2
--- pkgsrc/devel/MoarVM/distinfo:1.1    Fri Apr  8 20:26:46 2016
+++ pkgsrc/devel/MoarVM/distinfo        Sun Feb  5 12:36:59 2017
@@ -1,8 +1,8 @@
-$NetBSD: distinfo,v 1.1 2016/04/08 20:26:46 bsiegert Exp $
+$NetBSD: distinfo,v 1.2 2017/02/05 12:36:59 bsiegert Exp $
 
-SHA1 (MoarVM-2016.03.tar.gz) = e0b92cad450b35b1db1c63e676bf94dfa5c742d1
-RMD160 (MoarVM-2016.03.tar.gz) = 0c627a21dc7dbaca2f0846520cb9719240623969
-SHA512 (MoarVM-2016.03.tar.gz) = 358c48f6b1803e39841e9f287f2fe3cdde743962d9d9c34e629c18b1b7290fd8a713917f24233dd092ed68adf9a6d0aadb25bda22b3e353d40bd57ae58d70216
-Size (MoarVM-2016.03.tar.gz) = 3318299 bytes
-SHA1 (patch-build_probe.pm) = 8ea9313c1d2329239f123fb4f6c28e1f7de566c2
+SHA1 (MoarVM-2017.01.tar.gz) = 12664fa7e2e9160e36fe55fc806b7c960fe99224
+RMD160 (MoarVM-2017.01.tar.gz) = 3f0cb18b1f5d8f6da168cd15a6db6be84214a29c
+SHA512 (MoarVM-2017.01.tar.gz) = d31b292edc4468c4dd26a367944230ce02286cc601c231b175273b0c61d1fb907aced5b63c1f7426eabfd9148b17aee63dea12c74829206fb38250cd4193804f
+Size (MoarVM-2017.01.tar.gz) = 3718510 bytes
+SHA1 (patch-build_probe.pm) = 9cf0bbd99ff38715c0b8b57bcd227df18d9adfb4
 SHA1 (patch-build_setup.pm) = fd53d47bd3126b2c89d4e8d82c6bc0b6db5f4fdf

Index: pkgsrc/devel/MoarVM/patches/patch-build_probe.pm
diff -u pkgsrc/devel/MoarVM/patches/patch-build_probe.pm:1.1 pkgsrc/devel/MoarVM/patches/patch-build_probe.pm:1.2
--- pkgsrc/devel/MoarVM/patches/patch-build_probe.pm:1.1        Fri Apr  8 20:26:46 2016
+++ pkgsrc/devel/MoarVM/patches/patch-build_probe.pm    Sun Feb  5 12:36:59 2017
@@ -1,8 +1,8 @@
-$NetBSD: patch-build_probe.pm,v 1.1 2016/04/08 20:26:46 bsiegert Exp $
+$NetBSD: patch-build_probe.pm,v 1.2 2017/02/05 12:36:59 bsiegert Exp $
 
 Add LDFLAGS for Configure.pl probes so e.g. finding libffi works.
 
---- build/probe.pm.orig        2015-12-25 12:37:32.000000000 +0000
+--- build/probe.pm.orig        2016-11-16 14:34:28.000000000 +0000
 +++ build/probe.pm
 @@ -57,7 +57,7 @@ sub compile {
          push @objs, $obj;
@@ -13,3 +13,12 @@ Add LDFLAGS for Configure.pl probes so e
      system $command
          and return;
      return 1;
+@@ -102,7 +102,7 @@ EOT
+     }
+ 
+     if ($can_compile) {
+-        $command = "$config->{ld} $config->{ldout}$leaf $obj $config->{ldlibs} 2>&1";
++        $command = "$config->{ld} $ENV{'LDFLAGS'} $config->{ldout}$leaf $obj $config->{ldlibs} 2>&1";
+         $output  = `$command` || $!;
+         if ($? >> 8 == 0) {
+             $can_link = 1;