[pkgsrc/trunk]: pkgsrc/databases/cstore Initial import of cstore-0.2, a "colu...

[agc <agc%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/1aa6af9525c5
branches:  trunk
changeset: 535195:1aa6af9525c5
user:      agc <agc%pkgsrc.org@localhost>
date:      Sat Nov 10 11:33:00 2007 +0000

description:
Initial import of cstore-0.2, a "column store" relational database,
into the Packages Collection. This package is meant to support data
warehousing in a much more efficient manner than traditional RDBMSs.

        C-Store is a read-optimized relational DBMS that contrasts sharply
        with most current systems, which are write-optimized.  Among the many
        differences in its design are:

        + storage of data by column rather than by row,

        + careful coding and packing of objects into storage including main
        memory during query processing,

        + storing an overlapping collection of column-oriented projections,
        rather than the current fare of tables and indexes,

        + a non-traditional implementation of transactions which includes high
        availability and snapshot isolation for read-only transactions,

        + and the extensive use of bitmap indexes to complement B-tree
        structures.

Please note that the storage required for this package could be viewed
as being quite extreme - the standard build and data occupies in excess
of 20 GB, and that is all installed into ${PREFIX} at installation
time.

diffstat:

 databases/cstore/DESCR            |  17 +++++++++
 databases/cstore/MESSAGE          |   8 ++++
 databases/cstore/Makefile         |  73 +++++++++++++++++++++++++++++++++++++++
 databases/cstore/PLIST            |  69 ++++++++++++++++++++++++++++++++++++
 databases/cstore/distinfo         |  26 +++++++++++++
 databases/cstore/patches/patch-aa |  33 +++++++++++++++++
 databases/cstore/patches/patch-ab |  15 ++++++++
 databases/cstore/patches/patch-ac |  15 ++++++++
 databases/cstore/patches/patch-ad |  15 ++++++++
 databases/cstore/patches/patch-ae |  13 ++++++
 databases/cstore/patches/patch-af |  13 ++++++
 databases/cstore/patches/patch-ag |  16 ++++++++
 databases/cstore/patches/patch-ah |  17 +++++++++
 databases/cstore/patches/patch-ai |  13 ++++++
 databases/cstore/patches/patch-aj |   9 ++++
 databases/cstore/patches/patch-ak |  16 ++++++++
 databases/cstore/patches/patch-al |  33 +++++++++++++++++
 databases/cstore/patches/patch-ao |  23 ++++++++++++
 databases/cstore/patches/patch-ap |  13 ++++++
 databases/cstore/patches/patch-aq |  13 ++++++
 20 files changed, 450 insertions(+), 0 deletions(-)

diffs (truncated from 530 to 300 lines):

diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/DESCR
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/DESCR    Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,17 @@
+C-Store is a read-optimized relational DBMS that contrasts sharply
+with most current systems, which are write-optimized.  Among the many
+differences in its design are:
+
++ storage of data by column rather than by row,
+
++ careful coding and packing of objects into storage including main
+memory during query processing,
+
++ storing an overlapping collection of column-oriented projections,
+rather than the current fare of tables and indexes,
+
++ a non-traditional implementation of transactions which includes high
+availability and snapshot isolation for read-only transactions,
+
++ and the extensive use of bitmap indexes to complement B-tree
+structures.
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/MESSAGE
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/MESSAGE  Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,8 @@
+===========================================================================
+$NetBSD: MESSAGE,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+
+The storage requirements for this package could be considered to
+be quite large - on the order of 20 Gigabytes for the complete
+data.
+
+===========================================================================
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/Makefile
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/Makefile Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,73 @@
+# $NetBSD: Makefile,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+
+DISTNAME=      cstore0.2
+PKGNAME=       cstore-0.2
+CATEGORIES=    databases
+MASTER_SITES=  http://db.lcs.mit.edu/projects/cstore/
+MASTER_SITES+= http://db.csail.mit.edu/data/
+DISTFILES=     ${DISTNAME}${EXTRACT_SUFX} data4.tar.gz D6.data.ros.gz
+
+MAINTAINER=    agc%NetBSD.org@localhost
+HOMEPAGE=      http://db.lcs.mit.edu/projects/cstore/
+COMMENT=       Read-optimised column store relational DBMS
+
+USE_TOOLS+=    gmake perl
+USE_LANGUAGES+=        c c++
+WRKSRC=                ${WRKDIR}/cstore
+
+REPLACE_PERL=  Build/BSsrcs
+
+SUBST_CLASSES+=                prefix
+SUBST_STAGE.prefix=    post-patch
+SUBST_FILES.prefix=    Build/makefile.init
+SUBST_SED.prefix+=     -e 's|@PREFIX@|'${PREFIX:Q}'|g'
+
+DATAFILES=     D1.data.mini
+DATAFILES+=    D1.data.retsupp50sort
+DATAFILES+=    D1.shipdate.mini
+DATAFILES+=    D1.suppkey.mini
+DATAFILES+=    D10.data.ros
+DATAFILES+=    D10.data.wos
+DATAFILES+=    D2.data
+DATAFILES+=    D2.data.mini
+DATAFILES+=    D3.custkey.mini
+DATAFILES+=    D3.data.mini
+DATAFILES+=    D4.data
+DATAFILES+=    D4.data.mini
+DATAFILES+=    D5.data.mini
+DATAFILES+=    D6.data.ros
+DATAFILES+=    D6.data.wos
+DATAFILES+=    D7.data.ros
+DATAFILES+=    D7.data.wos
+DATAFILES+=    D8.data.ros
+DATAFILES+=    D8.data.wos
+DATAFILES+=    D9.data.ros
+DATAFILES+=    D9.data.wos
+DATAFILES+=    d1.lsuppkey.full
+DATAFILES+=    deltaPos.test
+DATAFILES+=    deltaPosTestBig2
+DATAFILES+=    lineitem.sorted.full
+DATAFILES+=    lshipdate.sorted.mini
+DATAFILES+=    lshipdate.sorted.tiny
+DATAFILES+=    supp.data
+
+INSTALLATION_DIRS_FROM_PLIST=  yes
+
+post-extract:
+       cd ${WRKDIR} && mv ${DATAFILES} ${WRKSRC}/data
+
+do-build:
+       cd ${WRKSRC}/src && ${SETENV} ${BUILD_ENV} ${MAKE} debug2
+       cd ${WRKSRC}/src &&  ./cstoreqptest 0 createData.cnf global.cnf
+
+do-install:
+       ${INSTALL_PROGRAM} ${WRKSRC}/src/cstoreqptest ${PREFIX}/bin/cstoreqp
+       ${INSTALL_DATA_DIR} ${PREFIX}/cstore/data
+       cd ${WRKSRC}/data && pax -rwpe . ${PREFIX}/cstore/data
+       ${INSTALL_DATA_DIR} ${PREFIX}/share/doc/cstore
+       cd ${WRKSRC}/doc && pax -rwpe . ${PREFIX}/share/doc/cstore
+
+.include "../../archivers/lzo/buildlink3.mk"
+.include "../../databases/db4/buildlink3.mk"
+
+.include "../../mk/bsd.pkg.mk"
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/PLIST
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/PLIST    Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,69 @@
+@comment $NetBSD: PLIST,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+bin/cstoreqp
+cstore/data/D1.data.mini
+cstore/data/D1.data.retsupp50sort
+cstore/data/D1.shipdate.mini
+cstore/data/D1.suppkey.mini
+cstore/data/D10.data.ros
+cstore/data/D10.data.ros.1
+cstore/data/D10.data.ros.2
+cstore/data/D10.data.wos
+cstore/data/D2.data
+cstore/data/D2.data.1
+cstore/data/D2.data.2
+cstore/data/D2.data.3
+cstore/data/D2.data.mini
+cstore/data/D3.custkey.mini
+cstore/data/D3.data.mini
+cstore/data/D4.data
+cstore/data/D4.data.1
+cstore/data/D4.data.2
+cstore/data/D4.data.3
+cstore/data/D4.data.mini
+cstore/data/D5.data.mini
+cstore/data/D6.data.ros
+cstore/data/D6.data.ros.1
+cstore/data/D6.data.ros.2
+cstore/data/D6.data.ros.3
+cstore/data/D6.data.ros.4
+cstore/data/D6.data.ros.5
+cstore/data/D6.data.ros.6
+cstore/data/D6.data.ros.7
+cstore/data/D6.data.ros.8
+cstore/data/D6.data.wos
+cstore/data/D7.data.ros
+cstore/data/D7.data.ros.1
+cstore/data/D7.data.ros.2
+cstore/data/D7.data.ros.3
+cstore/data/D7.data.wos
+cstore/data/D8.data.ros
+cstore/data/D8.data.ros.1
+cstore/data/D8.data.ros.2
+cstore/data/D8.data.wos
+cstore/data/D9.data.ros
+cstore/data/D9.data.ros.1
+cstore/data/D9.data.ros.2
+cstore/data/D9.data.ros.3
+cstore/data/D9.data.wos
+cstore/data/d1.lsuppkey.full
+cstore/data/d1.lsuppkey.full.1
+cstore/data/deltaPos.test
+cstore/data/deltaPosTestBig2
+cstore/data/flushCache
+cstore/data/getData
+cstore/data/lineitem.sorted.full
+cstore/data/lineitem.sorted.full.1
+cstore/data/lshipdate.sorted.mini
+cstore/data/lshipdate.sorted.tiny
+cstore/data/makeLargeFile
+cstore/data/supp.data
+share/doc/cstore/CatalogPersistent.txt
+share/doc/cstore/PlanGenerator_20050628.doc
+share/doc/cstore/README
+share/doc/cstore/ReleaseOverview.txt
+share/doc/cstore/Union.doc
+share/doc/cstore/thesis.pdf
+share/doc/cstore/vldb.pdf
+@dirrm share/doc/cstore
+@dirrm cstore/data
+@dirrm cstore
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/distinfo
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/distinfo Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,26 @@
+$NetBSD: distinfo,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+
+SHA1 (D6.data.ros.gz) = 3efe5c0b4675dcdc2c89cf9a1b6f305663d153a5
+RMD160 (D6.data.ros.gz) = 20b99b066b953424dcb0b5435e9731806fac5d52
+Size (D6.data.ros.gz) = 992143667 bytes
+SHA1 (cstore0.2.tar.gz) = 51f3eac1adde49f72c454ed3895f5954ee17ad46
+RMD160 (cstore0.2.tar.gz) = bb510ddcd3dbc5683059f313e6fb0335ad7c71dd
+Size (cstore0.2.tar.gz) = 4292918 bytes
+SHA1 (data4.tar.gz) = aa07edc8a13bb194774af49cd6a1f5a95574aefd
+RMD160 (data4.tar.gz) = 9914cae55abfef9d880f14a004f4f608180ec905
+Size (data4.tar.gz) = 1348695152 bytes
+SHA1 (patch-aa) = 52f482fa451a16f85c727cd9953510ccb04836ab
+SHA1 (patch-ab) = 92bb9855f834ffb7b54f0ca99efd8f1bf3c9c65e
+SHA1 (patch-ac) = b495ba01f725c3b35ac1cd92a640a5978510dc19
+SHA1 (patch-ad) = 4abd94414fb16281db481626a048fcd12d38a3eb
+SHA1 (patch-ae) = c5fa606caafa61c0feac0ce12c3b479a124f392d
+SHA1 (patch-af) = 8dbdc8d20bc56643a86bcbdff7ee631daadbad9a
+SHA1 (patch-ag) = eb72568f71ec1372e0b69cad3e8b2f7d830597e4
+SHA1 (patch-ah) = fca6811c7a53662e5cf41956867b39c974a96876
+SHA1 (patch-ai) = a294ac30adf46ec031925363a0174c8dcd8b4c2a
+SHA1 (patch-aj) = d99dfbd23487bce2ae97f48474790d341437d7d4
+SHA1 (patch-ak) = 3c57bc19f340b49e88b4ae605cb0e04ec8f3a0ce
+SHA1 (patch-al) = a0683ed97a61438bfa4cae734955905cc97d6410
+SHA1 (patch-ao) = ad649ccb0c599b8ad46ba7120d0b074e7eabda27
+SHA1 (patch-ap) = c7058ab97051c8421e26344431929fb420ca0076
+SHA1 (patch-aq) = 827992abf73b976d9ac2dd121d14212f7a61452c
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/patches/patch-aa
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/patches/patch-aa Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,33 @@
+$NetBSD: patch-aa,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+
+--- Build/makefile.init        2007/10/01 11:20:34     1.1
++++ Build/makefile.init        2007/10/01 11:24:09
+@@ -1,10 +1,11 @@
+ CC:=$(WTF) g++
+ 
+ CSTORE_INCLUDE := src
+-SLEEPYCAT_DIRECTORY := /usr/local/BerkeleyDB.4.2
+-SLEEPYCAT_INCLUDE := $(SLEEPYCAT_DIRECTORY)/include/
++SLEEPYCAT_DIRECTORY := @PREFIX@
++SLEEPYCAT_INCLUDE := $(SLEEPYCAT_DIRECTORY)/include/db4
+ SLEEPYCAT_LIB := $(SLEEPYCAT_DIRECTORY)/lib
+-LZO_LIB := /home/dna/huff/lzo-1.08/src/.libs
++LZO_INCLUDE := @PREFIX@/include/lzo
++LZO_LIB := @PREFIX@/lib
+ 
+ CFLAGS  := -O0 -g3 -Wall -c -Wno-deprecated -fmessage-length=0
+ #override CFLAGS with optimization flags if so desired
+@@ -17,10 +18,10 @@
+ endif
+ 
+ RULEFLAGS := -MM -MG -P -w 
+-IFLAGS        := -I$(SLEEPYCAT_INCLUDE) -I$(CSTORE_INCLUDE) 
+-LFLAGS        := -L$(SLEEPYCAT_LIB) -L$(LZO_LIB)
++IFLAGS        := -I$(SLEEPYCAT_INCLUDE) -I$(CSTORE_INCLUDE) -I$(LZO_INCLUDE)
++LFLAGS        := -L$(SLEEPYCAT_LIB) -L$(LZO_LIB) -Wl,-R$(SLEEPYCAT_LIB) -Wl,-R$(LZO_LIB)
+ OPTS    := -Wl,--demangle
+-LIBS  := -ldb-4.2 -ldb_cxx-4.2  -lpthread -llzo
++LIBS  := -ldb4-4.5 -ldb4_cxx-4.5  -lpthread -llzo2
+ #override OPTS with gprof flags
+ ifdef __CSTORE_GPROF__
+       OPTS    := -pg -Wl,--demangle
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/patches/patch-ab
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/patches/patch-ab Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,15 @@
+$NetBSD: patch-ab,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+
+--- src/UnitTests/BAndOpTest.h 2007/11/07 20:18:13     1.1
++++ src/UnitTests/BAndOpTest.h 2007/11/07 20:19:06
+@@ -60,8 +60,8 @@
+       BAndOpTest();
+       virtual ~BAndOpTest();
+       bool run(Globals* g, const vector<string>&);
+-  bool BAndOpTest::runTestCase1(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
+-  bool BAndOpTest::runTestCase2(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
++  bool runTestCase1(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
++  bool runTestCase2(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
+ };
+ 
+ #endif //_BANDOPTEST_H_
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/patches/patch-ac
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/patches/patch-ac Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,15 @@
+$NetBSD: patch-ac,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+
+--- src/UnitTests/BOrOpTest.h  2007/11/07 20:19:59     1.1
++++ src/UnitTests/BOrOpTest.h  2007/11/07 20:20:14
+@@ -60,8 +60,8 @@
+       BOrOpTest();
+       virtual ~BOrOpTest();
+       bool run(Globals* g, const vector<string>&);
+-  bool BOrOpTest::runTestCase1(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
+-  bool BOrOpTest::runTestCase2(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
++  bool runTestCase1(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
++  bool runTestCase2(unsigned int leftRunLengthDiff_, unsigned int rightRunLengthDiff_, unsigned int leftStartPosDiff_, unsigned int rightStartPosDiff_);
+ };
+ 
+ #endif //_BOROPTEST_H_
diff -r c2314d1c1a3d -r 1aa6af9525c5 databases/cstore/patches/patch-ad
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/databases/cstore/patches/patch-ad Sat Nov 10 11:33:00 2007 +0000
@@ -0,0 +1,15 @@
+$NetBSD: patch-ad,v 1.1.1.1 2007/11/10 11:33:00 agc Exp $
+
+--- src/UnitTests/BNotOpTest.h 2007/11/07 20:21:01     1.1
++++ src/UnitTests/BNotOpTest.h 2007/11/07 20:21:17
+@@ -59,8 +59,8 @@
+       BNotOpTest();
+       virtual ~BNotOpTest();
+       bool run(Globals* g, const vector<string>&);