pkgsrc-Changes-HG archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
[pkgsrc/trunk]: pkgsrc/pkgtools/pbulk Initial import of pbulk, the new pkgsrc...
details: https://anonhg.NetBSD.org/pkgsrc/rev/1e5108b1988b
branches: trunk
changeset: 530237:1e5108b1988b
user: joerg <joerg%pkgsrc.org@localhost>
date: Tue Jun 19 19:49:55 2007 +0000
description:
Initial import of pbulk, the new pkgsrc bulk build framework.
Discussion of various parts of the design with jlam@, wiz@, tls@
and many other developers. Special thanks to David Maxwell for
testing the initial prototype and finding some bugs with
Coverity Prevent.
Supported by Google's Summer of Code 2007 project.
OK for import during the freeze: jlam@
>From DESCR:
pbulk is the modular bulk build framework for pkgsrc.
This package contains:
- pbulk-scan, a program to scan the entire pkgsrc tree
or a list of directories therein for packages and
dependencies. Distributed operation using a master/client
mode is supported.
- pbulk-resolve, a program to resolve the dependencies
from the output of pbulk-scan
- pbulk-build, the build scheduler. Distributed builds via TCP
are supported.
- bulkbuild and related scripts to implement full and limited
bulk builds on top of those programs.
The pbulk framework is considered experimental.
diffstat:
pkgtools/pbulk/DESCR | 19 +
pkgtools/pbulk/Makefile | 53 +
pkgtools/pbulk/PLIST | 29 +
pkgtools/pbulk/files/pbulk/Makefile | 5 +
pkgtools/pbulk/files/pbulk/Makefile.inc | 18 +
pkgtools/pbulk/files/pbulk/lib/Makefile | 11 +
pkgtools/pbulk/files/pbulk/lib/alloc.c | 95 ++
pkgtools/pbulk/files/pbulk/lib/atomic.c | 83 +
pkgtools/pbulk/files/pbulk/lib/event.c | 146 +++
pkgtools/pbulk/files/pbulk/lib/exec.c | 82 +
pkgtools/pbulk/files/pbulk/lib/match.c | 482 +++++++++++
pkgtools/pbulk/files/pbulk/lib/netaddr.c | 74 +
pkgtools/pbulk/files/pbulk/lib/pbulk.h | 65 +
pkgtools/pbulk/files/pbulk/lib/read_child.c | 84 +
pkgtools/pbulk/files/pbulk/lib/read_file.c | 71 +
pkgtools/pbulk/files/pbulk/pbuild/Makefile | 6 +
pkgtools/pbulk/files/pbulk/pbuild/client.c | 116 ++
pkgtools/pbulk/files/pbulk/pbuild/jobs.c | 555 +++++++++++++
pkgtools/pbulk/files/pbulk/pbuild/master.c | 290 ++++++
pkgtools/pbulk/files/pbulk/pbuild/pbuild.c | 230 +++++
pkgtools/pbulk/files/pbulk/pbuild/pbuild.h | 92 ++
pkgtools/pbulk/files/pbulk/pbuild/pbulk-build.1 | 132 +++
pkgtools/pbulk/files/pbulk/pbuild/stat.c | 108 ++
pkgtools/pbulk/files/pbulk/pbulk.conf | 69 +
pkgtools/pbulk/files/pbulk/presolve/Makefile | 8 +
pkgtools/pbulk/files/pbulk/presolve/pbulk-resolve.1 | 77 +
pkgtools/pbulk/files/pbulk/presolve/presolve.c | 426 +++++++++
pkgtools/pbulk/files/pbulk/pscan/Makefile | 6 +
pkgtools/pbulk/files/pbulk/pscan/client.c | 120 ++
pkgtools/pbulk/files/pbulk/pscan/jobs.c | 267 ++++++
pkgtools/pbulk/files/pbulk/pscan/master.c | 240 +++++
pkgtools/pbulk/files/pbulk/pscan/pbulk-scan.1 | 95 ++
pkgtools/pbulk/files/pbulk/pscan/pscan.c | 243 +++++
pkgtools/pbulk/files/pbulk/pscan/pscan.h | 60 +
pkgtools/pbulk/files/pbulk/scripts/Makefile | 13 +
pkgtools/pbulk/files/pbulk/scripts/build | 64 +
pkgtools/pbulk/files/pbulk/scripts/build-client-start | 10 +
pkgtools/pbulk/files/pbulk/scripts/bulkbuild | 12 +
pkgtools/pbulk/files/pbulk/scripts/client-clean | 11 +
pkgtools/pbulk/files/pbulk/scripts/compute-packages.awk | 73 +
pkgtools/pbulk/files/pbulk/scripts/create-broken-graph.awk | 91 ++
pkgtools/pbulk/files/pbulk/scripts/create-report-html.awk | 244 +++++
pkgtools/pbulk/files/pbulk/scripts/create-report-txt.awk | 197 ++++
pkgtools/pbulk/files/pbulk/scripts/create-report.awk | 93 ++
pkgtools/pbulk/files/pbulk/scripts/pkg-build | 102 ++
pkgtools/pbulk/files/pbulk/scripts/pkg-up-to-date | 64 +
pkgtools/pbulk/files/pbulk/scripts/pre-build | 55 +
pkgtools/pbulk/files/pbulk/scripts/report | 103 ++
pkgtools/pbulk/files/pbulk/scripts/scan | 88 ++
pkgtools/pbulk/files/pbulk/scripts/scan-client-start | 10 +
pkgtools/pbulk/files/pbulk/scripts/upload | 45 +
51 files changed, 5732 insertions(+), 0 deletions(-)
diffs (truncated from 5936 to 300 lines):
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/DESCR
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/DESCR Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,19 @@
+pbulk is the modular bulk build framework for pkgsrc.
+
+This package contains:
+
+- pbulk-scan, a program to scan the entire pkgsrc tree
+ or a list of directories therein for packages and
+ dependencies. Distributed operation using a master/client
+ mode is supported.
+
+- pbulk-resolve, a program to resolve the dependencies
+ from the output of pbulk-scan
+
+- pbulk-build, the build scheduler. Distributed builds via TCP
+ are supported.
+
+- bulkbuild and related scripts to implement full and limited
+ bulk builds on top of those programs.
+
+The pbulk framework is considered experimental.
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/Makefile
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/Makefile Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,53 @@
+# $NetBSD: Makefile,v 1.1.1.1 2007/06/19 19:49:55 joerg Exp $
+
+DISTNAME= pbulk-0.1
+CATEGORIES= pkgtools
+MASTER_SITES= # empty
+DISTFILES= # empty
+
+MAINTAINER= joerg%NetBSD.org@localhost
+HOMEPAGE= ftp://ftp.NetBSD.org/pub/NetBSD/packages/pkgsrc/doc/pkgsrc.html
+COMMENT= Modular bulk build framework
+
+PKG_DESTDIR_SUPPORT= user-destdir
+
+WRKSRC= ${WRKDIR}/pbulk
+EXTRACT_ONLY= # empty
+NO_CHECKSUM= YES
+
+USE_TOOLS+= awk:run bzip2:run digest:run gzip:run ident:run make:run \
+ mail:run
+DEPENDS+= rsync-[0-9]*:../../net/rsync
+
+SUBST_CLASSES+= tools
+SUBST_STAGE.tools= pre-configure
+SUBST_MESSAGE.tools= Fixing references to tools
+SUBST_FILES.tools= pbulk.conf scripts/build scripts/build-client-start \
+ scripts/bulkbuild scripts/client-clean \
+ scripts/pkg-build scripts/pkg-up-to-date scripts/pre-build \
+ scripts/report scripts/scan scripts/scan-client-start scripts/upload \
+ scripts/compute-packages.awk scripts/create-broken-graph.awk \
+ scripts/create-report-html.awk scripts/create-report-txt.awk \
+ scripts/create-report.awk
+SUBST_VARS.tools= AWK BZIP2 DIGEST GZIP_CMD IDENT MAKE MAIL_CMD \
+ PBULK_CONFIG PKG_INFO PREFIX SH
+
+CONF_FILES+= share/examples/pbulk/pbulk.conf ${PKG_SYSCONFDIR}/pbulk.conf
+
+PBULK_CONFIG= ${PKG_SYSCONFDIR}/pbulk.conf
+
+INSTALLATION_DIRS= bin libexec/pbulk \
+ ${PKGMANDIR}/cat1 ${PKGMANDIR}/man1 \
+ share/examples/pbulk
+INSTALL_ENV+= MANDIR=${PREFIX}/${PKGMANDIR}
+
+.include "../../mk/bsd.prefs.mk"
+
+do-extract:
+ ${CP} -r ${FILESDIR}/pbulk ${WRKDIR}
+
+post-install:
+ ${INSTALL_DATA} ${WRKSRC}/pbulk.conf ${DESTDIR}${PREFIX}/share/examples/pbulk/pbulk.conf
+
+.include "../../devel/libevent/buildlink3.mk"
+.include "../../mk/bsd.pkg.mk"
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/PLIST
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/PLIST Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,29 @@
+@comment $NetBSD: PLIST,v 1.1.1.1 2007/06/19 19:49:55 joerg Exp $
+bin/bulkbuild
+bin/pbulk-build
+bin/pbulk-resolve
+bin/pbulk-scan
+libexec/pbulk/build
+libexec/pbulk/build-client-start
+libexec/pbulk/client-clean
+libexec/pbulk/compute-packages
+libexec/pbulk/create-broken-graph
+libexec/pbulk/create-report
+libexec/pbulk/create-report-html
+libexec/pbulk/create-report-txt
+libexec/pbulk/pkg-build
+libexec/pbulk/pkg-up-to-date
+libexec/pbulk/pre-build
+libexec/pbulk/report
+libexec/pbulk/scan
+libexec/pbulk/scan-client-start
+libexec/pbulk/upload
+man/cat1/pbulk-build.0
+man/cat1/pbulk-resolve.0
+man/cat1/pbulk-scan.0
+man/man1/pbulk-build.1
+man/man1/pbulk-resolve.1
+man/man1/pbulk-scan.1
+share/examples/pbulk/pbulk.conf
+@dirrm share/examples/pbulk
+@dirrm libexec/pbulk
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/files/pbulk/Makefile
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/files/pbulk/Makefile Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,5 @@
+# $NetBSD: Makefile,v 1.1.1.1 2007/06/19 19:49:55 joerg Exp $
+
+SUBDIR= lib presolve pscan pbuild scripts
+
+.include <bsd.subdir.mk>
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/files/pbulk/Makefile.inc
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/files/pbulk/Makefile.inc Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,18 @@
+# $NetBSD: Makefile.inc,v 1.1.1.1 2007/06/19 19:49:55 joerg Exp $
+
+BINDIR?= ${PREFIX}/bin
+
+WARNS= 4
+
+.if !defined(LIBPBULK_ONLY)
+CPPFLAGS+= -I${.CURDIR}/../lib
+DPADD+= ${.OBJDIR}/../lib/libpbulk.a
+LDADD+= -L${.OBJDIR}/../lib -lpbulk
+.endif
+
+.if !defined(NO_LIBEVENT_NEEDED)
+CPPFLAGS+=
+.if !defined(LIBPBULK_ONLY)
+LDADD+= -levent
+.endif
+.endif
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/files/pbulk/lib/Makefile
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/files/pbulk/lib/Makefile Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,11 @@
+# $NetBSD: Makefile,v 1.1.1.1 2007/06/19 19:49:59 joerg Exp $
+
+LIB= pbulk
+MKPRIVATELIB= yes
+MKLINKLIB= no
+MKPIC= no
+SRCS= alloc.c atomic.c event.c exec.c match.c read_child.c read_file.c netaddr.c
+
+LIBPBULK_ONLY= # defined
+
+.include <bsd.lib.mk>
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/files/pbulk/lib/alloc.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/files/pbulk/lib/alloc.c Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,95 @@
+/* $NetBSD: alloc.c,v 1.1.1.1 2007/06/19 19:49:58 joerg Exp $ */
+
+/*-
+ * Copyright (c) 2007 Joerg Sonnenberger <joerg%NetBSD.org@localhost>.
+ * All rights reserved.
+ *
+ * This code was developed as part of Google's Summer of Code 2007 program.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
+ * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
+ * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <err.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "pbulk.h"
+
+char *
+xasprintf(const char *fmt, ...)
+{
+ va_list ap;
+ char *buf;
+
+ va_start(ap, fmt);
+ if (vasprintf(&buf, fmt, ap) == -1)
+ err(1, "asprintf failed");
+ va_end(ap);
+ return buf;
+}
+
+void *
+xmalloc(size_t len)
+{
+ void *ptr;
+
+ if ((ptr = malloc(len)) == NULL)
+ err(1, "malloc failed");
+ return ptr;
+}
+
+void *
+xrealloc(void *buf, size_t len)
+{
+ void *ptr;
+
+ if ((ptr = realloc(buf, len)) == NULL)
+ err(1, "realloc failed");
+ return ptr;
+}
+
+char *
+xstrdup(const char *str)
+{
+ char *buf;
+
+ if ((buf = strdup(str)) == NULL)
+ err(1, "strdup failed");
+ return buf;
+}
+
+char *
+xstrndup(const char *str, size_t len)
+{
+ char *buf;
+
+ buf = xmalloc(len + 1);
+ strncpy(buf, str, len);
+ buf[len] = '\0';
+
+ return buf;
+}
diff -r 2044588ec02b -r 1e5108b1988b pkgtools/pbulk/files/pbulk/lib/atomic.c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/pkgtools/pbulk/files/pbulk/lib/atomic.c Tue Jun 19 19:49:55 2007 +0000
@@ -0,0 +1,83 @@
+/* $NetBSD: atomic.c,v 1.1.1.1 2007/06/19 19:49:59 joerg Exp $ */
+
+/*-
+ * Copyright (c) 2007 Joerg Sonnenberger <joerg%NetBSD.org@localhost>.
+ * All rights reserved.
+ *
+ * This code was developed as part of Google's Summer of Code 2007 program.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING,
+ * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
+ * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/stat.h>
+#include <err.h>
+#include <fcntl.h>
+#include <limits.h>
+#include <string.h>
Home |
Main Index |
Thread Index |
Old Index