Re: "hg clone https://anonhg.netbsd.org/src/" still aborts, but...

[Steffen Nurpmeso <steffen%sdaoden.eu@localhost>]

Greg A. Woods wrote in
<m1jjfD9-0036tR0@more.local>:
 |At Fri, 12 Jun 2020 00:26:26 -0700, "Greg A. Woods" <woods%planix.ca@localhost> \
 |wrote:
 |Subject: Re: "hg clone https://anonhg.netbsd.org/src/"; still aborts, but...
 |>
 |> I'll now fire up a new "git clone" next for a more up-to-date
 |> comparison.  There will be another rsync && cvs start during this, just
 |> to be fair.  :-)
 |
 |And it's done.  The whole "git clone" completed far faster than just the
 |last "hg checkout trunk" step (i.e. in just under 1.5hrs, 36mins faster
 |than the HG checkout alone, and 3.5hrs faster than the whole "hg clone"):
 |
 |00:01 [685] $ time git clone https://github.com/NetBSD/src g-NetBSD-src-\
 |test
 |Cloning into 'g-NetBSD-src-test'...
 |remote: Enumerating objects: 1274, done.
 |remote: Counting objects: 100% (1274/1274), done.
 |remote: Compressing objects: 100% (875/875), done.
 |remote: Total 5117234 (delta 642), reused 675 (delta 396), pack-reused \
 |5115960
 |Receiving objects: 100% (5117234/5117234), 1.87 GiB | 2.42 MiB/s, done.
 |Resolving deltas: 100% (3876350/3876350), done.
 |Checking out files: 100% (171999/171999), done.
 | 5061.00s real   510.15s user   409.84s system
 |01:26 [686] $
 |
 |The last step, the checkout, took most of the time, at least an hour, so
 |on a local fast disk that wasn't otherwise being bombarded by background
 |activity, it would have been decently fast and arguably "usable", all
 |things considered.

Fwiw, i also cloned NetBSD git from github again, just yesterday!
I edited config (added netbsd-8 and netbsd-9 which i had not yet,
in the old config i carried along):

  -rw-r----- 1 steffen code 322 Jun 11 16:40 config

then cloned:

  #?0|kent:net-src.git$ git fetch
  remote: Enumerating objects: 1586, done.
  remote: Counting objects: 100% (1586/1586), done.
  remote: Compressing objects: 100% (1090/1090), done.
  remote: Total 3139135 (delta 730), reused 717 (delta 494), pack-reused 3137549
  Receiving objects: 100% (3139135/3139135), 1.27 GiB | 1.31 MiB/s, done.
  Resolving deltas: 100% (2458013/2458013), done.
  From https://github.com/NetBSD/src
   * [new branch]      trunk      -> origin/trunk
   * [new branch]      netbsd-8   -> origin/netbsd-8
   * [new branch]      netbsd-9   -> origin/netbsd-9
  #?0|kent:net-src.git$ ll .git/objects/pack/
  total 1426248K
  drwxr-s--T 1 steffen code         16 Apr 27  2019 ../

(Had it all the time, but not cloned, restricted internet
bandwidth, etc. etc.  Old box could not handle such large repos at
all.  Now i am complete again.)  (But 9front is missing, it uses
Mercurial that everybody drops support for, i.e., even bitbucket.)

  -r--r----- 1 steffen code 1372573771 Jun 11 16:58 pack-70bb3164693bdcc104ac5f8e57d93e28c63fd187.pack
  -r--r----- 1 steffen code   87896852 Jun 11 16:58 pack-70bb3164693bdcc104ac5f8e57d93e28c63fd187.idx
  drwxr-s--T 1 steffen code        198 Jun 11 16:58 ./

Sometimes 2 Mbit/s, really good connection i had.  18 Minutes!!

  #?0|kent:net-src.git$ git gcap
  Enumerating objects: 3139135, done.
  Counting objects: 100% (3139135/3139135), done.
  Delta compression using up to 4 threads
  Compressing objects: 100% (3105124/3105124), done.
  Writing objects: 100% (3139135/3139135), done.
  Total 3139135 (delta 2501467), reused 610835 (delta 0), pack-reused 0
  Expanding reachable commits in commit graph: 280837, done.
  #?0|kent:net-src.git$ ll .git/objects/pack/
  total 1188348K
  drwxr-s--T 1 steffen code         16 Apr 27  2019 ../
  -r--r----- 1 steffen code 1128965592 Jun 11 17:19 pack-4dba9b43ee8e134c8a884eed2356916d7f3aabdd.pack
  -r--r----- 1 steffen code   87896852 Jun 11 17:19 pack-4dba9b43ee8e134c8a884eed2356916d7f3aabdd.idx
  drwxr-s--T 1 steffen code        198 Jun 11 17:19 ./

Hell i am *so* happy to have this new (well, a 14 months) box with
four processors and one of these unbelievable NVME SSDs which
scrubes at 1.2 GB/s!!  <20 Minutes to fully garbage collect and
prune a repository of over three millions objects and over
a Gigabyte of data, that is sheer grazy.  In the past i had git
dying after six and more hours, because it could not make its way
through on the old 2 GB RAM machine.

And it is always astonishing to see that only FreeBSD manages to
create a garbage collected repository that is larger than the
Linux kernel (of which i have linux-4.19.y and linux-5.4.y only).

I have not used it yet, but git now offers partial checkouts, so
with a shallow clone and a partial one-directory checkout the last
point of criticism i had has finally vanished.  I for one am so
happy to have this version control system, it cannot be said.
Really.  About ten years now, and the feeling stays, and even gets
better.  Yes.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)