Re: wc(1) Output and POSIX

[Alistair Crooks <agc%pkgsrc.org@localhost>]

FYI,

I use a very low-tech awk script to create tabular output into json. I've
yet to find a use-case for converting into html, but I'm sure it exists and
is just as easily implemented in a few lines of awk.

https://agc.ac/images/json-20161228.png

For those of you who can't see the image, script is:

awk -v opts="$@" '

BEGIN { pretty = (opts ~ /pretty/) ? "\n" : ""; printf("[%s", pretty) }

END { print "]" }

{

        printf("%s%s[", (NR == 1) ? "" : ",", (NR == 1) ? "" : pretty);

        for (i = 1 ; i <= NF ; i++) {

                printf("\"%s\"%s", $i, (i < NF) ? "," : "");

        }

        printf("]");

}

'

and sample usage is:

[17:14:25] agc@netbsd-002 ~/local/2json-20150618 [13879] > /bin/df -h |
./json
[["Filesystem","Size","Used","Avail","%Cap","Mounted","on"],["/dev/wd0a","19G","15G","3.7G","79%","/"],["/dev/wd1a","49G","19G","28G","40%","/d"],["kernfs","1.0K","1.0K","0B","100%","/kern"],["ptyfs","1.0K","1.0K","0B","100%","/dev/pts"],["procfs","4.0K","4.0K","0B","100%","/proc"],["tmpfs","256M","0B","256M","0%","/var/shm"]]

This works on most tabular output commands I've found, since all it's doing
is omitting the space. It's up to the other end to reconstruct or parse
appropriately.

Regards,
Alistair

PS. As the original poster wanted wc(1) output, here's another example:

[17:22:29] agc@netbsd-002 /usr/share/dict [13894] > l
total 5944
drwxr-xr-x   4 root  wheel      512 Sep 13 17:02 .
drwxr-xr-x  33 root  wheel      512 Apr 29  2016 ..
-r--r--r--   1 root  wheel      516 Jun  5  2016 README
-r--r--r--   1 root  wheel     3317 Jun  5  2016 american
-r--r--r--   1 root  wheel     3512 Jun  5  2016 british
-r--r--r--   1 root  wheel      620 Jun  5  2016 eign
drwxr-xr-x   2 root  wheel      512 Apr 29  2016 papers
-r--r--r--   1 root  wheel     8640 Jun  5  2016 propernames
drwxr-xr-x   2 root  wheel      512 Sep 13 17:02 special
-r--r--r--   1 root  wheel     8216 Jun  5  2016 stop
-r--r--r--   2 root  wheel  2487343 Jun 27  2016 web2
-r--r--r--   1 root  wheel  1012730 Jun  5  2016 web2a
-r--r--r--   2 root  wheel  2487343 Jun 27  2016 words
[17:22:30] agc@netbsd-002 /usr/share/dict [13895] > wc * | json
[["12","69","516","README"],["353","353","3317","american"],["353","353","3512","british"],["133","133","620","eign"],["0","2","512","papers"],["1323","1323","8640","propernames"],["0","4","512","special"],["968","968","8216","stop"],["235001","235001","2487343","web2"],["76205","121847","1012730","web2a"],["235001","235001","2487343","words"],["549349","595054","6013261","total"]]
[17:22:39] agc@netbsd-002 /usr/share/dict [13896] > wc * | json | jq .
[
  [
    "12",
    "69",
    "516",
    "README"
  ],
  [
    "353",
    "353",
    "3317",
    "american"
  ],
  [
    "353",
    "353",
    "3512",
    "british"
  ],
  [
    "133",
    "133",
    "620",
    "eign"
  ],
  [
    "0",
    "2",
    "512",
    "papers"
  ],
  [
    "1323",
    "1323",
    "8640",
    "propernames"
  ],
  [
    "0",
    "4",
    "512",
    "special"
  ],
  [
    "968",
    "968",
    "8216",
    "stop"
  ],
  [
    "235001",
    "235001",
    "2487343",
    "web2"
  ],
  [
    "76205",
    "121847",
    "1012730",
    "web2a"
  ],
  [
    "235001",
    "235001",
    "2487343",
    "words"
  ],
  [
    "549349",
    "595054",
    "6013261",
    "total"
  ]
]
[17:23:10] agc@netbsd-002 /usr/share/dict [13897] >



On 27 December 2016 at 22:15, Kamil Rytarowski <n54%gmx.com@localhost> wrote:

>
>
> On 28.12.2016 04:14, David Holland wrote:
> > On Wed, Dec 28, 2016 at 02:54:00AM +0000, David Holland wrote:
> >  > I... think the library itself looks pretty gross,
> >
> > to wit, it's doing almost exactly the same things as proplib or any of
> > the proposed proplib replacements, just with a very slightly different
> > goal.
> >
>
> The libxo library as it's done in FreeBSD is an improvement for my
> use-case, however it's not ideal. I mean, outputing from system/native
> tools stream in the standarized format like json/xml/yaml/etc.
>
> I want to serialize the state of a running system and to achieve this I
> need to serialize all system tools from *bin, that are not trivial
> filesystem accessors (I would treat wc(1) as a program in this group -
> but if there is a use-case to render such property of a file, why not);
> and render everything into unified format and with a tool like osquery
> from Facebook.
>
>    Performant Endpoint Visibility
>
>    osquery allows you to easily ask questions about your Linux,
>    Windows, and OS X infrastructure. Whether your goal is intrusion
>    detection, infrastructure reliability, or compliance, osquery gives
>    you the ability to empower and inform a broad set of organizations
>    within your company.
>
>    -- https://osquery.io/
>
> My goal is to make frontend with Lua (meta)tables and (lib)SQL (reuse
> sqlite?), while translating it to osquery should be straight-forward.
>
> I know a potential user interested to use graphql & json for the same
> task to introspect system and orchestrate system utilities like debuggers