From rminnich at gmail.com  Tue May  2 03:25:33 2017
From: rminnich at gmail.com (ron minnich)
Date: Mon, 01 May 2017 17:25:33 +0000
Subject: [TUHS] dmr note on BSD's sins
Message-ID: <CAP6exYK-iikmM1p4f-bNHh0_HSCwPGPCusp1f1S45vhEYmZbXg@mail.gmail.com>

OK, I recall a note dmr wrote probably in the late 70s/early 80s when folks
at UCB had (iirc) extended the symbol name size in C programs to
essentially unlimited. This followed on (iirc) file names going beyond 14
characters.

The rough outline was that dmr was calling out the revisions for being too
general, and the phrase "BSD sins" sticks in my head (sins as a verb).

I'm reminded of this by something that happened with some interns recently,
as they wanted to make something immensely complex to cover a case that
basically never happened. I was trying to point out that you can go
overboard on that sort of thing, and it would have been nice to have such a
quote handy -- anyone else remember it?

ron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170501/836c2a70/attachment.html>

From toby at telegraphics.com.au  Tue May  2 04:09:07 2017
From: toby at telegraphics.com.au (Toby Thain)
Date: Mon, 1 May 2017 14:09:07 -0400
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <CAP6exYK-iikmM1p4f-bNHh0_HSCwPGPCusp1f1S45vhEYmZbXg@mail.gmail.com>
References: <CAP6exYK-iikmM1p4f-bNHh0_HSCwPGPCusp1f1S45vhEYmZbXg@mail.gmail.com>
Message-ID: <1189db4c-a1f5-6d03-45ed-81f1a2c51685@telegraphics.com.au>

On 2017-05-01 1:25 PM, ron minnich wrote:
> OK, I recall a note dmr wrote probably in the late 70s/early 80s when
> folks at UCB had (iirc) extended the symbol name size in C programs to
> essentially unlimited. This followed on (iirc) file names going beyond
> 14 characters.
>
> The rough outline was that dmr was calling out the revisions for being
> too general, and the phrase "BSD sins" sticks in my head (sins as a verb).
>
> I'm reminded of this by something that happened with some interns
> recently, as they wanted to make something immensely complex to cover a
> case that basically never happened. I was trying to point out that you
> can go overboard on that sort of thing, and it would have been nice to
> have such a quote handy -- anyone else remember it?

Not what you're looking for, but in recent times we'd just say YAGNI - 
You Ain't Gonna Need It.

--T

>
> ron



From scj at yaccman.com  Wed May  3 02:11:52 2017
From: scj at yaccman.com (Steve Johnson)
Date: Tue, 02 May 2017 09:11:52 -0700
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <1189db4c-a1f5-6d03-45ed-81f1a2c51685@telegraphics.com.au>
Message-ID: <a6533db70fc0fc45073e5de11cf9ba5311d1416b@webmail.yaccman.com>

I recall a paper Dennis wrote (maybe more like a note) that was titled
     echo -c considered harmful
(I think it was -c).   It decried the tendency, now completely out
of control, for everybody and their dog to piddle on perfectly good
code just because it's "open". 

Let's face it, programming languages today are more like shells --
they rise or fall on the quality of their libraries, and the
expression of algorithms (or even simplicity or efficiency) are
secondary.  And operating systems seem to exist mainly to host window
systems that vie for how deeply and obscurely they can hide vital
options, and pride themselves that striking any key at any time, no
matter how inadvertent, will cause some unexpected and unwanted
action.

I'm reminded of a friend of mine who took a math course, and said that
the professor "proved theorems by induction on the number of lemmas".

And yes, I'm sounding like a grumpy old fart this morning...

Steve

----- Original Message -----
From: "Toby Thain" <toby at telegraphics.com.au>
To:<tuhs at minnie.tuhs.org>
Cc:
Sent:Mon, 1 May 2017 14:09:07 -0400
Subject:Re: [TUHS] dmr note on BSD's sins

 On 2017-05-01 1:25 PM, ron minnich wrote:
 > OK, I recall a note dmr wrote probably in the late 70s/early 80s
when
 > folks at UCB had (iirc) extended the symbol name size in C programs
to
 > essentially unlimited. This followed on (iirc) file names going
beyond
 > 14 characters.
 >
 > The rough outline was that dmr was calling out the revisions for
being
 > too general, and the phrase "BSD sins" sticks in my head (sins as a
verb).
 >
 > I'm reminded of this by something that happened with some interns
 > recently, as they wanted to make something immensely complex to
cover a
 > case that basically never happened. I was trying to point out that
you
 > can go overboard on that sort of thing, and it would have been nice
to
 > have such a quote handy -- anyone else remember it?

 Not what you're looking for, but in recent times we'd just say YAGNI
- 
 You Ain't Gonna Need It.

 --T

 >
 > ron


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170502/feaee61b/attachment.html>

From fair-tuhs at netbsd.org  Wed May  3 02:39:23 2017
From: fair-tuhs at netbsd.org (Erik E. Fair)
Date: Tue, 02 May 2017 09:39:23 -0700
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <a6533db70fc0fc45073e5de11cf9ba5311d1416b@webmail.yaccman.com>
References: <1189db4c-a1f5-6d03-45ed-81f1a2c51685@telegraphics.com.au>
Message-ID: <3980.1493743163@cesium.clock.org>

There is also Rob Pike's USENIX paper "UNIX Style, or cat -v Considered Harmful"

http://harmful.cat-v.org/cat-v/

which definitely called out UCB CSRG "sins" in the view of Pike & Kernighan. Perhaps this is what you're thinking of?

	Erik Fair


From rminnich at gmail.com  Wed May  3 02:43:43 2017
From: rminnich at gmail.com (ron minnich)
Date: Tue, 02 May 2017 16:43:43 +0000
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <3980.1493743163@cesium.clock.org>
References: <1189db4c-a1f5-6d03-45ed-81f1a2c51685@telegraphics.com.au>
 <a6533db70fc0fc45073e5de11cf9ba5311d1416b@webmail.yaccman.com>
 <3980.1493743163@cesium.clock.org>
Message-ID: <CAP6exY+nvrwM9P0vnK255iJn2-XipfVLX=6tLF2gz1Np9qYFkg@mail.gmail.com>

well, my memory is dmr. But it was a long time ago ...

On Tue, May 2, 2017 at 9:39 AM Erik E. Fair <fair-tuhs at netbsd.org> wrote:

> There is also Rob Pike's USENIX paper "UNIX Style, or cat -v Considered
> Harmful"
>
> http://harmful.cat-v.org/cat-v/
>
> which definitely called out UCB CSRG "sins" in the view of Pike &
> Kernighan. Perhaps this is what you're thinking of?
>
>         Erik Fair
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170502/e158639c/attachment.html>

From dds at aueb.gr  Wed May  3 06:28:25 2017
From: dds at aueb.gr (Diomidis Spinellis)
Date: Tue, 2 May 2017 23:28:25 +0300
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <a6533db70fc0fc45073e5de11cf9ba5311d1416b@webmail.yaccman.com>
References: <a6533db70fc0fc45073e5de11cf9ba5311d1416b@webmail.yaccman.com>
Message-ID: <a4f90d5b-ceb8-fa94-2cf2-332a5ed0707b@aueb.gr>

On 02/05/2017 19:11, Steve Johnson wrote:
> I recall a paper Dennis wrote (maybe more like a note) that was titled
>      echo -c considered harmful
> (I think it was -c).   It decried the tendency, now completely out of
> control, for everybody and their dog to piddle on perfectly good code
> just because it's "open".

There's definitely Rob Pike's talk "UNIX Style, or cat -v Considered 
Harmful", which he delivered at the 1983 Usenix Association Conference 
and Software Tools USers Group Summer Conference.  Unfortunately, I 
can't find it online.  It's interesting that the talk's date is now 
closer to the birth of Unix than to the present.

Diomidis


From bigatojj at gmail.com  Wed May  3 06:34:03 2017
From: bigatojj at gmail.com (=?UTF-8?Q?Daniel_Camol=C3=AAs?=)
Date: Tue, 2 May 2017 17:34:03 -0300
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <CAM4FNStH7KH6+ab0o_fYKfzutvPQ__0w0Zq5_6hk4AFP=+aSAw@mail.gmail.com>
References: <a6533db70fc0fc45073e5de11cf9ba5311d1416b@webmail.yaccman.com>
 <a4f90d5b-ceb8-fa94-2cf2-332a5ed0707b@aueb.gr>
 <CAM4FNStH7KH6+ab0o_fYKfzutvPQ__0w0Zq5_6hk4AFP=+aSAw@mail.gmail.com>
Message-ID: <CAM4FNStfvA7MqCuTh3K+pedbTzyhyyRmMN_hDbbTwq52+RqkkA@mail.gmail.com>

There you go:
http://harmful.cat-v.org/cat-v/

Em 2 de mai de 2017 17:29, "Diomidis Spinellis" <dds at aueb.gr> escreveu:

On 02/05/2017 19:11, Steve Johnson wrote:

> I recall a paper Dennis wrote (maybe more like a note) that was titled
>      echo -c considered harmful
> (I think it was -c).   It decried the tendency, now completely out of
> control, for everybody and their dog to piddle on perfectly good code
> just because it's "open".
>

There's definitely Rob Pike's talk "UNIX Style, or cat -v Considered
Harmful", which he delivered at the 1983 Usenix Association Conference and
Software Tools USers Group Summer Conference.  Unfortunately, I can't find
it online.  It's interesting that the talk's date is now closer to the
birth of Unix than to the present.

Diomidis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170502/f075c99f/attachment.html>

From scj at yaccman.com  Wed May  3 06:34:38 2017
From: scj at yaccman.com (Steve Johnson)
Date: Tue, 02 May 2017 13:34:38 -0700
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <a4f90d5b-ceb8-fa94-2cf2-332a5ed0707b@aueb.gr>
Message-ID: <c13f9d32e0121cf319ba434e4cd4b7695f10ca53@webmail.yaccman.com>

Thanks for the correction -- that is indeed the paper I was thinking
of...

Steve

----- Original Message -----
From: "Diomidis Spinellis" <dds at aueb.gr>
To:"Steve Johnson" <scj at yaccman.com>, "Toby Thain"
<toby at telegraphics.com.au>, <tuhs at minnie.tuhs.org>
Cc:
Sent:Tue, 2 May 2017 23:28:25 +0300
Subject:Re: [TUHS] dmr note on BSD's sins

 On 02/05/2017 19:11, Steve Johnson wrote:
 > I recall a paper Dennis wrote (maybe more like a note) that was
titled
 > echo -c considered harmful
 > (I think it was -c). It decried the tendency, now completely out of
 > control, for everybody and their dog to piddle on perfectly good
code
 > just because it's "open".

 There's definitely Rob Pike's talk "UNIX Style, or cat -v Considered 
 Harmful", which he delivered at the 1983 Usenix Association
Conference 
 and Software Tools USers Group Summer Conference. Unfortunately, I 
 can't find it online. It's interesting that the talk's date is now 
 closer to the birth of Unix than to the present.

 Diomidis

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170502/6c7c6aba/attachment.html>

From akosela at andykosela.com  Wed May  3 07:22:36 2017
From: akosela at andykosela.com (Andy Kosela)
Date: Tue, 2 May 2017 16:22:36 -0500
Subject: [TUHS]  dmr note on BSD's sins
In-Reply-To: <a4f90d5b-ceb8-fa94-2cf2-332a5ed0707b@aueb.gr>
References: <a6533db70fc0fc45073e5de11cf9ba5311d1416b@webmail.yaccman.com>
 <a4f90d5b-ceb8-fa94-2cf2-332a5ed0707b@aueb.gr>
Message-ID: <CALMnNGg4wEFdxdPSU+VcuyOhywpU1GpY=Mm3xi4itP1fn+zLbw@mail.gmail.com>

On Tuesday, May 2, 2017, Diomidis Spinellis <dds at aueb.gr> wrote:

> On 02/05/2017 19:11, Steve Johnson wrote:
>
>> I recall a paper Dennis wrote (maybe more like a note) that was titled
>>      echo -c considered harmful
>> (I think it was -c).   It decried the tendency, now completely out of
>> control, for everybody and their dog to piddle on perfectly good code
>> just because it's "open".
>>
>
> There's definitely Rob Pike's talk "UNIX Style, or cat -v Considered
> Harmful", which he delivered at the 1983 Usenix Association Conference and
> Software Tools USers Group Summer Conference.  Unfortunately, I can't find
> it online.  It's interesting that the talk's date is now closer to the
> birth of Unix than to the present.
>
>
This talk is today as legendary as Plato's "On the Good" lecture[1].

[1] https://www.jstor.org/stable/4182081

--Andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170502/17f217f2/attachment.html>

From doug at cs.dartmouth.edu  Wed May  3 10:52:51 2017
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Tue, 02 May 2017 20:52:51 -0400
Subject: [TUHS] dmr note on BSD's sins
Message-ID: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>

With Steve's eloquent grump and cat -v on the table, I can't
help re-citing the peerless cardinal sin of
        less --help | wc

Doug


From dave at horsfall.org  Wed May  3 10:59:13 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Wed, 3 May 2017 10:59:13 +1000 (EST)
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
References: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
Message-ID: <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>

On Tue, 2 May 2017, Doug McIlroy wrote:

> With Steve's eloquent grump and cat -v on the table, I can't help 
> re-citing the peerless cardinal sin of
>         less --help | wc

Speaking of "cat", what really drives me nuts is "cat file | cmd"...

What's wrong with "cmd < file" (or to really confuse newbies, "< file 
cmd")?

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From crossd at gmail.com  Wed May  3 22:54:19 2017
From: crossd at gmail.com (Dan Cross)
Date: Wed, 3 May 2017 08:54:19 -0400
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
References: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
 <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
Message-ID: <CAEoi9W5hz74awSWhR5Mi3pgKKkFxo4OGOS8hGzbxSC6Sq2XMXg@mail.gmail.com>

On Tue, May 2, 2017 at 8:59 PM, Dave Horsfall <dave at horsfall.org> wrote:

> On Tue, 2 May 2017, Doug McIlroy wrote:
> > With Steve's eloquent grump and cat -v on the table, I can't help
> > re-citing the peerless cardinal sin of
> >         less --help | wc
>
> Speaking of "cat", what really drives me nuts is "cat file | cmd"...
>
> What's wrong with "cmd < file" (or to really confuse newbies, "< file
> cmd")?


Ooo! Ooo! Ooo! I've actually got something for this....

First of all, there's nothing strictly speaking *wrong* with 'cmd < file'
and cat 'cat file | cmd' is definitely overused, often unintentionally and
out of ignorance.

However, 'cmd <file' requires 'file' as a literal string in the command;
`cat` can be useful when the file parameter may be optional. E.g., 'cat
"$@" | ...'. Now some folks will immediately respond by saying, "many
commands will read from stdin if a filename is not presented on the command
line, so why not, 'cmd "$@"'?" And that's certainly a valid question, to
which I would answer that the semantics of a command sometimes subtly
change when presented with one or more filenames as argument (e.g. 'grep'),
so using `cat` may suppress that behavior if desired. "But `grep` has the
`-h` option to tell it not to print the filename!" Yes, but `grep` is just
*one command* and not *all* of them do. The point being that 'cat' in a
pipeline has it's place, even if that place is rarely the place we see it.

Another, related use to cover up one of the more odious of recent design
decisions in Unix-like systems is to use `cat` at the *end* of the
pipeline. Some programs change behavior if they know that they are writing
into a tty; one can suppress that if one terminates the pipeline in `cat`.
This is surely a case of mis-using a feature to mask a bug, but it's often
useful regardless.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170503/8eae661e/attachment.html>

From krewat at kilonet.net  Wed May  3 23:09:52 2017
From: krewat at kilonet.net (Arthur Krewat)
Date: Wed, 3 May 2017 09:09:52 -0400
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <CAEoi9W5hz74awSWhR5Mi3pgKKkFxo4OGOS8hGzbxSC6Sq2XMXg@mail.gmail.com>
References: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
 <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
 <CAEoi9W5hz74awSWhR5Mi3pgKKkFxo4OGOS8hGzbxSC6Sq2XMXg@mail.gmail.com>
Message-ID: <58cf42d9-b153-7831-5012-16e5544faf2f@kilonet.net>

Not to mention, you can cat multiple files - as in concatenate :)

On 5/3/2017 8:54 AM, Dan Cross wrote:
> On Tue, May 2, 2017 at 8:59 PM, Dave Horsfall <dave at horsfall.org 
> <mailto:dave at horsfall.org>> wrote:
>
>     On Tue, 2 May 2017, Doug McIlroy wrote:
>     > With Steve's eloquent grump and cat -v on the table, I can't help
>     > re-citing the peerless cardinal sin of
>     >         less --help | wc
>
>     Speaking of "cat", what really drives me nuts is "cat file | cmd"...
>
>     What's wrong with "cmd < file" (or to really confuse newbies, "< file
>     cmd")?
>
>
> Ooo! Ooo! Ooo! I've actually got something for this....
>
> First of all, there's nothing strictly speaking *wrong* with 'cmd < 
> file' and cat 'cat file | cmd' is definitely overused, often 
> unintentionally and out of ignorance.
>
> However, 'cmd <file' requires 'file' as a literal string in the 
> command; `cat` can be useful when the file parameter may be optional. 
> E.g., 'cat "$@" | ...'. Now some folks will immediately respond by 
> saying, "many commands will read from stdin if a filename is not 
> presented on the command line, so why not, 'cmd "$@"'?" And that's 
> certainly a valid question, to which I would answer that the semantics 
> of a command sometimes subtly change when presented with one or more 
> filenames as argument (e.g. 'grep'), so using `cat` may suppress that 
> behavior if desired. "But `grep` has the `-h` option to tell it not to 
> print the filename!" Yes, but `grep` is just *one command* and not 
> *all* of them do. The point being that 'cat' in a pipeline has it's 
> place, even if that place is rarely the place we see it.
>
> Another, related use to cover up one of the more odious of recent 
> design decisions in Unix-like systems is to use `cat` at the *end* of 
> the pipeline. Some programs change behavior if they know that they are 
> writing into a tty; one can suppress that if one terminates the 
> pipeline in `cat`. This is surely a case of mis-using a feature to 
> mask a bug, but it's often useful regardless.
>
>         - Dan C.
>



From cym224 at gmail.com  Wed May  3 23:41:11 2017
From: cym224 at gmail.com (Nemo)
Date: Wed, 3 May 2017 09:41:11 -0400
Subject: [TUHS] dmr note on BSD's sins
Message-ID: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>

On 3 May 2017 at 09:09, Arthur Krewat <krewat at kilonet.net> wrote:
> Not to mention, you can cat multiple files - as in concatenate :)

Along these lines, who said "Cat went to Berkely, came back waving flags."

N.


From michael at kjorling.se  Wed May  3 23:46:32 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Wed, 3 May 2017 13:46:32 +0000
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
References: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
 <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
Message-ID: <20170503134632.GF26584@yeono.kjorling.se>

On 3 May 2017 10:59 +1000, from dave at horsfall.org (Dave Horsfall):
> What's wrong with "cmd < file" (or to really confuse newbies, "< file 
> cmd")?

If you want to _really_ confuse a newbie, stick an "<file" or ">file"
_in the middle of_ a long command, and watch hilarity ensue.

Or the "<sth>" that someone here mentioned, if you are so inclined...

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From clemc at ccc.com  Thu May  4 00:57:07 2017
From: clemc at ccc.com (Clem Cole)
Date: Wed, 3 May 2017 10:57:07 -0400
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
References: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
 <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
Message-ID: <CAC20D2Nk3+P67Z_OxDym4MEPJbne7GZW3tyK35B3ZfO_MfZ8PQ@mail.gmail.com>

below...


On Tue, May 2, 2017 at 8:59 PM, Dave Horsfall <dave at horsfall.org> wrote:

> On Tue, 2 May 2017, Doug McIlroy wrote:
>
> > With Steve's eloquent grump and cat -v on the table, I can't help
> > re-citing the peerless cardinal sin of
> >         less --help | wc
>

To Doug -- touché




> Speaking of "cat", what really drives me nuts is "cat file | cmd"...
>
> What's wrong with "cmd < file" (or to really confuse newbies, "< file
> cmd")?


Not to be argumentative, but Dave, please be careful.  I agree that
idiom is misused, but there are times when I think it is appropriate, which
is one of the reason why I love the Unix guiding principle of not "forcing"
the user to do one thing or another because we know better, but letting
education and good 'taste' be the guide.

​I think Justice Potter Stewart describe​d pornography of knowing it when
he sees it.  IMO: there are time when using cat is in better taste that a
file redirection.  For instance, I can think of a few examples when I need
to direct input from multiple files, using a more complex 'wye' style pipe
line or a redirection from backward quotes, that the cat foo sequence seems
like its easier to understand/debug.  [Maybe it shows my inner
schizophrenia and heard voices from multiple places that I kinda like the
'wye' command].

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170503/9465a193/attachment.html>

From arnold at skeeve.com  Thu May  4 01:34:28 2017
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Wed, 03 May 2017 09:34:28 -0600
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <CAC20D2Nk3+P67Z_OxDym4MEPJbne7GZW3tyK35B3ZfO_MfZ8PQ@mail.gmail.com>
References: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
 <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
 <CAC20D2Nk3+P67Z_OxDym4MEPJbne7GZW3tyK35B3ZfO_MfZ8PQ@mail.gmail.com>
Message-ID: <201705031534.v43FYSCY028025@freefriends.org>

> [Maybe it shows my inner
> schizophrenia and heard voices from multiple places that I kinda like the
> 'wye' command].

Um, what is the 'wye' command? Never heard of it...

Thanks,

Arnold


From clemc at ccc.com  Thu May  4 01:44:38 2017
From: clemc at ccc.com (Clem Cole)
Date: Wed, 3 May 2017 11:44:38 -0400
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <201705031534.v43FYSCY028025@freefriends.org>
References: <201705030052.v430qp2K004255@coolidge.cs.Dartmouth.EDU>
 <alpine.BSF.2.20.1705031055190.11767@aneurin.horsfall.org>
 <CAC20D2Nk3+P67Z_OxDym4MEPJbne7GZW3tyK35B3ZfO_MfZ8PQ@mail.gmail.com>
 <201705031534.v43FYSCY028025@freefriends.org>
Message-ID: <CAC20D2PNFsdedd+KUNn34YHm7cQPZP3nvjHQdR=uwuQtOVS3LQ@mail.gmail.com>

Early line at time version of paste

On Wed, May 3, 2017 at 11:34 AM, <arnold at skeeve.com> wrote:

> > [Maybe it shows my inner
> > schizophrenia and heard voices from multiple places that I kinda like the
> > 'wye' command].
>
> Um, what is the 'wye' command? Never heard of it...
>
> Thanks,
>
> Arnold
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170503/aa1dcdb7/attachment.html>

From dave at horsfall.org  Thu May  4 08:18:56 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Thu, 4 May 2017 08:18:56 +1000 (EST)
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
References: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
Message-ID: <alpine.BSF.2.20.1705040815080.11767@aneurin.horsfall.org>

On Wed, 3 May 2017, Nemo wrote:

> Along these lines, who said "Cat went to Berkely, came back waving 
> flags."

How did the expression go?  LSD and BSD were both produced at Berkeley; 
this is thought not to be a coincidence.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From tfb at tfeb.org  Fri May  5 00:59:22 2017
From: tfb at tfeb.org (Tim Bradshaw)
Date: Thu, 4 May 2017 15:59:22 +0100
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
References: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
Message-ID: <93E87A49-523F-4BCE-9D89-E62351410A8C@tfeb.org>

On 3 May 2017, at 14:41, Nemo <cym224 at gmail.com> wrote:
> 
> Along these lines, who said "Cat went to Berkely, came back waving flags."

And anyone who lived through the SunOS 4 -- SunOS 5 transition will know that some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful.  I remember spending really a long time finding and building BSD/GNU versions of utilities which actually had the options you needed on early SunOS 5 machines: later on Sun themselves put some of them back.

From lm at mcvoy.com  Fri May  5 09:14:36 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Thu, 4 May 2017 16:14:36 -0700
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <93E87A49-523F-4BCE-9D89-E62351410A8C@tfeb.org>
References: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
 <93E87A49-523F-4BCE-9D89-E62351410A8C@tfeb.org>
Message-ID: <20170504231436.GC2932@mcvoy.com>

On Thu, May 04, 2017 at 03:59:22PM +0100, Tim Bradshaw wrote:
> On 3 May 2017, at 14:41, Nemo <cym224 at gmail.com> wrote:
> > 
> > Along these lines, who said "Cat went to Berkely, came back waving flags."
> 
> And anyone who lived through the SunOS 4 -- SunOS 5 transition will know that some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful.  I remember spending really a long time finding and building BSD/GNU versions of utilities which actually had the options you needed on early SunOS 5 machines: later on Sun themselves put some of them back.

As someone who was at Sun, loved SunOS 4 and hated Solaris, yup.  Even the
diehard Solaris people like Bryan Cantrill say that they stick /opt/gnu/bin
in their path first.

It boggles my mind that Sun was so strict on being compat with SVR4 and
they knew that everyone hated that.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 


From doug at cs.dartmouth.edu  Fri May  5 23:06:56 2017
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Fri, 05 May 2017 09:06:56 -0400
Subject: [TUHS] [Groff] ASCII Minus Sign in man Pages
Message-ID: <201705051306.v45D6uQr006114@coolidge.cs.Dartmouth.EDU>

> I believe I was the last person to modify the Linux man page macros.
> Their current maintainer is not the kind of groff expert to whom it
> would occur to modify them; it would work as well to ask me questions

Question #1. Which tmac file to they use? If it's not in the groff
package, where can it be found?

Doug


From random832 at fastmail.com  Sat May  6 00:01:55 2017
From: random832 at fastmail.com (Random832)
Date: Fri, 05 May 2017 10:01:55 -0400
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <20170504231436.GC2932@mcvoy.com>
References: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
 <93E87A49-523F-4BCE-9D89-E62351410A8C@tfeb.org>
 <20170504231436.GC2932@mcvoy.com>
Message-ID: <1493992915.3765984.966912328.38F55A2B@webmail.messagingengine.com>

On Thu, May 4, 2017, at 19:14, Larry McVoy wrote:
> On Thu, May 04, 2017 at 03:59:22PM +0100, Tim Bradshaw wrote:
> > On 3 May 2017, at 14:41, Nemo <cym224 at gmail.com> wrote:
> > > 
> > > Along these lines, who said "Cat went to Berkely, came back waving flags."
> > 
> > And anyone who lived through the SunOS 4 -- SunOS 5 transition will know that some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful.  I remember spending really a long time finding and building BSD/GNU versions of utilities which actually had the options you needed on early SunOS 5 machines: later on Sun themselves put some of them back.

I mean, there's a legitimate argument that some of them would have been
just as useful or more as separate utilities than as flags. But to some
extent the fully generalized version seems questionable.

I mean, to apply the design philosophy to, say, ls -t (nevermind that
this particular flag has in fact existed since Unix V1, it's exactly the
*kind* of thing that should prompt someone who is consistently applying
Pike's idea of the unix philosophy to say "sorting should be the sort
tool"), you would have to have A) a basic "ls" command which does all
the filesystem-accessing work and prints timestamps and all other
relevant output in a format that can be sorted lexicographically, B) run
it through sort(1), C) a command [or maybe an awk script] that will pare
it back down to the usual human-readable formats (names only, ls -l, or
maybe a timestamp-and-name format, with human-friendly timestamps). And
then D) another tool to sort it into columns.

And you'd have to type in all of those every time you want a directory
listing, or maybe have it packaged up in a script (which would then need
a name, and unique names for every combination of behaviors you might
want to use).

At that point, who gets to decide that ls -t is useful enough to be
"proper unix" but cat -v is "some ridiculous idea from berkeley"?



Anyway, on the original subject of unlimited symbol name and filename
lengths - ultimately there's a tradeoff - your symbols or filenames
probably don't have to get very large before it makes more sense to use
a variable-length format to save space, and the smallest reasonable
limit for a variable-length format is 255.


From clemc at ccc.com  Sat May  6 01:20:02 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 5 May 2017 11:20:02 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point [was dmr note on BSD's sins]
Message-ID: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>

On Thu, May 4, 2017 at 7:14 PM, Larry McVoy <lm at mcvoy.com> wrote:

> some of those Berkeley flags (not specifically for cat, but almost
> certainly including those for cat) were really quite useful.


​Amen!!! I think that this is key point.   What is in good taste or good
style?  Doug's distain for the results of:
               less --help | wc
is found in bad design, a poor user interface, little fore thought,  *etc.*
  Most of what is there has been added over Eric Scheinbrood's
original  more(1)
I do not believe are used that often, but some of them are of course, and
you can not tell who uses what!! So how do you decide what get rid of?

How do you learn what people really need --    IMO: much of that is
* experience *and this thing we call 'good taste.'  As opposed to what I
think happened with less(1) and many others similar programs (programmers
peeing on the code because they code and the source was available -- I can
add this feature to it and I think it is cool.  As opposed to asking,
what do really get and not having a 'master builder' arbitrating or vetting
things).

The problem we have is that we don't yet have a way of defining good taste.
  One might suggest that it takes years of civilization and also that
tastes do change over time.   Pike's minimalistic view (which I think is
taken to the extreme in the joke about automobile dashboard on Brian
Kernighan's car) sets the bar to one end and probably one of the reason why
UNIX had a bad reputation, certainly by non-computer scientists, when it
first appeared as being difficult.  Another extreme is systems that put in
you a box and never let you do anything but what we told you do; which I
find just as frighten and frustration builds there when I use them.
Clearly systems that are so noisy that can not find what really want, or
need is another dimension of the same bad design.  So what to do? [more in
a minute...]

Larry is right.  Many of the 'features' added to UNIX (and Linux) over time
have been and *are useful*.   Small and simple as it was (and I really
admire Ken, Dennis and the Team for its creation), but in 2017 I really
don't want to run the Sixth Edition for  my day to day work anymore - which
I happily did in 1977.     But the problem is, as we got 'unlimited address
space' of 32 and 64 bits, and more room for more 'useful' things, we also
got a great deal of rubbish and waste.    I am interpreting Doug's point
about less --help | wc is that is that are so many thorns and weeds, it
hard to see to see the flowers in the garden.

I'd like to observe that is that most college courses in CS I have seen
talk about how to construct a programs, algorithms, structures - i.e. the
mechanics of some operation.   But this discussion is about the human
element.   What we feel is good or bad and how it related to how to use
the program.

I think about my friends that have degrees in literature, art and
architecture.   In all cases, they spend a lot of time examining past
examples, of good and bad - and thinking and discussing what makes them so.
  I'm actually happy to see it was Professor McIlroy that is one of the
folks taking a stand on current craziness.   I think this is only going to
get better if a new crop of students that have been trained in 'good
taste.'  So, I wonder do any of the schools like Darthmouth and the like
teach courses that study 'style' and taste in CS.   Yes, it is a young
field but we have been around long enough that we do a body of work good
and bad to consider.

I think there is a book or two and a few lectures in there somewhere.
Thoughts?

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/582a0bfb/attachment.html>

From bakul at bitblocks.com  Sat May  6 01:37:03 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Fri, 5 May 2017 08:37:03 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
	user stand point [was dmr note on BSD's sins]
In-Reply-To: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
References: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
Message-ID: <1C6D6D3E-058F-4743-8DA0-17CEC09B64E9@bitblocks.com>

I think the key issue is not as much minimalism as composability. BSD often prioritized convenience over composability. Each command doing one thing well and doing line oriented makes them more composability. You can always package up convenient combinations in a script. Plan9 has lc which prints like unix ls -C but it is an rc script. Trying to achieve composability can result in leaner systems.

> On May 5, 2017, at 8:20 AM, Clem Cole <clemc at ccc.com> wrote:
> 
> 
>> On Thu, May 4, 2017 at 7:14 PM, Larry McVoy <lm at mcvoy.com> wrote:
>> some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful. 
> 
> ​Amen!!! I think that this is key point.   What is in good taste or good style?  Doug's distain for the results of:
>                less --help | wc
> is found in bad design, a poor user interface, little fore thought,  etc.   Most of what is there has been added over Eric Scheinbrood's original  more(1) I do not believe are used that often, but some of them are of course, and you can not tell who uses what!! So how do you decide what get rid of?
> 
> How do you learn what people really need --    IMO: much of that is experience and this thing we call 'good taste.'  As opposed to what I think happened with less(1) and many others similar programs (programmers peeing on the code because they code and the source was available -- I can add this feature to it and I think it is cool.  As opposed to asking, what do really get and not having a 'master builder' arbitrating or vetting things). 
> 
> The problem we have is that we don't yet have a way of defining good taste.   One might suggest that it takes years of civilization and also that tastes do change over time.   Pike's minimalistic view (which I think is taken to the extreme in the joke about automobile dashboard on Brian Kernighan's car) sets the bar to one end and probably one of the reason why UNIX had a bad reputation, certainly by non-computer scientists, when it first appeared as being difficult.  Another extreme is systems that put in you a box and never let you do anything but what we told you do; which I find just as frighten and frustration builds there when I use them.  Clearly systems that are so noisy that can not find what really want, or need is another dimension of the same bad design.  So what to do? [more in a minute...]
> 
> Larry is right.  Many of the 'features' added to UNIX (and Linux) over time have been and are useful.   Small and simple as it was (and I really admire Ken, Dennis and the Team for its creation), but in 2017 I really don't want to run the Sixth Edition for  my day to day work anymore - which I happily did in 1977.     But the problem is, as we got 'unlimited address space' of 32 and 64 bits, and more room for more 'useful' things, we also got a great deal of rubbish and waste.    I am interpreting Doug's point about less --help | wc is that is that are so many thorns and weeds, it hard to see to see the flowers in the garden.
> 
> I'd like to observe that is that most college courses in CS I have seen talk about how to construct a programs, algorithms, structures - i.e. the mechanics of some operation.   But this discussion is about the human element.   What we feel is good or bad and how it related to how to use the program.   
> 
> I think about my friends that have degrees in literature, art and architecture.   In all cases, they spend a lot of time examining past examples, of good and bad - and thinking and discussing what makes them so.   I'm actually happy to see it was Professor McIlroy that is one of the folks taking a stand on current craziness.   I think this is only going to get better if a new crop of students that have been trained in 'good taste.'  So, I wonder do any of the schools like Darthmouth and the like teach courses that study 'style' and taste in CS.   Yes, it is a young field but we have been around long enough that we do a body of work good and bad to consider.
> 
> I think there is a book or two and a few lectures in there somewhere.   Thoughts?
> 
> Clem
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/d75dcba3/attachment.html>

From tfb at tfeb.org  Sat May  6 03:33:19 2017
From: tfb at tfeb.org (Tim Bradshaw)
Date: Fri, 5 May 2017 18:33:19 +0100
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <20170504231436.GC2932@mcvoy.com>
References: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
 <93E87A49-523F-4BCE-9D89-E62351410A8C@tfeb.org>
 <20170504231436.GC2932@mcvoy.com>
Message-ID: <6A6951FC-D4C8-48AC-A926-BBE27E9334F7@tfeb.org>

I asked my wife (who is a longer-term Sun person than me and also went through this) if my memory was correct (I now keep elaborate and deeply unpublishable notes but I didn't then) and she immediately said 'echo -n': this was presumably either in /(usr/)?bin/echo in BSD/SunOS 4 or in the /bin/sh builtin, and not in the versions for SunOS 5 with resulting horrible failures of scripts all over the place.

I think I can sort of see why the serious-standards-compliance-even-when-the-standard-is-horrible thing might have seemed like it would appeal to the kind of organisations Sun wanted to sell to (ones where people wear suits), but it also drove their traditional userbase to Linux/*BSD and this did not help them in the long term.  (Also: do the standards say 'thou shall have no other options but these', because if they do they are terrible standards).

> On 5 May 2017, at 00:14, Larry McVoy <lm at mcvoy.com> wrote:
> 
>> On Thu, May 04, 2017 at 03:59:22PM +0100, Tim Bradshaw wrote:
>>> On 3 May 2017, at 14:41, Nemo <cym224 at gmail.com> wrote:
>>> 
>>> Along these lines, who said "Cat went to Berkely, came back waving flags."
>> 
>> And anyone who lived through the SunOS 4 -- SunOS 5 transition will know that some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful.  I remember spending really a long time finding and building BSD/GNU versions of utilities which actually had the options you needed on early SunOS 5 machines: later on Sun themselves put some of them back.
> 
> As someone who was at Sun, loved SunOS 4 and hated Solaris, yup.  Even the
> diehard Solaris people like Bryan Cantrill say that they stick /opt/gnu/bin
> in their path first.
> 
> It boggles my mind that Sun was so strict on being compat with SVR4 and
> they knew that everyone hated that.
> -- 
> ---
> Larry McVoy                     lm at mcvoy.com             http://www.mcvoy.com/lm 



From a.phillip.garcia at gmail.com  Sat May  6 04:56:06 2017
From: a.phillip.garcia at gmail.com (A. P. Garcia)
Date: Fri, 5 May 2017 13:56:06 -0500
Subject: [TUHS] dmr note on BSD's sins
In-Reply-To: <1493992915.3765984.966912328.38F55A2B@webmail.messagingengine.com>
References: <CAJfiPzzuHAb0S9j-sY0XyHfWQSR1BVcLMy-d7ee7YgUwAs-hdQ@mail.gmail.com>
 <93E87A49-523F-4BCE-9D89-E62351410A8C@tfeb.org>
 <20170504231436.GC2932@mcvoy.com>
 <1493992915.3765984.966912328.38F55A2B@webmail.messagingengine.com>
Message-ID: <CAFCBnZv6QEcmGfvdnLnwC9OWsbzwc6jVanYp6uYXoTZ=UguUAw@mail.gmail.com>

On Fri, May 5, 2017 at 9:01 AM, Random832 <random832 at fastmail.com> wrote:
> On Thu, May 4, 2017, at 19:14, Larry McVoy wrote:
>> On Thu, May 04, 2017 at 03:59:22PM +0100, Tim Bradshaw wrote:
>> > On 3 May 2017, at 14:41, Nemo <cym224 at gmail.com> wrote:
>> > >
>> > > Along these lines, who said "Cat went to Berkely, came back waving flags."
>> >
>> > And anyone who lived through the SunOS 4 -- SunOS 5 transition will know that some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful.  I remember spending really a long time finding and building BSD/GNU versions of utilities which actually had the options you needed on early SunOS 5 machines: later on Sun themselves put some of them back.
>
> I mean, there's a legitimate argument that some of them would have been
> just as useful or more as separate utilities than as flags. But to some
> extent the fully generalized version seems questionable.
>
> I mean, to apply the design philosophy to, say, ls -t (nevermind that
> this particular flag has in fact existed since Unix V1, it's exactly the
> *kind* of thing that should prompt someone who is consistently applying
> Pike's idea of the unix philosophy to say "sorting should be the sort
> tool"), you would have to have A) a basic "ls" command which does all
> the filesystem-accessing work and prints timestamps and all other
> relevant output in a format that can be sorted lexicographically, B) run
> it through sort(1), C) a command [or maybe an awk script] that will pare
> it back down to the usual human-readable formats (names only, ls -l, or
> maybe a timestamp-and-name format, with human-friendly timestamps). And
> then D) another tool to sort it into columns.

have you been using powershell lately? it makes things like this
rather enjoyable. yes, powershell is open source, and yes, it's been
ported to linux, but i think unix needs its own home-grown "object
oriented shell".


<snip>


From doug at cs.dartmouth.edu  Sat May  6 12:02:21 2017
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Fri, 05 May 2017 22:02:21 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point [was dmr note on BSD's sins]
In-Reply-To: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
References: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
Message-ID: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>

Clem wrote, "So, I wonder do any of the schools like Darthmouth and the like
teach courses that study 'style' and taste in CS."

Some time around 1970, Bob Rosen instituted a program-reading
course at the University of Iowa, by overt analogy to the
study of literature. Style was certainly a central concern;
I'm not sure about the deeper matter of taste. I'm not aware
of other examples, except perhaps for a collaboration between
Don Kunth and a teachaer of (I think) creative writing--a
course whose content I can't recall.

Doug


From noel.hunt at gmail.com  Sat May  6 12:16:53 2017
From: noel.hunt at gmail.com (Noel Hunt)
Date: Sat, 6 May 2017 12:16:53 +1000
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point [was dmr note on BSD's sins]
In-Reply-To: <1C6D6D3E-058F-4743-8DA0-17CEC09B64E9@bitblocks.com>
References: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
 <1C6D6D3E-058F-4743-8DA0-17CEC09B64E9@bitblocks.com>
Message-ID: <CAGfO01ytR+g09Mp+YQ3fR2j=QnHZsfcZM-xQQKr+NNvtESypcA@mail.gmail.com>

What you call 'composability' is nothing more than the 'software
tools' approach.

I recall reading an interview with Dennis Ritchie, but can no
longer find it, where he talks about this approach versus the
monolithic program model, as for example demonstrated by
'perl', which does everything. Dennis remarked that the
'tools' approach required careful thought (which programs to
use, how to connect them with pipes) and so was more difficult
to use, hence the popularity of 'perl'.

I'd be delighted if someone can point me to where this
interview might be.

P.S.

A most beautiful example of this approach is from Doug McIlroy
in his critique of Donald Knuth's solution to a problem posed
by Jon Bentley in his 'Programming Pearls' column:

    Read a file of text, determine the n most frequently used
    words, and print out a sorted list of those words along with
    their frequencies.

Donald Knuth wrote a long, beautifully crafted program in some
pseudo-code.  Doug McIlroy provided an alternative solution:

tr -cs A-Za-z '\n' |
tr A-Z a-z |
sort |
uniq -c |
sort -rn |
sed ${1}q

This is real genius.


On Sat, May 6, 2017 at 1:37 AM, Bakul Shah <bakul at bitblocks.com> wrote:

> I think the key issue is not as much minimalism as composability. BSD
> often prioritized convenience over composability. Each command doing one
> thing well and doing line oriented makes them more composability. You can
> always package up convenient combinations in a script. Plan9 has lc which
> prints like unix ls -C but it is an rc script. Trying to achieve
> composability can result in leaner systems.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/350640c1/attachment.html>

From toby at telegraphics.com.au  Sat May  6 12:40:12 2017
From: toby at telegraphics.com.au (Toby Thain)
Date: Fri, 5 May 2017 22:40:12 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point [was dmr note on BSD's sins]
In-Reply-To: <CAGfO01ytR+g09Mp+YQ3fR2j=QnHZsfcZM-xQQKr+NNvtESypcA@mail.gmail.com>
References: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
 <1C6D6D3E-058F-4743-8DA0-17CEC09B64E9@bitblocks.com>
 <CAGfO01ytR+g09Mp+YQ3fR2j=QnHZsfcZM-xQQKr+NNvtESypcA@mail.gmail.com>
Message-ID: <8b552139-8e5e-a32f-fd47-6a863642ecab@telegraphics.com.au>

On 2017-05-05 10:16 PM, Noel Hunt wrote:
> ...
> Donald Knuth wrote a long, beautifully crafted program in some
> pseudo-code.  Doug McIlroy provided an alternative solution:
>

Much as I love Professor Knuth, he is still doing this. He always was a 
low level coder*, and his programs are longer and more complicated than 
the high level equivalents, by the expected factors. (If anyone doubts 
it, note that he adopted C after Pascal, and many of his programs are 
linked from his web pages.) No doubt he has solid rationalisations for 
taking this approach for his purposes, but of course it's not ideal for 
all situations.

--Toby


* - Many of us will remember we were expected to understand TAOCP 
through MIX. And he started out like this:
http://www.catonmat.net/blog/donald-knuths-first-computer/


From scj at yaccman.com  Sat May  6 15:33:02 2017
From: scj at yaccman.com (Steve Johnson)
Date: Fri, 05 May 2017 22:33:02 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
	user stand point [was dmr note on BSD's sins]
In-Reply-To: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
Message-ID: <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>

In 1973, I spent 9 months at the University of Waterloo, home of
Watfor.   I taught a course in "Advanced Application Techniques" --
the language was FORTRAN, and the techniques included such things as
using libraries and handling errors in input data...

When I left for Canada, there had been a lot of discussion about
programming style at the labs, and so in my first lecture I told the
students that I would award points for style, and briefly talked about
what that was and how I would assess the style, and I did so
throughout the course.

What amazed me was that some students seemed to instantly get the
idea, and became quite good at producing excellent, clean programs,
while others were violently allergic to the whole notion that anything
"subjective" should stand between them and their grade in the class.
 I remember several interchanges with students, often angry, who had
a lot of difficulty understanding why their 35-line solution to a
problem should get a lower grade than a 15-line version (excluding
comment lines).

in fairness, I didn't (and don't) know any better way to learn style
than doing a lot of reading, a lot of writing, and getting criticized
constructively by those with more experience.  For me, a lot of what
I learned was from Stan Brown at the Labs, who read piles of my
(atrocious) FORTRAN code and repeatedly pointed out that when you
wrote a program, you were not just communicating with the computer,
but also other humans (including yourself) who would read (and perhaps
modify) the code in the future...

Steve

----- Original Message -----
From: "Doug McIlroy" <doug at cs.dartmouth.edu>
To:<lm at mcvoy.com>, <doug at cs.dartmouth.edu>, <clemc at ccc.com>
Cc:<tuhs at minnie.tuhs.org>
Sent:Fri, 05 May 2017 22:02:21 -0400
Subject:Re: [TUHS] Discuss of style and design of computer programs
from a user stand point [was dmr note on BSD's sins]

 Clem wrote, "So, I wonder do any of the schools like Darthmouth and
the like
 teach courses that study 'style' and taste in CS."

 Some time around 1970, Bob Rosen instituted a program-reading
 course at the University of Iowa, by overt analogy to the
 study of literature. Style was certainly a central concern;
 I'm not sure about the deeper matter of taste. I'm not aware
 of other examples, except perhaps for a collaboration between
 Don Kunth and a teachaer of (I think) creative writing--a
 course whose content I can't recall.

 Doug

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/4b496d7b/attachment.html>

From bakul at bitblocks.com  Sat May  6 16:07:27 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Fri, 05 May 2017 23:07:27 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
	user stand point [was dmr note on BSD's sins]
In-Reply-To: Your message of "Sat, 06 May 2017 12:16:53 +1000."
 <CAGfO01ytR+g09Mp+YQ3fR2j=QnHZsfcZM-xQQKr+NNvtESypcA@mail.gmail.com>
References: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
 <1C6D6D3E-058F-4743-8DA0-17CEC09B64E9@bitblocks.com>
 <CAGfO01ytR+g09Mp+YQ3fR2j=QnHZsfcZM-xQQKr+NNvtESypcA@mail.gmail.com>
Message-ID: <20170506060727.A00C0124AEFE@mail.bitblocks.com>

On Sat, 06 May 2017 12:16:53 +1000 Noel Hunt <noel.hunt at gmail.com> wrote:
> 
> What you call 'composability' is nothing more than the 'software
> tools' approach.

Composable components can have a much tigher coupling.  vi,
rsync, git and gdb are s/w tools but they are not composable
in the sense of sed, grep, uniq, awk, head, tail, tr, sort,
ls, find etc.

> I recall reading an interview with Dennis Ritchie, but can no
> longer find it, where he talks about this approach versus the
> monolithic program model, as for example demonstrated by
> 'perl', which does everything. Dennis remarked that the
> 'tools' approach required careful thought (which programs to
> use, how to connect them with pipes) and so was more difficult
> to use, hence the popularity of 'perl'.

This is the "modularity" approach, where one program does one
thing well rather than doing everything (e.g IDE). But
moudlarity does not necessarily give you composability.

Also, it is not enough to have composable components; you need
a language/environment in which they can be used.

> A most beautiful example of this approach is from Doug McIlroy
> in his critique of Donald Knuth's solution to a problem posed
> by Jon Bentley in his 'Programming Pearls' column:
> 
>     Read a file of text, determine the n most frequently used
>     words, and print out a sorted list of those words along with
>     their frequencies.
> 
> Donald Knuth wrote a long, beautifully crafted program in some
> pseudo-code.  Doug McIlroy provided an alternative solution:
> 
> tr -cs A-Za-z '\n' |
> tr A-Z a-z |
> sort |
> uniq -c |
> sort -rn |
> sed ${1}q
> 
> This is real genius.

Indeed. 

Note that a similar approach was implemented by Iverson in
APL. In fact a shell pipeline has a lot in common with array
programming!

Iverson in essence built a whole workshop with a set of simple
but powerful standard tools (functions and higher order
functions).  The same has been carried on in follow on array
languages such as j and k and q languages.  Many of these
array functions are directly applicable to stream processing.
In contrast unix shells provide a very shallow glue layer.

> On Sat, May 6, 2017 at 1:37 AM, Bakul Shah <bakul at bitblocks.com> wrote:
> 
> > I think the key issue is not as much minimalism as composability. BSD
> > often prioritized convenience over composability. Each command doing one
> > thing well and doing line oriented makes them more composability. You can
> > always package up convenient combinations in a script. Plan9 has lc which
> > prints like unix ls -C but it is an rc script. Trying to achieve
> > composability can result in leaner systems.


From michael at kjorling.se  Sat May  6 19:18:57 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 6 May 2017 09:18:57 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
Message-ID: <20170506091857.GE12539@yeono.kjorling.se>

On 5 May 2017 22:33 -0700, from scj at yaccman.com (Steve Johnson):
> For me, a lot of what
> I learned was from Stan Brown at the Labs, who read piles of my
> (atrocious) FORTRAN code and repeatedly pointed out that when you
> wrote a program, you were not just communicating with the computer,
> but also other humans (including yourself) who would read (and perhaps
> modify) the code in the future...

I would actually take that one step further: When you are writing
code, you are _first and foremost_ communicating with whatever human
will need to read or modify the code later. That human might be you, a
colleague, or the violent psychopath who knows both where you live and
where your little kids go to school (might as well be you). You should
strive to write the code accordingly, _even if_ the odds of the threat
ever materializing are slim at most. Style matters a lot, there.

Sure, some code really needs to be heavily optimized, and might end up
obscure for it. In operating systems, interrupt routines are probably
one of the major candidates, but first stage bootloaders need their
fair share of it too for other reasons. Tight loops or code that needs
to be able to operate on lots of data quickly are other examples. But
_most code isn't like that_ and can allow for some extra verbosity and
some performance penalty if that means making it significantly easier
to read and modify. That's not to say code should be verbose _just
because_, but if you don't need ultimate performance, I say aim for
readability and modifyability first; processing and memory usage
second. You can always go back and optimize bits that turn out to need
it.

And of course, I find Doug McIlroy's shell snippet mentioned by Noel
interesting, _because I did something very similar not all that long
ago_; in my case, stringing together a bunch of shell tools to get a
list of all URLs from an e-mail message for further processing.
(Something like a composable _urlview_; if urlview had even had a
"dump to stdout" option, that would have done all I needed it for.)
Works like a charm, and isn't even terribly complicated. It could
probably be done in a single line of perl or awk instead of piping
through a grand total of six commands, but that's okay. And of course,
stringing together a bunch of commands was the first approach that
occured to me...

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From cym224 at gmail.com  Sat May  6 23:09:27 2017
From: cym224 at gmail.com (Nemo)
Date: Sat, 6 May 2017 09:09:27 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <20170506091857.GE12539@yeono.kjorling.se>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se>
Message-ID: <CAJfiPzxKN09Y_8w7tPvAXjC=VZh-qJYpPezW40UV6B40K=5a5g@mail.gmail.com>

On 06/05/2017, Michael Kjörling <michael at kjorling.se> wrote:
> I would actually take that one step further: When you are writing
> code, you are _first and foremost_ communicating with whatever human
> will need to read or modify the code later.

Something similar appears in the front of almost every Wirth book
(though sometimes honoured in the breach).  That being said, there are
times when the code is transformed by optimization into something very
difficult to read.  (We used noweb to document the transformation.)

N.


From doug at cs.dartmouth.edu  Sat May  6 23:30:58 2017
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Sat, 06 May 2017 09:30:58 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point [was dmr note on BSD's sins]
Message-ID: <201705061330.v46DUwlH020977@coolidge.cs.Dartmouth.EDU>

> tr -cs A-Za-z '\n' |
> tr A-Z a-z |
> sort |
> uniq -c |
> sort -rn |
> sed ${1}q
>
> This is real genius.

Not genius. Experience. In the Bentley/Knuth/McIlroy paper I said,
"[Old] Unix hands know instinctively how to solve this one in a jiffy."
While that is certainly true, the script was informed by my having
written "spell", which itself was an elaboration of a model
pioneered by Steve Johnson. By 1986, when BKM was published,
the lore was baked in: word-processing scripts in a similar
vein were stock in trade.

A very early exercise of this sort was Dennis Ritchie's
enumeration of anagrams in the unabridged Merriam-Webster.
Since the word list barely fit on the tiny disk of the time,
the job entailed unimaginable marshalling of resources. I
was mightily impressed then, and still am.

Doug


From michael at kjorling.se  Sat May  6 23:44:55 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 6 May 2017 13:44:55 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <CAJfiPzxKN09Y_8w7tPvAXjC=VZh-qJYpPezW40UV6B40K=5a5g@mail.gmail.com>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se>
 <CAJfiPzxKN09Y_8w7tPvAXjC=VZh-qJYpPezW40UV6B40K=5a5g@mail.gmail.com>
Message-ID: <20170506134455.GG12539@yeono.kjorling.se>

On 6 May 2017 09:09 -0400, from cym224 at gmail.com (Nemo):
>> I would actually take that one step further: When you are writing
>> code, you are _first and foremost_ communicating with whatever human
>> will need to read or modify the code later.
> 
> ..., there are
> times when the code is transformed by optimization into something very
> difficult to read.  (We used noweb to document the transformation.)

True, optimized code can be difficult to read. But very little code
really _needs_ to be optimized to that level, and when you come across
it, you will generally know -- at least after running the application
through a profiler. (I have to admit we do have it easy these days.)

I've been working on optimizing some SQL code recently -- I know,
that's not exactly operating system kernel C or assembler -- and it's
amazing at times how the code can end up being both easier to
understand _as well as_ faster to execute, if you just put some
thought into what the code really needs to do. And not just minor
speedups either, but wallclock runtime reductions on the order of _two
to four nines_.

I think there's been only one time in the last several years when I
have really _needed_ to optimize code to the point that its workings
were not obvious.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From toby at telegraphics.com.au  Sun May  7 00:16:03 2017
From: toby at telegraphics.com.au (Toby Thain)
Date: Sat, 6 May 2017 10:16:03 -0400
Subject: [TUHS] The Elements of Programming Style (book) - was Re: Discuss
 of style and design of computer programs
In-Reply-To: <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
References: <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
Message-ID: <f3bc4461-5a08-b227-2868-eb4aeb247289@telegraphics.com.au>

On 2017-05-06 1:33 AM, Steve Johnson wrote:
> In 1973, I spent 9 months at the University of Waterloo, home of Watfor.
>   I taught a course in "Advanced Application Techniques" -- the language
> was FORTRAN, and the techniques included such things as using libraries
> and handling errors in input data...
> ...
> in fairness, I didn't (and don't) know any better way to learn style
> than doing a lot of reading, a lot of writing, and getting criticized
> constructively by those with more experience.  For me, a lot of what I
> learned was from Stan Brown at the Labs, who read piles of my
> (atrocious) FORTRAN code and repeatedly pointed out that when you wrote
> a program, you were not just communicating with the computer, but also
> other humans (including yourself) who would read (and perhaps modify)
> the code in the future...

As a longtime maintenance programmer: I could not agree more. Thankfully 
this idea is fairly well accepted today. Although, I am working on a 
project right now that makes me think the author simply never considered 
that anyone else might ever read or maintain it.

In this context, one of the timeless classics is The Elements of 
Programming Style (Kernighan and Plauger). It had a huge effect on my C 
when I read it years ago and I still try to apply its lessons every day 
in any language.

--Toby


>
> Steve
>
>
>
>     ----- Original Message -----
>     From:
>     "Doug McIlroy" <doug at cs.dartmouth.edu>
>
>     To:
>     <lm at mcvoy.com>, <doug at cs.dartmouth.edu>, <clemc at ccc.com>
>     Cc:
>     <tuhs at minnie.tuhs.org>
>     Sent:
>     Fri, 05 May 2017 22:02:21 -0400
>     Subject:
>     Re: [TUHS] Discuss of style and design of computer programs from a
>     user stand point [was dmr note on BSD's sins]
>
>
>     Clem wrote, "So, I wonder do any of the schools like Darthmouth and
>     the like
>     teach courses that study 'style' and taste in CS."
>
>     Some time around 1970, Bob Rosen instituted a program-reading
>     course at the University of Iowa, by overt analogy to the
>     study of literature. Style was certainly a central concern;
>     I'm not sure about the deeper matter of taste. I'm not aware
>     of other examples, except perhaps for a collaboration between
>     Don Kunth and a teachaer of (I think) creative writing--a
>     course whose content I can't recall.
>
>     Doug
>



From lm at mcvoy.com  Sun May  7 00:40:11 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Sat, 6 May 2017 07:40:11 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <20170506091857.GE12539@yeono.kjorling.se>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se>
Message-ID: <20170506144011.GF28787@mcvoy.com>

On Sat, May 06, 2017 at 09:18:57AM +0000, Michael Kj??rling wrote:
> On 5 May 2017 22:33 -0700, from scj at yaccman.com (Steve Johnson):
> > For me, a lot of what
> > I learned was from Stan Brown at the Labs, who read piles of my
> > (atrocious) FORTRAN code and repeatedly pointed out that when you
> > wrote a program, you were not just communicating with the computer,
> > but also other humans (including yourself) who would read (and perhaps
> > modify) the code in the future...
> 
> I would actually take that one step further: When you are writing
> code, you are _first and foremost_ communicating with whatever human
> will need to read or modify the code later. That human might be you, a
> colleague, or the violent psychopath who knows both where you live and
> where your little kids go to school (might as well be you). 

When I run engineering teams the point I make is if you wrote the code
more than 6 months ago it might as well be someone else's code.  So
write it in a way that someone else can debug / extend it.

I also made the point that "clever" sucks.  At least most of the time.
All "clever" means is "hard to understand".  The brief joy one gets 
from clever code is stomped on by the frustration one gets from having
to fix it.  

When they wouldn't listen to that I'd say "I'm the dumbest guy here, right?
And I'm the boss.  And I debug could.  The path to a big bonus does not 
go through clever code".  That seemed to work.

Personally, I find code that is clean, straightforward, obvious to be
beautiful.  The clever stuff usually strikes an odd note, not a good
one.

--lm



From corey at lod.com  Sun May  7 01:09:13 2017
From: corey at lod.com (Corey Lindsly)
Date: Sat, 6 May 2017 08:09:13 -0700 (PDT)
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506144011.GF28787@mcvoy.com>
Message-ID: <20170506150913.57571411A@lod.com>


> Personally, I find code that is clean, straightforward, obvious to be
> beautiful.  The clever stuff usually strikes an odd note, not a good
> one.
> 
> --lm

I am not a programmer. Almost four decades ago, my first computer was a 
TRS80 Model 1 with 16KB RAM. I spent one month disassembling and stepping 
through the Z80 code for the resident Microsoft BASIC interpreter. The 
entire thing fit in a 12KB PROM so it was originally written in assembly 
and tightly optimized. It was fascinating and extremely instructive. All 
these years later, I could probably still slap together a Z80 program if I 
needed to. 

Anyway, I reached one point in the assembly code that I simply could not 
understand. It seemed like a mistake, and I went through it again and 
again until I finally realized what it was doing. There was a branch/loop 
that jumped to the middle of a multi-byte machine instruction, so that 
branch had to be disassembled and stepped separately until it "synced" up 
with the other branch again. Maybe this is standard practice in 
programming (I don't know) but at the time I thought, what kind of evil 
genius devised this to save a few bytes of memory?

--corey


From michael at kjorling.se  Sun May  7 01:20:53 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 6 May 2017 15:20:53 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506150913.57571411A@lod.com>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
Message-ID: <20170506152053.GI12539@yeono.kjorling.se>

On 6 May 2017 08:09 -0700, from corey at lod.com (Corey Lindsly):
> Anyway, I reached one point in the assembly code that I simply could not 
> understand. It seemed like a mistake, and I went through it again and 
> again until I finally realized what it was doing. There was a branch/loop 
> that jumped to the middle of a multi-byte machine instruction, so that 
> branch had to be disassembled and stepped separately until it "synced" up 
> with the other branch again. Maybe this is standard practice in 
> programming (I don't know) but at the time I thought, what kind of evil 
> genius devised this to save a few bytes of memory?

IIRC, that _was_ a common trick at least on machines of that class. It
did have the potential to save a few bytes, yes (more if the
instructions were such that you'd get some _other, desired_, behavior
by jumping into the middle of one with some specific state), but it
also foiled lots of disassemblers: Simply disassembling a binary from
start to finish would yield nonsense in those locations, as you
experienced. It thus basically forced you to single-step those
instructions to figure out what was going on from the binary.

I'm pretty sure it works on every architecture with variable-length
instructions and arbitrary jump capability, as long as you have
control over the specific machine instructions generated (such as if
you are programming in assembler). Of course, it _is_ also a total
nightmare to maintain such code.

I would absolutely not say that doing something like that is standard
practice in modern programming. Even in microcontrollers, where
program and data memory can be scarce even today, I would argue that
the costs would not outweigh the benefits by a long shot.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From rminnich at gmail.com  Sun May  7 01:23:23 2017
From: rminnich at gmail.com (ron minnich)
Date: Sat, 06 May 2017 15:23:23 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506150913.57571411A@lod.com>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
Message-ID: <CAP6exYKGXqJe7=LJyVXvbVS+w7TZipvybR4XefOnXpen+9hH2A@mail.gmail.com>

On Sat, May 6, 2017 at 8:09 AM Corey Lindsly <corey at lod.com> wrote:

> There was a branch/loop
> that jumped to the middle of a multi-byte machine instruction, so that
> branch had to be disassembled and stepped separately until it "synced" up
> with the other branch again. Maybe this is standard practice in
> programming (I don't know) but at the time I thought, what kind of evil
> genius devised this to save a few bytes of memory?
>
>
This was extremely common back then. I had a friend who worked on a gas
chromatograph project, names redacted here. It had a very advanced idea, a
thermal printer. It would print a banner when it started.  Getting that
print to work, in the ROM they had, was a nightmare that involved all these
tricks.

When the "A" version came out, they asked my friend to have it print the A
after the product number. His response: "NO". There was no way he could
ever pick apart the crazy code that had printed out the startup banner so
he could add an "A". The startup banner remained the same.

Executing code as data in the early startup was also common in those days,
and modifying that data and then rerunning it happened all the time.

This is why the things like the Therac 25 happened ...
https://en.wikipedia.org/wiki/Therac-25

Note the reference to "Cargo coding", reusing code you don't understand. In
modern terms we call this software reuse and it's taught at all the best
uni's. Google a package, pull it down, compile it in, done.

Lest you think things are better now, Linux uses self modifying code to
optimize certain critical operations, and at one talk I heard the speaker
say that he'd like to put more self modifying code into Linux, "because
it's fun". Oh boy.

ron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/b0f8b7d4/attachment.html>

From lm at mcvoy.com  Sun May  7 01:24:11 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Sat, 6 May 2017 08:24:11 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506152053.GI12539@yeono.kjorling.se>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
 <20170506152053.GI12539@yeono.kjorling.se>
Message-ID: <20170506152411.GJ28787@mcvoy.com>

On Sat, May 06, 2017 at 03:20:53PM +0000, Michael Kj??rling wrote:
> I would absolutely not say that doing something like that is standard
> practice in modern programming. Even in microcontrollers, where
> program and data memory can be scarce even today, I would argue that
> the costs would not outweigh the benefits by a long shot.

It strikes as being similar to Duff's device (1).  Which is a niche thing
but I still use that from time to time.  Not to save memory, just because
as a C programmer it seems pretty natural to do it.

--lm

(1) https://en.wikipedia.org/wiki/Duff's_device


From michael at kjorling.se  Sun May  7 01:44:48 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 6 May 2017 15:44:48 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <CAP6exYKGXqJe7=LJyVXvbVS+w7TZipvybR4XefOnXpen+9hH2A@mail.gmail.com>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
 <CAP6exYKGXqJe7=LJyVXvbVS+w7TZipvybR4XefOnXpen+9hH2A@mail.gmail.com>
Message-ID: <20170506154447.GJ12539@yeono.kjorling.se>

On 6 May 2017 15:23 +0000, from rminnich at gmail.com (ron minnich):
> This is why the things like the Therac 25 happened ...
> https://en.wikipedia.org/wiki/Therac-25

Or the Ariane 5 flight 501, which according to Wikipedia cost more
than 370 million dollars (and resulted in the total loss of the
spacecraft). I would say a few million or even a few tens of millions
of dollars to double-check the software, or even write new software
specifically designed for the Ariane 5 rather than reuse software
designed for the Ariane 4 outside of that software's design limits,
might not have been a bad way to spend money there.

And that was in 1996. Hardly that long ago.

https://en.wikipedia.org/wiki/Ariane_5#Notable_launches

https://en.wikipedia.org/wiki/Cluster_%28spacecraft%29

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From michael at kjorling.se  Sun May  7 01:51:24 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 6 May 2017 15:51:24 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506152411.GJ28787@mcvoy.com>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
 <20170506152053.GI12539@yeono.kjorling.se>
 <20170506152411.GJ28787@mcvoy.com>
Message-ID: <20170506155124.GK12539@yeono.kjorling.se>

On 6 May 2017 08:24 -0700, from lm at mcvoy.com (Larry McVoy):
>> I would absolutely not say that doing something like that is standard
>> practice in modern programming. Even in microcontrollers, where
>> program and data memory can be scarce even today, I would argue that
>> the costs would not outweigh the benefits by a long shot.
> 
> It strikes as being similar to Duff's device (1).  Which is a niche thing
> but I still use that from time to time.  Not to save memory, just because
> as a C programmer it seems pretty natural to do it.
> 
> --lm
> 
> (1) https://en.wikipedia.org/wiki/Duff's_device

I disagree; loop unrolling and jumping to the beginning of some
instruction inside that unrolled loop is not at all the same thing as
jumping _into the middle of a machine language instruction_.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From lm at mcvoy.com  Sun May  7 01:53:44 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Sat, 6 May 2017 08:53:44 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506155124.GK12539@yeono.kjorling.se>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
 <20170506152053.GI12539@yeono.kjorling.se>
 <20170506152411.GJ28787@mcvoy.com>
 <20170506155124.GK12539@yeono.kjorling.se>
Message-ID: <20170506155344.GK28787@mcvoy.com>

On Sat, May 06, 2017 at 03:51:24PM +0000, Michael Kj??rling wrote:
> On 6 May 2017 08:24 -0700, from lm at mcvoy.com (Larry McVoy):
> >> I would absolutely not say that doing something like that is standard
> >> practice in modern programming. Even in microcontrollers, where
> >> program and data memory can be scarce even today, I would argue that
> >> the costs would not outweigh the benefits by a long shot.
> > 
> > It strikes as being similar to Duff's device (1).  Which is a niche thing
> > but I still use that from time to time.  Not to save memory, just because
> > as a C programmer it seems pretty natural to do it.
> > 
> > --lm
> > 
> > (1) https://en.wikipedia.org/wiki/Duff's_device
> 
> I disagree; loop unrolling and jumping to the beginning of some
> instruction inside that unrolled loop is not at all the same thing as
> jumping _into the middle of a machine language instruction_.

That's fine, I feel no need to argue about it.  Seemed similar to me but
I'm not the sharpest tool in the shed :)


From khm at sciops.net  Sun May  7 02:40:02 2017
From: khm at sciops.net (Kurt H Maier)
Date: Sat, 6 May 2017 09:40:02 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <20170506091857.GE12539@yeono.kjorling.se>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se>
Message-ID: <20170506164002.GD38580@wopr>

On Sat, May 06, 2017 at 09:18:57AM +0000, Michael Kjörling wrote:
>
> I would actually take that one step further: When you are writing     
> code, you are _first and foremost_ communicating with whatever human  
> will need to read or modify the code later. 
         
This can be, and is, taken too far.  It is currently fashionable in the
Ruby world to erect testing gantries that look like
       
describe 'service' do
  it 'is listening' do
    expect(port 443).to be_listening
  end
end
       
and then drag in a couple hundred thousand lines of extra code to make  
this cuteness approximate instructions for the computer.
       
It's all well and good to write for the person who comes behind you, but
at some point it will break, and the closer you've flown to this sun the
worse it will be for the poor sod who has to pick up the pieces.  I'd   
take durable code I have to work to understand over fragile artwork     
any day.
       
khm


From david at kdbarto.org  Sun May  7 03:52:11 2017
From: david at kdbarto.org (David)
Date: Sat, 6 May 2017 10:52:11 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
	user stand point
In-Reply-To: <mailman.821.1494062349.3779.tuhs@minnie.tuhs.org>
References: <mailman.821.1494062349.3779.tuhs@minnie.tuhs.org>
Message-ID: <D211CAF7-1ECF-4F83-8F7E-055C9EF7FE7D@kdbarto.org>


> From: Michael Kjörling <michael at kjorling.se>
> To: tuhs at minnie.tuhs.org
> Subject: Re: [TUHS] Discuss of style and design of computer programs
> 	from a user stand point
> Message-ID: <20170506091857.GE12539 at yeono.kjorling.se>
> Content-Type: text/plain; charset=utf-8
> 
> I would actually take that one step further: When you are writing
> code, you are _first and foremost_ communicating with whatever human
> will need to read or modify the code later. That human might be you, a
> colleague, or the violent psychopath who knows both where you live and
> where your little kids go to school (might as well be you). You should
> strive to write the code accordingly, _even if_ the odds of the threat
> ever materializing are slim at most. Style matters a lot, there.
> 
Interesting, I was going to say about the same thing about the violent psychopath
who has to maintain your code after you leave. When I lectured at UCSD or was
giving talks on style for ViaSat I always said the same thing:

  Whatever you write, the fellow who is going to wind up maintaining it is a known
  axe killer, now released from prison, completely reformed. He learned computer
  programming on MS/DOS 3.1 and a slightly broken version of Pascal. He will be
  given your home phone number and address so if he has any questions about the
  code you wrote he can get in contact with you.

This always got a few chuckles. I then pointed out that whenever anyone gets code
that someone else wrote, the recipient always thinks that they can ‘clean up’ what
is there because the original author clearly doesn’t understand what proper code
looks like.

Over time, I’ve learned that everyone has a style when writing code, just like handwriting
and given enough time, I can spot who the author of a block of code is just from the
indenting, placement of ( and ) around a statement and other small traits.

What makes good code is the ability to convey the meaning of the algorithm
from the original author to all those who come after. Sometimes even the most
unusual code can be quite clear, while the most cleanly formatted and commented
code can be opaque to all.

	David




From dave at horsfall.org  Sun May  7 04:43:33 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sun, 7 May 2017 04:43:33 +1000 (EST)
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <20170506144011.GF28787@mcvoy.com>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se> <20170506144011.GF28787@mcvoy.com>
Message-ID: <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>

On Sat, 6 May 2017, Larry McVoy wrote:

> When I run engineering teams the point I make is if you wrote the code 
> more than 6 months ago it might as well be someone else's code.  So 
> write it in a way that someone else can debug / extend it.

A principle I've tried to follow over the years is "write code as though 
the next person to maintain it is an axe-wielding murderer who knows where 
you live".

> I also made the point that "clever" sucks.  At least most of the time. 
> All "clever" means is "hard to understand".  The brief joy one gets from 
> clever code is stomped on by the frustration one gets from having to fix 
> it.

Apropos the above, I do confess to writing "clever code" from time to time 
(PERL is a beautiful language for such sins)...

Of course, in the olden days it was the APL "one-liners" :-)

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From bakul at bitblocks.com  Sun May  7 05:50:49 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Sat, 06 May 2017 12:50:49 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
	user stand point
In-Reply-To: Your message of "Sun, 07 May 2017 04:43:33 +1000."
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se> <20170506144011.GF28787@mcvoy.com>
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
Message-ID: <20170506195049.8FBEA124AEA4@mail.bitblocks.com>

On Sun, 07 May 2017 04:43:33 +1000 Dave Horsfall <dave at horsfall.org> wrote:
> 
> > I also made the point that "clever" sucks.  At least most of the time. 
> > All "clever" means is "hard to understand".  The brief joy one gets from 
> > clever code is stomped on by the frustration one gets from having to fix 
> > it.

> Apropos the above, I do confess to writing "clever code" from time to time 
> (PERL is a beautiful language for such sins)...
> 
> Of course, in the olden days it was the APL "one-liners" :-)

On the other hand, you have to grok only one line of code
instead reading pages of Java/C++ with surprises lurking
everywhere :-)

Since you need one line to do the work of many many lines of
c++/Java code, you should expect to spend more time on that
one line!

The same is true with shell one liners as well. We don't
(usually) consider constructs such as the following "clever"
because we expect its reader to have a certain level of code
literacy and context (or an ability to gain such).

tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q


From usotsuki at buric.co  Sun May  7 06:00:52 2017
From: usotsuki at buric.co (Steve Nickolas)
Date: Sat, 6 May 2017 16:00:52 -0400 (EDT)
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506152053.GI12539@yeono.kjorling.se>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com> <20170506152053.GI12539@yeono.kjorling.se>
Message-ID: <alpine.BSF.2.02.1705061554080.4621@frieza.hoshinet.org>

On Sat, 6 May 2017, Michael Kjörling wrote:

> On 6 May 2017 08:09 -0700, from corey at lod.com (Corey Lindsly):
>> Anyway, I reached one point in the assembly code that I simply could not
>> understand. It seemed like a mistake, and I went through it again and
>> again until I finally realized what it was doing. There was a branch/loop
>> that jumped to the middle of a multi-byte machine instruction, so that
>> branch had to be disassembled and stepped separately until it "synced" up
>> with the other branch again. Maybe this is standard practice in
>> programming (I don't know) but at the time I thought, what kind of evil
>> genius devised this to save a few bytes of memory?
>
> IIRC, that _was_ a common trick at least on machines of that class. It
> did have the potential to save a few bytes, yes (more if the
> instructions were such that you'd get some _other, desired_, behavior
> by jumping into the middle of one with some specific state), but it
> also foiled lots of disassemblers: Simply disassembling a binary from
> start to finish would yield nonsense in those locations, as you
> experienced. It thus basically forced you to single-step those
> instructions to figure out what was going on from the binary.
>
> I'm pretty sure it works on every architecture with variable-length
> instructions and arbitrary jump capability, as long as you have
> control over the specific machine instructions generated (such as if
> you are programming in assembler). Of course, it _is_ also a total
> nightmare to maintain such code.
>
> I would absolutely not say that doing something like that is standard
> practice in modern programming. Even in microcontrollers, where
> program and data memory can be scarce even today, I would argue that
> the costs would not outweigh the benefits by a long shot.

In 6502 code, it's not uncommon to do something like

foo1:     lda      #$00
           .byte    $2C       ; 3-byte BIT
foo2:     lda      #$01
            .
            .
            .

to save a byte (and probably still done for the few who write in ASM). 
The "2C" operand would cause it to disassemble as something like...

1000-     LDA      #$00
1002-     BIT      $01A9

which is the route you'd go down if you called "foo1".  Apart diddling a 
few CPU flags, and an unneeded read on $01A9, harmless.

(Most 6502 programmers would probably see a strange BIT instruction as an 
attempt to do this.)

It's probably not a good idea to still do this unless you're really REALLY 
crunched for space.

-uso.

From michael at kjorling.se  Sun May  7 07:45:19 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 6 May 2017 21:45:19 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <alpine.BSF.2.02.1705061554080.4621@frieza.hoshinet.org>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
 <20170506152053.GI12539@yeono.kjorling.se>
 <alpine.BSF.2.02.1705061554080.4621@frieza.hoshinet.org>
Message-ID: <20170506214519.GN12539@yeono.kjorling.se>

On 6 May 2017 16:00 -0400, from usotsuki at buric.co (Steve Nickolas):
> In 6502 code, it's not uncommon to do something like
> 
> foo1:     lda      #$00
>           .byte    $2C       ; 3-byte BIT
> foo2:     lda      #$01
>            .
>            .
>            .
> 
> to save a byte (and probably still done for the few who write in
> ASM). The "2C" operand would cause it to disassemble as something
> like...
> 
> 1000-     LDA      #$00
> 1002-     BIT      $01A9
> 
> which is the route you'd go down if you called "foo1".  Apart
> diddling a few CPU flags, and an unneeded read on $01A9, harmless.

You could do something quite similar on the 8086, which I am somewhat
more familiar with. For example, in some kind of pseudo 8086
assembler, with $ denoting the value of the instruction pointer at the
beginning of the current instruction:

        jmp short $+3
        int 90h ; hex: cd 90

would almost certainly decompile to the above, if the decompiler
doesn't barf when it can't create a jump target label mid-instruction,
but it would certainly _execute_ as

        jmp short l1
        db 0cdh ; dummy byte
    l1: nop ; hex: 90

resulting in a jump over one byte followed by a no-op instruction. Of
course, invoking interrupt 90h would be a perfectly legal thing to do,
assuming that your interrupt tables are set up correctly. If you want
to mess with someone trying to figure out what the code is doing,
write a (nonsense or meaningful) value to interrupt vector 90h before
you do that. For example, assuming that it isn't being used, you could
point interrupt 90h at the reboot jump location, which IIRC on the IBM
PC would mean point it at FFFFh:FFF0h or absolute address FFFF0h.
Anyone trying to actually execute the INT 90h instruction would see
their computer reboot, but anyone actually executing the code would
see little more than a CPU cache flush due to the jump.

Or, something slightly more "useful"

        jmp short $+3
        int 0 ; hex: cd 00
        db 0
        jz somewhere

which, again IIRC, would clear the accumulator (AX) register because
0000 hex is XOR AX,AX, which as a side effect sets the zero flag
because the result is zero, and then executes a conditional jump if
the zero flag is set (which it will be). In other words, it _executes_
as

        jmp short l1
        db 0cdh ; dummy byte
    l1: xor ax,ax ; hex: 00 00
        jz somewhere

but a naiive decompiler will see the CDh byte after the jump and take
that as the first byte of the two-byte interrupt instruction. I don't
know what 00 followed by the first byte of the JZ instruction would
be, but probably a register/register XOR with a register store. Make
it a far jump and you have a few extra bytes to play with, to befuddle
anyone trying to figure out just what on Earth your code is really
doing.

The above, of course, is really just scratching the surface. With how
many variable-length instructions the x86 has, with careful selection
of opcodes and possibly use of undocumented but functionally valid
combinations, I wouldn't be the least bit surprised if it's
technically possible to write a 8086 program that does one thing when
started normally, and something utterly different but still useful
when started at an offset that results in execution beginning in the
middle of an instruction. Bonus points if the first thing that program
does is jump into itself _at that offset_. Now, the _work involved in
doing so_, never mind maintaining it...

Of course, I wouldn't do anything like the above in any real-world
code base meant to run on modern systems unless obfuscation really
_was_ a design goal. Honest. If I was tasked to write something that
really needed to do something non-trivial in 16 KiB RAM on an original
IBM 5150 PC or clone? I'd probably spend quite a bit of time in front
of the whiteboard and with reference manuals before writing even one
line of code...

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From scj at yaccman.com  Sun May  7 08:11:32 2017
From: scj at yaccman.com (Steve Johnson)
Date: Sat, 06 May 2017 15:11:32 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
	user stand point [was dmr note on BSD's sins]
In-Reply-To: <20170506060727.A00C0124AEFE@mail.bitblocks.com>
Message-ID: <5abeeef795d15ae5b9a1eaca3bcb4f684ecb1c4d@webmail.yaccman.com>

 few more comments about style and optimization:First, the thing that
make optimized code hard to understand is that (at least when a
compiler does it) the optimized program goes through a different set
of states than the user expected from the source code.  When a loop
is unrolled, or invariant expressions moved out of a loop, or indexing
is turned into pointer manipulation, if you don't know what's
happening its mysterious.  But once you'v seen this a few times, it's
only a momentary hitch...Also, one lesson I learned from Unix is that
if you can make a program 10x faster it becomes a qualitatively
different experience.  The first version of Yacc I wrote, using the
textbook algorithms, took 20 minutes to process a 30-rule grammar.
 On the shared PDP-11, when I ran Yacc everyone else's jobs slowed to
a crawl, and people were heard to mutter "*%&#!  Johnson's running
Yacc again!"   A couple of years later, Yacc produced parsers faster
than the C compiler could compile them.Consider what using Google
would be like if it took 7 seconds rather than .7 seconds to respond.
 Probably still useful, but a different user experience.We had many
decades to be spoiled as far as program performance was concerned.
 As our tools got more bloated, faster processors and more memory hid
the true effect from most of us.  Now we have to rethink everything
to make it parallel.  And complexity is enemy number one when making
something parallel...
Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/0b9f37a0/attachment.html>

From lm at mcvoy.com  Sun May  7 09:35:56 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Sat, 6 May 2017 16:35:56 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point [was dmr note on BSD's sins]
In-Reply-To: <5abeeef795d15ae5b9a1eaca3bcb4f684ecb1c4d@webmail.yaccman.com>
References: <20170506060727.A00C0124AEFE@mail.bitblocks.com>
 <5abeeef795d15ae5b9a1eaca3bcb4f684ecb1c4d@webmail.yaccman.com>
Message-ID: <20170506233556.GG9278@mcvoy.com>

On Sat, May 06, 2017 at 03:11:32PM -0700, Steve Johnson wrote:
> And complexity is enemy number one when making something parallel...

+1 Couldn't agree more, for multiple reasons.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 


From cym224 at gmail.com  Sun May  7 10:51:05 2017
From: cym224 at gmail.com (Nemo)
Date: Sat, 6 May 2017 20:51:05 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
Message-ID: <CAJfiPzyVPzyYN4--5RuVt+nAcw4ppm3_ejdZmWC_oGv9gxjDYw@mail.gmail.com>

On 6 May 2017 at 11:23, ron minnich <rminnich at gmail.com> wrote (in part):
[...]
> Lest you think things are better now, Linux uses self modifying code to
> optimize certain critical operations, and at one talk I heard the speaker
> say that he'd like to put more self modifying code into Linux, "because it's
> fun". Oh boy.

Fun, indeed!  Even self-modifying chips are being touted -- Yikes!

N.


From imp at bsdimp.com  Sun May  7 11:15:43 2017
From: imp at bsdimp.com (Warner Losh)
Date: Sat, 6 May 2017 19:15:43 -0600
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <20170506195049.8FBEA124AEA4@mail.bitblocks.com>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se> <20170506144011.GF28787@mcvoy.com>
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
 <20170506195049.8FBEA124AEA4@mail.bitblocks.com>
Message-ID: <CANCZdfrvwJwsUQjbBdudJ4Ro_uo_yeatVAhYxGZ+rkesUB56Dg@mail.gmail.com>

On Sat, May 6, 2017 at 1:50 PM, Bakul Shah <bakul at bitblocks.com> wrote:
> tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q

The cool thing about this thread is that I learned two things: what tr
-s does, and the Nq does for sed...

Sadly, this doesn't work so well for text that isn't ASCII-7 english...

Warner


From noel.hunt at gmail.com  Sun May  7 11:42:31 2017
From: noel.hunt at gmail.com (Noel Hunt)
Date: Sun, 7 May 2017 11:42:31 +1000
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <CANCZdfrvwJwsUQjbBdudJ4Ro_uo_yeatVAhYxGZ+rkesUB56Dg@mail.gmail.com>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se> <20170506144011.GF28787@mcvoy.com>
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
 <20170506195049.8FBEA124AEA4@mail.bitblocks.com>
 <CANCZdfrvwJwsUQjbBdudJ4Ro_uo_yeatVAhYxGZ+rkesUB56Dg@mail.gmail.com>
Message-ID: <CAGfO01x_pDxSTtyKFgAu9FYAJBQDtqgdSRmkXAO4TO==73GmYw@mail.gmail.com>

I was about to suggest using the Plan9 port utilities of the
same name but it seems 'uniq' is not coded to handle Runes
(aka utf-8). I don't imagine it would be hard to re-write it to
handle utf-8.

On Sun, May 7, 2017 at 11:15 AM, Warner Losh <imp at bsdimp.com> wrote:

> On Sat, May 6, 2017 at 1:50 PM, Bakul Shah <bakul at bitblocks.com> wrote:
> > tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q
>
> The cool thing about this thread is that I learned two things: what tr
> -s does, and the Nq does for sed...
>
> Sadly, this doesn't work so well for text that isn't ASCII-7 english...
>
> Warner
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170507/78dce325/attachment.html>

From crossd at gmail.com  Sun May  7 14:06:48 2017
From: crossd at gmail.com (Dan Cross)
Date: Sun, 7 May 2017 00:06:48 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point [was dmr note on BSD's sins]
In-Reply-To: <20170506060727.A00C0124AEFE@mail.bitblocks.com>
References: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
 <1C6D6D3E-058F-4743-8DA0-17CEC09B64E9@bitblocks.com>
 <CAGfO01ytR+g09Mp+YQ3fR2j=QnHZsfcZM-xQQKr+NNvtESypcA@mail.gmail.com>
 <20170506060727.A00C0124AEFE@mail.bitblocks.com>
Message-ID: <CAEoi9W4BtA_D2R-kQ08Zoe5gYUu+EL+3zD66FND7_H+p_M03Tw@mail.gmail.com>

On Sat, May 6, 2017 at 2:07 AM, Bakul Shah <bakul at bitblocks.com> wrote:

> Note that a similar approach was implemented by Iverson in
> APL. In fact a shell pipeline has a lot in common with array
> programming!
>
> Iverson in essence built a whole workshop with a set of simple
> but powerful standard tools (functions and higher order
> functions).  The same has been carried on in follow on array
> languages such as j and k and q languages.  Many of these
> array functions are directly applicable to stream processing.
> In contrast unix shells provide a very shallow glue layer.


Interesting analogy. I always think that function composition in lazy
languages like Haskell reminds me of shell pipelines. I know that Doug
swims in those waters; perhaps he will correct me if he feels that's a poor
analogy?

Carrying it forward a bit, however, I'm always amazed (and a little
appalled) at how the kids always think that they invented Rock n' Roll.
Ideas are continually recycled, but it's the rare recycler indeed who seeks
out the prior art on rediscovering one of the good ones. "Look! You can
filter and transform this data!" "...but I've been doing that at the shell
nearly every day for the past quarter of a century...." "...this one goes
to 11."

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170507/887dbd48/attachment.html>

From steve at sk2.org  Sun May  7 17:42:27 2017
From: steve at sk2.org (Stephen Kitt)
Date: Sun, 7 May 2017 09:42:27 +0200
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <20170506214519.GN12539@yeono.kjorling.se>
References: <20170506144011.GF28787@mcvoy.com>
 <20170506150913.57571411A@lod.com>
 <20170506152053.GI12539@yeono.kjorling.se>
 <alpine.BSF.2.02.1705061554080.4621@frieza.hoshinet.org>
 <20170506214519.GN12539@yeono.kjorling.se>
Message-ID: <20170507094227.4af8e074@heffalump.sk2.org>

On Sat, 6 May 2017 21:45:19 +0000, Michael Kjörling <michael at kjorling.se>
wrote:
> On 6 May 2017 16:00 -0400, from usotsuki at buric.co (Steve Nickolas):
> > In 6502 code, it's not uncommon to do something like
> > 
> > foo1:     lda      #$00
> >           .byte    $2C       ; 3-byte BIT
> > foo2:     lda      #$01
> >            .
> >            .
> >            .
> > 
> > to save a byte (and probably still done for the few who write in
> > ASM). The "2C" operand would cause it to disassemble as something
> > like...
> > 
> > 1000-     LDA      #$00
> > 1002-     BIT      $01A9
> > 
> > which is the route you'd go down if you called "foo1".  Apart
> > diddling a few CPU flags, and an unneeded read on $01A9, harmless.  
> 
> You could do something quite similar on the 8086, which I am somewhat
> more familiar with.
[...]

One possible equivalent 16-bit x86 is

	foo1:	mov al, 0
		db 035h		; XOR AX, imm16
	foo2:	mov al, 1

which implements a branchless fall-through. (But you’d probably just use
SALC...)

> Or, something slightly more "useful"
> 
>         jmp short $+3
>         int 0 ; hex: cd 00
>         db 0
>         jz somewhere
> 
> which, again IIRC, would clear the accumulator (AX) register because
> 0000 hex is XOR AX,AX,

According to my disassembler 0000 is "add byte ptr [bx+si], al"; "xor ax, ax"
is either 31c0 or 33c0.

> which as a side effect sets the zero flag
> because the result is zero, and then executes a conditional jump if
> the zero flag is set (which it will be). In other words, it _executes_
> as
> 
>         jmp short l1
>         db 0cdh ; dummy byte
>     l1: xor ax,ax ; hex: 00 00
>         jz somewhere
> 
> but a naiive decompiler will see the CDh byte after the jump and take
> that as the first byte of the two-byte interrupt instruction. I don't
> know what 00 followed by the first byte of the JZ instruction would
> be, but probably a register/register XOR with a register store. Make
> it a far jump and you have a few extra bytes to play with, to befuddle
> anyone trying to figure out just what on Earth your code is really
> doing.

My memories of disassembling x86 code in the early 90s at least is that,
because of all this, disassemblers were pretty good at restarting streams
from jump destinations, so it never really was an issue. You need to get
creative with single-stepping mode and self-decrypting code to really annoy
people trying to understand your code ;-).

> The above, of course, is really just scratching the surface. With how
> many variable-length instructions the x86 has, with careful selection
> of opcodes and possibly use of undocumented but functionally valid
> combinations, I wouldn't be the least bit surprised if it's
> technically possible to write a 8086 program that does one thing when
> started normally, and something utterly different but still useful
> when started at an offset that results in execution beginning in the
> middle of an instruction. Bonus points if the first thing that program
> does is jump into itself _at that offset_. Now, the _work involved in
> doing so_, never mind maintaining it...

I have seen some short sections of code which do this. The x86 ISA is so rich
that you can make it adapt to many different constraints — one I rather liked
is assembling code which is valid ASCII (amaze your friends by typing in
a .COM using COPY CON!). There’s a C compiler which does that on GitHub
somewhere.

> Of course, I wouldn't do anything like the above in any real-world
> code base meant to run on modern systems unless obfuscation really
> _was_ a design goal. Honest. If I was tasked to write something that
> really needed to do something non-trivial in 16 KiB RAM on an original
> IBM 5150 PC or clone? I'd probably spend quite a bit of time in front
> of the whiteboard and with reference manuals before writing even one
> line of code...

Yes, and combine that with assemblers which output all possible encodings of
a given instruction!

Regards,

Stephen


From michael at kjorling.se  Sun May  7 23:49:43 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sun, 7 May 2017 13:49:43 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <CAEoi9W4BtA_D2R-kQ08Zoe5gYUu+EL+3zD66FND7_H+p_M03Tw@mail.gmail.com>
References: <CAC20D2MFq8DWeOZznMxFfmj05YiXdu0SBOb+Zmhm=5pk+=ZoAA@mail.gmail.com>
 <1C6D6D3E-058F-4743-8DA0-17CEC09B64E9@bitblocks.com>
 <CAGfO01ytR+g09Mp+YQ3fR2j=QnHZsfcZM-xQQKr+NNvtESypcA@mail.gmail.com>
 <20170506060727.A00C0124AEFE@mail.bitblocks.com>
 <CAEoi9W4BtA_D2R-kQ08Zoe5gYUu+EL+3zD66FND7_H+p_M03Tw@mail.gmail.com>
Message-ID: <20170507134943.GR12539@yeono.kjorling.se>

On 7 May 2017 00:06 -0400, from crossd at gmail.com (Dan Cross):
> "Look! You can
> filter and transform this data!" "...but I've been doing that at the shell
> nearly every day for the past quarter of a century...." "...this one goes
> to 11."

"Look! This code can execute code that it creates itself!" "LISP has
been around for a while."

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From michael at kjorling.se  Sun May  7 23:54:07 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sun, 7 May 2017 13:54:07 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <CAGfO01x_pDxSTtyKFgAu9FYAJBQDtqgdSRmkXAO4TO==73GmYw@mail.gmail.com>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se>
 <20170506144011.GF28787@mcvoy.com>
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
 <20170506195049.8FBEA124AEA4@mail.bitblocks.com>
 <CANCZdfrvwJwsUQjbBdudJ4Ro_uo_yeatVAhYxGZ+rkesUB56Dg@mail.gmail.com>
 <CAGfO01x_pDxSTtyKFgAu9FYAJBQDtqgdSRmkXAO4TO==73GmYw@mail.gmail.com>
Message-ID: <20170507135407.GS12539@yeono.kjorling.se>

On 7 May 2017 11:42 +1000, from noel.hunt at gmail.com (Noel Hunt):
> I don't imagine it would be hard to re-write [uniq] to
> handle utf-8.

It does look like at least GNU coreutils 8.13 uniq is broken in that
regard, which frankly surprised me. That version isn't _that_ old.

    $ uniq --version
    uniq (GNU coreutils) 8.13
    Copyright (C) 2011 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later
    <http://gnu.org/licenses/gpl.html>.
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.

    Written by Richard M. Stallman and David MacKenzie.
    $ ( echo $'\u1234' ; echo $'\u2345' ; echo $'\u1234' )
    ሴ
    ⍅
    ሴ
    $ ( echo $'\u1234' ; echo $'\u2345' ; echo $'\u1234' ) | uniq
    ሴ
    $

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From arnold at skeeve.com  Mon May  8 00:58:24 2017
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Sun, 07 May 2017 08:58:24 -0600
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <20170507135407.GS12539@yeono.kjorling.se>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se>
 <20170506144011.GF28787@mcvoy.com>
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
 <20170506195049.8FBEA124AEA4@mail.bitblocks.com>
 <CANCZdfrvwJwsUQjbBdudJ4Ro_uo_yeatVAhYxGZ+rkesUB56Dg@mail.gmail.com>
 <CAGfO01x_pDxSTtyKFgAu9FYAJBQDtqgdSRmkXAO4TO==73GmYw@mail.gmail.com>
 <20170507135407.GS12539@yeono.kjorling.se>
Message-ID: <201705071458.v47EwOtp014162@freefriends.org>

Michael Kjörling <michael at kjorling.se> wrote:

> On 7 May 2017 11:42 +1000, from noel.hunt at gmail.com (Noel Hunt):
> > I don't imagine it would be hard to re-write [uniq] to
> > handle utf-8.
>
> It does look like at least GNU coreutils 8.13 uniq is broken in that
> regard, which frankly surprised me. That version isn't _that_ old.

Are your LC_* env variables set correctly?

If you file a bug report, it should get fixed.

Thanks,

Arnold


From imp at bsdimp.com  Mon May  8 01:13:11 2017
From: imp at bsdimp.com (Warner Losh)
Date: Sun, 7 May 2017 09:13:11 -0600
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <CAGfO01x_pDxSTtyKFgAu9FYAJBQDtqgdSRmkXAO4TO==73GmYw@mail.gmail.com>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se> <20170506144011.GF28787@mcvoy.com>
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
 <20170506195049.8FBEA124AEA4@mail.bitblocks.com>
 <CANCZdfrvwJwsUQjbBdudJ4Ro_uo_yeatVAhYxGZ+rkesUB56Dg@mail.gmail.com>
 <CAGfO01x_pDxSTtyKFgAu9FYAJBQDtqgdSRmkXAO4TO==73GmYw@mail.gmail.com>
Message-ID: <CANCZdfonzdbH4dX_HKbtWpGaJtC=-DxZXD3VbFFtKxNNuEZHgA@mail.gmail.com>

On Sat, May 6, 2017 at 7:42 PM, Noel Hunt <noel.hunt at gmail.com> wrote:
> I was about to suggest using the Plan9 port utilities of the
> same name but it seems 'uniq' is not coded to handle Runes
> (aka utf-8). I don't imagine it would be hard to re-write it to
> handle utf-8.

I guess I should have been clearer on what wouldn't work. It can't
possibly work for Japanese and Chinese where words aren't separated by
whitespace. Would cause problems in hybrid languages where words can
be composed of logograms and sonograms (say Japanese which often use a
few Kanji with hiragana endings that then run into hiragana particles
or other grammar elements). Can't work without modification (using
class names) for Cyrillic because there's no A or Z in words there.
Won't work in any language that has a discontiguous set of letters,
which includes many western european languages since all the accented
or otherwise decorated letters aren't in the range A-Z.

So whether or not the underlying tools can handle UTF-8 encoding,
there are problems with the original.

If you used:

 tr -cs "[:alpha:]" '\n' | tr "[:upper:]" "[:lower:]" | sort | uniq -c
| sort -rn | sed ${1}q

you'd still have issues with languages that don't use word separators,
or write non-alphabetically.

Warner

> On Sun, May 7, 2017 at 11:15 AM, Warner Losh <imp at bsdimp.com> wrote:
>>
>> On Sat, May 6, 2017 at 1:50 PM, Bakul Shah <bakul at bitblocks.com> wrote:
>> > tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q
>>
>> The cool thing about this thread is that I learned two things: what tr
>> -s does, and the Nq does for sed...
>>
>> Sadly, this doesn't work so well for text that isn't ASCII-7 english...
>>
>> Warner
>
>


From michael at kjorling.se  Mon May  8 02:33:08 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sun, 7 May 2017 16:33:08 +0000
Subject: [TUHS] Discuss of style and design of computer programs from a
 user stand point
In-Reply-To: <201705071458.v47EwOtp014162@freefriends.org>
References: <201705060202.v4622L1J013430@coolidge.cs.Dartmouth.EDU>
 <5a2d6cc957c2efcd968f35aa5557c7a0e309dd27@webmail.yaccman.com>
 <20170506091857.GE12539@yeono.kjorling.se>
 <20170506144011.GF28787@mcvoy.com>
 <alpine.BSF.2.20.1705070430400.11767@aneurin.horsfall.org>
 <20170506195049.8FBEA124AEA4@mail.bitblocks.com>
 <CANCZdfrvwJwsUQjbBdudJ4Ro_uo_yeatVAhYxGZ+rkesUB56Dg@mail.gmail.com>
 <CAGfO01x_pDxSTtyKFgAu9FYAJBQDtqgdSRmkXAO4TO==73GmYw@mail.gmail.com>
 <20170507135407.GS12539@yeono.kjorling.se>
 <201705071458.v47EwOtp014162@freefriends.org>
Message-ID: <20170507163308.GT12539@yeono.kjorling.se>

On 7 May 2017 08:58 -0600, from arnold at skeeve.com:
>>> I don't imagine it would be hard to re-write [uniq] to handle utf-8.
>> 
>> It does look like at least GNU coreutils 8.13 uniq is broken in that
>> regard, which frankly surprised me. That version isn't _that_ old.
> 
> Are your LC_* env variables set correctly?

I believe so (I don't recall seeing any other UTF-8-related weirdness
for a very long time), but I would want to verify the behavior on a
system that doesn't have a gazillion customizations accumulated over
years _and_ has the most recent version of coreutils before I file an
actual bug report. (It's not like Debian is ever bleeding edge, to
begin with.) I'll see if I spin up a VM or three one of these days to
check.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From doug at cs.dartmouth.edu  Mon May  8 04:19:02 2017
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Sun, 07 May 2017 14:19:02 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user standpoint
Message-ID: <201705071819.v47IJ24H002013@coolidge.cs.Dartmouth.EDU>

> tr -cs A-Za-z '\n' |
> tr A-Z a-z |
> sort |
> uniq -c |
> sort -rn |
> sed ${1}q
>
> This is real genius.

Not genius. Experience. In the Bentley/Knuth/McIlroy paper I said,
"[Old] Unix hands know instinctively how to solve this one in a jiffy."
While that is certainly true, the script was informed by my having
written "spell", which itself was an elaboration of a model
pioneered by Steve Johnson. By 1986, when BKM was published,
the lore was baked in: word-processing scripts in a similar
vein were stock in trade.

A very early exercise of this sort was Dennis Ritchie's
enumeration of anagrams in the unabridged Merriam-Webster.
Since the word list barely fit on the tiny disk of the time,
the job entailed unimaginable marshalling of resources. I
was mightily impressed then, and still am.

Doug


From ches at cheswick.com  Mon May  8 04:50:52 2017
From: ches at cheswick.com (William Cheswick)
Date: Sun, 7 May 2017 14:50:52 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
 user standpoint
In-Reply-To: <201705071819.v47IJ24H002013@coolidge.cs.Dartmouth.EDU>
References: <201705071819.v47IJ24H002013@coolidge.cs.Dartmouth.EDU>
Message-ID: <5D559802-6D6E-4668-9CA5-141B071BC535@cheswick.com>


> On 7May 2017, at 2:19 PM, Doug McIlroy <doug at cs.dartmouth.edu> wrote:
> 
> Not genius. Experience.

It’s the pipe that was genius.  When I first heard the idea, I thought it was preposterous. My
programs’ outputs were designed for the line printer, with carriage control characters.  How
could this idea be useful?

These tools are powerful--I co-founded a company out of network probe software mostly designed
around filters.   And it scales beautifully to multi-core computing.  I had movie visualization software
suddenly nearly double in speed one day.  They had doubled the number of CPUs in the ferric cluster.

My favorite filter:  the unsort (or scramble) command.  
Honeyman’s “idiom” command:
sort | uniq -c | sort -rn



From lm at mcvoy.com  Mon May  8 08:03:47 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Sun, 7 May 2017 15:03:47 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
 user standpoint
In-Reply-To: <5D559802-6D6E-4668-9CA5-141B071BC535@cheswick.com>
References: <201705071819.v47IJ24H002013@coolidge.cs.Dartmouth.EDU>
 <5D559802-6D6E-4668-9CA5-141B071BC535@cheswick.com>
Message-ID: <20170507220347.GQ9278@mcvoy.com>

On Sun, May 07, 2017 at 02:50:52PM -0400, William Cheswick wrote:
> My favorite filter:  the unsort (or scramble) command.  
> sort | uniq -c | sort -rn

I use that one all the time.  Super handy.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 


From dot at dotat.at  Mon May  8 23:39:36 2017
From: dot at dotat.at (Tony Finch)
Date: Mon, 8 May 2017 14:39:36 +0100
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <CAJfiPzyVPzyYN4--5RuVt+nAcw4ppm3_ejdZmWC_oGv9gxjDYw@mail.gmail.com>
References: <CAJfiPzyVPzyYN4--5RuVt+nAcw4ppm3_ejdZmWC_oGv9gxjDYw@mail.gmail.com>
Message-ID: <alpine.DEB.2.11.1705081424500.24591@grey.csi.cam.ac.uk>

Nemo <cym224 at gmail.com> wrote:
> On 6 May 2017 at 11:23, ron minnich <rminnich at gmail.com> wrote (in part):
> [...]
> > Lest you think things are better now, Linux uses self modifying code to
> > optimize certain critical operations, and at one talk I heard the speaker
> > say that he'd like to put more self modifying code into Linux, "because it's
> > fun". Oh boy.
>
> Fun, indeed!  Even self-modifying chips are being touted -- Yikes!

You reminded me of these comments on a bug in NVidia's Tegra
"Project Denver" dynamic JIT firmware:

https://twitter.com/FioraAeterna/status/855445075341398017
>
> small brain: bug in your code
> big brain:  bug in the compiler
> cosmic brain: bug in the cpu's on-chip recompiler
> https://github.com/golang/go/issues/19809#issuecomment-290804472

https://twitter.com/eqe/status/855533948931252224
>
> This happened with TransMeta back in the day, and now with Tegra. I
> wonder if NVidia has a update deployment strategy...

(Marginally topical relevance is that Linus Torvalds worked for Transmeta)

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/  -  I xn--zr8h punycode
Lundy, Fastnet, Irish Sea, Shannon, Rockall, Malin, South Hebrides: Easterly
or northeasterly 4 or 5, occasionally 6 at first, becoming variable 3 at times
later. Slight or moderate. Fair. Good.


From scj at yaccman.com  Tue May  9 02:21:47 2017
From: scj at yaccman.com (Steve Johnson)
Date: Mon, 08 May 2017 09:21:47 -0700
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <alpine.DEB.2.11.1705081424500.24591@grey.csi.cam.ac.uk>
Message-ID: <0aab8f20d7cbcc18f830870cb361db844c229d28@webmail.yaccman.com>

That's a very interesting bug, and Fiora is a genius...

I too worked at Transmeta, although our recompiler was not on chip...

We designed the chip originally based on Spec benchmarks.  This
turned out to be a mistake because Windows doesn't look a bit like
Spec.

For one thing, when booting Windows, 50% of all the instructions were
executed once only.  When we found all of these instructions and
JITted their
basic blocks, it ran like a turtle.  The solution was to put in an
X86 interpreter, and only JIT when the instruction was executed 5
times.  The resulting performance
was more than acceptable.  But there was a problem demonstrating this
with a number of benchmarks, namely those that

	* Ran a small problem and timed it.
	* Based on that, made a bigger problem that was scaled by #1 -- the
slower the small problem, the smaller the big problem.
	* Added 1 and 2 to get the score.

	The small problem often ran mostly in the interpreter.  So the "big"
problem was really pretty small.  That ran fast, but overall, #1
dominated.

	If we actually ran the big problem from the getgo, we looked pretty
decent.

	Among the interesting facts we learned:  Windows executed a billion
instructions between detecting a reason to crash and putting up the
blue screen...

	We also ran into problems with self modifying code.  To their
credit, Microsoft quickly fixed the things we pointed out (most in
3rd-party drivers).

	Transmeta actually had some of the most interesting technology I've
ever worked with as a compiler writer, including a checkpoint/restart
in hardware that let the JIT do out-of-order execution and then do it
over serially if a fault appeared.
Steve

----- Original Message -----
From: "Tony Finch" <dot at dotat.at>
To:"Nemo" <cym224 at gmail.com>
Cc:"TUHS main list" <tuhs at minnie.tuhs.org>
Sent:Mon, 8 May 2017 14:39:36 +0100
Subject:Re: [TUHS] Discuss of style and design of computer programs
from a

 Nemo <cym224 at gmail.com> wrote:
 > On 6 May 2017 at 11:23, ron minnich <rminnich at gmail.com> wrote (in
part):
 > [...]
 > > Lest you think things are better now, Linux uses self modifying
code to
 > > optimize certain critical operations, and at one talk I heard the
speaker
 > > say that he'd like to put more self modifying code into Linux,
"because it's
 > > fun". Oh boy.
 >
 > Fun, indeed! Even self-modifying chips are being touted -- Yikes!

 You reminded me of these comments on a bug in NVidia's Tegra
 "Project Denver" dynamic JIT firmware:

 https://twitter.com/FioraAeterna/status/855445075341398017
 >
 > small brain: bug in your code
 > big brain: bug in the compiler
 > cosmic brain: bug in the cpu's on-chip recompiler
 > https://github.com/golang/go/issues/19809#issuecomment-290804472

 https://twitter.com/eqe/status/855533948931252224
 >
 > This happened with TransMeta back in the day, and now with Tegra. I
 > wonder if NVidia has a update deployment strategy...

 (Marginally topical relevance is that Linus Torvalds worked for
Transmeta)

 Tony.
 -- 
 f.anthony.n.finch <dot at dotat.at> http://dotat.at/ - I xn--zr8h
punycode
 Lundy, Fastnet, Irish Sea, Shannon, Rockall, Malin, South Hebrides:
Easterly
 or northeasterly 4 or 5, occasionally 6 at first, becoming variable 3
at times
 later. Slight or moderate. Fair. Good.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170508/7d5fd7ae/attachment.html>

From crossd at gmail.com  Tue May  9 03:01:33 2017
From: crossd at gmail.com (Dan Cross)
Date: Mon, 8 May 2017 13:01:33 -0400
Subject: [TUHS] Discuss of style and design of computer programs from a
In-Reply-To: <0aab8f20d7cbcc18f830870cb361db844c229d28@webmail.yaccman.com>
References: <alpine.DEB.2.11.1705081424500.24591@grey.csi.cam.ac.uk>
 <0aab8f20d7cbcc18f830870cb361db844c229d28@webmail.yaccman.com>
Message-ID: <CAEoi9W7c8d-JbiBF_S8SgiUXqLMCbPY2=94Sm_M-dKz-WQf_zQ@mail.gmail.com>

On Mon, May 8, 2017 at 12:21 PM, Steve Johnson <scj at yaccman.com> wrote:

> That's a very interesting bug, and Fiora is a genius...
>

That bug actually came from Dave Chase at Google. He mentioned it to me in
passing when he found it, "Hey, want to hear about this crazy hardware bug
we found?" I remember just sort of looking at him with this, "you've got to
be kidding me..." expression.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170508/99f6fd7d/attachment.html>

From dds at aueb.gr  Thu May 11 00:08:15 2017
From: dds at aueb.gr (Diomidis Spinellis)
Date: Wed, 10 May 2017 17:08:15 +0300
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <583620cb-6856-b9ee-a5ee-32885c09362b@aueb.gr>

I've made available on GitHub a series of tables showing the evolution 
of Unix facilities (as documented in the man pages) over the system's 
lifetime [1] and two diagrams where I attempted to draw the 
corresponding architecture [2].  I've also documented the process in a 
short blog post [3].  I'd welcome any suggestions for corrections and 
improvements you may have, particularly for the architecture diagrams.

[1] https://dspinellis.github.io/unix-history-man/
[2] https://dspinellis.github.io/unix-architecture/
[3] https://www.spinellis.gr/blog/20170510/

Cheers,

Diomidis



From steffen at sdaoden.eu  Thu May 11 00:38:13 2017
From: steffen at sdaoden.eu (Steffen Nurpmeso)
Date: Wed, 10 May 2017 16:38:13 +0200
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <583620cb-6856-b9ee-a5ee-32885c09362b@aueb.gr>
References: <583620cb-6856-b9ee-a5ee-32885c09362b@aueb.gr>
Message-ID: <20170510143813.YrQ5R%steffen@sdaoden.eu>

Yes, hi,

Diomidis Spinellis <dds at aueb.gr> wrote:
 |I've made available on GitHub a series of tables showing the evolution 
 |of Unix facilities (as documented in the man pages) over the system's 
 |lifetime [1] and two diagrams where I attempted to draw the 
 |corresponding architecture [2].  I've also documented the process in a 
 |short blog post [3].  I'd welcome any suggestions for corrections and 
 |improvements you may have, particularly for the architecture diagrams.
 |
 |[1] https://dspinellis.github.io/unix-history-man/

i am confident groff was part of (Free)BSD even before it was
released.  That is, i have used groff with FreeBSD 4.9 (?) base
systems onwards, and i know from the CSRG history that they worked
to get there.  (It is not part of the repo, though, it is only
deduced.  I still don't have the McKusick CD's, shame on me.)

 |[2] https://dspinellis.github.io/unix-architecture/
 |[3] https://www.spinellis.gr/blog/20170510/

--steffen
|
|Ralph says i must not use signatures which spread the light!


From erik at ono-sendai.com  Thu May 11 09:09:09 2017
From: erik at ono-sendai.com (Erik Berls)
Date: Wed, 10 May 2017 16:09:09 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170510143813.YrQ5R%steffen@sdaoden.eu>
References: <583620cb-6856-b9ee-a5ee-32885c09362b@aueb.gr>
 <20170510143813.YrQ5R%steffen@sdaoden.eu>
Message-ID: <CAEFkZw-ygO1mD22_cpCbxN7Ms_QP3znoukkgP+JMb-C2xv1oLw@mail.gmail.com>

groff came into NetBSD (of which, FreeBSD 1.0 incorporated a
pre-release NetBSD 0.8 tarball), with the 386bsd + patchkit initial
import.  Sun Mar 21 09:45:37 1993 UTC, by cgd.  It was upgraded to
groff release 1.08 about 4 months later, by jtc.

Source: NetBSD’s cvsweb


-----Original Message-----
From: Steffen Nurpmeso <steffen at sdaoden.eu>
Reply: tuhs at minnie.tuhs.org <tuhs at minnie.tuhs.org>
Date: May 10, 2017 at 2:16:16 PM
To: Diomidis Spinellis <dds at aueb.gr>
Cc: tuhs at minnie.tuhs.org <tuhs at minnie.tuhs.org>
Subject:  Re: [TUHS] The evolution of Unix facilities and architecture

> Yes, hi,
>
> Diomidis Spinellis wrote:
> |I've made available on GitHub a series of tables showing the evolution
> |of Unix facilities (as documented in the man pages) over the system's
> |lifetime [1] and two diagrams where I attempted to draw the
> |corresponding architecture [2]. I've also documented the process in a
> |short blog post [3]. I'd welcome any suggestions for corrections and
> |improvements you may have, particularly for the architecture diagrams.
> |
> |[1] https://dspinellis.github.io/unix-history-man/
>
> i am confident groff was part of (Free)BSD even before it was
> released. That is, i have used groff with FreeBSD 4.9 (?) base
> systems onwards, and i know from the CSRG history that they worked
> to get there. (It is not part of the repo, though, it is only
> deduced. I still don't have the McKusick CD's, shame on me.)
>
> |[2] https://dspinellis.github.io/unix-architecture/
> |[3] https://www.spinellis.gr/blog/20170510/
>
> --steffen
> |
> |Ralph says i must not use signatures which spread the light!
>

--
Erik Berls


From clemc at ccc.com  Thu May 11 10:49:37 2017
From: clemc at ccc.com (Clem Cole)
Date: Wed, 10 May 2017 20:49:37 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <583620cb-6856-b9ee-a5ee-32885c09362b@aueb.gr>
References: <583620cb-6856-b9ee-a5ee-32885c09362b@aueb.gr>
Message-ID: <CAC20D2P_KBeDQO-q9dk84zP8CXC10C5S4fdLaxH0iLZaoBN+Uw@mail.gmail.com>

On Wed, May 10, 2017 at 10:08 AM, Diomidis Spinellis <dds at aueb.gr> wrote:

> I've made available on GitHub a series of tables showing the evolution of
> Unix facilities (as documented in the man pages) over the system's lifetime
> [1] and two diagrams where I attempted to draw the corresponding
> architecture [2].  I've also documented the process in a short blog post
> [3].  I'd welcome any suggestions for corrections and improvements you may
> have, particularly for the architecture diagrams.
>
> [1] https://dspinellis.github.io/unix-history-man/

​fsck(8) is not a BSD program.   It is a CMU program originally, although
Michigan can sort of claim it also. Ted Kowalski is (was) the primary
author of fsck.  He started writing an earlier idea for it at Michigan,
which came from something like it he had seen on MTS, I believe called
"Scavenger" and IIRC we had also had TSS called 'Vulture' - which cleaned
up the 'carrion' after a disk crash (MTS and TSS are brothers for the IBM
360/67 and a number of early UNIX hacker also cut their teeth).   Anyway,
fsck, scavenger and vulture were all of the same idea.   The primary work
was done on the CMU 11/34 EE Digital lab system, in first floor of
Hammershag Hall in the mid-1970s.   Note, I had a >>very<< small hand in
fsck, as Ted was teaching me about C at the time (you can blame me for the
upper case error messages - that how MTS and TSS worked in those days).

If you look at edits and style it's clearly Ted's code.  BTW:  Ted took the
sources to fsck back to the labs when he finished it, and the program was
first released via the Summit streams but I can not tell which one [I think
PWB 2.0 was the first 'official' AT&T version - aps his old office mate at
Summit, might know].
I believe one of the AT&T features was support for RP06, which took
'swapping' and the temporary paging file stuff on the PDP-11.   That was
not need at CMU because we did not have disk that large on our PDP-11s.

Note it was was originally written for the for the 6th edition FS (which I
hope I have still have the sources in my files) as well as later for the
7th edition (or as Ted would called it UNIX/TS - which we had had a CMU
shortly after he had upgraded us - we ran a hybrid system until the 1979
when V7 was finally released).   I believe Noel recovered a copy in his
files recently.     As I have said previously, fsck migrated to a number of
sites independently - via mag tape most likely; Noel has pointed out that
MIT had it in the late 1970s also, probably brought back from BTL by one of
their summer students.   That said, it went more main stream via the BSD
4.x tapes when Joy passed it on, but all of that pre-dates BSD 4.x.

How UCB (Joy) got it is unknown, although I have also pointed out that Ted
was Joy's housemate at Michigan when they were undergrads, and Ted quite
likely sent him a tape or someone like Ken or any number of other BTL folks
could have brought it with him when they were there.​

​Clem​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170510/c8944aee/attachment.html>

From steffen at sdaoden.eu  Thu May 11 22:40:02 2017
From: steffen at sdaoden.eu (Steffen Nurpmeso)
Date: Thu, 11 May 2017 14:40:02 +0200
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAEFkZw-ygO1mD22_cpCbxN7Ms_QP3znoukkgP+JMb-C2xv1oLw@mail.gmail.com>
References: <583620cb-6856-b9ee-a5ee-32885c09362b@aueb.gr>
 <20170510143813.YrQ5R%steffen@sdaoden.eu>
 <CAEFkZw-ygO1mD22_cpCbxN7Ms_QP3znoukkgP+JMb-C2xv1oLw@mail.gmail.com>
Message-ID: <20170511124002.kJmQX%steffen@sdaoden.eu>

Erik Berls <erik at ono-sendai.com> wrote:
 |groff came into NetBSD (of which, FreeBSD 1.0 incorporated a
 |pre-release NetBSD 0.8 tarball), with the 386bsd + patchkit initial
 |import.  Sun Mar 21 09:45:37 1993 UTC, by cgd.  It was upgraded to
 |groff release 1.08 about 4 months later, by jtc.
 |
 |Source: NetBSD’s cvsweb

So come, ey, Mr. Spinellis has a devilish red focus on FreeBSD.
No other BSD in sight.  For the git clone of FreeBSD that i have
i see groff-1.09 import as a series of committs starting with
[b4b083cfbe2] around 1995-01-17.  But i better should not have
looked, back then perl was still part of the base system, and
today Lua is missing, too.

P.S.: depite the fact that my VServer (still) runs Alpine happily.
At least there is awk, i currently have a mail-server rotation
period of ~12 hours because of otherwise valid nonsense
connections that blacklistd is not covering, and isn't it absurd
that i need to parse log files to recollect state that the server
has readily available.  But why complain and not coding something
better, yes.
Ciao.

--steffen
|
|Ralph says i must not use signatures which spread the light!

From jnc at mercury.lcs.mit.edu  Fri May 12 00:07:29 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Thu, 11 May 2017 10:07:29 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>

    > From: Clem Cole

    > it was was originally written for the for the 6th edition FS (which I
    > hope I have still have the sources in my files) ...
    > I believe Noel recovered a copy in his files recently.

Well, I have _something_. It's called 'fcheck', not 'fsck', but it looks like
what we're talking about - maybe it was originally named, or renamed, to be in
the same series as {d,i,n}check? But it does have the upper-case error
messages... :-) Anyway, here it is:

  http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/fcheck.c
  http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/fcheck.8

Interestingly, the man page for it makes reference to a 'check' command, which
I didn't recall at all; here it is:

  http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/check.c
  http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/check.8

for those who are interested.


    > Noel has pointed out that MIT had it in the late 1970s also, probably
    > brought back from BTL by one of their summer students.

I think most of the Unix stuff we got from Bell (e.g. the OS, which is clearly
PWB1, not V6) came from someone who was in a Scout unit there in high school,
of all bizarre connections! ISTR this came the same way, but maybe I'm wrong.
It definitely arrived later than the OS - we'd be using icheck/dcheck for
quite a while before it arrived - so maybe it was another channel?

The only thing that for sure (that I recall) that didn't come this way was
Emacs. Since the author had been a grad student in our group at MIT, I think
you all can guess how we got that!

	Noel



From lm at mcvoy.com  Fri May 12 00:21:42 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Thu, 11 May 2017 07:21:42 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
Message-ID: <20170511142142.GA4341@mcvoy.com>

On Thu, May 11, 2017 at 10:07:29AM -0400, Noel Chiappa wrote:
>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/fcheck.c

Is this style of declarations common?

char
        *bbit, 
        *abbit,
        *state, 
        *lc, 
        pathname[200], 
        *pp, 
        *name, 
        sflag, 
        nflag, 
        yflag, 
;

unsigned
        dsize,
        fmin,
        fmax
;

I've not seen that before, if it's fairly unique then we might be able to
figure out who wrote this stuff (or did I miss that and we know already?)

--lm



From clemc at ccc.com  Fri May 12 02:15:09 2017
From: clemc at ccc.com (Clem Cole)
Date: Thu, 11 May 2017 12:15:09 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
Message-ID: <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>

On Thu, May 11, 2017 at 10:07 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

>     > From: Clem Cole
>
>     > it was was originally written for the for the 6th edition FS (which I
>     > hope I have still have the sources in my files) ...
>     > I believe Noel recovered a copy in his files recently.
>
> Well, I have _something_. It's called 'fcheck', not 'fsck', but it looks
> like
> what we're talking about - maybe it was originally named, or renamed, to
> be in
> the same series as {d,i,n}check? But it does have the upper-case error
> messages... :-) Anyway, here it is:
>
>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/fcheck.c
>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/fcheck.8


fcheck ---> aka fsick  -- aka fsck -- that's it.


>
>
> Interestingly, the man page for it makes reference to a 'check' command,
> which
> I didn't recall at all; here it is:
>
>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/check.c
>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/check.8
>
> for those who are interested.
>
>
>     > Noel has pointed out that MIT had it in the late 1970s also, probably
>     > brought back from BTL by one of their summer students.
>
> I think most of the Unix stuff we got from Bell (e.g. the OS, which is
> clearly
> PWB1, not V6) came from someone who was in a Scout unit there in high
> school,
>
Jon Stienhart maybe???   He & Paul Rubinfield were in that scout group
years ago and were both long time UNIX hackers, but I've forgotten where
Stienhart did his undergrad.

of all bizarre connections! ISTR this came the same way, but maybe I'm
> wrong.
> It definitely arrived later than the OS - we'd be using icheck/dcheck for
> quite a while before it arrived - so maybe it was another channel?
>
​ This is Ted's code and my error messages.


> The only thing that for sure (that I recall) that didn't come this way was
> Emacs. Since the author had been a grad student in our group at MIT, I
> think
> you all can guess how we got that!
>
>         Noel
>
> ​Clem​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170511/f56580e6/attachment.html>

From clemc at ccc.com  Fri May 12 02:17:20 2017
From: clemc at ccc.com (Clem Cole)
Date: Thu, 11 May 2017 12:17:20 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511142142.GA4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <20170511142142.GA4341@mcvoy.com>
Message-ID: <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>

Ted certainly did that a lot.
(It drove me nuts.   I hated it and argued a bit about it.)  One of the
reasons I hated C when I first learned it.
Clem

On Thu, May 11, 2017 at 10:21 AM, Larry McVoy <lm at mcvoy.com> wrote:

> On Thu, May 11, 2017 at 10:07:29AM -0400, Noel Chiappa wrote:
> >   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/fcheck.c
>
> Is this style of declarations common?
>
> char
>         *bbit,
>         *abbit,
>         *state,
>         *lc,
>         pathname[200],
>         *pp,
>         *name,
>         sflag,
>         nflag,
>         yflag,
> ;
>
> unsigned
>         dsize,
>         fmin,
>         fmax
> ;
>
> I've not seen that before, if it's fairly unique then we might be able to
> figure out who wrote this stuff (or did I miss that and we know already?)
>
> --lm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170511/6e7938cb/attachment.html>

From imp at bsdimp.com  Fri May 12 02:52:55 2017
From: imp at bsdimp.com (Warner Losh)
Date: Thu, 11 May 2017 10:52:55 -0600
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
Message-ID: <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>

On Thu, May 11, 2017 at 10:15 AM, Clem Cole <clemc at ccc.com> wrote:
>
>
>
>
>
>
> On Thu, May 11, 2017 at 10:07 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
> wrote:
>>
>>     > From: Clem Cole
>>
>>     > it was was originally written for the for the 6th edition FS (which
>> I
>>     > hope I have still have the sources in my files) ...
>>     > I believe Noel recovered a copy in his files recently.
>>
>> Well, I have _something_. It's called 'fcheck', not 'fsck', but it looks
>> like
>> what we're talking about - maybe it was originally named, or renamed, to
>> be in
>> the same series as {d,i,n}check? But it does have the upper-case error
>> messages... :-) Anyway, here it is:
>>
>>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/fcheck.c
>>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/fcheck.8
>
>
> fcheck ---> aka fsick  -- aka fsck -- that's it.

There's a dcheck.c in the TUHS v7 sources. How's that related?

Warner


>> Interestingly, the man page for it makes reference to a 'check' command,
>> which
>> I didn't recall at all; here it is:
>>
>>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/check.c
>>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/check.8
>>
>> for those who are interested.
>>
>>
>>     > Noel has pointed out that MIT had it in the late 1970s also,
>> probably
>>     > brought back from BTL by one of their summer students.
>>
>> I think most of the Unix stuff we got from Bell (e.g. the OS, which is
>> clearly
>> PWB1, not V6) came from someone who was in a Scout unit there in high
>> school,
>
> Jon Stienhart maybe???   He & Paul Rubinfield were in that scout group years
> ago and were both long time UNIX hackers, but I've forgotten where Stienhart
> did his undergrad.
>
>> of all bizarre connections! ISTR this came the same way, but maybe I'm
>> wrong.
>> It definitely arrived later than the OS - we'd be using icheck/dcheck for
>> quite a while before it arrived - so maybe it was another channel?
>
> This is Ted's code and my error messages.
>
>>
>> The only thing that for sure (that I recall) that didn't come this way was
>> Emacs. Since the author had been a grad student in our group at MIT, I
>> think
>> you all can guess how we got that!
>>
>>         Noel
>>
> Clem
>


From jnc at mercury.lcs.mit.edu  Fri May 12 03:08:03 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Thu, 11 May 2017 13:08:03 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170511170803.1F46518C09A@mercury.lcs.mit.edu>

    > From: Warner Losh

    > There's a dcheck.c in the TUHS v7 sources. How's that related?

That was one of the earlier tools - not sure how far back it goes, but it's in
V6, but not V5. It consistency checks the directory tree. Another tool, 'icheck',
consistency checks file blocks and the free list.

	Noel



From michael at kjorling.se  Fri May 12 03:11:00 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Thu, 11 May 2017 17:11:00 +0000
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <20170511142142.GA4341@mcvoy.com>
 <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>
Message-ID: <20170511171100.GA9980@yeono.kjorling.se>

On 11 May 2017 12:17 -0400, from clemc at ccc.com (Clem Cole):
> On Thu, May 11, 2017 at 10:21 AM, Larry McVoy <lm at mcvoy.com> wrote:
>> Is this style of declarations common?
>> 
>> char
>>         *bbit,
>>         *abbit,
>>         *state,
>>         *lc,
>>         pathname[200],
>> /.../
> 
> Ted certainly did that a lot.
> (It drove me nuts.   I hated it and argued a bit about it.)  One of the
> reasons I hated C when I first learned it.

On the flip side, it certainly does beat `char* x, y, z[100];` or
`FILE* fpsrc, fpdst;`. I wonder how many aspiring C programmers have
been tripped up by constructs like those? It's perfectly reasonable
_once you know about it_, but if you don't, then, well...

It's even more fun if your system doesn't have memory protection. No,
I'm not speaking from experience; what made you think that I did? ;-)

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From clemc at ccc.com  Fri May 12 03:12:20 2017
From: clemc at ccc.com (Clem Cole)
Date: Thu, 11 May 2017 13:12:20 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
Message-ID: <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>

On Thu, May 11, 2017 at 12:52 PM, Warner Losh <imp at bsdimp.com> wrote:

> On Thu, May 11, 2017 at 10:15 AM, Clem Cole <clemc at ccc.com> wrote:
> >
> >
> >
> >
> >
> >
> > On Thu, May 11, 2017 at 10:07 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
> > wrote:
> >>
> >>     > From: Clem Cole
> >>
> >>     > it was was originally written for the for the 6th edition FS
> (which
> >> I
> >>     > hope I have still have the sources in my files) ...
> >>     > I believe Noel recovered a copy in his files recently.
> >>
> >> Well, I have _something_. It's called 'fcheck', not 'fsck', but it looks
> >> like
> >> what we're talking about - maybe it was originally named, or renamed, to
> >> be in
> >> the same series as {d,i,n}check? But it does have the upper-case error
> >> messages... :-) Anyway, here it is:
> >>
> >>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/fcheck.c
> >>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/fcheck.8
> >
> >
> > fcheck ---> aka fsick  -- aka fsck -- that's it.
>
> There's a dcheck.c in the TUHS v7 sources. How's that related?
>
> ​Directory CHECK - was a pass down the upper level pathname structure of
the FS.
In fact it was the model for fsck.   Ted had me steer at it.  One of the
passes in pretty much pulled from that code directly.

The problem was that originally there were a couple of tools to put things
back together, but until Ted wrote fsck there was not one single tool that
pretty much did what you wanted and got it right most of the time.

Clem​





> Warner
>
>
> >> Interestingly, the man page for it makes reference to a 'check' command,
> >> which
> >> I didn't recall at all; here it is:
> >>
> >>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/s1/check.c
> >>   http://ana-3.lcs.mit.edu/~jnc/tech/unix/man8/check.8
> >>
> >> for those who are interested.
> >>
> >>
> >>     > Noel has pointed out that MIT had it in the late 1970s also,
> >> probably
> >>     > brought back from BTL by one of their summer students.
> >>
> >> I think most of the Unix stuff we got from Bell (e.g. the OS, which is
> >> clearly
> >> PWB1, not V6) came from someone who was in a Scout unit there in high
> >> school,
> >
> > Jon Stienhart maybe???   He & Paul Rubinfield were in that scout group
> years
> > ago and were both long time UNIX hackers, but I've forgotten where
> Stienhart
> > did his undergrad.
> >
> >> of all bizarre connections! ISTR this came the same way, but maybe I'm
> >> wrong.
> >> It definitely arrived later than the OS - we'd be using icheck/dcheck
> for
> >> quite a while before it arrived - so maybe it was another channel?
> >
> > This is Ted's code and my error messages.
> >
> >>
> >> The only thing that for sure (that I recall) that didn't come this way
> was
> >> Emacs. Since the author had been a grad student in our group at MIT, I
> >> think
> >> you all can guess how we got that!
> >>
> >>         Noel
> >>
> > Clem
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170511/a124548c/attachment.html>

From ron at ronnatalie.com  Fri May 12 06:37:29 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Thu, 11 May 2017 16:37:29 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
Message-ID: <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>

I remember the pre-fsck days.   It was part of my test to become an operator at the UNIX site at JHU that I could run the various manual checks.

The V6 file system wasn’t exactly stable during crashes (lousy database behavior), so there was almost certainly something to clean up.

 

The first thing we’d run was icheck.   This runs down the superblock freelist and all the allocated blocks in the inodes.     If there were missing blocks (not in a file or the free list), you could use icheck –s

to rebuild it.    Similarly, if you had duplicated allocations in the freelist or between the freelist and a single file.   Anything more complicated required some clever patching (typically, we’d just mount readonly, copy the files, and then blow them away with clri).

 

Then you’d run dcheck.   As mentioned dcheck walks the directory path from the top of the disk counting inode references that it reconciles with the link count in the inode.   Occasionally we’d end up with a 0-0 inode (no directory entires, but allocated…typically this is caused by people removing a file while it is still open, a regular practice of some programs for their /tmp files.).    clri again blew these away.

 

Clri wrote zeros all over the inode.   This had the effect of wiping out the file, but it was dangerous if you got the i-number wrong.    We replaced it with “clrm” which just cleared the allocated bit, a lot easy to reverse.

 

If you really had a mess of a file system, you might get a piece of the directory tree broken off from a path to the root.   Or you’d have an inode that icheck reported dups.   ncheck would try to reconcile an inumber into an absolute path.

 

After a while a program called fsdb came around that allowed you to poke at the various file system structures.    We didn’t use it much because by the time we had it, fsck was fast on its heals.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170511/929937aa/attachment.html>

From dave at horsfall.org  Fri May 12 07:34:12 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Fri, 12 May 2017 07:34:12 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511170803.1F46518C09A@mercury.lcs.mit.edu>
References: <20170511170803.1F46518C09A@mercury.lcs.mit.edu>
Message-ID: <alpine.BSF.2.20.1705120730390.67621@aneurin.horsfall.org>

On Thu, 11 May 2017, Noel Chiappa wrote:

> Another tool, 'icheck', consistency checks file blocks and the free 
> list.

Named so because it checked the inode list (for which the sledge-hammer 
fix was "clri").

Somewhere in the AUUGN archives is a paper I wrote detailing the proper 
use of "[ind]check" and "clri".

And then along came "fsdb", wherein you could edit the inode directly...

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From dave at horsfall.org  Fri May 12 07:44:32 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Fri, 12 May 2017 07:44:32 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511171100.GA9980@yeono.kjorling.se>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <20170511142142.GA4341@mcvoy.com>
 <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>
 <20170511171100.GA9980@yeono.kjorling.se>
Message-ID: <alpine.BSF.2.20.1705120736210.67621@aneurin.horsfall.org>

On Thu, 11 May 2017, Michael Kjörling wrote:

> On the flip side, it certainly does beat `char* x, y, z[100];` or `FILE* 
> fpsrc, fpdst;`. I wonder how many aspiring C programmers have been 
> tripped up by constructs like those? It's perfectly reasonable _once you 
> know about it_, but if you don't, then, well...

Am I the only one here who thinks that e.g. a char pointer should be 
"char* cp1, cp2" instead of "char *cp1, *cp2"?  I.e. the fundamental type 
is "char*", not "char", and to this day I still write:

    char*	cp1;
    char*	cp2;

etc, which IMHO makes it clear (which is every programmer's duty).  I used 
to write that way in a previous life, and the boss didn't complain.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

From ron at ronnatalie.com  Fri May 12 07:49:31 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Thu, 11 May 2017 17:49:31 -0400
Subject: [TUHS] C declarations.
Message-ID: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>

Bjarne agrees with you.   He put the * (and the &) with the type name to emphasize it is part of the type.
This works fine as long as you only use one declaration per statement.

The problem with that is that * doesn't really bind to the type name.   It binds to the variable.

char* cp1, cp2;   // cp1 is pointer to char,   cp2 is just a char.

I always found it confusing that the * is used to indicate an pointer here, where as when you want to change an lvalue to a pointer, you use &.

But if we're going to gripe about the evolution of C.   My biggest gripe is when they fixed structs to be real types, they didn't also do so for arrays.
Arrays and their  degeneration to poitners is one of the biggest annoyances in C.

> Am I the only one here who thinks that e.g. a char pointer should be
> "char* cp1, cp2" instead of "char *cp1, *cp2"?  I.e. the fundamental type is "char*", not "char", and to this day I still write:




From krewat at kilonet.net  Fri May 12 08:01:36 2017
From: krewat at kilonet.net (Arthur Krewat)
Date: Thu, 11 May 2017 18:01:36 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
Message-ID: <e84dcab4-4d45-5cc3-ca1b-34d786e9c87c@kilonet.net>


But wouldn't that imply you didn't need to use the * when referencing 
the actual value?

For example, if you do this:

char *p;
char c;

The correct way to get the character it's pointing to dereference like this:

c=*p;

If the type was char* (not char)

Wouldn't that imply I wouldn't need the * to dereference the pointer?




On 5/11/2017 5:49 PM, Ron Natalie wrote:
> Bjarne agrees with you.   He put the * (and the &) with the type name to emphasize it is part of the type.
> This works fine as long as you only use one declaration per statement.
>
> The problem with that is that * doesn't really bind to the type name.   It binds to the variable.
>
> char* cp1, cp2;   // cp1 is pointer to char,   cp2 is just a char.
>
> I always found it confusing that the * is used to indicate an pointer here, where as when you want to change an lvalue to a pointer, you use &.
>
> But if we're going to gripe about the evolution of C.   My biggest gripe is when they fixed structs to be real types, they didn't also do so for arrays.
> Arrays and their  degeneration to poitners is one of the biggest annoyances in C.
>
>> Am I the only one here who thinks that e.g. a char pointer should be
>> "char* cp1, cp2" instead of "char *cp1, *cp2"?  I.e. the fundamental type is "char*", not "char", and to this day I still write:
>
>
>



From imp at bsdimp.com  Fri May 12 08:06:45 2017
From: imp at bsdimp.com (Warner Losh)
Date: Thu, 11 May 2017 16:06:45 -0600
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705120736210.67621@aneurin.horsfall.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <20170511142142.GA4341@mcvoy.com>
 <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>
 <20170511171100.GA9980@yeono.kjorling.se>
 <alpine.BSF.2.20.1705120736210.67621@aneurin.horsfall.org>
Message-ID: <CANCZdfp2JzXeu7b_2t0WCptMdEfnXvQzgMyYOao6udmojtOKPg@mail.gmail.com>

On Thu, May 11, 2017 at 3:44 PM, Dave Horsfall <dave at horsfall.org> wrote:
> On Thu, 11 May 2017, Michael Kjörling wrote:
>
>> On the flip side, it certainly does beat `char* x, y, z[100];` or `FILE*
>> fpsrc, fpdst;`. I wonder how many aspiring C programmers have been
>> tripped up by constructs like those? It's perfectly reasonable _once you
>> know about it_, but if you don't, then, well...
>
> Am I the only one here who thinks that e.g. a char pointer should be
> "char* cp1, cp2" instead of "char *cp1, *cp2"?  I.e. the fundamental type
> is "char*", not "char", and to this day I still write:
>
>     char*       cp1;
>     char*       cp2;
>
> etc, which IMHO makes it clear (which is every programmer's duty).  I used
> to write that way in a previous life, and the boss didn't complain.

I've encountered several people with that world view, so you aren't
alone. I take a contrary view. Since C doesn't behave that way, it
encourages people to think that char* cp1, cp2 is equivalent to what
you wrote, which it's not. * is a modifier of char rather than char *
being a fundamental type. Been burned too many times by it I guess
over the years.

Warner


From davida at pobox.com  Fri May 12 08:03:28 2017
From: davida at pobox.com (David Arnold)
Date: Fri, 12 May 2017 08:03:28 +1000
Subject: [TUHS] C declarations.
In-Reply-To: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
Message-ID: <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>

> On 12 May 2017, at 07:49, Ron Natalie <ron at ronnatalie.com> wrote:
> 
> <…>
> But if we're going to gripe about the evolution of C.   My biggest gripe is when they fixed structs to be real types, they didn't also do so for arrays.
> Arrays and their  degeneration to poitners is one of the biggest annoyances in C.

And, conversely (perversely?) one of its greatest joys.



d



From lm at mcvoy.com  Fri May 12 08:25:47 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Thu, 11 May 2017 15:25:47 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
Message-ID: <20170511222547.GJ4341@mcvoy.com>

This is one place where I think Linux kicked Unix's ass.  And I am not
really sure how they did it, I have an idea but am not positive.  Unix
file systems up through UFS as shipped by Sun, were all vulnerable to
what I call the power out test.  Untar some big tarball and power off
the machine in the middle of it.  Reboot.  Hilarity ensues (not).

You were dropped into some stand alone shell after fsck threw up its
hands and it was up to you to fix it.  Dozens and dozens of errors.
It was almost always faster to go to backups because figuring that 
stuff out, file by file (which I have done more than once), gets you
to the point that your run "fsck -y" and go poke at lost+found when
fsck is done, realize that there is no hope, and reach for backups.

Try the same thing with Linux.  The file system will come back, starting
with, I believe, ext2.

My belief is that Linux orders writes such that while you may lose data
(as in, a process created a file, the OS said it was OK, but that file 
will not be in the file system after a crash), but the rest of the file 
system will be consistent.  I think it's as if you powered off the
machine a few seconds earlier than you actually did, some stuff is in
flight and until they can write stuff out in the proper order you may
lose data on a hard reset.

But it doesn't leave your file system in a mess and it's not the brute
force slow way that DOS does it.

I copied Ted, who had his fingers deep in that code, maybe he can correct
me where I got it wrong.  Details aside, I think this is a place where
Linux moved the state of the art significantly forward.  There are other
places but this one is a big deal IMHO, maybe the biggest deal.

--lm

On Thu, May 11, 2017 at 04:37:29PM -0400, Ron Natalie wrote:
> I remember the pre-fsck days.   It was part of my test to become an operator at the UNIX site at JHU that I could run the various manual checks.
> 
> The V6 file system wasn???t exactly stable during crashes (lousy database behavior), so there was almost certainly something to clean up.
> 
>  
> 
> The first thing we???d run was icheck.   This runs down the superblock freelist and all the allocated blocks in the inodes.     If there were missing blocks (not in a file or the free list), you could use icheck ???s
> 
> to rebuild it.    Similarly, if you had duplicated allocations in the freelist or between the freelist and a single file.   Anything more complicated required some clever patching (typically, we???d just mount readonly, copy the files, and then blow them away with clri).
> 
>  
> 
> Then you???d run dcheck.   As mentioned dcheck walks the directory path from the top of the disk counting inode references that it reconciles with the link count in the inode.   Occasionally we???d end up with a 0-0 inode (no directory entires, but allocated???typically this is caused by people removing a file while it is still open, a regular practice of some programs for their /tmp files.).    clri again blew these away.
> 
>  
> 
> Clri wrote zeros all over the inode.   This had the effect of wiping out the file, but it was dangerous if you got the i-number wrong.    We replaced it with ???clrm??? which just cleared the allocated bit, a lot easy to reverse.
> 
>  
> 
> If you really had a mess of a file system, you might get a piece of the directory tree broken off from a path to the root.   Or you???d have an inode that icheck reported dups.   ncheck would try to reconcile an inumber into an absolute path.
> 
>  
> 
> After a while a program called fsdb came around that allowed you to poke at the various file system structures.    We didn???t use it much because by the time we had it, fsck was fast on its heals.
> 

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 


From ron at ronnatalie.com  Fri May 12 08:30:07 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Thu, 11 May 2017 18:30:07 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511222547.GJ4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
Message-ID: <015901d2caa6$2594b410$70be1c30$@ronnatalie.com>

I got a phone call one day from one of the operators that "fsck was stuck in
a loop."    I thought maybe it was wedged or it was just printing out
continuously.
I got there and find that Fsck had repaired the root partition and rebooted
the machine and then come up and decided the root needed fixing again and
then rebooted...
The operator didn't mention that the "fsck loop" involved a reboot every
time through.

The other one I got was that "fsck has been printing for an hour."   After
getting them to read me one of the errors I suggested they scroll back the
printout and find the first message.
RP01 OFFLINE.

Now, go over to the disk drive and look in the glass top and tell me...is it
spinning?   No?

Now put the run/stop switch to stop.   Now put it to Run.     Now reboot the
machine and DON'T LEAN ON THE FRONT OF THE DRIVE AGAIN.
(This one RP06 if you leaned on the lid it would spin down the disk).
 



From lm at mcvoy.com  Fri May 12 08:32:32 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Thu, 11 May 2017 15:32:32 -0700
Subject: [TUHS] C declarations.
In-Reply-To: <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
Message-ID: <20170511223232.GM4341@mcvoy.com>

On Fri, May 12, 2017 at 08:03:28AM +1000, David Arnold wrote:
> > On 12 May 2017, at 07:49, Ron Natalie <ron at ronnatalie.com> wrote:
> > 
> > <???>
> > But if we're going to gripe about the evolution of C.   My biggest gripe is when they fixed structs to be real types, they didn't also do so for arrays.
> > Arrays and their  degeneration to poitners is one of the biggest annoyances in C.
> 
> And, conversely (perversely?) one of its greatest joys.

I dunno if it is one of its greatest joys but pointers in C have always 
made sense to me.

I'm curious as to what is busted about arrays in C?  To me they just
seemed like a way to define how to look at a wad of memory and they
seem to work for me.  About the only thing I don't like about them is
that there is no late binding as to the size, Ada has late binding and
I thought it could be useful (I only know because Rob Netzer and I 
wrote an Ada compiler for CS736 at UW-Madison that did a lot of Ada
but exceptions and late binding we did not do).

--lm


From ron at ronnatalie.com  Fri May 12 08:41:47 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Thu, 11 May 2017 18:41:47 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <20170511223232.GM4341@mcvoy.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
Message-ID: <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>


>  I'm curious as to what is busted about arrays in C?  To me they just
seemed like a way to define how to look at a wad of memory and they seem to
work for me.  

The problem is they don't work like every other type.    Somewhere around
the phototypesetter release or V7, they fixed structs to be
assignable/passable to functions.
They didn't do that for arrays.   They retained their half-assed
quasipointer status.    It would have been easy to bite the bullet back in
those days before people codified the inane behavior.

For example:

struct foo {
    int a;
    int b;
};

struct foo x;
x.a = 1;
x.b = 1;
struct foo y;
y = x; // works fine.

struct foo myfunc(struct foo arg) {
    arg.a = 5;
    return 5;
}
y = myfunc(x);   // does not modify x


Now look at arrays:

char x[4] = { 1, 2, 3, 4};

char y[4];
y = x ; // illegal

void myfunc(char arg[4]) {
    arg[0] = 1000;
}

myfunc(x);  // CHANGES X!

If had my way, y = x and passing and returning arrays by value would work
just like every other C type.




From dave at horsfall.org  Fri May 12 09:44:40 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Fri, 12 May 2017 09:44:40 +1000 (EST)
Subject: [TUHS] C declarations.
In-Reply-To: <e84dcab4-4d45-5cc3-ca1b-34d786e9c87c@kilonet.net>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <e84dcab4-4d45-5cc3-ca1b-34d786e9c87c@kilonet.net>
Message-ID: <alpine.BSF.2.20.1705120938230.67621@aneurin.horsfall.org>

On Thu, 11 May 2017, Arthur Krewat wrote:

> But wouldn't that imply you didn't need to use the * when referencing 
> the actual value?
> 
> For example, if you do this:
> 
> char *p;
> char c;
> 
> The correct way to get the character it's pointing to dereference like 
> this:
> 
> c=*p;
> 
> If the type was char* (not char)
> 
> Wouldn't that imply I wouldn't need the * to dereference the pointer?

No, because you'd be assigning the contents i.e. the address (or however 
references are implemented on a given architecture) to a character; 
whether the fundamental type is "char*" or ("mostly char but it's really a 
pointer") is the same.

Then again, I learned ALGOLW (and PASCAL) before I learned C, and it has a 
"REFERENCE" type...

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From dave at horsfall.org  Fri May 12 09:47:01 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Fri, 12 May 2017 09:47:01 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511222547.GJ4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
Message-ID: <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>

On Thu, 11 May 2017, Larry McVoy wrote:

[...]

> Try the same thing with Linux.  The file system will come back, starting 
> with, I believe, ext2.

That's a journalled FS, isn't it?  In which case the transactions get
replayed.

> My belief is that Linux orders writes such that while you may lose data 
> (as in, a process created a file, the OS said it was OK, but that file 
> will not be in the file system after a crash), but the rest of the file 
> system will be consistent.  I think it's as if you powered off the 
> machine a few seconds earlier than you actually did, some stuff is in 
> flight and until they can write stuff out in the proper order you may 
> lose data on a hard reset.

And FreeBSD (at least) has been doing ordered writes for quite some time.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From ron at ronnatalie.com  Fri May 12 09:48:27 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Thu, 11 May 2017 19:48:27 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
Message-ID: <017001d2cab1$16bcf4b0$4436de10$@ronnatalie.com>

Ordered writes go back to the original BSD fast file system, no?   I  seem
to recall that when we switched from our V6/V7 disks,
the filesystem got a lot more stable in crashes.

-----Original Message-----
From: TUHS [mailto:tuhs-bounces at minnie.tuhs.org] On Behalf Of Dave Horsfall
Sent: Thursday, May 11, 2017 7:47 PM
To: The Eunuchs Hysterical Society
Subject: Re: [TUHS] The evolution of Unix facilities and architecture

On Thu, 11 May 2017, Larry McVoy wrote:

[...]

> Try the same thing with Linux.  The file system will come back, 
> starting with, I believe, ext2.

That's a journalled FS, isn't it?  In which case the transactions get
replayed.

> My belief is that Linux orders writes such that while you may lose 
> data (as in, a process created a file, the OS said it was OK, but that 
> file will not be in the file system after a crash), but the rest of 
> the file system will be consistent.  I think it's as if you powered 
> off the machine a few seconds earlier than you actually did, some 
> stuff is in flight and until they can write stuff out in the proper 
> order you may lose data on a hard reset.

And FreeBSD (at least) has been doing ordered writes for quite some time.

--
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will
suffer."



From bakul at bitblocks.com  Fri May 12 10:15:20 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Thu, 11 May 2017 17:15:20 -0700
Subject: [TUHS] C declarations.
In-Reply-To: Your message of "Thu, 11 May 2017 15:32:32 PDT."
 <20170511223232.GM4341@mcvoy.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
Message-ID: <20170512001520.38000124AEA4@mail.bitblocks.com>

On Thu, 11 May 2017 15:32:32 PDT Larry McVoy <lm at mcvoy.com> wrote:
> 
> I dunno if it is one of its greatest joys but pointers in C have always 
> made sense to me.
> 
> I'm curious as to what is busted about arrays in C?  To me they just
> seemed like a way to define how to look at a wad of memory and they
> seem to work for me.  About the only thing I don't like about them is
> that there is no late binding as to the size, Ada has late binding and
> I thought it could be useful (I only know because Rob Netzer and I 
> wrote an Ada compiler for CS736 at UW-Madison that did a lot of Ada
> but exceptions and late binding we did not do).

Coming from a Pascal background I really liked the terseness
of C. But theere were three things that bothered me.

a) K&R style argument declarations. This got fixed in ANSI C.
   C declarations were messy but that did not bother me much.

b) No nested procedures. Gnu C had added them but the
   implementation was a bit screwy and most people didn't care
   in any case so there was no hope of this getting fixed.
   Partly because most people used it as a portable assembly
   language!

c) arrays were not first class objects.  Given "T v[N];" v[i]
   is of type T and i+v or v+i is of type T* -- this is
   perfectly well defined and fine. What is not fine is that
   an array is a second class object. Thus you can not do
   for example,

    int v[5];
    struct foo {
	int w[5];
	int x;
    } y;
	...
	y.w = v;

   You can not pass a whole array to a function (or return
   one) without enclosing it in a struct. You can not take an
   address of an array, only an element of it. There is no way
   to declare or pass a subarray. A function that operates on a
   arbitray sized  multi dim. array can be written but can get
   messy.

   One big impact of current array behavior is that the onus
   to do boundary condition checks is on the programmer and
   can't be done by the compiler.

Just as Pascal got "conformant arrays" in its 1990 standard,
arrays could've been made first class (but without "ref"
parameters code can look messy). For instance, consider the
folloowing:

    double a[5,20], b[20,10], c[5,10];
    int err = mat_multiply(c, a, b);

    int mat_mult(ref double c[int cx, cy], a[int ax, ay], b[int bx, by]) {
	if (cx != ax || cy != by || ay != bx) return -1;
	...
	return 0;
    }

This can be implemented without much trouble and allows
full boundary condition checking.  Currently the user will
have to manually pass {a,b,c}{x,y} parameters and he would
have to either create auxiliary vectors to point to each row
or have the function to do all the index arithmetic explicitly
(and the compiler can't check if your code has no boundary
condition bugs).

A slightly more difficult situation arises when you want to
pass sub arrays. For instance,

	mat_mutl(c[5,0:9], a[5,10:19], b[0:9,0:5])

Here you will have to somehow pass stride and offset (or use
iliffe vectors or some such).


From lm at mcvoy.com  Fri May 12 10:16:26 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Thu, 11 May 2017 17:16:26 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
Message-ID: <20170512001626.GQ4341@mcvoy.com>

On Fri, May 12, 2017 at 09:47:01AM +1000, Dave Horsfall wrote:
> On Thu, 11 May 2017, Larry McVoy wrote:
> 
> [...]
> 
> > Try the same thing with Linux.  The file system will come back, starting 
> > with, I believe, ext2.
> 
> That's a journalled FS, isn't it?  In which case the transactions get
> replayed.

My memory is ext2 is not journaled, I think that happened in ext3.  Or 
maybe it was an option on ext2?  Either way, I think ext2 did the right
thing without the journal.


From lm at mcvoy.com  Fri May 12 10:21:12 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Thu, 11 May 2017 17:21:12 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <017001d2cab1$16bcf4b0$4436de10$@ronnatalie.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <017001d2cab1$16bcf4b0$4436de10$@ronnatalie.com>
Message-ID: <20170512002112.GR4341@mcvoy.com>

Yeah, I get ordered writes, I taught a CS course at Stanford and I made
my students learn all about them.  I'm a UFS guy, so far as I know I'm
the last guy to push UFS/FFS forward (which is sort of sad).  

The Linux stuff is better.  It just is.  And we should all respect that,
I know we sit around and love on ancient Unix, and believe me, I love 
that stuff it changed the world, but we should respect people who have
moved it past what Unix did.  And I think Linux moved the file system
past what Unix did.

--lm

On Thu, May 11, 2017 at 07:48:27PM -0400, Ron Natalie wrote:
> Ordered writes go back to the original BSD fast file system, no?   I  seem
> to recall that when we switched from our V6/V7 disks,
> the filesystem got a lot more stable in crashes.
> 
> -----Original Message-----
> From: TUHS [mailto:tuhs-bounces at minnie.tuhs.org] On Behalf Of Dave Horsfall
> Sent: Thursday, May 11, 2017 7:47 PM
> To: The Eunuchs Hysterical Society
> Subject: Re: [TUHS] The evolution of Unix facilities and architecture
> 
> On Thu, 11 May 2017, Larry McVoy wrote:
> 
> [...]
> 
> > Try the same thing with Linux.  The file system will come back, 
> > starting with, I believe, ext2.
> 
> That's a journalled FS, isn't it?  In which case the transactions get
> replayed.
> 
> > My belief is that Linux orders writes such that while you may lose 
> > data (as in, a process created a file, the OS said it was OK, but that 
> > file will not be in the file system after a crash), but the rest of 
> > the file system will be consistent.  I think it's as if you powered 
> > off the machine a few seconds earlier than you actually did, some 
> > stuff is in flight and until they can write stuff out in the proper 
> > order you may lose data on a hard reset.
> 
> And FreeBSD (at least) has been doing ordered writes for quite some time.
> 
> --
> Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will
> suffer."

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 


From toby at telegraphics.com.au  Fri May 12 11:05:55 2017
From: toby at telegraphics.com.au (Toby Thain)
Date: Thu, 11 May 2017 21:05:55 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
Message-ID: <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>

On 2017-05-11 7:47 PM, Dave Horsfall wrote:
> On Thu, 11 May 2017, Larry McVoy wrote:
>
> [...]
>
>> Try the same thing with Linux.  The file system will come back, starting
>> with, I believe, ext2.
>
> That's a journalled FS, isn't it?  In which case the transactions get
> replayed.

No, I think ext3fs was the first version that was journaled.

So was reiserfs. With pull-plug tests I could get ext3fs to toss cookies 
but not reiserfs.

Now of course the state of the art is copy-on-write, like ZFS.

--Toby


>
>> My belief is that Linux orders writes such that while you may lose data
>> (as in, a process created a file, the OS said it was OK, but that file
>> will not be in the file system after a crash), but the rest of the file
>> system will be consistent.  I think it's as if you powered off the
>> machine a few seconds earlier than you actually did, some stuff is in
>> flight and until they can write stuff out in the proper order you may
>> lose data on a hard reset.
>
> And FreeBSD (at least) has been doing ordered writes for quite some time.
>



From wes.parish at paradise.net.nz  Fri May 12 11:41:51 2017
From: wes.parish at paradise.net.nz (Wesley Parish)
Date: Fri, 12 May 2017 13:41:51 +1200 (NZST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512001626.GQ4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <20170512001626.GQ4341@mcvoy.com>
Message-ID: <1494553311.591512df1d8a4@www.paradise.net.nz>

Consistent with what I remember of running fsck on Slackware in the 90s after unscheduled shutdowns. 
I longed for the time I'd get back when ext3 was incorporated into Linux and ext2 was relegated to 
legacy.

As far as I can remember ext2 was never journaled. I remembered getting quite excited reading about 
the log-structured file system in the O'Reilly 4.4BSD-Lite CD. That made so much sense to me.

Quoting Larry McVoy <lm at mcvoy.com>:

> On Fri, May 12, 2017 at 09:47:01AM +1000, Dave Horsfall wrote:
> > On Thu, 11 May 2017, Larry McVoy wrote:
> > 
> > [...]
> > 
> > > Try the same thing with Linux. The file system will come back,
> starting 
> > > with, I believe, ext2.
> > 
> > That's a journalled FS, isn't it? In which case the transactions get
> > replayed.
> 
> My memory is ext2 is not journaled, I think that happened in ext3. Or 
> maybe it was an option on ext2? Either way, I think ext2 did the right
> thing without the journal.
>  



"I have supposed that he who buys a Method means to learn it." - Ferdinand Sor,
Method for Guitar

"A verbal contract isn't worth the paper it's written on." -- Samuel Goldwyn


From theopav at gmail.com  Fri May 12 12:41:51 2017
From: theopav at gmail.com (Theo Pavlidis)
Date: Thu, 11 May 2017 22:41:51 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <20170512001520.38000124AEA4@mail.bitblocks.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <20170512001520.38000124AEA4@mail.bitblocks.com>
Message-ID: <CAJVMwZT9j2-=bBWHkm9qHS+KDshe-gmveGNh7No2=+-HN_KKQQ@mail.gmail.com>

I tend to stay away from using arrays as arguments of procedures, too many
surprises. Instead I use structures.
For example:

typedef struct {
.........
xteam NoSo[MAXB], EsWe[MAXB];
} xboard;
............
xboard TB;
............
       ScoreBoard(..., &TB, ...);


Of course, in my old age I only do simple game programming
​


Theo Pavlidis
(
​Member of 1127: 1980-86)
​
On Thu, May 11, 2017 at 8:15 PM, Bakul Shah <bakul at bitblocks.com> wrote:

> On Thu, 11 May 2017 15:32:32 PDT Larry McVoy <lm at mcvoy.com> wrote:
> >
> > I dunno if it is one of its greatest joys but pointers in C have always
> > made sense to me.
> >
> > I'm curious as to what is busted about arrays in C?  To me they just
> > seemed like a way to define how to look at a wad of memory and they
> > seem to work for me.  About the only thing I don't like about them is
> > that there is no late binding as to the size, Ada has late binding and
> > I thought it could be useful (I only know because Rob Netzer and I
> > wrote an Ada compiler for CS736 at UW-Madison that did a lot of Ada
> > but exceptions and late binding we did not do).
>
> Coming from a Pascal background I really liked the terseness
> of C. But theere were three things that bothered me.
>
> a) K&R style argument declarations. This got fixed in ANSI C.
>    C declarations were messy but that did not bother me much.
>
> b) No nested procedures. Gnu C had added them but the
>    implementation was a bit screwy and most people didn't care
>    in any case so there was no hope of this getting fixed.
>    Partly because most people used it as a portable assembly
>    language!
>
> c) arrays were not first class objects.  Given "T v[N];" v[i]
>    is of type T and i+v or v+i is of type T* -- this is
>    perfectly well defined and fine. What is not fine is that
>    an array is a second class object. Thus you can not do
>    for example,
>
>     int v[5];
>     struct foo {
>         int w[5];
>         int x;
>     } y;
>         ...
>         y.w = v;
>
>    You can not pass a whole array to a function (or return
>    one) without enclosing it in a struct. You can not take an
>    address of an array, only an element of it. There is no way
>    to declare or pass a subarray. A function that operates on a
>    arbitray sized  multi dim. array can be written but can get
>    messy.
>
>    One big impact of current array behavior is that the onus
>    to do boundary condition checks is on the programmer and
>    can't be done by the compiler.
>
> Just as Pascal got "conformant arrays" in its 1990 standard,
> arrays could've been made first class (but without "ref"
> parameters code can look messy). For instance, consider the
> folloowing:
>
>     double a[5,20], b[20,10], c[5,10];
>     int err = mat_multiply(c, a, b);
>
>     int mat_mult(ref double c[int cx, cy], a[int ax, ay], b[int bx, by]) {
>         if (cx != ax || cy != by || ay != bx) return -1;
>         ...
>         return 0;
>     }
>
> This can be implemented without much trouble and allows
> full boundary condition checking.  Currently the user will
> have to manually pass {a,b,c}{x,y} parameters and he would
> have to either create auxiliary vectors to point to each row
> or have the function to do all the index arithmetic explicitly
> (and the compiler can't check if your code has no boundary
> condition bugs).
>
> A slightly more difficult situation arises when you want to
> pass sub arrays. For instance,
>
>         mat_mutl(c[5,0:9], a[5,10:19], b[0:9,0:5])
>
> Here you will have to somehow pass stride and offset (or use
> iliffe vectors or some such).
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170511/0b5b8993/attachment.html>

From imp at bsdimp.com  Fri May 12 12:42:54 2017
From: imp at bsdimp.com (Warner Losh)
Date: Thu, 11 May 2017 20:42:54 -0600
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512002112.GR4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <017001d2cab1$16bcf4b0$4436de10$@ronnatalie.com>
 <20170512002112.GR4341@mcvoy.com>
Message-ID: <CANCZdfpi7V1CsJ+PmOiSPrhuYV_sB+17GEz8q0whfQFneA10og@mail.gmail.com>

On Thu, May 11, 2017 at 6:21 PM, Larry McVoy <lm at mcvoy.com> wrote:
> Yeah, I get ordered writes, I taught a CS course at Stanford and I made
> my students learn all about them.  I'm a UFS guy, so far as I know I'm
> the last guy to push UFS/FFS forward (which is sort of sad).
>
> The Linux stuff is better.  It just is.  And we should all respect that,
> I know we sit around and love on ancient Unix, and believe me, I love
> that stuff it changed the world, but we should respect people who have
> moved it past what Unix did.  And I think Linux moved the file system
> past what Unix did.

The Linux stuff was incrementally better in that it didn't fry the
filesystem, but instead would flush large amounts of changes if there
was no synchronous write before the power failed. This was progress.
However, it got its speed by doing async writes even of the metadata,
and that lead to the famous story where Linus was able to recover an
entire kernel he'd just accidentally rm -rf'd by hitting reset on his
machine before the syncer ran. Better than a scrambled filesystem?
Sure. But consistent and robust? Not so much. Plus, driver bugs could
still wreck havoc.

Soon after things like soft-updates came along in the BSD world which
solved the scrambled filesystem problems (for the most part), then SU
Journalling to make it robust. On the Linux side, ext2fs begat ext3
and then ext4fs came along and added journalling. ZFS came into being.
As did btrfs and many others. Some on Linux, some on Solaris, some on
BSD. The competition between them all has helped to make them all
better.

I've been running a v7 derivative (Venix) on my Rainbow lately.
There's issues with the motherboard that causes many insta-panics (it
turns off the interrupts due to something bad on the mobo --- a new
mobo doesn't do that). I've yet to lose anything, but the performance
isn't that great. So it's not an 'all the time' sort of thing.

Warner

> --lm
>
> On Thu, May 11, 2017 at 07:48:27PM -0400, Ron Natalie wrote:
>> Ordered writes go back to the original BSD fast file system, no?   I  seem
>> to recall that when we switched from our V6/V7 disks,
>> the filesystem got a lot more stable in crashes.
>>
>> -----Original Message-----
>> From: TUHS [mailto:tuhs-bounces at minnie.tuhs.org] On Behalf Of Dave Horsfall
>> Sent: Thursday, May 11, 2017 7:47 PM
>> To: The Eunuchs Hysterical Society
>> Subject: Re: [TUHS] The evolution of Unix facilities and architecture
>>
>> On Thu, 11 May 2017, Larry McVoy wrote:
>>
>> [...]
>>
>> > Try the same thing with Linux.  The file system will come back,
>> > starting with, I believe, ext2.
>>
>> That's a journalled FS, isn't it?  In which case the transactions get
>> replayed.
>>
>> > My belief is that Linux orders writes such that while you may lose
>> > data (as in, a process created a file, the OS said it was OK, but that
>> > file will not be in the file system after a crash), but the rest of
>> > the file system will be consistent.  I think it's as if you powered
>> > off the machine a few seconds earlier than you actually did, some
>> > stuff is in flight and until they can write stuff out in the proper
>> > order you may lose data on a hard reset.
>>
>> And FreeBSD (at least) has been doing ordered writes for quite some time.
>>
>> --
>> Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will
>> suffer."
>
> --
> ---
> Larry McVoy                  lm at mcvoy.com             http://www.mcvoy.com/lm


From hellwig.geisse at mni.thm.de  Fri May 12 16:24:32 2017
From: hellwig.geisse at mni.thm.de (Hellwig Geisse)
Date: Fri, 12 May 2017 08:24:32 +0200
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705120736210.67621@aneurin.horsfall.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <20170511142142.GA4341@mcvoy.com>
 <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>
 <20170511171100.GA9980@yeono.kjorling.se>
 <alpine.BSF.2.20.1705120736210.67621@aneurin.horsfall.org>
Message-ID: <1494570272.2218.180.camel@mni.thm.de>

On Fr, 2017-05-12 at 07:44 +1000, Dave Horsfall wrote:
> 
> Am I the only one here who thinks that e.g. a char pointer should be 
> "char* cp1, cp2" instead of "char *cp1, *cp2"?  I.e. the fundamental
> type is "char*", not "char", and to this day I still write:
> 
>     char*	cp1;
>     char*	cp2;
> 
> etc, which IMHO makes it clear (which is every programmer's duty).
> I used  to write that way in a previous life, and the boss didn't 
> complain.

This view does not work well with more complicated
declarations like "void (*p)(int)". What is the
"fundamental type" here? One could argue that the
real culprit is the list construction, which does
not mix well with C declarations.

Hellwig


From harald at skogtun.org  Fri May 12 18:15:49 2017
From: harald at skogtun.org (Harald Arnesen)
Date: Fri, 12 May 2017 10:15:49 +0200
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
Message-ID: <59156F35.2040508@skogtun.org>

Dave Horsfall [2017-05-12 01:47]:

> On Thu, 11 May 2017, Larry McVoy wrote:
>> Try the same thing with Linux.  The file system will come back, starting 
>> with, I believe, ext2.
> 
> That's a journalled FS, isn't it?  In which case the transactions get
> replayed.

No, journalling started with ext3. However, ext2 was (and is) a very
solid fs, which survived many power-outs and crashes when I used it back
in the 90s.
-- 
Hilsen Harald


From michael at kjorling.se  Fri May 12 18:17:13 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Fri, 12 May 2017 08:17:13 +0000
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
Message-ID: <20170512081713.GD7265@yeono.kjorling.se>

On 11 May 2017 21:05 -0400, from toby at telegraphics.com.au (Toby Thain):
>>> ext2
>> 
>> That's a journalled FS, isn't it?  In which case the transactions get
>> replayed.
> 
> No, I think ext3fs was the first version that was journaled.

Correct. ext2 doesn't have journaling, but if you do `mkfs.ext2 -j` on
Linux it creates a _journaled ext2_ A.K.A. _ext3_ file system.

With the resulting selling point of ext3 being mainly _much_ shorter
fsck times.


> So was reiserfs. With pull-plug tests I could get ext3fs to toss
> cookies but not reiserfs.

ReiserFS seems to be one of those where peoples' experiences really
differ. I've had massive crashes involving reiserfs myself, but don't
think I have ever actually lost any significant amounts of data to
ext2/3. Maybe it was just bad luck, but I have been bit sufficiently
badly by it to relegate it to the scrap heap of history. At least at
the time lack of good recovery tools didn't help (but then again if
ZFS breaks sufficiently that the pool doesn't import, you're pretty
hosed, too). Backups, backups.

These days, for me, it's pretty much all ZFS where I have any say in
it, and in addition even my home desktop is on a UPS (with automated,
controlled shutdown when approaching battery depletion), so unplanned
shutdowns are far less likely to happen. About the only way that can
realistically happen is either a kernel crash, or the UPS misreporting
(overestimating) the remaining battery time, resulting in a hard
shutdown before or during the shutdown process. Neither is impossible,
of course.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From tfb at tfeb.org  Fri May 12 23:56:59 2017
From: tfb at tfeb.org (Tim Bradshaw)
Date: Fri, 12 May 2017 14:56:59 +0100
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512081713.GD7265@yeono.kjorling.se>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
 <20170512081713.GD7265@yeono.kjorling.se>
Message-ID: <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>

On 12 May 2017, at 09:17, Michael Kjörling <michael at kjorling.se> wrote:
> 
> These days, for me, it's pretty much all ZFS

One of ZFS's particularly lovely features was that there was no offline filesystem checker at all.  So if you have a filesystem (pool, whatever) which you think something bad might have happened to, you check it *by mounting it*, where the checker runs *in the kernel, so any serious error in the code means a panic, if you're lucky and something worse if you're not.

When I found out about this I thought seriously of shorting Sun's stock (if I knew how to do that).  I would have made money.  As it was we stuck with logged UFS which, by 2007 or so was seriously bulletproof.

--tim

From michael at kjorling.se  Sat May 13 00:22:46 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Fri, 12 May 2017 14:22:46 +0000
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
 <20170512081713.GD7265@yeono.kjorling.se>
 <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>
Message-ID: <20170512142246.GH7265@yeono.kjorling.se>

On 12 May 2017 14:56 +0100, from tfb at tfeb.org (Tim Bradshaw):
> So if you have a filesystem (pool, whatever) which you think
> something bad might have happened to, you check it *by mounting it*,
> where the checker runs *in the kernel,

Easy peasy! No need to remember obscure fsck parameters; zpool import
is all you need.

Irony aside, I didn't say it was perfect.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From lm at mcvoy.com  Sat May 13 00:30:47 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Fri, 12 May 2017 07:30:47 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
 <20170512081713.GD7265@yeono.kjorling.se>
 <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>
Message-ID: <20170512143047.GV4341@mcvoy.com>

On Fri, May 12, 2017 at 02:56:59PM +0100, Tim Bradshaw wrote:
> When I found out about this I thought seriously of shorting Sun's
> stock (if I knew how to do that).  I would have made money.  As it was
> we stuck with logged UFS which, by 2007 or so was seriously bulletproof.

Wait, someone added logging to UFS?  Is there a writeup of that anywhere?
That would stomp all over my claim that nobody has hacked on UFS since
I did (which would be fine with me, I liked UFS, be cool if someone moved
it forward).

(pause while I google)

Yep, someone did.  I'd like to know who.  I found this:

http://www.oracle.com/technetwork/systems/linux/fs-performance-149840.pdf

Can anyone confirm those results?  That would be the first I've heard
of Solaris being faster than Linux.  If that's true has Linux tried 
to implement the same sort of logging?  

--lm

P.S.  I realize this isn't ancient Unix so I could move this to the 
linux-kernel mailing list.  Though maybe it is appropriate, it's tech
from the 1990's - is that ancient enough?


From richard at inf.ed.ac.uk  Sat May 13 00:04:45 2017
From: richard at inf.ed.ac.uk (Richard Tobin)
Date: Fri, 12 May 2017 15:04:45 +0100 (BST)
Subject: [TUHS] C declarations.
In-Reply-To: Ron Natalie's message of Thu, 11 May 2017 17:49:31 -0400
Message-ID: <20170512140445.EA5DD126E33E@macaroni.inf.ed.ac.uk>

> The problem with that is that * doesn't really bind to the type name. 
> It binds to the variable.
> 
> char* cp1, cp2;   // cp1 is pointer to char,   cp2 is just a char.
> 
> I always found it confusing that the * is used to indicate an pointer
> here, where as when you want to change an lvalue to a pointer, you use
> &.

The way to read it is that you are declaring *cp1 as a char.

-- Richard

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



From tfb at tfeb.org  Sat May 13 01:11:23 2017
From: tfb at tfeb.org (Tim Bradshaw)
Date: Fri, 12 May 2017 16:11:23 +0100
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512143047.GV4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
 <20170512081713.GD7265@yeono.kjorling.se>
 <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>
 <20170512143047.GV4341@mcvoy.com>
Message-ID: <276AEA9A-1584-4D52-AEA1-18725991D93E@tfeb.org>

On 12 May 2017, at 15:30, Larry McVoy <lm at mcvoy.com> wrote:
> 
> Wait, someone added logging to UFS?

Oh, yes.  I forget when it came in (Solaris 2.5?).  It's been the default (ie you need to turn it off in vfstab rather than turn it on) for some time, perhaps since Solaris 9?

> 
> Yep, someone did.  I'd like to know who.  I found this:
> 
> http://www.oracle.com/technetwork/systems/linux/fs-performance-149840.pdf
> 
> Can anyone confirm those results?

I can't confirm them, but I can confirm that a lot of rubbish has been talked about filesystem performance by people with various agendas.  I had an argument (well over 10 years ago now) with someone who claimed that ext2 (I guess, might have been ext3 by then) was just way faster than UFS for various operations (lots of file creattion/deletion I think).  It was ... if you left logging off.  If you turned logging on, not so much.

--tim

From jnc at mercury.lcs.mit.edu  Sat May 13 01:12:56 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Fri, 12 May 2017 11:12:56 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170512151256.B27CE18C09A@mercury.lcs.mit.edu>

    > From: "Ron Natalie"

    > Ordered writes go back to the original BSD fast file system, no?  I seem
    > to recall that when we switched from our V6/V7 disks, the filesystem got
    > a lot more stable in crashes.

I had a vague memory of reading about that, so I looked in the canonical FFS
paper (McKusick et al, "A Fast File System for UNIX" [1984)]) but found no
mention of it.

I did find a paper about 'fsck' (McKusick, Kowalski, "Fsck: The UNIX File
System Check Program") which talks (in Section 2.5. "Updates to the file
system") about how "problem[s] with asynchronous inode updates can be avoided
by doing all inode deallocations synchronously", but it's not clear if they're
talking about something that was actually done, or just saying
(hypothetically) that that's how one would fix it.

Is is possible that the changes to the file system (e.g. the way free blocks
were kept) made it more crash-proof?

     Noel



From clemc at ccc.com  Sat May 13 01:17:02 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 11:17:02 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512151256.B27CE18C09A@mercury.lcs.mit.edu>
References: <20170512151256.B27CE18C09A@mercury.lcs.mit.edu>
Message-ID: <CAC20D2M81oWA6MHbXCXcwUDxhU+q18USAK9Ek6m4QzF3o36BJw@mail.gmail.com>

George Gobble of Purdue did the FS work to V7/4.1 to fix the FS corruption
issues.   That was taken back by Kirk (wnj) and incorporated in 4.1A.    It
may have been before USENIX was creating proceedings.   I'll have to look
on my shelf at home or maybe ask George.

Clem

On Fri, May 12, 2017 at 11:12 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

>     > From: "Ron Natalie"
>
>     > Ordered writes go back to the original BSD fast file system, no?  I
> seem
>     > to recall that when we switched from our V6/V7 disks, the filesystem
> got
>     > a lot more stable in crashes.
>
> I had a vague memory of reading about that, so I looked in the canonical
> FFS
> paper (McKusick et al, "A Fast File System for UNIX" [1984)]) but found no
> mention of it.
>
> I did find a paper about 'fsck' (McKusick, Kowalski, "Fsck: The UNIX File
> System Check Program") which talks (in Section 2.5. "Updates to the file
> system") about how "problem[s] with asynchronous inode updates can be
> avoided
> by doing all inode deallocations synchronously", but it's not clear if
> they're
> talking about something that was actually done, or just saying
> (hypothetically) that that's how one would fix it.
>
> Is is possible that the changes to the file system (e.g. the way free
> blocks
> were kept) made it more crash-proof?
>
>      Noel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/05b363ef/attachment.html>

From clemc at ccc.com  Sat May 13 01:18:53 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 11:18:53 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2M81oWA6MHbXCXcwUDxhU+q18USAK9Ek6m4QzF3o36BJw@mail.gmail.com>
References: <20170512151256.B27CE18C09A@mercury.lcs.mit.edu>
 <CAC20D2M81oWA6MHbXCXcwUDxhU+q18USAK9Ek6m4QzF3o36BJw@mail.gmail.com>
Message-ID: <CAC20D2NZ2_6RhrgBj87Ljb8fEO=sEnD9AXWc-Z+FzjRmkWn=DQ@mail.gmail.com>

I should have said -- it was not hypothetical -- George implemented it and
published the code and we all picked it up,

On Fri, May 12, 2017 at 11:17 AM, Clem Cole <clemc at ccc.com> wrote:

> George Gobble of Purdue did the FS work to V7/4.1 to fix the FS corruption
> issues.   That was taken back by Kirk (wnj) and incorporated in 4.1A.    It
> may have been before USENIX was creating proceedings.   I'll have to look
> on my shelf at home or maybe ask George.
>
> Clem
>
> On Fri, May 12, 2017 at 11:12 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
> wrote:
>
>>     > From: "Ron Natalie"
>>
>>     > Ordered writes go back to the original BSD fast file system, no?  I
>> seem
>>     > to recall that when we switched from our V6/V7 disks, the
>> filesystem got
>>     > a lot more stable in crashes.
>>
>> I had a vague memory of reading about that, so I looked in the canonical
>> FFS
>> paper (McKusick et al, "A Fast File System for UNIX" [1984)]) but found no
>> mention of it.
>>
>> I did find a paper about 'fsck' (McKusick, Kowalski, "Fsck: The UNIX File
>> System Check Program") which talks (in Section 2.5. "Updates to the file
>> system") about how "problem[s] with asynchronous inode updates can be
>> avoided
>> by doing all inode deallocations synchronously", but it's not clear if
>> they're
>> talking about something that was actually done, or just saying
>> (hypothetically) that that's how one would fix it.
>>
>> Is is possible that the changes to the file system (e.g. the way free
>> blocks
>> were kept) made it more crash-proof?
>>
>>      Noel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/b4c29ece/attachment.html>

From clemc at ccc.com  Sat May 13 01:46:08 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 11:46:08 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2NZ2_6RhrgBj87Ljb8fEO=sEnD9AXWc-Z+FzjRmkWn=DQ@mail.gmail.com>
References: <20170512151256.B27CE18C09A@mercury.lcs.mit.edu>
 <CAC20D2M81oWA6MHbXCXcwUDxhU+q18USAK9Ek6m4QzF3o36BJw@mail.gmail.com>
 <CAC20D2NZ2_6RhrgBj87Ljb8fEO=sEnD9AXWc-Z+FzjRmkWn=DQ@mail.gmail.com>
Message-ID: <CAC20D2MhK-WR-RBVBsuEZMACkZTwxe6_ijMgqJYzJiO-u_ONVQ@mail.gmail.com>

I just did a small amount of hunting.  The oldest printed USENIX Proceeding
seems to be 1983 [which was the one Rob gave 'cat -v considered harmful' -
although only the abstract is in it].

George did the ordered writes work earlier as I was still at Tektronix,
because I remember getting a tape from him a putting the changes into our
V7 system.

If we hunt around for a 'Purdue-EE' distribution circa '79-'81 we should be
able to find it.  BTW: as a piece of History for Diomidis on that same tape
is fix for one of the first '0-day' UNIX exploits I can remember.  I'll see
if I can find it and identify it for you.   That would be a good piece of
history to call out.

The story is this ...

George was very upset when he found it.  But this was during the time when
UNIX was fighting a bit for it's life in the press as not being a 'real'
OS.   DEC and IBM making claims that it was a toy, *etc*.   So most of the
the hacker community took it pretty seriously.   It is funny, today we
would react in the opposite manner.,  But, there was a big 'hush-hush'
meeting at a Summer USENIX that was very exclusive to be invited too.    We
were in a private conference room, the door was locked etc.   I remember
that Dennis was there, Joy was there. Ron's old friend Mike must have been
in it.  I think a couple of the Rand folks.   Anyway - it was an issue with
profile(2) -- surprise, surprise.   Pretty easy fix.   We all took the code
back and promised to get patches out ASAP and not tell any one about it.

Clem

On Fri, May 12, 2017 at 11:18 AM, Clem Cole <clemc at ccc.com> wrote:

> I should have said -- it was not hypothetical -- George implemented it
> and published the code and we all picked it up,
>
> On Fri, May 12, 2017 at 11:17 AM, Clem Cole <clemc at ccc.com> wrote:
>
>> George Gobble of Purdue did the FS work to V7/4.1 to fix the FS
>> corruption issues.   That was taken back by Kirk (wnj) and incorporated in
>> 4.1A.    It may have been before USENIX was creating proceedings.   I'll
>> have to look on my shelf at home or maybe ask George.
>>
>> Clem
>>
>> On Fri, May 12, 2017 at 11:12 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
>> wrote:
>>
>>>     > From: "Ron Natalie"
>>>
>>>     > Ordered writes go back to the original BSD fast file system, no?
>>> I seem
>>>     > to recall that when we switched from our V6/V7 disks, the
>>> filesystem got
>>>     > a lot more stable in crashes.
>>>
>>> I had a vague memory of reading about that, so I looked in the canonical
>>> FFS
>>> paper (McKusick et al, "A Fast File System for UNIX" [1984)]) but found
>>> no
>>> mention of it.
>>>
>>> I did find a paper about 'fsck' (McKusick, Kowalski, "Fsck: The UNIX File
>>> System Check Program") which talks (in Section 2.5. "Updates to the file
>>> system") about how "problem[s] with asynchronous inode updates can be
>>> avoided
>>> by doing all inode deallocations synchronously", but it's not clear if
>>> they're
>>> talking about something that was actually done, or just saying
>>> (hypothetically) that that's how one would fix it.
>>>
>>> Is is possible that the changes to the file system (e.g. the way free
>>> blocks
>>> were kept) made it more crash-proof?
>>>
>>>      Noel
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/e573253f/attachment.html>

From chet.ramey at case.edu  Sat May 13 01:52:50 2017
From: chet.ramey at case.edu (Chet Ramey)
Date: Fri, 12 May 2017 11:52:50 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512143047.GV4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
 <20170512081713.GD7265@yeono.kjorling.se>
 <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>
 <20170512143047.GV4341@mcvoy.com>
Message-ID: <3aa087e2-9305-3d5b-123e-dc2d7833baf2@case.edu>

On 5/12/17 10:30 AM, Larry McVoy wrote:
> On Fri, May 12, 2017 at 02:56:59PM +0100, Tim Bradshaw wrote:
>> When I found out about this I thought seriously of shorting Sun's
>> stock (if I knew how to do that).  I would have made money.  As it was
>> we stuck with logged UFS which, by 2007 or so was seriously bulletproof.
> 
> Wait, someone added logging to UFS?  Is there a writeup of that anywhere?

You could look at the soft updates paper from 1999 for Kirk's perspective.

https://www.usenix.org/legacy/event/usenix99/full_papers/mckusick/mckusick.pdf

There was a paper about journaled soft updates, too:

https://www.mckusick.com/softdep/suj.pdf

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet at case.edu    http://cnswww.cns.cwru.edu/~chet/


From imp at bsdimp.com  Sat May 13 02:21:16 2017
From: imp at bsdimp.com (Warner Losh)
Date: Fri, 12 May 2017 10:21:16 -0600
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <3aa087e2-9305-3d5b-123e-dc2d7833baf2@case.edu>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <alpine.BSF.2.20.1705120944510.67621@aneurin.horsfall.org>
 <1873b923-6f5e-83a2-753d-6d7a2f8b580a@telegraphics.com.au>
 <20170512081713.GD7265@yeono.kjorling.se>
 <0121AE71-2930-40AB-ABFD-552E207AF979@tfeb.org>
 <20170512143047.GV4341@mcvoy.com>
 <3aa087e2-9305-3d5b-123e-dc2d7833baf2@case.edu>
Message-ID: <CANCZdfqAfwgO0XKG1pg8Y500Wmo9Aw8Ci9h-L37KqRdi=emuwQ@mail.gmail.com>

On Fri, May 12, 2017 at 9:52 AM, Chet Ramey <chet.ramey at case.edu> wrote:
> On 5/12/17 10:30 AM, Larry McVoy wrote:
>> On Fri, May 12, 2017 at 02:56:59PM +0100, Tim Bradshaw wrote:
>>> When I found out about this I thought seriously of shorting Sun's
>>> stock (if I knew how to do that).  I would have made money.  As it was
>>> we stuck with logged UFS which, by 2007 or so was seriously bulletproof.
>>
>> Wait, someone added logging to UFS?  Is there a writeup of that anywhere?
>
> You could look at the soft updates paper from 1999 for Kirk's perspective.
>
> https://www.usenix.org/legacy/event/usenix99/full_papers/mckusick/mckusick.pdf
>
> There was a paper about journaled soft updates, too:
>
> https://www.mckusick.com/softdep/suj.pdf

There's also Margo's paper on LFS, which added logging to UFS, though
by the time it was over it wasn't recognizable.

https://www.usenix.org/publications/library/proceedings/sd93/seltzer.pdf

The Journaled updates, though, is that someone added an intent log to
UFS + SoftUpdates. This was used to replay the last few operations
when doing fsck coming back up after a crash. There's also something
called gjournal, which adds a different type of journaling to UFS, but
the less said about that train-wreck the better.

Warner


From doug at cs.dartmouth.edu  Sat May 13 04:43:14 2017
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Fri, 12 May 2017 14:43:14 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>


>  We all took the code back and promised to get patches out ASAP and not tell any one about it.

Fascinating. Chnages were installed frequently in the Unix lab, mostly
at night without fanfare. But an actual zero-day should have been big
enough news for me to have heard about. I'm pretty sure I didn't; Dennis
evidently kept his counsel.

Doug


From crossd at gmail.com  Sat May 13 04:56:37 2017
From: crossd at gmail.com (Dan Cross)
Date: Fri, 12 May 2017 14:56:37 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>
References: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>
Message-ID: <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>

On Fri, May 12, 2017 at 2:43 PM, Doug McIlroy <doug at cs.dartmouth.edu> wrote:
>
> >  We all took the code back and promised to get patches out ASAP and not
> tell any one about it.
>
> Fascinating. Chnages were installed frequently in the Unix lab, mostly
> at night without fanfare. But an actual zero-day should have been big
> enough news for me to have heard about. I'm pretty sure I didn't; Dennis
> evidently kept his counsel.


I wonder if such a thing would have been treated the same way within Bell
Labs as outside?

Presumably you didn't have to worry about hordes of undergraduates picking
over your systems looking for ways to get root access. Or, indeed,
undergraduates doing anything on your systems, save for the occasional
intern or precocious child of an employee. For that matter, this raises a
question: what was the attitude towards root access within the labs? Was it
constrained to the anointed few or did a large-ish number of people have it?

Anyway, I could well imagine a scenario where Dennis comes back but thinks
fairly little of it and makes vague mention of a fairly serious bug but
gives it little more thought than any other fairly serious bug. It's
patched and folks go on with their lives, since it's much less likely to be
the source of irritation in a corporate search department than it would be
in, say, a university.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/992e4ccc/attachment.html>

From clemc at ccc.com  Sat May 13 05:43:00 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 15:43:00 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>
References: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>
 <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>
Message-ID: <CAC20D2PMM98AR5Pt_ZkqERHN1EiD7o-3UKgGXf_R_YiGKM-2Fw@mail.gmail.com>

Could be.   The scare was that the anti-UNIX folks would get wind of it and
it would used in the fight as to why VMS was 'better.' The CS Research
community has not yet made the switch off the 36 bit world to Vaxen, so the
Arpanet community is still pretty much PDP-10 central; but it was also
right around the time when DARPA was defunding the PDP-10; had chosen the
VAX but was arguing VMS vs UNIX.

I don't thing CSRG had been funded as a group yet. Joy might have done his
'fast vax' paper to show that UNIX was just as good as VMS, but that work
might be on the horizon.  Certainly all of 4.1a/b/c, 4.2, 4.3, NET-x was
years away.

The point is that you didn't (yet) have a mass of students on the systems
'in the field', but some folks had that as a vision (and want it to be that
way and are scared it something bad happens 'in the press' - it would cause
a set back.

At the that time, think a couple of Universities are >>starting<< to use
UNIX for general CS classes/teaching (Purdue & UCB being two of them),
maybe Michigan and U of I, but I think CMU and Stanford are still using
PDP-20's [not sure about MIT] (where Princeton and UCLA I think were still
IBM shops for undergrads).

So the whole reason to keep it quiet @ the USENOX conference was because it
was felt at the time, the folks in that room were the primary people
hacking the kernel and if we all took the couple of lines of fix back to
our shops, the problem was solved.

It sort of blows my mind if Doug never knew about it, in hind sight it
seems George got his wish!!

Clem

On Fri, May 12, 2017 at 2:56 PM, Dan Cross <crossd at gmail.com> wrote:

> On Fri, May 12, 2017 at 2:43 PM, Doug McIlroy <doug at cs.dartmouth.edu>
> wrote:
>>
>> >  We all took the code back and promised to get patches out ASAP and not
>> tell any one about it.
>>
>> Fascinating. Chnages were installed frequently in the Unix lab, mostly
>> at night without fanfare. But an actual zero-day should have been big
>> enough news for me to have heard about. I'm pretty sure I didn't; Dennis
>> evidently kept his counsel.
>
>
> I wonder if such a thing would have been treated the same way within Bell
> Labs as outside?
>
> Presumably you didn't have to worry about hordes of undergraduates picking
> over your systems looking for ways to get root access. Or, indeed,
> undergraduates doing anything on your systems, save for the occasional
> intern or precocious child of an employee. For that matter, this raises a
> question: what was the attitude towards root access within the labs? Was it
> constrained to the anointed few or did a large-ish number of people have it?
>
> Anyway, I could well imagine a scenario where Dennis comes back but thinks
> fairly little of it and makes vague mention of a fairly serious bug but
> gives it little more thought than any other fairly serious bug. It's
> patched and folks go on with their lives, since it's much less likely to be
> the source of irritation in a corporate search department than it would be
> in, say, a university.
>
>         - Dan C.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/7af6e776/attachment.html>

From clemc at ccc.com  Sat May 13 06:06:34 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 16:06:34 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2PMM98AR5Pt_ZkqERHN1EiD7o-3UKgGXf_R_YiGKM-2Fw@mail.gmail.com>
References: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>
 <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>
 <CAC20D2PMM98AR5Pt_ZkqERHN1EiD7o-3UKgGXf_R_YiGKM-2Fw@mail.gmail.com>
Message-ID: <CAC20D2MYtH6QEkpkpSBKf2fQ-EmuLfeiyRoja0gdDuko06+Kxw@mail.gmail.com>

BTW:  As I think more about it, I believe that it is probable that George
found it on the PDP-11 first (V7) not yet BSD 3.0/4.0 (altlhough I've
forgotten the date or which USENIX we had the meeting).  I'm not sure why,
but I remember putting the fix into a multiple kernels due to specifics of
the different MMUs).   Because of that, memory I'm going to date it as
probably 1981, because I would have had 3 different kernels to play with:
PDP-11, Vax and Glaser and I were writing Magix an OS for the Tek Magonlia,
a 68000 based workstation being build in Tek Labs.

But I'm sure if we go hunting through different people's records around
that time, changes to the profile kernel code for security will show up.
As I said, George originally found it and we all grabbed his fix... so I
would expect to see a lot of updated kernels with a similar changes around
the same time; but all put in by different people - but I fear, it was also
earlier enough that this was before SCCS was widely being used; so tracking
it can only be done by looking at distribution tapes from those times.

Clem

On Fri, May 12, 2017 at 3:43 PM, Clem Cole <clemc at ccc.com> wrote:

> Could be.   The scare was that the anti-UNIX folks would get wind of it
> and it would used in the fight as to why VMS was 'better.' The CS Research
> community has not yet made the switch off the 36 bit world to Vaxen, so the
> Arpanet community is still pretty much PDP-10 central; but it was also
> right around the time when DARPA was defunding the PDP-10; had chosen the
> VAX but was arguing VMS vs UNIX.
>
> I don't thing CSRG had been funded as a group yet. Joy might have done his
> 'fast vax' paper to show that UNIX was just as good as VMS, but that work
> might be on the horizon.  Certainly all of 4.1a/b/c, 4.2, 4.3, NET-x was
> years away.
>
> The point is that you didn't (yet) have a mass of students on the systems
> 'in the field', but some folks had that as a vision (and want it to be that
> way and are scared it something bad happens 'in the press' - it would cause
> a set back.
>
> At the that time, think a couple of Universities are >>starting<< to use
> UNIX for general CS classes/teaching (Purdue & UCB being two of them),
> maybe Michigan and U of I, but I think CMU and Stanford are still using
> PDP-20's [not sure about MIT] (where Princeton and UCLA I think were still
> IBM shops for undergrads).
>
> So the whole reason to keep it quiet @ the USENOX conference was because
> it was felt at the time, the folks in that room were the primary people
> hacking the kernel and if we all took the couple of lines of fix back to
> our shops, the problem was solved.
>
> It sort of blows my mind if Doug never knew about it, in hind sight it
> seems George got his wish!!
>
> Clem
>
> On Fri, May 12, 2017 at 2:56 PM, Dan Cross <crossd at gmail.com> wrote:
>
>> On Fri, May 12, 2017 at 2:43 PM, Doug McIlroy <doug at cs.dartmouth.edu>
>> wrote:
>>>
>>> >  We all took the code back and promised to get patches out ASAP and
>>> not tell any one about it.
>>>
>>> Fascinating. Chnages were installed frequently in the Unix lab, mostly
>>> at night without fanfare. But an actual zero-day should have been big
>>> enough news for me to have heard about. I'm pretty sure I didn't; Dennis
>>> evidently kept his counsel.
>>
>>
>> I wonder if such a thing would have been treated the same way within Bell
>> Labs as outside?
>>
>> Presumably you didn't have to worry about hordes of undergraduates
>> picking over your systems looking for ways to get root access. Or, indeed,
>> undergraduates doing anything on your systems, save for the occasional
>> intern or precocious child of an employee. For that matter, this raises a
>> question: what was the attitude towards root access within the labs? Was it
>> constrained to the anointed few or did a large-ish number of people have it?
>>
>> Anyway, I could well imagine a scenario where Dennis comes back but
>> thinks fairly little of it and makes vague mention of a fairly serious bug
>> but gives it little more thought than any other fairly serious bug. It's
>> patched and folks go on with their lives, since it's much less likely to be
>> the source of irritation in a corporate search department than it would be
>> in, say, a university.
>>
>>         - Dan C.
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/505437b9/attachment.html>

From reed at reedmedia.net  Sat May 13 06:40:03 2017
From: reed at reedmedia.net (Jeremy C. Reed)
Date: Fri, 12 May 2017 15:40:03 -0500 (CDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2MYtH6QEkpkpSBKf2fQ-EmuLfeiyRoja0gdDuko06+Kxw@mail.gmail.com>
References: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>
 <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>
 <CAC20D2PMM98AR5Pt_ZkqERHN1EiD7o-3UKgGXf_R_YiGKM-2Fw@mail.gmail.com>
 <CAC20D2MYtH6QEkpkpSBKf2fQ-EmuLfeiyRoja0gdDuko06+Kxw@mail.gmail.com>
Message-ID: <alpine.NEB.2.20.1705121526290.28884@t1.m.reedmedia.net>

profil code somewhere in here?
https://github.com/weiss/original-bsd/commits/master/sys/kern/kern_clock.c?after=b44636d7febc9dcf553118bd320571864188351d+104

that has the sccs history back to April 1980 for src/sys/sys/clock.c




From dave at horsfall.org  Sat May 13 07:12:14 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sat, 13 May 2017 07:12:14 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <1494570272.2218.180.camel@mni.thm.de>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <20170511142142.GA4341@mcvoy.com>
 <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>
 <20170511171100.GA9980@yeono.kjorling.se>
 <alpine.BSF.2.20.1705120736210.67621@aneurin.horsfall.org>
 <1494570272.2218.180.camel@mni.thm.de>
Message-ID: <alpine.BSF.2.20.1705130708410.67621@aneurin.horsfall.org>

On Fri, 12 May 2017, Hellwig Geisse wrote:

> >     char*	cp1;
> >     char*	cp2;
> > 
> > etc, which IMHO makes it clear (which is every programmer's duty). I 
> > used  to write that way in a previous life, and the boss didn't 
> > complain.
> 
> This view does not work well with more complicated declarations like 
> "void (*p)(int)". What is the "fundamental type" here? One could argue 
> that the real culprit is the list construction, which does not mix well 
> with C declarations.

Let's see:

    aneurin% cdecl
    Type `help' or `?' for help
    explain void (*p)(int)
    declare p as pointer to function (int) returning void

So the "fundamental" type (if there was such a thing) would be a pointer 
to a function, I guess i.e. don't treat it as anything else.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

From ron at ronnatalie.com  Sat May 13 07:29:16 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Fri, 12 May 2017 17:29:16 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>
References: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>
 <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>
Message-ID: <022a01d2cb66$cfa7f780$6ef7e680$@ronnatalie.com>

Allegedly at one point Dennis Mumaugh over at the NSA had a list of UNIX security bugs that became classified.

Mike Muuss (and the rest of us BRLers)  decided not to request a copy because we figured we’d be better off not being constrained by knowing some bug we were disseminating information on (to other system adminsitrators) was classified.    We did have a little informal meeting with Dennis in the hall and ran through our list of known issues at the time.

 

My favorite isn’t so much a UNIX bug but a PDP-11 hardware bug.    It’s just that I think it may be impossible on a DEC operating system to create a program that manifests it (you have to fill your entire address space with SPL instructions).

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/1f7bc028/attachment.html>

From clemc at ccc.com  Sat May 13 07:29:20 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 17:29:20 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.NEB.2.20.1705121526290.28884@t1.m.reedmedia.net>
References: <201705121843.v4CIhE68027515@coolidge.cs.Dartmouth.EDU>
 <CAEoi9W4__qopm1yQpW-0ErGOYn97eYuj87jwCP4u-D-5Hf6SYQ@mail.gmail.com>
 <CAC20D2PMM98AR5Pt_ZkqERHN1EiD7o-3UKgGXf_R_YiGKM-2Fw@mail.gmail.com>
 <CAC20D2MYtH6QEkpkpSBKf2fQ-EmuLfeiyRoja0gdDuko06+Kxw@mail.gmail.com>
 <alpine.NEB.2.20.1705121526290.28884@t1.m.reedmedia.net>
Message-ID: <CAC20D2O1tbaWQdoqKPRCz3zeKfM9q6yde5KCZ5OoHGHk0FKC1Q@mail.gmail.com>

Excuse me - I said -- profil - I intended to say  ptrace(2)....   long day
-- aging brain bits.

http://man.cat-v.org/unix_7th/2/ptrace

The debugger system call that allows you to a processed memory.

On Fri, May 12, 2017 at 4:40 PM, Jeremy C. Reed <reed at reedmedia.net> wrote:

> profil code somewhere in here?
> https://github.com/weiss/original-bsd/commits/master/
> sys/kern/kern_clock.c?after=b44636d7febc9dcf553118bd320571864188351d+104
>
> that has the sccs history back to April 1980 for src/sys/sys/clock.c
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/4680e758/attachment.html>

From hellwig.geisse at mni.thm.de  Sat May 13 09:25:42 2017
From: hellwig.geisse at mni.thm.de (Hellwig Geisse)
Date: Sat, 13 May 2017 01:25:42 +0200
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705130708410.67621@aneurin.horsfall.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <20170511142142.GA4341@mcvoy.com>
 <CAC20D2M+kTnQJ0+q0EF-RmURTA8aq4R-2w98Ac3Cx9pB_Dc4AA@mail.gmail.com>
 <20170511171100.GA9980@yeono.kjorling.se>
 <alpine.BSF.2.20.1705120736210.67621@aneurin.horsfall.org>
 <1494570272.2218.180.camel@mni.thm.de>
 <alpine.BSF.2.20.1705130708410.67621@aneurin.horsfall.org>
Message-ID: <1494631542.2218.215.camel@mni.thm.de>

On Sa, 2017-05-13 at 07:12 +1000, Dave Horsfall wrote:
> 
> Let's see:
> 
>     aneurin% cdecl
>     Type `help' or `?' for help
>     explain void (*p)(int)
>     declare p as pointer to function (int) returning void
> 
> So the "fundamental" type (if there was such a thing) would be a
> pointer to a function, I guess i.e. don't treat it as anything else.
> 

Yes, of course. What I was aiming at: If you try
to declare two of these variables, neither
"void (*p,q)(int)" nor "void (*(p,q))(int)"
is allowed, so you cannot use the "fundamental
type" to declare more than one variable of this
type in a single declaration list (as you had
suggested with "char* cp1, cp2").

"void (*p)(int), (*q)(int)" in contrast is legal,
but I wouldn't call "void" the fundamental type
in these declarations. Thus my statement "list
construction (in declarations) and C declarations
don't mix well" - IMO one of the difficulties in
reading/writing C declarations, and the starting
point of this discussion.

Hellwig


From jnc at mercury.lcs.mit.edu  Sat May 13 09:30:12 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Fri, 12 May 2017 19:30:12 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>

    > From: Clem Cole

    > I said -- profil - I intended to say  ptrace(2)

Is that the one where running an SUID program under the debugger allowed one
to patch the in-core image of said program?

If so, I have a story, and a puzzle, about that.


A couple of us, including Jim Gettys (later of X-windows fame) were on out way
out to dinner one evening (I don't recall when, alas, but I didn't meet him
until '80 or so), and he mentioned this horrible Unix security bug that had
just been found. All he would tell me about it (IIRC) was that it involved
ptrace.

So, over dinner (without the source) I figured out what it had to be:
patching SUID programs. So I asked him if that was what it was, and I don't
recall his exact answer, but I vaguely recall he hemmed and hawed in a way
that let me know I'd worked it out.

So when we got back from dinner, I looked at the source to our system to see
if I was right, and.... it had already been fixed! Here's the code:

	if (xp->x_count!=1 || xp->x_iptr->i_mode&ISVTX)
		goto error;

Now, we'd been running that system since '77 (when I joined CSR), without any
changes to that part of the OS, so I'm pretty sure this fix pre-dates your
story?

So when I saw your email about this, I wondered 'did that bug get fixed at
MIT when some undergrad used it to break in' (I _think_ ca. '77 is when they
switched from an OS called Delphi on the -11/45 used for the undergrad CS
programming course - I _think_ they switched that machine from Delphi to
Unix), or did it come with PWB1? (Like I said, that system was mostly PWB1.)

So I just looked in the PWB1 sources, and... there it is, the _exact_ same
fix. So we must have got it from PWB1.

So now the question is: did the PWB guys find and fix this, and forget to
tell the research guys? Or did they tell them, and the research guys blew
them off? Or what?

	Noel


From dave at horsfall.org  Sat May 13 09:38:49 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sat, 13 May 2017 09:38:49 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
References: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
Message-ID: <alpine.BSF.2.20.1705130937330.67621@aneurin.horsfall.org>

On Fri, 12 May 2017, Noel Chiappa wrote:

> So when we got back from dinner, I looked at the source to our system to see
> if I was right, and.... it had already been fixed! Here's the code:
> 
> 	if (xp->x_count!=1 || xp->x_iptr->i_mode&ISVTX)
> 		goto error;

Err, isn't that the sticky bit, not the setuid bit?

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From random832 at fastmail.com  Sat May 13 09:52:49 2017
From: random832 at fastmail.com (Random832)
Date: Fri, 12 May 2017 19:52:49 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705130937330.67621@aneurin.horsfall.org>
References: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
 <alpine.BSF.2.20.1705130937330.67621@aneurin.horsfall.org>
Message-ID: <1494633169.1599957.975050752.2FB9B2DA@webmail.messagingengine.com>

On Fri, May 12, 2017, at 19:38, Dave Horsfall wrote:
> On Fri, 12 May 2017, Noel Chiappa wrote:
> 
> > So when we got back from dinner, I looked at the source to our system to see
> > if I was right, and.... it had already been fixed! Here's the code:
> > 
> > 	if (xp->x_count!=1 || xp->x_iptr->i_mode&ISVTX)
> > 		goto error;
> 
> Err, isn't that the sticky bit, not the setuid bit?

The sticky bit makes it keep the image in memory when there are no
processes using it. I assume x_count is determining whether there are
processes using it. So, taken together, these checks are "is there or
might there be in the future a process, other than the one being
debugged, using this exact copy of the image rather than loading it from
the disk".

The next line is "xp->x_iptr->i_flag &= ~ITEXT", which I assume prevents
the image from being reused for other processes started while this one
is running.

I am looking at 7th edition in the UnixTree site, the whole fix is:

		/*
		 * If text, must assure exclusive use
		 */
		if (xp = u.u_procp->p_textp) {
			if (xp->x_count!=1 || xp->x_iptr->i_mode&ISVTX)
				goto error;
			xp->x_iptr->i_flag &= ~ITEXT;
		}

The equivalent section to the one this appears in in 6th edition doesn't
have the fix, and the comment claims, doesn't work at all:

	/* write user I (for now, always an error) */
	case 4:
		if (suiword(ipc.ip_addr, 0) < 0)
			goto error;
		suiword(ipc.ip_addr, ipc.ip_data);
		break;

This is clearly PDP-11 specific, maybe a similar bug reappeared with
demand-paged virtual memory. 


From clemc at ccc.com  Sat May 13 10:22:23 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 20:22:23 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
References: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
Message-ID: <CAC20D2P28DukBiky3D6NTGkjm+vwYuhrNHFz416bv1oGV82cjg@mail.gmail.com>

Interesting...   I don't remember Gettys being at the meeting (I would get
to know Jim a few years later when he was at Princeton before he came back
to MIT to work on X) and he's a been a friend of mine for a number of years
(actually lives in the next town over).

I do not remember all the details of the bug at this point, to many beers
ago; but yes the jist of the issue was being able to write to user memory
with ptraced process with SUID being involved.

The only thing that worries me about your response is I thought remembered
that MMU was somehow involved.   Just turning off SUID was not the only
part of the solution.

I do remember that the bug was in the Research kernel at the time and
Dennis had not known about it until that meeting so if PWB had it fixed,
that's an example of something that did not go back, which I would find
surprising.

I suspect MIT found and fixed it independently, but it never got passed it
back for whatever reason.

We should try to look in the PWB 1.0 kernel.

Clem

On Fri, May 12, 2017 at 7:30 PM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

>     > From: Clem Cole
>
>     > I said -- profil - I intended to say  ptrace(2)
>
> Is that the one where running an SUID program under the debugger allowed
> one
> to patch the in-core image of said program?
>
> If so, I have a story, and a puzzle, about that.
>
>
> A couple of us, including Jim Gettys (later of X-windows fame) were on out
> way
> out to dinner one evening (I don't recall when, alas, but I didn't meet him
> until '80 or so), and he mentioned this horrible Unix security bug that had
> just been found. All he would tell me about it (IIRC) was that it involved
> ptrace.
>
> So, over dinner (without the source) I figured out what it had to be:
> patching SUID programs. So I asked him if that was what it was, and I don't
> recall his exact answer, but I vaguely recall he hemmed and hawed in a way
> that let me know I'd worked it out.
>
> So when we got back from dinner, I looked at the source to our system to
> see
> if I was right, and.... it had already been fixed! Here's the code:
>
>         if (xp->x_count!=1 || xp->x_iptr->i_mode&ISVTX)
>                 goto error;
>
> Now, we'd been running that system since '77 (when I joined CSR), without
> any
> changes to that part of the OS, so I'm pretty sure this fix pre-dates your
> story?
>
> So when I saw your email about this, I wondered 'did that bug get fixed at
> MIT when some undergrad used it to break in' (I _think_ ca. '77 is when
> they
> switched from an OS called Delphi on the -11/45 used for the undergrad CS
> programming course - I _think_ they switched that machine from Delphi to
> Unix), or did it come with PWB1? (Like I said, that system was mostly
> PWB1.)
>
> So I just looked in the PWB1 sources, and... there it is, the _exact_ same
> fix. So we must have got it from PWB1.
>
> So now the question is: did the PWB guys find and fix this, and forget to
> tell the research guys? Or did they tell them, and the research guys blew
> them off? Or what?
>
>         Noel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/fc322093/attachment.html>

From clemc at ccc.com  Sat May 13 10:23:55 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 12 May 2017 20:23:55 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2P28DukBiky3D6NTGkjm+vwYuhrNHFz416bv1oGV82cjg@mail.gmail.com>
References: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
 <CAC20D2P28DukBiky3D6NTGkjm+vwYuhrNHFz416bv1oGV82cjg@mail.gmail.com>
Message-ID: <CAC20D2P5+pQh_qEQ8LPWZiqMD1+wbVtkJK4bcLhwAarHHK3c6g@mail.gmail.com>

On Fri, May 12, 2017 at 8:22 PM, Clem Cole <clemc at ccc.com> wrote:

> We should try to look in the PWB 1.0 kernel.
>

​As you said, you found it in the PWB1.0 sources... which is really
interesting.​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170512/8ddf0c55/attachment.html>

From dave at horsfall.org  Sat May 13 10:26:12 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sat, 13 May 2017 10:26:12 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <1494633169.1599957.975050752.2FB9B2DA@webmail.messagingengine.com>
References: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
 <alpine.BSF.2.20.1705130937330.67621@aneurin.horsfall.org>
 <1494633169.1599957.975050752.2FB9B2DA@webmail.messagingengine.com>
Message-ID: <alpine.BSF.2.20.1705131024360.67621@aneurin.horsfall.org>

On Fri, 12 May 2017, Random832 wrote:

> > > 	if (xp->x_count!=1 || xp->x_iptr->i_mode&ISVTX)
> > > 		goto error;
> > 
> > Err, isn't that the sticky bit, not the setuid bit?
> 
> The sticky bit makes it keep the image in memory when there are no 
> processes using it. I assume x_count is determining whether there are 
> processes using it. So, taken together, these checks are "is there or 
> might there be in the future a process, other than the one being 
> debugged, using this exact copy of the image rather than loading it from 
> the disk".

I know that, but the discussion was about the SUID bit, and the ability to 
modify the in-core image of a set-uid program being run...

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From jnc at mercury.lcs.mit.edu  Sat May 13 10:44:23 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Fri, 12 May 2017 20:44:23 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>

    > From: Dave Horsfall

    > Err, isn't that the sticky bit, not the setuid bit?

Oh, right you are. I just looked in the code for ptrace(), and assumed that
was it.

The fix is _actually_ in sys1$exec() (in V6) and sys1$getxfile() (in PWB1 and
the MIT system:

	/*
	 * set SUID/SGID protections, if no tracing
	 */

	if ((u.u_procp->p_flag&STRC)==0) {
                if(ip->i_mode&ISUID)
			if(u.u_uid != 0) {
				u.u_uid = ip->i_uid;
				u.u_procp->p_uid = ip->i_uid;
				}

The thing is, this code is identical in V6, PWB1, and MIT system!?

So now I'm wondering - was this really the bug? Or was there some
bug in ptrace I don't see, which was the actual bug that's being
discussed here.

Because is sure looks like this would prevent the exploitation that I
described (start an SUID program under the debugger, then patch the code).

Or perhaps somehow this fix was broken by some other feature,, and that
introduced the exploit?

	  Noel


From random832 at fastmail.com  Sat May 13 10:48:35 2017
From: random832 at fastmail.com (Random832)
Date: Fri, 12 May 2017 20:48:35 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705131024360.67621@aneurin.horsfall.org>
References: <20170512233012.7B9DB18C099@mercury.lcs.mit.edu>
 <alpine.BSF.2.20.1705130937330.67621@aneurin.horsfall.org>
 <1494633169.1599957.975050752.2FB9B2DA@webmail.messagingengine.com>
 <alpine.BSF.2.20.1705131024360.67621@aneurin.horsfall.org>
Message-ID: <1494636515.1613066.975082816.77F24338@webmail.messagingengine.com>



On Fri, May 12, 2017, at 20:26, Dave Horsfall wrote:
> On Fri, 12 May 2017, Random832 wrote:
> 
> > > > 	if (xp->x_count!=1 || xp->x_iptr->i_mode&ISVTX)
> > > > 		goto error;
> > > 
> > > Err, isn't that the sticky bit, not the setuid bit?
> > 
> > The sticky bit makes it keep the image in memory when there are no 
> > processes using it. I assume x_count is determining whether there are 
> > processes using it. So, taken together, these checks are "is there or 
> > might there be in the future a process, other than the one being 
> > debugged, using this exact copy of the image rather than loading it from 
> > the disk".
> 
> I know that, but the discussion was about the SUID bit, and the ability
> to 
> modify the in-core image of a set-uid program being run...

It seems to me that this check is central to being able to (or not)
modify the in-core image of any process at all other than the one being
traced (say, by attaching to a SUID program that has already dropped
privileges, and making changes that will affect the next time it is
run).


From random832 at fastmail.com  Sat May 13 10:51:43 2017
From: random832 at fastmail.com (Random832)
Date: Fri, 12 May 2017 20:51:43 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
Message-ID: <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>

On Fri, May 12, 2017, at 20:44, Noel Chiappa wrote:
> So now I'm wondering - was this really the bug? Or was there some
> bug in ptrace I don't see, which was the actual bug that's being
> discussed here.

Ah. There's the other piece. You start the SUID program under the
debugger, and rather than kicking off the debugger, it simply starts it
non-suid. *However*, in the presence of shared text (either of the two
cases being checked for in the other place), you can make changes to the
text image (e.g. put whatever code you want at the entry point), which
will be reused the *next* time it is started *without* the debugger.


From dave at horsfall.org  Sat May 13 10:55:34 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sat, 13 May 2017 10:55:34 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
Message-ID: <alpine.BSF.2.20.1705131053430.67621@aneurin.horsfall.org>

On Fri, 12 May 2017, Random832 wrote:

> Ah. There's the other piece. You start the SUID program under the 
> debugger, and rather than kicking off the debugger, it simply starts it 
> non-suid. *However*, in the presence of shared text (either of the two 
> cases being checked for in the other place), you can make changes to the 
> text image (e.g. put whatever code you want at the entry point), which 
> will be reused the *next* time it is started *without* the debugger.

Cripes!  I think you're right...  If so, well done!

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From torek at torek.net  Sat May 13 11:17:02 2017
From: torek at torek.net (Chris Torek)
Date: Fri, 12 May 2017 18:17:02 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: Your message of "Fri, 12 May 2017 20:51:43 -0400."
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
Message-ID: <201705130117.v4D1H26I087121@elf.torek.net>

>Ah.  There's the other piece.  You start the SUID program under
>the debugger, and rather than kicking off the debugger, it simply
>starts it non-suid.  *However*, in the presence of shared text
>(either of the two cases being checked for in the other place),
>you can make changes to the text image (e.g.  put whatever code
>you want at the entry point), which will be reused the *next*
>time it is started *without* the debugger.

Right.  Some of this was not a problem with demand-paged files
(4.xBSD) since you would not share swap images.  But there's more,
or rather, *was* more, once people added PT_ATTACH and new system
calls or behavior ... specifically, the setreuid / setregid calls
from 4.2BSD or the saved setuid behavior in System V.  This is one
I personally touched.

Suppose a process starts out setuid or setgid.  This means it has
alternative privileges (maybe super-user, maybe not).  With these
it can do things like open some files or transit some directories.
Afterward, using setreuid() and setregid() in 4.2BSD, the process
can swap its real and effective IDs, or give up its effective UID
or GID entirely, to give up its privileges.  So some program
could, for instance, chdir past a "lock" directory -- this was the
MDQS trick -- and now exist in a tree that it had no access to, or
open a file with secrets, that it no longer has permission to
open.

Once a process had given up any special privileges, though, it
could be ptraced again (via PT_ATTACH).  Now you can swap file
descriptor variables around in it, or otherwise make it do bad
things with the privileges it gained while it was setuid.  On
SysV, with "saved setuid", it is even worse: you can make the
process *regain* privileges and do whatever you want.

The fix we used was an extra bit in the process flags: "process
has had or used privileges". Once set (cleared only on exec()),
the process could no longer be ptraced except by root.  Processes
did -- and still do -- have to be careful about files they may
leave open, or other entities that survive exec().  (One should
also consider mmap() and shared regions.)

Chris


From lm at mcvoy.com  Sat May 13 11:24:15 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Fri, 12 May 2017 18:24:15 -0700
Subject: [TUHS] C declarations.
In-Reply-To: <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
Message-ID: <20170513012415.GZ4341@mcvoy.com>

On Thu, May 11, 2017 at 06:41:47PM -0400, Ron Natalie wrote:
> If had my way, y = x and passing and returning arrays by value would work
> just like every other C type.

Maybe, just maybe, now that makes sense.  But even now that would be a 
perf hit unless you added some magical copy on write semantics like 
Tcl has so it can have pass by value semantics but pass by reference
performance.

Back then, I think the perf hit would have been so bad everyone would be
passing arrays as a reference anyway.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 


From jnc at mercury.lcs.mit.edu  Sat May 13 11:25:30 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Fri, 12 May 2017 21:25:30 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170513012530.EF62518C099@mercury.lcs.mit.edu>

    > From: Random832

    > It seems to me that this check is central to being able to (or not)
    > modify the in-core image of any process at all other than the one being
    > traced (say, by attaching to a SUID program that has already dropped
    > privileges, and making changes that will affect the next time it is
    > run).

Right, good catch: if you have a program that was _both_ sticky and SUID, when
the system is idle (so the text copy in the swap area won't get recycled),
call up a copy under the debugger, patch it, exit (leaving the patched copy),
and then re-run it without the debugger.

I'd have to check the handling of patched sticky pure texts - to see if they
are retained or not.

{Checks code.}

Well, the code to do with pure texts is _very_ different between V6 and
PWB1.

The exact approach above might not work in V6, because the modified (in-core)
copy of pure texts are simply deleted when the last user exits them. But it
might be possible for a slight variant to work; leave the copy under the
debugger (which will prevent the in-core copy from being discarded), and then
run it again without the debugger. That might do it.

Under PWB1, I'm not sure if any variant would work (very complicated, and I'm
fading). There's an extra flag bit, XWRIT, which is set when a pure text is
written into; when the last user stops using the in-code pure text, the
modified text is written to swap.  (It lools like the in-core copy is always
discarded when the last user stops using it.) But the check for sticky would
probably stop a sticky pure-text being modified? But maybe the approach that
seems like it would work under V6 (leave the patched, debugger copy running,
and start a new instance) looks like it should work here too.

So maybe the sticky thing is irrelevant? On both V6 and PWB1, it just needs a
pure text which is SETUID: start under the debugger, patch, leave running, and
start a _new_ copy, which will run the patched version as the SUID user.

      Noel



From ron at ronnatalie.com  Sat May 13 12:45:02 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Fri, 12 May 2017 22:45:02 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <20170513012415.GZ4341@mcvoy.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
Message-ID: <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>

There's no performance "hit" because it doesn't work now.    There's really
no difference performance or computational complexity wise for

char x[4], y[4];
x = y;
and
struct { char a[4]; } x, y;
x = y;

About the only thing it would have broken (and would still break today), is
the fact that function parameters that are defined to be arrays, really pass
pointers.

char x[4];
void foo(char a[4]);
foo(x);

would be costlier than it is now doing the pass by value.    Of course you
could always fudge if you wanted to pass the pointer by actually doing it
that way...
char x[4]
void foo(char *a);
foo(x);

-----Original Message-----
From: Larry McVoy [mailto:lm at mcvoy.com] 
Sent: Friday, May 12, 2017 9:24 PM
To: Ron Natalie
Cc: 'Larry McVoy'; 'David Arnold'; 'The Eunuchs Hysterical Society'
Subject: Re: [TUHS] C declarations.

On Thu, May 11, 2017 at 06:41:47PM -0400, Ron Natalie wrote:
> If had my way, y = x and passing and returning arrays by value would 
> work just like every other C type.

Maybe, just maybe, now that makes sense.  But even now that would be a perf
hit unless you added some magical copy on write semantics like Tcl has so it
can have pass by value semantics but pass by reference performance.

Back then, I think the perf hit would have been so bad everyone would be
passing arrays as a reference anyway.
--
---
Larry McVoy            	     lm at mcvoy.com
http://www.mcvoy.com/lm 



From michael at kjorling.se  Sat May 13 22:20:50 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 13 May 2017 12:20:50 +0000
Subject: [TUHS] C declarations.
In-Reply-To: <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
Message-ID: <20170513122050.GF9980@yeono.kjorling.se>

On 12 May 2017 22:45 -0400, from ron at ronnatalie.com (Ron Natalie):
> About the only thing it would have broken (and would still break today), is
> the fact that function parameters that are defined to be arrays, really pass
> pointers.
> 
> char x[4];
> void foo(char a[4]);
> foo(x);
> 
> would be costlier than it is now doing the pass by value.

Correct me if I am wrong, but _pass by value_ as opposed to _pass by
reference_ requires making a copy, no? That's the whole point, to
allow the callee to poke at the value it is given at will. At the very
least, it would require making a copy when a part of the passed value
is modified. Where is that memory going to come from, when even
malloc() isn't a part of the language but rather a part of the
standard library? Where is all that going to come from if you pass a
large array on a memory-constrained system of specs common back in the
days when C was designed, especially one that lacks virtual memory
support?

What if (yes, somewhat contrived example) we had pass by value
semantics, and foo() is implemented as

    void foo(char a[4])
    {
        a[8] = 123;
    }

What exactly should happen? The function delcaration says we are
interested in four char, but the function uses far more than that. You
could of course invoke the almighty term _undefined behavior_, but you
really have to say _something_. And of course, nothing stops us from
calling that function as

    char x[100];
    foo(x);

To reliably prevent that, it seems to me you'd need to at least amend
arrays to know their own size, especially if you don't want the
compiler to throw its hands in the air as soon as it sees a malloc()
(and recognizes it).

Now consider instead something like

    struct foo_t { char a[4]; char b[10]; };
    union foo_u { char[14] x; struct foo_t s; };
    void foo(foo_u u)
    {
        u.s.a[8] = 123;
    }

(Please excuse any syntax errors there; unions have never been my
strong side.)

While I agree that _you shouldn't be doing that in the first place_,
the fact remains that doing it is not _disallowed_ by the language. It
could be disallowed by the language if the array knows its size, but
to reliably disallow it, you'd need to be able to detect it at compile
time. That would seem to significantly increase the complexity of the
compiler, which of course we wouldn't want on a low-spec system. (I
have a strong suspicion that my microwave oven has more computing
power and memory than some of the systems C was designed to run on...)


> Of course you
> could always fudge if you wanted to pass the pointer by actually doing it
> that way...
> char x[4]
> void foo(char *a);
> foo(x);

...in which case you need to know from somewhere else whether foo()
wants a zero-terminated char array or not. Not impossible -- we
already have a similar situation with, say, memcmp() and strncmp() --
but unless you're willing to introduce some additional syntax, it
seems to me that you lose expressiveness in the language by lumping
the two cases together.

All the above said, _please don't get me wrong_. Pass by value
semantics is a really nice feature to have, and I have been known to
curse at certain languages more than once where the normal behavior is
pass by reference and pass by value requires a good deal of extra
code. But for potentially large (or even undefined-size) types, pass
by value comes at a significant cost in terms of memory, and even the
copying itself. Copy-on-write solves some of that but requires
elaborate bookkeeping instead, to know what goes where (it's at least
one additional layer of indirection), and risks fragmentation. For
simplicity and predictability, it's hard to beat pass by reference.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From tfb at tfeb.org  Sat May 13 22:35:49 2017
From: tfb at tfeb.org (Tim Bradshaw)
Date: Sat, 13 May 2017 13:35:49 +0100
Subject: [TUHS] C declarations.
In-Reply-To: <20170513122050.GF9980@yeono.kjorling.se>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
Message-ID: <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>

On 13 May 2017, at 13:20, Michael Kjörling <michael at kjorling.se> wrote:
> 
> Correct me if I am wrong, but _pass by value_ as opposed to _pass by
> reference_ requires making a copy, no? That's the whole point, to
> allow the callee to poke at the value it is given at will. At the very
> least, it would require making a copy when a part of the passed value
> is modified. Where is that memory going to come from, when even
> malloc() isn't a part of the language but rather a part of the
> standard library? Where is all that going to come from if you pass a
> large array on a memory-constrained system of specs common back in the
> days when C was designed, especially one that lacks virtual memory
> support?

Are there languages that copy arrays in function calls defaultly?  Perhaps Fortran has some convention that allows that but I doubt it gets used very much, because it would be insane in most cases:  COMPUTE_MEAN_TEMPERATURE(ATMOS) is really *not* going to work very well if it involves copying the ATMOS array.

From michael at kjorling.se  Sat May 13 22:42:47 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Sat, 13 May 2017 12:42:47 +0000
Subject: [TUHS] C declarations.
In-Reply-To: <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
Message-ID: <20170513124247.GG9980@yeono.kjorling.se>

On 13 May 2017 13:35 +0100, from tfb at tfeb.org (Tim Bradshaw):
> Are there languages that copy arrays in function calls defaultly?
> Perhaps Fortran has some convention that allows that but I doubt it
> gets used very much, because it would be insane in most cases:
> COMPUTE_MEAN_TEMPERATURE(ATMOS) is really *not* going to work very
> well if it involves copying the ATMOS array.

I'm not completely sure about arrays, but at least Java has pass by
reference in some cases where you might expect pass by value. IIRC
function return values is a prime example.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From hellwig.geisse at mni.thm.de  Sat May 13 23:46:00 2017
From: hellwig.geisse at mni.thm.de (Hellwig Geisse)
Date: Sat, 13 May 2017 15:46:00 +0200
Subject: [TUHS] C declarations.
In-Reply-To: <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
Message-ID: <1494683160.2218.223.camel@mni.thm.de>

On Sa, 2017-05-13 at 13:35 +0100, Tim Bradshaw wrote:
> Are there languages that copy arrays in
> function calls defaultly?

Yes, Pascal is an example. If you do *not* want
an argument to be copied, its corresponding
parameter name must be preceded by the keyword
"var". The "User Manual and Report" explicitly
warns about the inefficiency incurred when
calling functions with parameters of structured
types (arrays or records) without "var".

Hellwig


From steve at quintile.net  Sun May 14 01:25:16 2017
From: steve at quintile.net (Steve Simon)
Date: Sat, 13 May 2017 16:25:16 +0100
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
Message-ID: <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>


hi,

this is (IMHO) a rather subtle bug,
the ones i remember where rather simpler. is it ok to discuss ancient security holes or is that still bad manners?

-Steve
 


From steve at sk2.org  Sun May 14 01:36:48 2017
From: steve at sk2.org (Stephen Kitt)
Date: Sat, 13 May 2017 17:36:48 +0200
Subject: [TUHS] C declarations.
In-Reply-To: <20170513124247.GG9980@yeono.kjorling.se>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
Message-ID: <20170513173648.2900e4cd@heffalump.sk2.org>

On Sat, 13 May 2017 12:42:47 +0000, Michael Kjörling <michael at kjorling.se>
wrote:
> On 13 May 2017 13:35 +0100, from tfb at tfeb.org (Tim Bradshaw):
> > Are there languages that copy arrays in function calls defaultly?
> > Perhaps Fortran has some convention that allows that but I doubt it
> > gets used very much, because it would be insane in most cases:
> > COMPUTE_MEAN_TEMPERATURE(ATMOS) is really *not* going to work very
> > well if it involves copying the ATMOS array.  
> 
> I'm not completely sure about arrays, but at least Java has pass by
> reference in some cases where you might expect pass by value. IIRC
> function return values is a prime example.

Technically (and this is serious nit-picking), Java is always pass-by-value,
but the value of an array or object *variable* is the reference to the array
or object, not the array or object itself. So the behaviour is
pass-by-reference for arrays and objects (although that’s a simplification
too, and unhelpful in some instances).

The same applies to return values: primitive types are returned by value
(copied), arrays and objects are returned via their reference (which is
copied too).

Regards,

Stephen


From clemc at ccc.com  Sun May 14 02:55:29 2017
From: clemc at ccc.com (Clem Cole)
Date: Sat, 13 May 2017 12:55:29 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
 <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
Message-ID: <CAC20D2NHZ0kzuGJmtBFxXwQSiT-33KiokV2Gi3pEyf6m5Nqp3A@mail.gmail.com>

On Sat, May 13, 2017 at 11:25 AM, Steve Simon <steve at quintile.net> wrote:

> hi,
>
> this is (IMHO) a rather subtle bug,
> the ones i remember where rather simpler. is it ok to discuss ancient
> security holes or is that still bad manners?
>
​Speaking for myself.....   I clearly don't think it is bad manners​ as
this stage - I brought it up!E
It was a different time when that occurred.  Today, I think *the general
security community*** pretty lives by the rules of if you find something,
notify the folks that fix it as quickly as possible and try to get a patch
out and figure out how to get that patch out.   Then make damned sure the
whole is well documented and published so: a) do we can test for it in the
wild, b) make sure it does not happen again.

It actually has always impressed me at how good UNIX was (is) when you
really get down to it.  IMHO, was less the 'thousand eyeballs'' and more
the 'eye balls that all of cared, could do something about it and most
importantly actually understood' the 'calculus' of the different problems
were want made UNIX secure and as good if not better than many 'commercial'
systems than its contemporaries.  *i.e.* the UNIX schemes used sensible
 human based security processes/mechanisms combined with basic math &
physics ( technology if you will) - as the higher order bits, not being
secret or obscure to protect.

Were there mistakes, yup.   But frankly, VMS had as many if not more and
some of them were far, far worse.   IBM's OS were considered good, but
their were documented exploits in the news there too.

Clem


** I note 'security community' because not all firm buy into this behavior.
  I speak for myself.   In the last few weeks my own employer (Intel)
recent has been mixed up in a bit over played issue with server chips sets,
AMT and Winders [its not my area/group etc but as I under the issue, the
bug does not seem to effect UNIX flavors nor systems that do not use AMT -
which is a server thingy].   Some outside of Intel people are have
complained that folks that own the bug @ my employer has been less that
forth coming.   I'll not defend nor comment because it's not mine to
comment on, other than to state I personally take an attitude of trying to
say a much as I can and when I am in a position for my job I will and do.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170513/f75501ad/attachment.html>

From pechter at gmail.com  Sun May 14 03:19:42 2017
From: pechter at gmail.com (William Pechter)
Date: Sat, 13 May 2017 13:19:42 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAC20D2NHZ0kzuGJmtBFxXwQSiT-33KiokV2Gi3pEyf6m5Nqp3A@mail.gmail.com>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
 <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
 <CAC20D2NHZ0kzuGJmtBFxXwQSiT-33KiokV2Gi3pEyf6m5Nqp3A@mail.gmail.com>
Message-ID: <1038e644-cef1-9424-f6fa-288941033bff@gmail.com>

Clem Cole wrote:
>
> On Sat, May 13, 2017 at 11:25 AM, Steve Simon <steve at quintile.net 
> <mailto:steve at quintile.net>> wrote:
>
>     hi,
>
>     this is (IMHO) a rather subtle bug,
>     the ones i remember where rather simpler. is it ok to discuss
>     ancient security holes or is that still bad manners?
>
> ​Speaking for myself.....   I clearly don't think it is bad manners​ 
> as this stage - I brought it up!E
> It was a different time when that occurred.  Today, I think /the 
> general security community**/ pretty lives by the rules of if you find 
> something, notify the folks that fix it as quickly as possible and try 
> to get a patch out and figure out how to get that patch out.   Then 
> make damned sure the whole is well documented and published so: a) do 
> we can test for it in the wild, b) make sure it does not happen again.
>
> It actually has always impressed me at how good UNIX was (is) when you 
> really get down to it.  IMHO, was less the 'thousand eyeballs'' and 
> more the 'eye balls that all of cared, could do something about it and 
> most importantly actually understood' the 'calculus' of the different 
> problems were want made UNIX secure and as good if not better than 
> many 'commercial' systems than its contemporaries. /i.e./ the UNIX 
> schemes used sensible  human based security 
> processes/mechanisms combined with basic math & physics ( technology 
> if you will) - as the higher order bits, not being secret or obscure 
> to protect.
>
The problem is once you got past "One true Unix" you were left hoping 
the vendor fixed their bugs.
I saw somethings on Solaris and HPUX which were pretty much as bad as 
Windows.

The thing about Research Unix was that the underlying security structure 
was well designed.
VMS wasn't too bad either.  The problem was the stuff layered on it.

When VMS went to 3.6 or so a friend of mine was almost fired by DEC for 
randomly testing boxes looking
to see DEC's internal boxes weren't running System/Manager, 
Field/Service and UETP/UETP User/password
combinations.  DEC had just implemented new security features and alerts 
and Mitnick had just recently
penetrated them (IIRC).  Next thing you know corporate security was all 
over my buddy who was just killing
time on night shift  temporarily covering someone's vacation time.

It was interesting to see the SysV security enhanced Unix from AT&T at 
Pyramid -- who was migrating to
SVR4 from their BSD/SysV hybrid.  ACL's, split root/system and security 
mgr stuff which had been added
to get VMS to C and B2...  Some of these things had me wondering if any 
commercial sites would implement
two sign-ins to authorize special root-type actions on an os.

| Were there mistakes, yup.   But frankly, VMS had as many if not more 
and some of them were far, far worse.   IBM's OS were considered good, 
but their were documented exploits in the news there too.

The loginout.exe one was bad.  Were there any structural ones past v3.6?
>
> Clem
>
>
> ** I note 'security community' because not all firm buy into this 
> behavior.   I speak for myself.   In the last few weeks my own 
> employer (Intel) recent has been mixed up in a bit over played issue 
> with server chips sets, AMT and Winders [its not my area/group etc but 
> as I under the issue, the bug does not seem to effect UNIX flavors nor 
> systems that do not use AMT - which is a server thingy].   Some 
> outside of Intel people are have complained that folks that own the 
> bug @ my employer has been less that forth coming.   I'll not defend 
> nor comment because it's not mine to comment on, other than to state I 
> personally take an attitude of trying to say a much as I can and when 
> I am in a position for my job I will and do.
>
Actually... I'd think AMT is an automated remote IT  Management thing 
rather than a server thing,
since it exists on all the business Thinkpads from my T61's Core 2's up 
to the T420 i5.  They couldn't
be considered servers except they do support Samba and NFS and ssh. They 
also dual boot which
is a major part of the risk.

Sorry for the pedantic add... but I just remediated my 5 laptops for 
crap that should've been fixed with
new vendor software -- but they can't be bothered.




-- 
Digital had it then.  Don't you wish you could buy it now!
pechter-at-gmail.com  http://xkcd.com/705/



From random832 at fastmail.com  Sun May 14 05:05:24 2017
From: random832 at fastmail.com (Random832)
Date: Sat, 13 May 2017 15:05:24 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <20170513122050.GF9980@yeono.kjorling.se>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
Message-ID: <1494702324.1901172.975588504.1BA76452@webmail.messagingengine.com>

On Sat, May 13, 2017, at 08:20, Michael Kjörling wrote:
> Correct me if I am wrong, but _pass by value_ as opposed to _pass by
> reference_ requires making a copy, no? That's the whole point, to
> allow the callee to poke at the value it is given at will.

If that were the issue, the callee could simply automatically make the
copy if-and-only-if it needs it.

The reason that it can't be done without making a copy is, rather, that
there might be some *other* reference in play, through which the
original array could be modified (thus improperly changing what the
callee sees).


From random832 at fastmail.com  Sun May 14 05:08:12 2017
From: random832 at fastmail.com (Random832)
Date: Sat, 13 May 2017 15:08:12 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
Message-ID: <1494702492.1902154.975591160.507FCE4C@webmail.messagingengine.com>

On Sat, May 13, 2017, at 08:35, Tim Bradshaw wrote:
> Are there languages that copy arrays in function calls defaultly? 
> Perhaps Fortran has some convention that allows that but I doubt it gets
> used very much, because it would be insane in most cases: 
> COMPUTE_MEAN_TEMPERATURE(ATMOS) is really *not* going to work very well
> if it involves copying the ATMOS array.

Fortran, for the record, passes nearly everything by reference - to call
a fortran function from C (or to make a C function callable from
fortran) you've got to mangle the name and change almost everything to
pointers.


From dave at horsfall.org  Sun May 14 09:01:41 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sun, 14 May 2017 09:01:41 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
 <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
Message-ID: <alpine.BSF.2.20.1705140854400.67621@aneurin.horsfall.org>

On Sat, 13 May 2017, Steve Simon wrote:

> this is (IMHO) a rather subtle bug, the ones i remember where rather 
> simpler. is it ok to discuss ancient security holes or is that still bad 
> manners?

Bring it on :-)  Any systems still vulnerable (and exposed to the 
Internet) deserves all that they get.

For example, I think I was the first one to mention the SPL bug here (note 
that I didn't find it); I'm still trying to find that little program which 
I think was published in a Usenix newsletter and exploited by, err, yours 
truly...

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From richard at inf.ed.ac.uk  Sun May 14 09:11:21 2017
From: richard at inf.ed.ac.uk (Richard Tobin)
Date: Sun, 14 May 2017 00:11:21 +0100 (BST)
Subject: [TUHS] C declarations.
In-Reply-To: Random832's message of Sat, 13 May 2017 15:08:12 -0400
Message-ID: <20170513231121.0DB3A1274787@macaroni.inf.ed.ac.uk>

> Fortran, for the record, passes nearly everything by reference

Sort of.  The Fortran 77 standard imposes restrictions that appear to
be intended to allow the implementation to pass by value-and-result
(i.e. values are copied in, and copied back at return).  In particular
it disallows aliasing that would allow you to distinguish between
the two methods:

  If a subprogram reference causes a dummy argument in the referenced
  subprogram to become associated with another dummy argument in the
  referenced subprogram, neither dummy argument may become defined
  during execution of that subprogram.

http://www.fortran.com/F77_std/rjcnf-15.html#sh-15.9.3.6

-- Richard

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



From dave at horsfall.org  Sun May 14 09:21:09 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sun, 14 May 2017 09:21:09 +1000 (EST)
Subject: [TUHS] C declarations.
In-Reply-To: <1494702492.1902154.975591160.507FCE4C@webmail.messagingengine.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <1494702492.1902154.975591160.507FCE4C@webmail.messagingengine.com>
Message-ID: <alpine.BSF.2.20.1705140913210.67621@aneurin.horsfall.org>

On Sat, 13 May 2017, Random832 wrote:

> Fortran, for the record, passes nearly everything by reference [...]

And if you passed a constant and the local argument was modified, so was 
the (single instance of) constant...

Actually, wasn't FORTRAN value-result i.e. all args were copied straight 
back again, or am I confusing that with ALGOL?  Blimey, it's been over 40 
years since I last submitted a card deck.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From dave at horsfall.org  Sun May 14 09:34:58 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sun, 14 May 2017 09:34:58 +1000 (EST)
Subject: [TUHS] Old Unix vulnerabilities
Message-ID: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>

OK, I'll kick it off.

A beauty in V6 (and possibly V7) was discovered by the kiddies in Elec 
Eng; by sending a signal with an appropriately-crafted negative value (as 
determined from inspecting <user.h>) you could overwrite u.u_uid with 
zero...  Needless to say I scrambled to fix that one on my 11/40 network!

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From stewart at serissa.com  Sun May 14 11:59:57 2017
From: stewart at serissa.com (Lawrence Stewart)
Date: Sat, 13 May 2017 21:59:57 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <20170513124247.GG9980@yeono.kjorling.se>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
Message-ID: <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>

I am not troubled at all by C’s conflation of arrays and pointers. That has always seemed natural - it is what is really going on under the covers.*

I’m the sort of person who fiddles with the source code until the compiler generates the code I want.  C is great for this.  Not always, but C compilers usually do the straightforward and obvious thing and generate the code you would write if you had to do it in assembler, only the compiler keeps track of the stupid details like frame offsets and inventing symbol names for branch targets.

I’ve written substantive things in Java, Modula-3, common lisp, python, and so forth, but when I write performance sensitive code, I do it in C**

I tend to work on communications code and when you are counting cycles in some HPC library, you write in C but think of every load, store, and shift.  My affinity for the language began the same way it probably did for a lot of people here: when I realized that *i++ was exactly a PDP11 addressing mode.

When the assistant rector at my church had a meeting about sacred texts, I brought in K&R (“the New Testament” second edition) and my prized copies of the BSTJ Unix issues.

-L

PS  

* - But I have never been able to remember the syntax for function pointers.  I always “man qsort” to refresh my memory.

** - If the code uses multidimensional arrays or subarrays, then maybe FORTRAN, because really x[i*stride+j] gets really old.



From dave at horsfall.org  Sun May 14 12:23:38 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Sun, 14 May 2017 12:23:38 +1000 (EST)
Subject: [TUHS] C declarations.
In-Reply-To: <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
Message-ID: <alpine.BSF.2.20.1705141220290.67621@aneurin.horsfall.org>

On Sat, 13 May 2017, Lawrence Stewart wrote:

> * - But I have never been able to remember the syntax for function 
> pointers.  I always “man qsort” to refresh my memory.

"cdecl" is your friend:

    % cdecl
    Type `help' or `?' for help
    declare foo as pointer to function (int, double) returning void
    void (*foo)(int , double )

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

From bakul at bitblocks.com  Sun May 14 14:24:11 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Sat, 13 May 2017 21:24:11 -0700
Subject: [TUHS] C declarations.
In-Reply-To: Your message of "Sat, 13 May 2017 21:59:57 EDT."
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
Message-ID: <20170514042411.93CCB124AEA4@mail.bitblocks.com>

On Sat, 13 May 2017 21:59:57 EDT Lawrence Stewart <stewart at serissa.com> wrote:
>
> * - But I have never been able to remember the syntax for function pointers.  > I always "man qsort" to refresh my memory.

The way I remember:

given *x[] or *x(), x /sticks/ to the /right/ first. Thus

int *x[];	// x is array of ptr to int
int *x();	// x is a function returning ptr to int

If you don't want a var to stick to its right, separate using
using parentheses.

int (*x)[];	// x is a ptr to array of ints
int (*x)();	// x is a ptr to function returning int
int *(*x)[];	// x is a ptr to array of ptrs to int

This sort of also works for multiple variables in one declaration. Given

int *x, y;	// * /sticks/ to the right (x) first, so not available for y.

Now if they'd allowed parenthesizing the type, as in

(int *)x, y;

we would see that both x & y are of type int *. That would've also allowed
declaring multiple vars in one declaration where the previous rule applies!

(int*)(*f,*g)();

Part of the confusion is * is tacked on at the front while []
& () at the back of a variable.  Someone (Chandy?) had
proposed unifying this syntax but it didn't go anywhere.  I
think the author used @ in a suffix place though I don't
recall any other details.  But if you put the var all the way
to the right and read a declaration right to left, it works:

int*[] x	// x is a array of ptr to ints
int() f		// f is a function returning it
int*()* f	// f is a ptr to function returning ptr to int
int*()* f,g	// f & g are ptrs to function returning ptr to int

This is only slightly weird if you are used to the current syntax
but you can easily retrain yourself.

IMHO, where evolution of C, Scheme and most programming
languages goes wrong is doing evolutionary design by
committee.  Most users grasp what is easier to use vs what is
hard but they don't have the aesthetic sense or imagination or
training to make things simple. And in large groups popularity
or force of personality or some other irrelevant attribute
wins not aesthetics.  Language evolution IMHO must be done by
benevolent dictatorship or a guild of like minded people who
have worked together for a long time!  Go seems to have
adapted that style....


From tytso at mit.edu  Sun May 14 14:30:24 2017
From: tytso at mit.edu (Theodore Ts'o)
Date: Sun, 14 May 2017 00:30:24 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170511222547.GJ4341@mcvoy.com>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
Message-ID: <20170514043024.pezg5eqslpcqsd5m@thunk.org>

On Thu, May 11, 2017 at 03:25:47PM -0700, Larry McVoy wrote:
> This is one place where I think Linux kicked Unix's ass.  And I am not
> really sure how they did it, I have an idea but am not positive.  Unix
> file systems up through UFS as shipped by Sun, were all vulnerable to
> what I call the power out test.  Untar some big tarball and power off
> the machine in the middle of it.  Reboot.  Hilarity ensues (not).
> 
> You were dropped into some stand alone shell after fsck threw up its
> hands and it was up to you to fix it.  Dozens and dozens of errors.
> It was almost always faster to go to backups because figuring that 
> stuff out, file by file (which I have done more than once), gets you
> to the point that your run "fsck -y" and go poke at lost+found when
> fsck is done, realize that there is no hope, and reach for backups.
> 
> Try the same thing with Linux.  The file system will come back, starting
> with, I believe, ext2.
> 
> My belief is that Linux orders writes such that while you may lose data
> (as in, a process created a file, the OS said it was OK, but that file 
> will not be in the file system after a crash), but the rest of the file 
> system will be consistent.  I think it's as if you powered off the
> machine a few seconds earlier than you actually did, some stuff is in
> flight and until they can write stuff out in the proper order you may
> lose data on a hard reset.

So the story is a bit complicated here, and may be an example of
"worse is better" --- which is ironically one of those things which is
used as an explanation for why BSD/Unix won ever though the Lisp was
technically superior[1] --- but in this case, it's Linux that did
something "dirty", and BSD that did something that was supposed to be
the "better" solution.

[1] https://www.jwz.org/doc/worse-is-better.html

So first let's talk about ext2 (which indeed, does not have file
system journalling; that came in ext3).  The BSD Fast File System goes
to a huge amount of effort to make sure that writes are sent to the
disk in exactly the right order so that fsck can actually fix things.
This requires that the disk not reorder writes (e.g., write caching is
disabled or in write-through mode).  Linux, in ext2, didn't bother
with trying to get the write order correct at all.  None.  Nada.  Zip.
Writes would go out in whatever order dictated by the elevator
scheduler, and so on a power failure or a kernel crash, the order in
which metadata writes would be sent to the disk was completely
unconstrained.

Sounds horrible, right?  In many ways, it was.  And I lost count of
how often NetBSD and FreeBSD users would talk about how primitive and
horrible ext2 was in comparison to FFS, which had all of this
excellent engineering work to make sure writes happened in the correct
order such that fsck was guaranteed to always be able to fix things.

So why did Linux get away with it?  When I wrote the fsck for ext2, I
knew that anything can and would happen, so it was implemented so that
it was extremely paranoid about not ever losing any data.  And if
there was a chance that an expert could recover the data, e2fsck would
stop and ask the system administrator to take a look.  In the case
that the user ran with fsck -y, the default was drop files into
lost+found, where as in some cases with the FFS fsck, it "knew" that
in a particular case, the order in which writes were staged out the
right thing to do was to let the unlink complete, so it would let the
refcount go to zero, or stay at zero.

The other thing that we did in Linux is that I made sure we had a
highly functional "debugfs" tool.  This tool served two purposes.  The
first was it made it very easy for me to creat a regression test suite
for fsck.  As far as I know, none of the other major file systems at
the time had an fsck with a regression test suite --- and I was
religious about adding tests as I added functionality, and as I fixed
bugs.  The debugfs tool made it easy for me to create test case file
systems that was corrupted in various interesting ways.  The other use
of debugfs was that it made it easy for experts to do file system
recovery after a crash, if there was some really precious file that
they needed to try to recover.

So this is why this is a great example of "worse is better".  In Linux,
ext2 was ***incredibly*** sloppy about how it handled write ordering
--- it didn't do anything at all.  But as a consequence we developed
tools that were extremely good to compensate, and in practice, it was
extremely rare (although it did happen on occasion) that files would
get lost or the file systme could end up in a state where fsck would
not be able to recover without manual intervention by a system
administrator using debugfs.

But the other thing to note here is that in the PC era, most disk
drives ran with write caching enabled, with writeback caching so that
the hard drive could do its own elevator shceduling.  So having a file
system that very carefully scheduled writes to make sure they happened
in the write order didn't help you a *bit* unless you configured your
hard drive to disable writeback caching --- at which point you would
take a massive speed hit.

This is ultimately also one weaknesses of Soft Updates --- it requires
that you disable writeback caching, since it works by letting the OS
control the order in which writes hit stable storage.  With
journalling you don't have to do that; but the tradeoff is that when
you do a journal commit, you need typically two cache flush
operations.  (Or a cache flush followed by a FUA write of the commit
block, if the disk supports FUA.)



There is another example of how Linux embraced the "worse is better"
philosophy in ext3, and that has to do with how we do journalling.
The sophisticated way to do journalling is to do logical journalling.
This is where what you write in the in journal is "set bit XXX in the
allocation bitmap", or "update the mtime to YYYY".  And in this way,
you can batch multiple file system operations into a single block
written to the journal.  Solaris/UFS and Irix uses this much more
sophisticated form of journalling.  (Actually, older versions of
Solaris did use volume-level journalling, which is basically what
ext3/ext4 uses, but they upgraded to the much more "right", more
advanced thing, which is logical journalling.)

Ext3 uses phyiscal, or volume-level journalling.  This journalling
works on the block level --- so if we flip a bit in an allocation
bitmap, we log the entire 4k block to the journal.  By default, we
only do a journal commit every five seconds (unless an fsync happens
first), so there could be multiple changes to a single inode table
blocks that can be batched together, but it's still true that for a
given metadata-heavy workload, a file system which uses logical
journalling will tend require many fewer blocks written to the journal
than a file system such as ext3/ext4 which uses physical block
journalling.

Why did Linux get away with it?  Number one, most workloads aren't
really modify metadata all _that_ intensively, and 12k of sequential
writes versus 32k of sequential writes doesn't actually take that much
more time.  Secondly, Ted's law of PC-class hardware ("most PC-class
hardware is crap") comes into play, and turns physical journalling
into an advantage.  PC class hardware tends not to have power fail
interrupts, and when power drops, and the voltage levels on the power
rails start drooping, DRAM tends to go insane and starts returning
garbage long before the DMA engine and the hard drive stops
functioning.

So if your system is doing logical journalling, after the file system
commits a transaction, it will start writing the inode table block to
the permanent location on disk.  If at that point you get a power
drop, garbage can get written to the inode table block, and if the
file system is using logical journalling, on reboot the mtime field
can get updated from the logical journal --- but the rest of the inode
table block is still garbage.

In contrast, since ext3 was using physical block journalling, even if
various metadata blocks get corrupted due to writes from failing DRAM
during a power drop, when we replay the journal, this will restore the
entire metadata block, and Things Just Worse.

I have talked to an XFS engineer from SGI, and this was definitely a
thing which SGI discovered the hard way.  After they discovered this
problem, they added extra capacitors to the power supply, added a
power fail interrupt, and taught Irix so that when the power fail
interrupt was triggered, it would frantically cancel DMA transfers in
order to avoid this problem.  I do not know how many of the other
Legacy Unix systems figured out this failure mode --- and I can't
claim that we were brilliant enough to design a system to avoid this
problem.  It just so happened that the brute-force design that we
chose was very well suited for crappy (but way cheaper than a Sun Fire
E10k :-) PC-class hardware.

> I copied Ted, who had his fingers deep in that code, maybe he can correct
> me where I got it wrong.  Details aside, I think this is a place where
> Linux moved the state of the art significantly forward.  There are other
> places but this one is a big deal IMHO, maybe the biggest deal.

So I'm not really sure we can claim to have "moved the state of the
art".  There certainly wasn't any brilliant computer science
innovations here.  That sort of thing is more like Soft Updates, of
which Valerie Aurora (formerly Henson) once wrote,

   "I've read this paper at least 15 times, and each time I when get
   to page 7, I'm feeling pretty good and thinking, "Yeah, okay, I
   must be smarter now than the last time I read this because I'm
   getting it this time," - and then I turn to page 8 and my head
   explodes." --- https://lwn.net/Articles/339337/

I will be the first to admit that with ext2/ext3/ext4, especially in
the early days, it was much more about brute force engineering, and
regression testing, and much less about "moving the state of the art".
Certainly those of us who were working on Linux weren't trying to get
papers published in peer reviewed journals or conferences!  (And I've
always thought that Greg Ganger was _way_ smarter than I.  :-)

And if the Lisp Machine hackers looked down on BSD, and complained
that BSD adopted the "Worse is Better" philosophy, while Lisp strived
for the true, elegant, Correct technical solution, it's perhaps
especially interesting to consider that if anything, Linux was an even
more radical example of the "Worse is Better" philosophy.

Cheers,

					- Ted

P.S.  There is yet another example of "Worse is Better" in how Linux
had PCMCIA support several years before FreeBSD/NetBSD.  However, if
you ejected a PCMCiA card in a Linux system, there was a chance (in
practice it worked out to be about in 1 in 5 times for a WiFI card, in
my experience) that the system would crash.  The *BSD's took a good
2-3 years longer to get PCMCIA support, but when they did, it was rock
solid.  Of course, if you are a laptop user, and are happy to keep
your 802.11 PCMCIA card permanently installed, guess which OS you were
likely to prefer --- "sloppy but works, mostly", or "it'll get there
eventually, and will be rock solid when it does, but zip, nada, right now"?


> 
> --lm
> 
> On Thu, May 11, 2017 at 04:37:29PM -0400, Ron Natalie wrote:
> > I remember the pre-fsck days.   It was part of my test to become an operator at the UNIX site at JHU that I could run the various manual checks.
> > 
> > The V6 file system wasn???t exactly stable during crashes (lousy database behavior), so there was almost certainly something to clean up.
> > 
> >  
> > 
> > The first thing we???d run was icheck.   This runs down the superblock freelist and all the allocated blocks in the inodes.     If there were missing blocks (not in a file or the free list), you could use icheck ???s
> > 
> > to rebuild it.    Similarly, if you had duplicated allocations in the freelist or between the freelist and a single file.   Anything more complicated required some clever patching (typically, we???d just mount readonly, copy the files, and then blow them away with clri).
> > 
> >  
> > 
> > Then you???d run dcheck.   As mentioned dcheck walks the directory path from the top of the disk counting inode references that it reconciles with the link count in the inode.   Occasionally we???d end up with a 0-0 inode (no directory entires, but allocated???typically this is caused by people removing a file while it is still open, a regular practice of some programs for their /tmp files.).    clri again blew these away.
> > 
> >  
> > 
> > Clri wrote zeros all over the inode.   This had the effect of wiping out the file, but it was dangerous if you got the i-number wrong.    We replaced it with ???clrm??? which just cleared the allocated bit, a lot easy to reverse.
> > 
> >  
> > 
> > If you really had a mess of a file system, you might get a piece of the directory tree broken off from a path to the root.   Or you???d have an inode that icheck reported dups.   ncheck would try to reconcile an inumber into an absolute path.
> > 
> >  
> > 
> > After a while a program called fsdb came around that allowed you to poke at the various file system structures.    We didn???t use it much because by the time we had it, fsck was fast on its heals.
> > 
> 
> -- 
> ---
> Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 


From random832 at fastmail.com  Sun May 14 15:52:30 2017
From: random832 at fastmail.com (Random832)
Date: Sun, 14 May 2017 01:52:30 -0400
Subject: [TUHS] Old Unix vulnerabilities
In-Reply-To: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>
References: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>
Message-ID: <1494741150.709655.975854064.7B3C6781@webmail.messagingengine.com>

On Sat, May 13, 2017, at 19:34, Dave Horsfall wrote:
> OK, I'll kick it off.
> 
> A beauty in V6 (and possibly V7) was discovered by the kiddies in Elec 
> Eng; by sending a signal with an appropriately-crafted negative value (as 
> determined from inspecting <user.h>) you could overwrite u.u_uid with 
> zero...  Needless to say I scrambled to fix that one on my 11/40 network!

V7 fixes it by changing the if(sig >= NSIG) in psignal to cast it to
unsigned. Kill still doesn't validate its parameter until SysIII and
4BSD. PWB1 does not have the fix.


From random832 at fastmail.com  Sun May 14 16:11:07 2017
From: random832 at fastmail.com (Random832)
Date: Sun, 14 May 2017 02:11:07 -0400
Subject: [TUHS] Old Unix vulnerabilities
In-Reply-To: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>
References: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>
Message-ID: <1494742267.712158.975859880.2F405BBE@webmail.messagingengine.com>

On Sat, May 13, 2017, at 19:34, Dave Horsfall wrote:
> OK, I'll kick it off.

Oh, and since we're doing this... I independently noticed a buffer
overflow vulnerability in mkdir, and later discovered that someone else
had actually published a working exploit for the same bug... in 2004.

http://archive.cert.uni-stuttgart.de/bugtraq/2004/06/msg00035.html


From scj at yaccman.com  Sun May 14 16:12:45 2017
From: scj at yaccman.com (Steve Johnson)
Date: Sat, 13 May 2017 23:12:45 -0700
Subject: [TUHS] C declarations.
In-Reply-To: <20170514042411.93CCB124AEA4@mail.bitblocks.com>
Message-ID: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>

I can't resist jumping in here.   My main point is that system coding
is a rather different beast than most application programming.  An
operating system must be able to handle memory allocation requests
from a user without knowing any details of what is requested.  Dennis
explicitly left out a heap and garbage collection code in C because he
didn't want OS code to be burdened with it.  Similarly with a "real"
string type (with concatenation, etc.).

An example that speaks to me is that of creating a symbol table.
 This isn't kernel programming, but consider what is needed:

	* A structure that holds the individual table entries
	* A tree of lookup tables that reflects the scoping rules of the
language -- several tables might need to be searched to find an entry
	* A function that looks up a name in a scope, returns the entry if
it's defined, and adds a new entry in the appropriate scope if it
isn't there yet.

Statistically a majority of the calls do not change the symbol table,
but some do, and you can't tell in advance.  And in the same way,
many of the possible scopes where a variable might reside need not be
searched.

So how much of this table should we copy into the lookup function?
 The answer seems obvious to me -- none of it...

Before Unix, Dennis and I and several other people worked on a
symbolic algebra system written in FORTRAN.  Dennis wrote a dynamic
storage allocator, which was a real feat in FORTRAN, complete with
garbage collector.  The goal was to have the program work on the 6
major manufacturers' FORTRANs.  We got it up and running on many of
the systems, but did not have easy access to OS 360.   Finally, we
were granted the third shift on the OS 360, and ran our program (that
had become rather solid by this time...).   It was a disaster -- the
brogram ran for a few second and then the OS dies and could not be
restarted (turns out the core dump area allotted on the disc was not
big enough to hold our program).  

The problem turned out to be that the IBM compiler did copy in/copy
out argument passing, while all the other compilers passed by
reference.  If you passed in an argument  that was dynamically
allocated, and the system did a garbage collection, all the elements
that were still live were moved around but kept their contents
intact... until the function returned, and tried to copy the changed
value out _where that block used to be at the time of the call._

It took us about 6 months to rewrite the system so we didn't do that
-- we had to allocate everything that would be needed first, then do
the work, and then free things...

None of us were fans of copy in/copy out after that experience...

Steve

----- Original Message -----
From: "Bakul Shah" <bakul at bitblocks.com>
To:"Lawrence Stewart" <stewart at serissa.com>
Cc:"tuhs" <tuhs at minnie.tuhs.org>
Sent:Sat, 13 May 2017 21:24:11 -0700
Subject:Re: [TUHS] C declarations.

 On Sat, 13 May 2017 21:59:57 EDT Lawrence Stewart
<stewart at serissa.com> wrote:
 >
 > * - But I have never been able to remember the syntax for function
pointers. > I always "man qsort" to refresh my memory.

 The way I remember:

 given *x[] or *x(), x /sticks/ to the /right/ first. Thus

 int *x[]; // x is array of ptr to int
 int *x(); // x is a function returning ptr to int

 If you don't want a var to stick to its right, separate using
 using parentheses.

 int (*x)[]; // x is a ptr to array of ints
 int (*x)(); // x is a ptr to function returning int
 int *(*x)[]; // x is a ptr to array of ptrs to int

 This sort of also works for multiple variables in one declaration.
Given

 int *x, y; // * /sticks/ to the right (x) first, so not available for
y.

 Now if they'd allowed parenthesizing the type, as in

 (int *)x, y;

 we would see that both x & y are of type int *. That would've also
allowed
 declaring multiple vars in one declaration where the previous rule
applies!

 (int*)(*f,*g)();

 Part of the confusion is * is tacked on at the front while []
 & () at the back of a variable. Someone (Chandy?) had
 proposed unifying this syntax but it didn't go anywhere. I
 think the author used @ in a suffix place though I don't
 recall any other details. But if you put the var all the way
 to the right and read a declaration right to left, it works:

 int*[] x // x is a array of ptr to ints
 int() f // f is a function returning it
 int*()* f // f is a ptr to function returning ptr to int
 int*()* f,g // f & g are ptrs to function returning ptr to int

 This is only slightly weird if you are used to the current syntax
 but you can easily retrain yourself.

 IMHO, where evolution of C, Scheme and most programming
 languages goes wrong is doing evolutionary design by
 committee. Most users grasp what is easier to use vs what is
 hard but they don't have the aesthetic sense or imagination or
 training to make things simple. And in large groups popularity
 or force of personality or some other irrelevant attribute
 wins not aesthetics. Language evolution IMHO must be done by
 benevolent dictatorship or a guild of like minded people who
 have worked together for a long time! Go seems to have
 adapted that style....

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170513/19d1420b/attachment.html>

From bakul at bitblocks.com  Sun May 14 16:48:20 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Sat, 13 May 2017 23:48:20 -0700
Subject: [TUHS] C declarations.
In-Reply-To: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
Message-ID: <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>

So the problem was your assumptions didn't match IBM's! Most likely the FORTRAN standard /allowed/ pass by value/result instead of pass by ref.

None of the changes I suggested in my previous email (that focused on arrays in C) would have required a GC or heap. It would've increased programmer convenience and reduced bugs though! 

> On May 13, 2017, at 11:12 PM, Steve Johnson <scj at yaccman.com> wrote:
> 
> I can't resist jumping in here.   My main point is that system coding is a rather different beast than most application programming.  An operating system must be able to handle memory allocation requests from a user without knowing any details of what is requested.  Dennis explicitly left out a heap and garbage collection code in C because he didn't want OS code to be burdened with it.  Similarly with a "real" string type (with concatenation, etc.).
> 
> An example that speaks to me is that of creating a symbol table.  This isn't kernel programming, but consider what is needed:
> A structure that holds the individual table entries
> A tree of lookup tables that reflects the scoping rules of the language -- several tables might need to be searched to find an entry
> A function that looks up a name in a scope, returns the entry if it's defined, and adds a new entry in the appropriate scope if it isn't there yet.
> Statistically a majority of the calls do not change the symbol table, but some do, and you can't tell in advance.  And in the same way, many of the possible scopes where a variable might reside need not be searched.
> 
> So how much of this table should we copy into the lookup function?  The answer seems obvious to me -- none of it...
> 
> Before Unix, Dennis and I and several other people worked on a symbolic algebra system written in FORTRAN.  Dennis wrote a dynamic storage allocator, which was a real feat in FORTRAN, complete with garbage collector.  The goal was to have the program work on the 6 major manufacturers' FORTRANs.  We got it up and running on many of the systems, but did not have easy access to OS 360.   Finally, we were granted the third shift on the OS 360, and ran our program (that had become rather solid by this time...).   It was a disaster -- the brogram ran for a few second and then the OS dies and could not be restarted (turns out the core dump area allotted on the disc was not big enough to hold our program).  
> 
> The problem turned out to be that the IBM compiler did copy in/copy out argument passing, while all the other compilers passed by reference.  If you passed in an argument  that was dynamically allocated, and the system did a garbage collection, all the elements that were still live were moved around but kept their contents intact... until the function returned, and tried to copy the changed value out where that block used to be at the time of the call.
> 
> It took us about 6 months to rewrite the system so we didn't do that -- we had to allocate everything that would be needed first, then do the work, and then free things...
> 
> None of us were fans of copy in/copy out after that experience...
> 
> Steve
> 
> 
> ----- Original Message -----
> From: "Bakul Shah" <bakul at bitblocks.com>
> To:"Lawrence Stewart" <stewart at serissa.com>
> Cc:"tuhs" <tuhs at minnie.tuhs.org>
> Sent:Sat, 13 May 2017 21:24:11 -0700
> Subject:Re: [TUHS] C declarations.
> 
> 
> On Sat, 13 May 2017 21:59:57 EDT Lawrence Stewart <stewart at serissa.com> wrote:
> >
> > * - But I have never been able to remember the syntax for function pointers. > I always "man qsort" to refresh my memory.
> 
> The way I remember:
> 
> given *x[] or *x(), x /sticks/ to the /right/ first. Thus
> 
> int *x[];	// x is array of ptr to int
> int *x();	// x is a function returning ptr to int
> 
> If you don't want a var to stick to its right, separate using
> using parentheses.
> 
> int (*x)[];	// x is a ptr to array of ints
> int (*x)();	// x is a ptr to function returning int
> int *(*x)[];	// x is a ptr to array of ptrs to int
> 
> This sort of also works for multiple variables in one declaration. Given
> 
> int *x, y;	// * /sticks/ to the right (x) first, so not available for y.
> 
> Now if they'd allowed parenthesizing the type, as in
> 
> (int *)x, y;
> 
> we would see that both x & y are of type int *. That would've also allowed
> declaring multiple vars in one declaration where the previous rule applies!
> 
> (int*)(*f,*g)();
> 
> Part of the confusion is * is tacked on at the front while []
> & () at the back of a variable. Someone (Chandy?) had
> proposed unifying this syntax but it didn't go anywhere. I
> think the author used @ in a suffix place though I don't
> recall any other details. But if you put the var all the way
> to the right and read a declaration right to left, it works:
> 
> int*[] x	// x is a array of ptr to ints
> int() f	// f is a function returning it
> int*()* f	// f is a ptr to function returning ptr to int
> int*()* f,g	// f & g are ptrs to function returning ptr to int
> 
> This is only slightly weird if you are used to the current syntax
> but you can easily retrain yourself.
> 
> IMHO, where evolution of C, Scheme and most programming
> languages goes wrong is doing evolutionary design by
> committee. Most users grasp what is easier to use vs what is
> hard but they don't have the aesthetic sense or imagination or
> training to make things simple. And in large groups popularity
> or force of personality or some other irrelevant attribute
> wins not aesthetics. Language evolution IMHO must be done by
> benevolent dictatorship or a guild of like minded people who
> have worked together for a long time! Go seems to have
> adapted that style....

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170513/46d94d99/attachment.html>

From dfawcus+lists-tuhs at employees.org  Sun May 14 22:55:43 2017
From: dfawcus+lists-tuhs at employees.org (Derek Fawcus)
Date: Sun, 14 May 2017 13:55:43 +0100
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <1038e644-cef1-9424-f6fa-288941033bff@gmail.com>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
 <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
 <CAC20D2NHZ0kzuGJmtBFxXwQSiT-33KiokV2Gi3pEyf6m5Nqp3A@mail.gmail.com>
 <1038e644-cef1-9424-f6fa-288941033bff@gmail.com>
Message-ID: <20170514125543.GA81508@cowbell.employees.org>

On Sat, May 13, 2017 at 01:19:42PM -0400, William Pechter wrote:
> When VMS went to 3.6 or so a friend of mine was almost fired by DEC for 
> randomly testing boxes looking
> to see DEC's internal boxes weren't running System/Manager, 
> Field/Service and UETP/UETP User/password
> combinations.

Those default account combinations were still being used to gain
access to VMS systems in the '87-'89 time frame;  although
user/password was less interesting by itself,  being an unpriviledged
account.

DF


From dfawcus+lists-tuhs at employees.org  Sun May 14 23:14:44 2017
From: dfawcus+lists-tuhs at employees.org (Derek Fawcus)
Date: Sun, 14 May 2017 14:14:44 +0100
Subject: [TUHS] C declarations.
In-Reply-To: <1494702324.1901172.975588504.1BA76452@webmail.messagingengine.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <1494702324.1901172.975588504.1BA76452@webmail.messagingengine.com>
Message-ID: <20170514131444.GB81508@cowbell.employees.org>

On Sat, May 13, 2017 at 03:05:24PM -0400, Random832 wrote:
> On Sat, May 13, 2017, at 08:20, Michael Kjörling wrote:
> > Correct me if I am wrong, but _pass by value_ as opposed to _pass by
> > reference_ requires making a copy, no? That's the whole point, to
> > allow the callee to poke at the value it is given at will.
> 
> If that were the issue, the callee could simply automatically make the
> copy if-and-only-if it needs it.

I recall reading a proposed ABI for C on MIPS which would have done
something similar for passing structs by value,  namely pass a pointer
but allocate local storage space in the callee.  If the callee ever
assigned to the passed struct,  it was to copy the actual values,
then update its pointer.  I don't think that ABI was ever implemented.

DF


From doug at cs.dartmouth.edu  Mon May 15 00:11:34 2017
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Sun, 14 May 2017 10:11:34 -0400
Subject: [TUHS] C declarations.
Message-ID: <201705141411.v4EEBYjv021607@coolidge.cs.Dartmouth.EDU>

> Are there languages that copy arrays in function calls defaultly?

> Pascal is an example.

Pascal's var convention, where the distinction between value
and reference is made once and for all for each argument of
each function, is sound. The flexibility of PL/I, where the
distinction is made at every call (parenthesize the name to
pass an array by value) is finicky, though utterly general.

> Where is all that [memory]  going to come from if you pass a
> large array on a memory-constrained system of specs common back in the
> days when C was designed

Amusingly, under the customary linkage method in the even earlier
days when Fortran was designed, pass-by-reference entailed a big
overhead that could easily dominate pass-by-value for small arrays.

[In the beginning, when CPUs had only one register, subroutine
preambles plugged the reference into every mention of that variable
throughout the body of the subroutine. This convention persisted
in Fortran, which was designed for a machine with three index
registered. Since reference variables were sometimes necessary
(think of swap(a,b) for example) they were made standard.] 

Doug


From cym224 at gmail.com  Mon May 15 00:48:49 2017
From: cym224 at gmail.com (Nemo)
Date: Sun, 14 May 2017 10:48:49 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <alpine.BSF.2.20.1705140913210.67621@aneurin.horsfall.org>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <1494702492.1902154.975591160.507FCE4C@webmail.messagingengine.com>
 <alpine.BSF.2.20.1705140913210.67621@aneurin.horsfall.org>
Message-ID: <CAJfiPzxoM6P8jz9yrMg9pyW4rRyBDX+4aHtat5BPYR5b6mM4ww@mail.gmail.com>

On 13 May 2017 at 19:21, Dave Horsfall <dave at horsfall.org> wrote:
> On Sat, 13 May 2017, Random832 wrote:
>
>> Fortran, for the record, passes nearly everything by reference [...]
>
> And if you passed a constant and the local argument was modified, so was
> the (single instance of) constant...

Probably depends on the implementation.  VAX-FORTRAN passed by
reference (but not on the stack).

Off-topic: I recall a working function that crashed when fed a
constant string.  Spelunking revealed that someone had replaced a
letter-by-letter case-insensitive comparison with a "faster version"
(written in VAX assembler, no less) that converted both arguments to
upper case and then block-compared.

N.


From usotsuki at buric.co  Mon May 15 00:58:18 2017
From: usotsuki at buric.co (Steve Nickolas)
Date: Sun, 14 May 2017 10:58:18 -0400 (EDT)
Subject: [TUHS] C declarations.
In-Reply-To: <201705141411.v4EEBYjv021607@coolidge.cs.Dartmouth.EDU>
References: <201705141411.v4EEBYjv021607@coolidge.cs.Dartmouth.EDU>
Message-ID: <alpine.BSF.2.02.1705141057160.64059@frieza.hoshinet.org>

On Sun, 14 May 2017, Doug McIlroy wrote:

>> Are there languages that copy arrays in function calls defaultly?
>
>> Pascal is an example.
>
> Pascal's var convention, where the distinction between value
> and reference is made once and for all for each argument of
> each function, is sound. The flexibility of PL/I, where the
> distinction is made at every call (parenthesize the name to
> pass an array by value) is finicky, though utterly general.

QuickBasic uses the ( ) trick too.

>> Where is all that [memory]  going to come from if you pass a
>> large array on a memory-constrained system of specs common back in the
>> days when C was designed
>
> Amusingly, under the customary linkage method in the even earlier
> days when Fortran was designed, pass-by-reference entailed a big
> overhead that could easily dominate pass-by-value for small arrays.
>
> [In the beginning, when CPUs had only one register, subroutine
> preambles plugged the reference into every mention of that variable
> throughout the body of the subroutine. This convention persisted
> in Fortran, which was designed for a machine with three index
> registered. Since reference variables were sometimes necessary
> (think of swap(a,b) for example) they were made standard.]
>
> Doug
>

Three! O_O That's a lot, if you're used to the 6502 and its mere *two* 
index registers.

-uso.


From clemc at ccc.com  Mon May 15 03:40:48 2017
From: clemc at ccc.com (Clem Cole)
Date: Sun, 14 May 2017 13:40:48 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170514043024.pezg5eqslpcqsd5m@thunk.org>
References: <20170511140729.2262B18C09A@mercury.lcs.mit.edu>
 <CAC20D2MDG0Mc3jJcbvZYXCo_FZcswt_no=3ajqvx45r-q9gKow@mail.gmail.com>
 <CANCZdfrbxzkMesjPPX3A0qcHL-Hz_g+k1F2PtBicOkZv3JB5Cw@mail.gmail.com>
 <CAC20D2Ndb54AqUx=qn48Rkc37b7pr9rwEVs+m+Aq1zR6ALjw6w@mail.gmail.com>
 <013b01d2ca96$6901b370$3b051a50$@ronnatalie.com>
 <20170511222547.GJ4341@mcvoy.com>
 <20170514043024.pezg5eqslpcqsd5m@thunk.org>
Message-ID: <CAC20D2OKF9drU_v8xcccbCJzMJpjDFpSnV-yGdC_KLpZ1LLs-Q@mail.gmail.com>

Ted -- thank you -- excellent write up.  Love it and I could not agree
more!!  Your 'worse is better' is the same idea as what is 'good enough,'
an argument I used to have at DEC, where being 'prefect' cost years and in
the end - and we lost because of it.

FWIW:  I put it slightly differently, make sure you pick a couple of things
that matter, and nail them... be the best on those >>few<< items but then
the rest needs to be 'good enough' and in time you can make those parts
better.   But if you wait for all parts to be great or worse 'perfect' - it
doesn't matter -- this says an ex-Alpha guy now working in INTEL*64 --
sigh...

BTW: Not to quibble, but you might also remember, traditional UNIX took the
same path as you did.   Ken's v6/v7 FS were not the great about write
ordering either.   Remember FFS is replacing Ken's work 15 years down the
road.   Kirk did not do all the careful ordered write stuff until after
George Gobble taught us all how to, which was a few years later.   When
Kirk implemented FFS, it was after the Purdue patches had been released to
make Unix's original FS more 'reliable' - and yes - Ted (Kowalski) and my
version of the original fsck was not nearly as careful as you were year
later.   But again, we were a huge step forward from what had been at the
time.  That said -- your stuff, as Larry has pointed, was rock solid and in
practice 'just worked.'  Certainly post ext3, I do not have memory of
losing any real user data on any of Linux boxes, and an error I made has
definitely be the cause of them to crashing over the years ;-)

Also, WRT to making the HW properly, and at lot of PC HW being trash -- yup
- which comes back to the what is good enough issue.   DEC did the same
thing and for long time in as SGI in making rock solid HW.  Heck DEC,
somewhat cornered the SCSI disk business for the mid-range and upper end
rack world in the 90s'.   Like Sgi's and Sun's OSses of the day, Tru64 has
very good DMA controllers under the covers that were hardened and lab
tested for corner cases.   In fact, it is one of the reasons why while
Tru64 could detect an Adaptec controller @ boot up and actually use it (I
had on my workstation), it was not officially in the 'SPD' as supported
device because the HW failed as you described and TruCluster's in
particular could not make a proper DLM do the failure modes of the Apaptec
(now as I used to point out to the marketing and HW weenies, none sane
person was going to put a $150 SCSI controller in their $.5M TruCluster
system - so we could have allowed it, just not allowed it some configs -
made it clear in the SPD -- Adaptec on in configs XXX).

As a result, the issue places out this way... an then DEC's VPs used to say
you could not make & sell an Alpha for under $5K (one person in particular
who I will leave nameless - those who were there - all know who I refer).
My last act before I left DEC/Compaq for Paceline, was to make the $1K
alpha using an $799 (end user) Compaq system with the K7 on the motherboard
swapped with an EV6 and some mechanical shims,  Adaptec SCSI BT (I still
have the motherboard @ home, and the EV6 in on my desk at Intel).  It was
built using PC parts - case, power supply et al.   The key was the Alpha
@5K was a better physically built system than the $799 based PC -- but who
cared ...   your closing para WRT to WiFi and PCMCIA summed up the issue
pretty well.

Clem

On Sun, May 14, 2017 at 12:30 AM, Theodore Ts'o <tytso at mit.edu> wrote:

> On Thu, May 11, 2017 at 03:25:47PM -0700, Larry McVoy wrote:
> > This is one place where I think Linux kicked Unix's ass.  And I am not
> > really sure how they did it, I have an idea but am not positive.  Unix
> > file systems up through UFS as shipped by Sun, were all vulnerable to
> > what I call the power out test.  Untar some big tarball and power off
> > the machine in the middle of it.  Reboot.  Hilarity ensues (not).
> >
> > You were dropped into some stand alone shell after fsck threw up its
> > hands and it was up to you to fix it.  Dozens and dozens of errors.
> > It was almost always faster to go to backups because figuring that
> > stuff out, file by file (which I have done more than once), gets you
> > to the point that your run "fsck -y" and go poke at lost+found when
> > fsck is done, realize that there is no hope, and reach for backups.
> >
> > Try the same thing with Linux.  The file system will come back, starting
> > with, I believe, ext2.
> >
> > My belief is that Linux orders writes such that while you may lose data
> > (as in, a process created a file, the OS said it was OK, but that file
> > will not be in the file system after a crash), but the rest of the file
> > system will be consistent.  I think it's as if you powered off the
> > machine a few seconds earlier than you actually did, some stuff is in
> > flight and until they can write stuff out in the proper order you may
> > lose data on a hard reset.
>
> So the story is a bit complicated here, and may be an example of
> "worse is better" --- which is ironically one of those things which is
> used as an explanation for why BSD/Unix won ever though the Lisp was
> technically superior[1] --- but in this case, it's Linux that did
> something "dirty", and BSD that did something that was supposed to be
> the "better" solution.
>
> [1] https://www.jwz.org/doc/worse-is-better.html
>
> So first let's talk about ext2 (which indeed, does not have file
> system journalling; that came in ext3).  The BSD Fast File System goes
> to a huge amount of effort to make sure that writes are sent to the
> disk in exactly the right order so that fsck can actually fix things.
> This requires that the disk not reorder writes (e.g., write caching is
> disabled or in write-through mode).  Linux, in ext2, didn't bother
> with trying to get the write order correct at all.  None.  Nada.  Zip.
> Writes would go out in whatever order dictated by the elevator
> scheduler, and so on a power failure or a kernel crash, the order in
> which metadata writes would be sent to the disk was completely
> unconstrained.
>
> Sounds horrible, right?  In many ways, it was.  And I lost count of
> how often NetBSD and FreeBSD users would talk about how primitive and
> horrible ext2 was in comparison to FFS, which had all of this
> excellent engineering work to make sure writes happened in the correct
> order such that fsck was guaranteed to always be able to fix things.
>
> So why did Linux get away with it?  When I wrote the fsck for ext2, I
> knew that anything can and would happen, so it was implemented so that
> it was extremely paranoid about not ever losing any data.  And if
> there was a chance that an expert could recover the data, e2fsck would
> stop and ask the system administrator to take a look.  In the case
> that the user ran with fsck -y, the default was drop files into
> lost+found, where as in some cases with the FFS fsck, it "knew" that
> in a particular case, the order in which writes were staged out the
> right thing to do was to let the unlink complete, so it would let the
> refcount go to zero, or stay at zero.
>
> The other thing that we did in Linux is that I made sure we had a
> highly functional "debugfs" tool.  This tool served two purposes.  The
> first was it made it very easy for me to creat a regression test suite
> for fsck.  As far as I know, none of the other major file systems at
> the time had an fsck with a regression test suite --- and I was
> religious about adding tests as I added functionality, and as I fixed
> bugs.  The debugfs tool made it easy for me to create test case file
> systems that was corrupted in various interesting ways.  The other use
> of debugfs was that it made it easy for experts to do file system
> recovery after a crash, if there was some really precious file that
> they needed to try to recover.
>
> So this is why this is a great example of "worse is better".  In Linux,
> ext2 was ***incredibly*** sloppy about how it handled write ordering
> --- it didn't do anything at all.  But as a consequence we developed
> tools that were extremely good to compensate, and in practice, it was
> extremely rare (although it did happen on occasion) that files would
> get lost or the file systme could end up in a state where fsck would
> not be able to recover without manual intervention by a system
> administrator using debugfs.
>
> But the other thing to note here is that in the PC era, most disk
> drives ran with write caching enabled, with writeback caching so that
> the hard drive could do its own elevator shceduling.  So having a file
> system that very carefully scheduled writes to make sure they happened
> in the write order didn't help you a *bit* unless you configured your
> hard drive to disable writeback caching --- at which point you would
> take a massive speed hit.
>
> This is ultimately also one weaknesses of Soft Updates --- it requires
> that you disable writeback caching, since it works by letting the OS
> control the order in which writes hit stable storage.  With
> journalling you don't have to do that; but the tradeoff is that when
> you do a journal commit, you need typically two cache flush
> operations.  (Or a cache flush followed by a FUA write of the commit
> block, if the disk supports FUA.)
>
>
>
> There is another example of how Linux embraced the "worse is better"
> philosophy in ext3, and that has to do with how we do journalling.
> The sophisticated way to do journalling is to do logical journalling.
> This is where what you write in the in journal is "set bit XXX in the
> allocation bitmap", or "update the mtime to YYYY".  And in this way,
> you can batch multiple file system operations into a single block
> written to the journal.  Solaris/UFS and Irix uses this much more
> sophisticated form of journalling.  (Actually, older versions of
> Solaris did use volume-level journalling, which is basically what
> ext3/ext4 uses, but they upgraded to the much more "right", more
> advanced thing, which is logical journalling.)
>
> Ext3 uses phyiscal, or volume-level journalling.  This journalling
> works on the block level --- so if we flip a bit in an allocation
> bitmap, we log the entire 4k block to the journal.  By default, we
> only do a journal commit every five seconds (unless an fsync happens
> first), so there could be multiple changes to a single inode table
> blocks that can be batched together, but it's still true that for a
> given metadata-heavy workload, a file system which uses logical
> journalling will tend require many fewer blocks written to the journal
> than a file system such as ext3/ext4 which uses physical block
> journalling.
>
> Why did Linux get away with it?  Number one, most workloads aren't
> really modify metadata all _that_ intensively, and 12k of sequential
> writes versus 32k of sequential writes doesn't actually take that much
> more time.  Secondly, Ted's law of PC-class hardware ("most PC-class
> hardware is crap") comes into play, and turns physical journalling
> into an advantage.  PC class hardware tends not to have power fail
> interrupts, and when power drops, and the voltage levels on the power
> rails start drooping, DRAM tends to go insane and starts returning
> garbage long before the DMA engine and the hard drive stops
> functioning.
>
> So if your system is doing logical journalling, after the file system
> commits a transaction, it will start writing the inode table block to
> the permanent location on disk.  If at that point you get a power
> drop, garbage can get written to the inode table block, and if the
> file system is using logical journalling, on reboot the mtime field
> can get updated from the logical journal --- but the rest of the inode
> table block is still garbage.
>
> In contrast, since ext3 was using physical block journalling, even if
> various metadata blocks get corrupted due to writes from failing DRAM
> during a power drop, when we replay the journal, this will restore the
> entire metadata block, and Things Just Worse.
>
> I have talked to an XFS engineer from SGI, and this was definitely a
> thing which SGI discovered the hard way.  After they discovered this
> problem, they added extra capacitors to the power supply, added a
> power fail interrupt, and taught Irix so that when the power fail
> interrupt was triggered, it would frantically cancel DMA transfers in
> order to avoid this problem.  I do not know how many of the other
> Legacy Unix systems figured out this failure mode --- and I can't
> claim that we were brilliant enough to design a system to avoid this
> problem.  It just so happened that the brute-force design that we
> chose was very well suited for crappy (but way cheaper than a Sun Fire
> E10k :-) PC-class hardware.
>
> > I copied Ted, who had his fingers deep in that code, maybe he can correct
> > me where I got it wrong.  Details aside, I think this is a place where
> > Linux moved the state of the art significantly forward.  There are other
> > places but this one is a big deal IMHO, maybe the biggest deal.
>
> So I'm not really sure we can claim to have "moved the state of the
> art".  There certainly wasn't any brilliant computer science
> innovations here.  That sort of thing is more like Soft Updates, of
> which Valerie Aurora (formerly Henson) once wrote,
>
>    "I've read this paper at least 15 times, and each time I when get
>    to page 7, I'm feeling pretty good and thinking, "Yeah, okay, I
>    must be smarter now than the last time I read this because I'm
>    getting it this time," - and then I turn to page 8 and my head
>    explodes." --- https://lwn.net/Articles/339337/
>
> I will be the first to admit that with ext2/ext3/ext4, especially in
> the early days, it was much more about brute force engineering, and
> regression testing, and much less about "moving the state of the art".
> Certainly those of us who were working on Linux weren't trying to get
> papers published in peer reviewed journals or conferences!  (And I've
> always thought that Greg Ganger was _way_ smarter than I.  :-)
>
> And if the Lisp Machine hackers looked down on BSD, and complained
> that BSD adopted the "Worse is Better" philosophy, while Lisp strived
> for the true, elegant, Correct technical solution, it's perhaps
> especially interesting to consider that if anything, Linux was an even
> more radical example of the "Worse is Better" philosophy.
>
> Cheers,
>
>                                         - Ted
>
> P.S.  There is yet another example of "Worse is Better" in how Linux
> had PCMCIA support several years before FreeBSD/NetBSD.  However, if
> you ejected a PCMCiA card in a Linux system, there was a chance (in
> practice it worked out to be about in 1 in 5 times for a WiFI card, in
> my experience) that the system would crash.  The *BSD's took a good
> 2-3 years longer to get PCMCIA support, but when they did, it was rock
> solid.  Of course, if you are a laptop user, and are happy to keep
> your 802.11 PCMCIA card permanently installed, guess which OS you were
> likely to prefer --- "sloppy but works, mostly", or "it'll get there
> eventually, and will be rock solid when it does, but zip, nada, right now"?
>
>
> >
> > --lm
> >
> > On Thu, May 11, 2017 at 04:37:29PM -0400, Ron Natalie wrote:
> > > I remember the pre-fsck days.   It was part of my test to become an
> operator at the UNIX site at JHU that I could run the various manual checks.
> > >
> > > The V6 file system wasn???t exactly stable during crashes (lousy
> database behavior), so there was almost certainly something to clean up.
> > >
> > >
> > >
> > > The first thing we???d run was icheck.   This runs down the superblock
> freelist and all the allocated blocks in the inodes.     If there were
> missing blocks (not in a file or the free list), you could use icheck ???s
> > >
> > > to rebuild it.    Similarly, if you had duplicated allocations in the
> freelist or between the freelist and a single file.   Anything more
> complicated required some clever patching (typically, we???d just mount
> readonly, copy the files, and then blow them away with clri).
> > >
> > >
> > >
> > > Then you???d run dcheck.   As mentioned dcheck walks the directory
> path from the top of the disk counting inode references that it reconciles
> with the link count in the inode.   Occasionally we???d end up with a 0-0
> inode (no directory entires, but allocated???typically this is caused by
> people removing a file while it is still open, a regular practice of some
> programs for their /tmp files.).    clri again blew these away.
> > >
> > >
> > >
> > > Clri wrote zeros all over the inode.   This had the effect of wiping
> out the file, but it was dangerous if you got the i-number wrong.    We
> replaced it with ???clrm??? which just cleared the allocated bit, a lot
> easy to reverse.
> > >
> > >
> > >
> > > If you really had a mess of a file system, you might get a piece of
> the directory tree broken off from a path to the root.   Or you???d have an
> inode that icheck reported dups.   ncheck would try to reconcile an inumber
> into an absolute path.
> > >
> > >
> > >
> > > After a while a program called fsdb came around that allowed you to
> poke at the various file system structures.    We didn???t use it much
> because by the time we had it, fsck was fast on its heals.
> > >
> >
> > --
> > ---
> > Larry McVoy                        lm at mcvoy.com
> http://www.mcvoy.com/lm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170514/644a2259/attachment.html>

From jnc at mercury.lcs.mit.edu  Mon May 15 07:44:19 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Sun, 14 May 2017 17:44:19 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170514214419.EFC1618C0A2@mercury.lcs.mit.edu>

    > From: Random832

    > Ah. There's the other piece. You start the SUID program under the
    > debugger, and ... it simply starts it non-suid. *However*, in the
    > presence of shared text ...  you can make changes to the text image
    > ... which will be reused the *next* time it is started *without* the
    > debugger.

So I actually tried to do this (on a V6 system running on an emulator), after
whipping up a tiny test program (which prints "1", and the real and current
UIDs): the plan was to patch it to print a different number.

However, after a variety of stubbed toes and hiccups (gory details below, if
anyone cares), including a semi-interesting issue with the debugger and pure
texts), I'm punting: when trying to set a breakpoint in a pure text, I get the
error message "Can't set breakpoint", which sort of correlates with the
comment in the V6 sig$ptrace(): "write user I (for now, always an error)".

So it's not at all clear that the technique we thought would work would, in
fact, work - unless people weren't using a stock V6 system, but rather one
that had been tweaked to e.g. allow use of debuggers on pure-text programs
(including split I+D).

It's interesting to speculate on what the 'right' fix would be, if somehow the
techique above did work. The 'simple' fix, on systems with a PWB1-line XWRIT
flag, would be to ignore SETUID bits when doing an exec() of a pure text that
had been modified. But probably 'the' right fix would be to give someone
debugging a pure-text program their own private copy of the text. (This would
also prevent people who try to run the program from hitting breakpoints while
it's being debugged. :-)


But anyway, it's clear that back when, when I thought I'd found the bug, I
clearly hadn't - which is why when I looked into the source, it looked like it
had been 'already' been fixed. (And why Jim G hemmed and hawed...)

But I'm kind of curious about that mod in PWB1 that writes a modified pure
text back to the swap area when the last process using it exits. What was the
thinking behind that? What's the value to allowing someone to patch the
in-core pure text, and then save those patches? And there's also the 'other
people who try and run a program beind debugged are going to hit breakpoints'
issue, if you do allow writing into pure texts...


	Noel


--------


For the gory details: to start with, attempting to run a pure-text program
(whether SUID or not) under the debugger produced a "Can't execute
{program-name} Process terminated."  error message.

'cdb' is printing this error message just after the call to exec() (if that
fails, and returns). I modified it to print the error number when that
happens, and it's ETXTBSY. I had a quick look at the V6 source, to see if I
could see what the problem is, and it seems to be be (in sys1$exec()):

    if(u.u_arg[1]!=0 && (ip->i_flag&ITEXT)==0 && ip->i_count!=1) {
		     u.u_error = ETXTBSY;
			       goto bad;
			       }

What that code does is a little obscure; I'm not sure I understand it. The
first term checks to see if the size of the text segment is non-zero (which it
is not, in both 0407 and 0410 files). The second is, I think, looking to see
if the inode is marked as being in use for a pure text (which it isn't, until
later in exec()). The third checks to make sure nobody else is using the file.

So I guess this prevents exec() of a file which is already open, and not for a
pure text. (Why this is the Right Thing is not instantly clear to me...)

Anyway, the reason this fails under 'cdb' is that the debugger already has it
open (to be able to read the code). So I munged the debugger to close it
before doing the exec(), and then the error went away.

Then I ran into a long series of issues, the details of which are not at all
interesting, connected with the fact that the version of 'cdb' I was using
(one I got off a Tim Shoppa modified V6 disk) doesn't correspond to either of
the sources I have for 'cdb'.

When I switched to the latest source (so I could fix the issue above), it had
some bug where it wouldn't work unless there was a 'core' file. But eventually
I kludged it enough to get the 'can't set breakpoints' message, at which point
I threw in the towel.



From dave at horsfall.org  Mon May 15 08:12:04 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Mon, 15 May 2017 08:12:04 +1000 (EST)
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170514125543.GA81508@cowbell.employees.org>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
 <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
 <CAC20D2NHZ0kzuGJmtBFxXwQSiT-33KiokV2Gi3pEyf6m5Nqp3A@mail.gmail.com>
 <1038e644-cef1-9424-f6fa-288941033bff@gmail.com>
 <20170514125543.GA81508@cowbell.employees.org>
Message-ID: <alpine.BSF.2.20.1705150810220.67621@aneurin.horsfall.org>

On Sun, 14 May 2017, Derek Fawcus wrote:

> > to see DEC's internal boxes weren't running System/Manager, 
> > Field/Service and UETP/UETP User/password combinations.
> 
> Those default account combinations were still being used to gain access 
> to VMS systems in the '87-'89 time frame;  although user/password was 
> less interesting by itself, being an unpriviledged account.

Wasn't there also Guest/Guest as well?  Admittedly it would also be pretty 
boring, but nonetheless still a toe-hold.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From ron at ronnatalie.com  Mon May 15 09:06:47 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Sun, 14 May 2017 19:06:47 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
Message-ID: <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>

You all are missing the point as to what the cost of passing arrays by value
or what other languages do.

 

C is implicitly a pass/return by value language for EVERY SINGLE POSSIBLE
DATATYPE WITH THE EXCEPTION OF ARRAYS.

Arrays are special for no reason other than it was thought to costly to do
it any otherway.   In fact, had it not been kludged up into a
pseudo-pass-by-reference,

nobody would have probably been the wiser.   It was too hard to pass/return
structs by value either in the initial compilers, but they fixed it to work
exactly that way.

 

So we have this array kludge that works differently than everything else.
Further, due to the kludgosity, it's not even exactly type safe.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170514/d7e5bfc5/attachment.html>

From krewat at kilonet.net  Mon May 15 09:34:01 2017
From: krewat at kilonet.net (Arthur Krewat)
Date: Sun, 14 May 2017 19:34:01 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
 <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
Message-ID: <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>

On 5/14/2017 7:06 PM, Ron Natalie wrote:
> array kludge
Is it really a kludge? It's a pointer.

int array[128];

Want to reference the array in a memcpy? Do: memcpy(destination, array, 
sizeof(array));

Want to reference an element in the array? array[element #]

Want to pass (a reference to) said array to a function? function(array)

Not sure what the kludge is.

I think C is more consistent when it comes to arrays than a lot of other 
languages. At least with C you KNOW it's a pointer to the original.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170514/acf3111e/attachment.html>

From crossd at gmail.com  Mon May 15 10:14:47 2017
From: crossd at gmail.com (Dan Cross)
Date: Sun, 14 May 2017 20:14:47 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
 <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
 <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>
Message-ID: <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>

On Sun, May 14, 2017 at 7:34 PM, Arthur Krewat <krewat at kilonet.net> wrote:

> On 5/14/2017 7:06 PM, Ron Natalie wrote:
>
> array kludge
>
> Is it really a kludge?
>

Yes.

It's a pointer.
>

Of course it is! But that doesn't mean that it SHOULD be, which is the
point.

int array[128];
>
> Want to reference the array in a memcpy? Do: memcpy(destination, array,
> sizeof(array));
>
> Want to reference an element in the array? array[element #]
>
> Want to pass (a reference to) said array to a function? function(array)
>
> Not sure what the kludge is.
>

The kludge is that it works different than everything else for no really
good reason.

I think C is more consistent when it comes to arrays than a lot of other
> languages. At least with C you KNOW it's a pointer to the original.
>

Sure! But if someone had told you, back when you first learned C, that it
was an array and had value semantics, would you have thought THAT was
strange? "If you want a pointer to the array, use the address operator on
the array and pass that; that'll behave like a 'foo*' pointer for an array
of foo. Most people do this for efficiency."

The point is that it's a thought exercise. It's not that folks don't
understand how C works now or the semantics of array-decay-to-pointer; it's
just that folks can also imagine a world where things had been done
differently. That world doesn't exist, but we can still imagine it.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170514/42e2d44f/attachment.html>

From ron at ronnatalie.com  Mon May 15 10:23:31 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Sun, 14 May 2017 20:23:31 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
 <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
 <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>
 <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>
Message-ID: <032701d2cd11$7be3f200$73abd600$@ronnatalie.com>

It's a pointer.

 

Ø   Of course it is! But that doesn't mean that it SHOULD be, which is the point.

 

Actually, it is **Not a pointer**, it gets passed as a pointer, but it is an array on both ends.

 

 

The implicit conversion of the array name to a pointer to its first item is a different issue.   This bothers me less, but it’s still hokie in my opinion.

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170514/6a91ab9a/attachment.html>

From lm at mcvoy.com  Mon May 15 10:40:19 2017
From: lm at mcvoy.com (Larry McVoy)
Date: Sun, 14 May 2017 17:40:19 -0700
Subject: [TUHS] C declarations.
In-Reply-To: <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
 <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
 <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>
 <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>
Message-ID: <20170515004019.GJ29107@mcvoy.com>

On Sun, May 14, 2017 at 08:14:47PM -0400, Dan Cross wrote:
> The kludge is that it works different than everything else for no really
> good reason.

I'd be curious what Steve thinks about all this, I know he weighed in 
a bit, but does he think that Dennis / Brian/ Ken regret this design choice?

Personally, I can easily see why they did it.  C doesn't really have a lot
of behind-the-scenes magic, it can easily be viewed as a pleasant way to
do portable code that sits directly on the hardware, no frameworks, no
garbage collector, no magic.  It's predictable in ways that higher level
languages are not.  That's one of the things I like about C, I can reason
about it.

But I suspect it was more basic than that.  They were running on 16 bit 
pretty slow machines and having a pass by value model for arrays just
didn't make any sense.  *Everyone* would pass by reference so why bother
having a pass by value model?  If you really, really wanted that you 
could get it

    int salaries[100];
    int payroll = add_them_up(malloc_and_copy(salaries, sizeof(salaries));

But any sane person in that day and age would go "why on God's green earth
would you do that?  It's slow.  Just don't change the salaries array and
you'll be fine".

Think about all the str* stuff - you really want to malloc and free that?
It would have killed performance.

Personally I think they looked at it, said "everyone will use references
for arrays anyway, let's make them references by default".

I get that it is not clean and pure and doesn't match how other types work,
I do see that, but I also see that their choice made a ton of sense at the
time.  And maybe still does.  It's not the same as the other types because
the other types are, typically, small.  Arrays can be big, really big.

--lm


From cym224 at gmail.com  Mon May 15 11:24:25 2017
From: cym224 at gmail.com (Nemo)
Date: Sun, 14 May 2017 21:24:25 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <alpine.BSF.2.20.1705150810220.67621@aneurin.horsfall.org>
References: <20170513004423.F3A9C18C099@mercury.lcs.mit.edu>
 <1494636703.1615694.975085000.13626EAC@webmail.messagingengine.com>
 <8B3FFD9C-FA5F-434F-A18E-FF59215C6DDD@quintile.net>
 <CAC20D2NHZ0kzuGJmtBFxXwQSiT-33KiokV2Gi3pEyf6m5Nqp3A@mail.gmail.com>
 <1038e644-cef1-9424-f6fa-288941033bff@gmail.com>
 <20170514125543.GA81508@cowbell.employees.org>
 <alpine.BSF.2.20.1705150810220.67621@aneurin.horsfall.org>
Message-ID: <CAJfiPzyatQQ7_+146ccs5+wDuNyb4fRQifMxeEdJKOYfuaZp2w@mail.gmail.com>

On 14 May 2017 at 18:12, Dave Horsfall <dave at horsfall.org> wrote:
> On Sun, 14 May 2017, Derek Fawcus wrote:
>
>> > to see DEC's internal boxes weren't running System/Manager,
>> > Field/Service and UETP/UETP User/password combinations.
>>
>> Those default account combinations were still being used to gain access
>> to VMS systems in the '87-'89 time frame;  although user/password was
>> less interesting by itself, being an unpriviledged account.
>
> Wasn't there also Guest/Guest as well?  Admittedly it would also be pretty
> boring, but nonetheless still a toe-hold.

I worked in a VAX shop once where a DEC FSE came by (on the wrong day
with the sysadmin out) and was rather upset that the default account
passwords had been changed.

N.


From nevin at eviloverlord.com  Mon May 15 12:00:40 2017
From: nevin at eviloverlord.com (Nevin Liber)
Date: Sun, 14 May 2017 21:00:40 -0500
Subject: [TUHS] C declarations.
In-Reply-To: <20170515004019.GJ29107@mcvoy.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
 <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
 <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>
 <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>
 <20170515004019.GJ29107@mcvoy.com>
Message-ID: <CAGg_6+P6uxsv5zwwAAqhw_H1q+REUFJPNE9aFg5xD39SqHFGdw@mail.gmail.com>

On Sun, May 14, 2017 at 7:40 PM, Larry McVoy <lm at mcvoy.com> wrote:

> I get that it is not clean and pure and doesn't match how other types work,
> I do see that, but I also see that their choice made a ton of sense at the
> time.  And maybe still does.  It's not the same as the other types because
> the other types are, typically, small.  Arrays can be big, really big.
>

IIRC, you couldn't pass or return structs either.  Also, no function
prototypes, so passing data of different sizes would have been a bit more
difficult.
-- 
 Nevin ":-)" Liber  <mailto:nevin at eviloverlord.com>  +1-847-691-1404
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170514/8912a30f/attachment.html>

From random832 at fastmail.com  Mon May 15 13:43:39 2017
From: random832 at fastmail.com (Random832)
Date: Sun, 14 May 2017 23:43:39 -0400
Subject: [TUHS] C declarations.
In-Reply-To: <032701d2cd11$7be3f200$73abd600$@ronnatalie.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
 <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
 <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>
 <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>
 <032701d2cd11$7be3f200$73abd600$@ronnatalie.com>
Message-ID: <1494819819.1845291.976532104.41B8F245@webmail.messagingengine.com>

On Sun, May 14, 2017, at 20:23, Ron Natalie wrote:
> Actually, it is **Not a pointer**, it gets passed as a pointer, but it is
> an array on both ends.

I'm not sure what you mean by "on both ends". It is a pointer in the
function. sizeof(dst) is 2, not 256, even if it were declared as int
dst[128].

On Sun, May 14, 2017, at 19:34, Arthur Krewat wrote:
> Is it really a kludge? It's a pointer.
> 
> int array[128];
> 
> Want to reference the array in a memcpy? Do: memcpy(destination, array, 
> sizeof(array));

The argument is that that should have to be memcpy(&destination, &array,
sizeof(array)), the same as if you were copying between two structs.


From dave at horsfall.org  Mon May 15 14:35:34 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Mon, 15 May 2017 14:35:34 +1000 (EST)
Subject: [TUHS] C declarations.
In-Reply-To: <20170514042411.93CCB124AEA4@mail.bitblocks.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
 <20170514042411.93CCB124AEA4@mail.bitblocks.com>
Message-ID: <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>

On Sat, 13 May 2017, Bakul Shah wrote:

> int*()* f	// f is a ptr to function returning ptr to int

Nope.

    aneurin% cdecl
    Type `help' or `?' for help
    explain int*()* f
    syntax error
    declare f as ptr to function returning ptr to int
    int *(*f)()

(Sigh; "cdecl" really needs to print a "> " prompt or something.)

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From bakul at bitblocks.com  Mon May 15 14:54:16 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Sun, 14 May 2017 21:54:16 -0700
Subject: [TUHS] C declarations.
In-Reply-To: Your message of "Mon, 15 May 2017 14:35:34 +1000."
 <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
 <20170514042411.93CCB124AEA4@mail.bitblocks.com>
 <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
Message-ID: <20170515045416.AF1CF124AEA4@mail.bitblocks.com>

On Mon, 15 May 2017 14:35:34 +1000 Dave Horsfall <dave at horsfall.org> wrote:
> On Sat, 13 May 2017, Bakul Shah wrote:
> 
> > int*()* f	// f is a ptr to function returning ptr to int
> 
> Nope.
> 
>     aneurin% cdecl
>     Type `help' or `?' for help
>     explain int*()* f
>     syntax error
>     declare f as ptr to function returning ptr to int
>     int *(*f)()
> 
> (Sigh; "cdecl" really needs to print a "> " prompt or something.)

Sorry, I was less than clear.  I was just doing a thought
experiment about an alternate declaration syntax.  Requoting
from my email:

                             But if you put the var all the way
  to the right and read a declaration right to left, it works:

Basically suggesting that if all type machinery *preceded* an
object declaration (including for functions), we don't need any
parentheses and declarations are very easy to understand --
you just read a declaration from right to left. So no need
for cdel!  Too late for this, of course!


From dave at horsfall.org  Mon May 15 15:01:58 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Mon, 15 May 2017 15:01:58 +1000 (EST)
Subject: [TUHS] C declarations.
In-Reply-To: <20170515045416.AF1CF124AEA4@mail.bitblocks.com>
References: <015401d2caa0$79762650$6c6272f0$@ronnatalie.com>
 <68E8DC0A-D1B8-4FF0-AD26-ACDC57E308AF@pobox.com>
 <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
 <20170514042411.93CCB124AEA4@mail.bitblocks.com>
 <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
 <20170515045416.AF1CF124AEA4@mail.bitblocks.com>
Message-ID: <alpine.BSF.2.20.1705151500090.67621@aneurin.horsfall.org>

On Sun, 14 May 2017, Bakul Shah wrote:

> > Nope.
> > 
> >     aneurin% cdecl
> >     Type `help' or `?' for help
> >     explain int*()* f
> >     syntax error
> >     declare f as ptr to function returning ptr to int
> >     int *(*f)()
> 
> Sorry, I was less than clear.  I was just doing a thought
> experiment about an alternate declaration syntax.  Requoting
> from my email:

Sorry - entirely my fault (which I'd realised just *after* I'd sent it, of 
course).  Yes, your idea is a lot better.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From tfb at tfeb.org  Mon May 15 16:46:21 2017
From: tfb at tfeb.org (Tim Bradshaw)
Date: Mon, 15 May 2017 07:46:21 +0100
Subject: [TUHS] C declarations.
In-Reply-To: <20170513231121.0DB3A1274787@macaroni.inf.ed.ac.uk>
References: <20170513231121.0DB3A1274787@macaroni.inf.ed.ac.uk>
Message-ID: <E0BCD6A5-33EC-413A-B590-B1F801D072F5@tfeb.org>

On 14 May 2017, at 00:11, Richard Tobin <richard at inf.ed.ac.uk> wrote:
> 
>  If a subprogram reference causes a dummy argument in the referenced
>  subprogram to become associated with another dummy argument in the
>  referenced subprogram, neither dummy argument may become defined
>  during execution of that subprogram.

I think the purpose of that is to allow compilers to assume that if you pass two array arguments (say) to a routine then they are really different chunks of memory, so it can optimize things (ie it does not need to worry about the various hazards that happen if they are secretly the same memory).

From dot at dotat.at  Mon May 15 19:46:08 2017
From: dot at dotat.at (Tony Finch)
Date: Mon, 15 May 2017 10:46:08 +0100
Subject: [TUHS] Old Unix vulnerabilities
In-Reply-To: <1494741150.709655.975854064.7B3C6781@webmail.messagingengine.com>
References: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>
 <1494741150.709655.975854064.7B3C6781@webmail.messagingengine.com>
Message-ID: <alpine.DEB.2.11.1705151035170.2058@grey.csi.cam.ac.uk>

Random832 <random832 at fastmail.com> wrote:
> On Sat, May 13, 2017, at 19:34, Dave Horsfall wrote:
> >
> > A beauty in V6 (and possibly V7) was discovered by the kiddies in Elec
> > Eng; by sending a signal with an appropriately-crafted negative value (as
> > determined from inspecting <user.h>) you could overwrite u.u_uid with
> > zero...  Needless to say I scrambled to fix that one on my 11/40 network!
>
> V7 fixes it by changing the if(sig >= NSIG) in psignal to cast it to
> unsigned.

Even without that check V7 wouldn't be vulnerable. In V6, the
vulnerability occurs in psig() when the signal action is reset:

http://minnie.tuhs.org/cgi-bin/utree.pl?file=V6/usr/sys/ken/sig.c

	rp = u.u_procp;
	n = rp->p_sig;
	rp->p_sig = 0;
	if((p=u.u_signal[n]) != 0) {
		u.u_error = 0;
		if(n != SIGINS && n != SIGTRC)
			u.u_signal[n] = 0;
		/* if n < 0 this can overwrite u.u_uid */

In V7, instead of a single pending signal, there is a bitmap of pending
signals, so the corresponding code is,

http://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/sys/sys/sig.c

	n = fsig(rp);
	if (n==0)
		return;
	rp->p_sig &= ~(1<<(n-1));
	if((p=u.u_signal[n]) != 0) {
		u.u_error = 0;
		if(n != SIGINS && n != SIGTRC)
			u.u_signal[n] = 0;
		/* always within the array bounds */

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/  -  I xn--zr8h punycode
Viking, North Utsire, South Utsire, Northeast Forties: Variable becoming
southeasterly 3 or 4, increasing 5 to 7, perhaps gale 8 later. Slight or
moderate becoming moderate or rough later. Fog patches, rain later. Moderate,
occasionally very poor.


From dot at dotat.at  Mon May 15 20:21:17 2017
From: dot at dotat.at (Tony Finch)
Date: Mon, 15 May 2017 11:21:17 +0100
Subject: [TUHS] C declarations.
In-Reply-To: <20170515004019.GJ29107@mcvoy.com>
References: <e07f3d95ccda301190c64337424f49d8a0713316@webmail.yaccman.com>
 <6D3DCC45-B2C6-4374-83EE-F745C0AF7F36@bitblocks.com>
 <030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com>
 <013240e6-cc66-12c7-325d-a1edf4107726@kilonet.net>
 <CAEoi9W5RQxi+atsMaOWhNBFo=jahYH-HeHJh_QqHCh5FHxXZfw@mail.gmail.com>
 <20170515004019.GJ29107@mcvoy.com>
Message-ID: <alpine.DEB.2.11.1705151116010.2058@grey.csi.cam.ac.uk>

Larry McVoy <lm at mcvoy.com> wrote:
> On Sun, May 14, 2017 at 08:14:47PM -0400, Dan Cross wrote:
> > The kludge is that it works different than everything else for no really
> > good reason.
>
> I'd be curious what Steve thinks about all this, I know he weighed in
> a bit, but does he think that Dennis / Brian/ Ken regret this design choice?

I wonder how much of it was due to evolving in stages from BCPL, where
everything is a word, and structs and arrays are just pointers (with maybe
an allocation of space for the pointer to point to).

And C couldn't pass structs by value for quite a long time.

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/  -  I xn--zr8h punycode
Fair Isle, Faeroes: Southeast 5 or 6, increasing 7 or gale 8 for a time.
Moderate or rough, occasionally very rough in Faeroes. Fair then rain with fog
patches. Good, becoming moderate or very poor.


From michael at kjorling.se  Mon May 15 22:58:40 2017
From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=)
Date: Mon, 15 May 2017 12:58:40 +0000
Subject: [TUHS] C declarations.
In-Reply-To: <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
References: <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
 <20170514042411.93CCB124AEA4@mail.bitblocks.com>
 <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
Message-ID: <20170515125840.GA14426@yeono.kjorling.se>

On 15 May 2017 14:35 +1000, from dave at horsfall.org (Dave Horsfall):
> (Sigh; "cdecl" really needs to print a "> " prompt or something.)

It does print "cdecl> " as a prompt on my system.

   $ cdecl
   Type `help' or `?' for help
   cdecl> explain int*()* f
   syntax error
   cdecl> declare f as ptr to function returning ptr to int
   int *(*f)()
   cdecl> ^D
   $ 

However, running it with -q turns this off:

   $ cdecl -q
   declare f as ptr to function returning ptr to int
   int *(*f)()
   ^D
   $ 

The man page for mine says 2.5, Jan 1996, and the Debian package
version says 2.5-11. 1996 in 2017 is old even for Debian.

Are you sure that you don't have a stray alias somewhere, or something
that might be causing it to misdetect as not running in a terminal?

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)


From dave at horsfall.org  Tue May 16 02:58:24 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Tue, 16 May 2017 02:58:24 +1000 (EST)
Subject: [TUHS] C declarations.
In-Reply-To: <20170515125840.GA14426@yeono.kjorling.se>
References: <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
 <20170514042411.93CCB124AEA4@mail.bitblocks.com>
 <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
 <20170515125840.GA14426@yeono.kjorling.se>
Message-ID: <alpine.BSF.2.20.1705160248430.67621@aneurin.horsfall.org>

On Mon, 15 May 2017, Michael Kjörling wrote:

> It does print "cdecl> " as a prompt on my system.
[...]
> The man page for mine says 2.5, Jan 1996, and the Debian package
> version says 2.5-11. 1996 in 2017 is old even for Debian.

I've got 2.5 as well (on FreeBSD 10.3), and "set prompt" has no effect.  
I even rebuilt it in case there was a later version.

> Are you sure that you don't have a stray alias somewhere, or something 
> that might be causing it to misdetect as not running in a terminal?

aneurin% cdecl
Type `help' or `?' for help
set prompt
set options
[...]
Current set values are:
	 noprompt
	   interactive

WTF?

I'll fire up GDB and have a poke around (once I'm properly awake; I only 
got up for a glass of water, and couldn't help but check my email).


-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

From scj at yaccman.com  Tue May 16 04:47:51 2017
From: scj at yaccman.com (Steve Johnson)
Date: Mon, 15 May 2017 11:47:51 -0700
Subject: [TUHS] C declarations.
In-Reply-To: &lt;030d01d2cd06$c44e2840$4cea78c0$@ronnatalie.com&gt;
Message-ID: <c993682baa397327f63f0a1dd43b1be698281791@webmail.yaccman.com>



	Some interesting comments:

	    "You all are missing the point as to what the cost of passing
arrays by value or what other languages do"

	I don't think so.  To me the issues is that the model of what it
means to compute has changed since the punch-card days.  When you
submitted a card deck in the early days, you had to include both the
function definition and the data--the function was compiled, the data
was read, and, for the most part there were no significant side
effects (just a printout, and maybe some stuff on mag tape).

	This was a model that had served mathematics well for centuries, and
it was very easy to understand.  Functional programming people still
like it a lot...

	However, with the introduction of permanent file systems, a new
paradigm came into being.  Now, interactions with the computer looked
more like database transactions:  Load your program, change a few
lines, put it back, and then call 'make'.  Trying to describe this
with a purely functional model leads to absurdities like:

	     file_system = edit( file_system, file_selector,
editing_commands );

	In fact, the editing commands can change files, create new ones, and
even delete files.  There is no reasonable way to handle any
realistic file systems with this model (let alone the Internet!)

	In C's early days, we were just getting into the new world.  Call by
value for arrays would have been expensive or impossible on the
machine with just a few kilobytes of memory for program + data.  So
we didn't do it.

	Structures were initially handled like arrays, but the compiler chose
to make a local copy when passed a structure pointer.  This copy was,
at one time, in static memory, which caused some problems.  Later, it
went on the stack.  It wasn't much used...

	This changed when the Blit terminal project was in place.  It was
just too attractive on a 68000 to write

	    struct pt = {  int x;  int y }        /* when int was
16-bits */

	and I made PCC pass small structures like this in registers, like
other arguments.  I seem to remember a dramatic speedup (2X or so)
from doing this...

"(did) Dennis / Brian/ Ken regret this design choice?

	Not that I recall.  Of course, we all had bugs in this area.  But I
think the lack of subscript range checking was a more serious problem
than using pointers in the first place.  And, indeed, for a few of
the pioneers, BCPL had done exactly the same thing.  

	Steve


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170515/3b53b088/attachment.html>

From bakul at bitblocks.com  Tue May 16 05:00:13 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Mon, 15 May 2017 12:00:13 -0700
Subject: [TUHS] cdecl (Re:  C declarations.
In-Reply-To: <alpine.BSF.2.20.1705160248430.67621@aneurin.horsfall.org>
References: <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
 <20170514042411.93CCB124AEA4@mail.bitblocks.com>
 <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
 <20170515125840.GA14426@yeono.kjorling.se>
 <alpine.BSF.2.20.1705160248430.67621@aneurin.horsfall.org>
Message-ID: <056F4192-940A-48E1-9478-46016F7FD26A@bitblocks.com>

pkg upgrade cdecl

2.5 didn't show a prompt but 3.1.3 does (though the prompt is in /green/. Sigh...)

> On May 15, 2017, at 9:58 AM, Dave Horsfall <dave at horsfall.org> wrote:
> 
>> On Mon, 15 May 2017, Michael Kjörling wrote:
>> 
>> It does print "cdecl> " as a prompt on my system.
> [...]
>> The man page for mine says 2.5, Jan 1996, and the Debian package
>> version says 2.5-11. 1996 in 2017 is old even for Debian.
> 
> I've got 2.5 as well (on FreeBSD 10.3), and "set prompt" has no effect.  
> I even rebuilt it in case there was a later version.
> 
>> Are you sure that you don't have a stray alias somewhere, or something 
>> that might be causing it to misdetect as not running in a terminal?
> 
> aneurin% cdecl
> Type `help' or `?' for help
> set prompt
> set options
> [...]
> Current set values are:
>     noprompt
>       interactive
> 
> WTF?
> 
> I'll fire up GDB and have a poke around (once I'm properly awake; I only 
> got up for a glass of water, and couldn't help but check my email).
> 
> 
> -- 
> Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."



From bakul at bitblocks.com  Tue May 16 05:54:25 2017
From: bakul at bitblocks.com (Bakul Shah)
Date: Mon, 15 May 2017 12:54:25 -0700
Subject: [TUHS] C declarations.
In-Reply-To: <c993682baa397327f63f0a1dd43b1be698281791@webmail.yaccman.com>
References: <c993682baa397327f63f0a1dd43b1be698281791@webmail.yaccman.com>
Message-ID: <27EF4DFC-5972-424F-BB22-0B2BD1593A45@bitblocks.com>

Pascal was designed in 1967-69 and its report came out in 1970. It is of similar complexity (the first self compiling compiler was about 4500 lines, IIRC). It had n dimensional fixed size arrays and structures (records). Allowing any parameter of any type to be a ref param meant you didn't have to copy if you didn't want to. And there is no hidden magic, no GC. I guess it wasn't much known in the US then so not much scope for stealing ideas.... I guess the difference was that Wirth started with Algol W while Ritchie stated with B and BCPL.

Unlike pascal C had the address of (&) operator so C could've only allow passing pointers to them. This would've been less "magic" and much cleaner. And from there a ref parameter is just a small step! To me your earlier comment made more sense - C was seen as a system programming language not an application programming one. For it passing strings and memory blocks was enough. Earlier I had wondered if dmr et al had a more Fortran like array model in mind and punted on it as it would complicate the language.

> On May 15, 2017, at 11:47 AM, Steve Johnson <scj at yaccman.com> wrote:
> 
> 
> Some interesting comments:
> 
>     "You all are missing the point as to what the cost of passing arrays by value or what other languages do"
> 
> I don't think so.  To me the issues is that the model of what it means to compute has changed since the punch-card days.  When you submitted a card deck in the early days, you had to include both the function definition and the data--the function was compiled, the data was read, and, for the most part there were no significant side effects (just a printout, and maybe some stuff on mag tape).
> 
> This was a model that had served mathematics well for centuries, and it was very easy to understand.  Functional programming people still like it a lot...
> 
> However, with the introduction of permanent file systems, a new paradigm came into being.  Now, interactions with the computer looked more like database transactions:  Load your program, change a few lines, put it back, and then call 'make'.  Trying to describe this with a purely functional model leads to absurdities like:
> 
>      file_system = edit( file_system, file_selector, editing_commands );
> 
> In fact, the editing commands can change files, create new ones, and even delete files.  There is no reasonable way to handle any realistic file systems with this model (let alone the Internet!)
> 
> In C's early days, we were just getting into the new world.  Call by value for arrays would have been expensive or impossible on the machine with just a few kilobytes of memory for program + data.  So we didn't do it.
> 
> Structures were initially handled like arrays, but the compiler chose to make a local copy when passed a structure pointer.  This copy was, at one time, in static memory, which caused some problems.  Later, it went on the stack.  It wasn't much used...
> 
> This changed when the Blit terminal project was in place.  It was just too attractive on a 68000 to write
> 
>     struct pt = {  int x;  int y }        /* when int was 16-bits */
> 
> and I made PCC pass small structures like this in registers, like other arguments.  I seem to remember a dramatic speedup (2X or so) from doing this...
> 
> 
> "(did) Dennis / Brian/ Ken regret this design choice?
> 
> Not that I recall.  Of course, we all had bugs in this area.  But I think the lack of subscript range checking was a more serious problem than using pointers in the first place.  And, indeed, for a few of the pioneers, BCPL had done exactly the same thing.  
> 
> 
> 
> Steve
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170515/8e372208/attachment.html>

From dave at horsfall.org  Tue May 16 08:52:43 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Tue, 16 May 2017 08:52:43 +1000 (EST)
Subject: [TUHS] cdecl (Re:  C declarations.
In-Reply-To: <056F4192-940A-48E1-9478-46016F7FD26A@bitblocks.com>
References: <20170511223232.GM4341@mcvoy.com>
 <015b01d2caa7$c658c020$530a4060$@ronnatalie.com>
 <20170513012415.GZ4341@mcvoy.com>
 <025701d2cb92$ec8182a0$c58487e0$@ronnatalie.com>
 <20170513122050.GF9980@yeono.kjorling.se>
 <0CF82AC1-E835-4C06-813F-D9EFD2C12290@tfeb.org>
 <20170513124247.GG9980@yeono.kjorling.se>
 <57639684-050B-4995-859F-876CAFE4BE9C@serissa.com>
 <20170514042411.93CCB124AEA4@mail.bitblocks.com>
 <alpine.BSF.2.20.1705151433050.67621@aneurin.horsfall.org>
 <20170515125840.GA14426@yeono.kjorling.se>
 <alpine.BSF.2.20.1705160248430.67621@aneurin.horsfall.org>
 <056F4192-940A-48E1-9478-46016F7FD26A@bitblocks.com>
Message-ID: <alpine.BSF.2.20.1705160850460.67621@aneurin.horsfall.org>

On Mon, 15 May 2017, Bakul Shah wrote:

> pkg upgrade cdecl

Didn't work; all my packages are up to date, so I guess you're running 
FreeBSD 11 or something?

> 2.5 didn't show a prompt but 3.1.3 does (though the prompt is in 
> /green/. Sigh...)

I'll wait for 3.1.3 to be ported then, although I might bash ahead with 
GDB anyway.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From scj at yaccman.com  Tue May 16 04:00:36 2017
From: scj at yaccman.com (Steve Johnson)
Date: Mon, 15 May 2017 11:00:36 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <CAJfiPzyatQQ7_+146ccs5+wDuNyb4fRQifMxeEdJKOYfuaZp2w@mail.gmail.com>
Message-ID: <d3bf73248162b9530b667787b7ac8dfbb59e3496@webmail.yaccman.com>

Early in the Unix days, a DEC repairperson showed up to do "preventive
maintenance" and managed to clobber the nascent file system.   Turns
out DEC didn't have any permanent file systems on machines that
small...

----- Original Message -----
From: "Nemo" <cym224 at gmail.com>
To:"The Eunuchs Hysterical Society" <tuhs at tuhs.org>
Cc:
Sent:Sun, 14 May 2017 21:24:25 -0400
Subject:Re: [TUHS] The evolution of Unix facilities and architecture

 On 14 May 2017 at 18:12, Dave Horsfall <dave at horsfall.org> wrote:
 > On Sun, 14 May 2017, Derek Fawcus wrote:
 >
 >> > to see DEC's internal boxes weren't running System/Manager,
 >> > Field/Service and UETP/UETP User/password combinations.
 >>
 >> Those default account combinations were still being used to gain
access
 >> to VMS systems in the '87-'89 time frame; although user/password
was
 >> less interesting by itself, being an unpriviledged account.
 >
 > Wasn't there also Guest/Guest as well? Admittedly it would also be
pretty
 > boring, but nonetheless still a toe-hold.

 I worked in a VAX shop once where a DEC FSE came by (on the wrong day
 with the sysadmin out) and was rather upset that the default account
 passwords had been changed.

 N.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170515/008815c8/attachment.html>

From gdmr at inf.ed.ac.uk  Tue May 16 17:25:20 2017
From: gdmr at inf.ed.ac.uk (George Ross)
Date: Tue, 16 May 2017 08:25:20 +0100
Subject: [TUHS] C declarations.
In-Reply-To: Your message of "Mon, 15 May 2017 11:47:51 PDT."
 <c993682baa397327f63f0a1dd43b1be698281791@webmail.yaccman.com>
Message-ID: <201705160725.v4G7PKiA009291@farg.inf.ed.ac.uk>

> ...  When you
> submitted a card deck in the early days, you had to include both the
> function definition and the data--the function was compiled, the data
> was read, and, for the most part there were no significant side
> effects (just a printout, and maybe some stuff on mag tape).

In the late 1970s I had a summer job which involved writing FORTRAN 
programs to analyse card decks.  This was on some IBM machine, which had a 
combi card punch/reader.  On one run the program loaded and compiled, and 
the first two data cards went through.  Then the machine crashed.

After a bit of head-scratching, I realised that it was because the punch 
station came immediately before the read station, and there was a small 
typo in the program.  It processed the first data card, but rather than 
logging to the printer it overpunched the second card instead resulting in 
an invalid hole combination.  That card then went into the read station, 
and the machine didn't like it...
-- 
George D M Ross MSc PhD CEng MBCS CITP, University of Edinburgh,
School of Informatics, 10 Crichton Street, Edinburgh, Scotland, EH8 9AB
Mail: gdmr at inf.ed.ac.uk   Voice: 0131 650 5147   Fax: 0131 650 6899
PGP: 1024D/AD758CC5  B91E D430 1E0D 5883 EF6A  426C B676 5C2B AD75 8CC5

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.




From jnc at mercury.lcs.mit.edu  Tue May 16 23:20:34 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Tue, 16 May 2017 09:20:34 -0400 (EDT)
Subject: [TUHS] The evolution of Unix facilities and architecture
Message-ID: <20170516132034.E998318C089@mercury.lcs.mit.edu>

    > From: "Steve Johnson"

    > a DEC repairperson showed up to do "preventive maintenance" and managed
    > to clobber the nascent file system.
    > Turns out DEC didn't have any permanent file systems on machines that
    > small...

A related story (possibly a different version of this one) which I read (can't
remember where, now) was that he trashed the contents of the RS04 fixed-head
hard disk, because on DEC OS's, those were only used for swapping.

	Noel


From clemc at ccc.com  Tue May 16 23:46:58 2017
From: clemc at ccc.com (Clem Cole)
Date: Tue, 16 May 2017 09:46:58 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <20170516132034.E998318C089@mercury.lcs.mit.edu>
References: <20170516132034.E998318C089@mercury.lcs.mit.edu>
Message-ID: <CAC20D2PdbVFtr+DBRD88DRfEV5bzv7bLapbdLOa3csrLmkTJ9A@mail.gmail.com>

I believe it is a true story...   The issue was that any early version of
the FE tools was set to use the RS04 as a temporary disk when running the
system exercisers if it found one.   The author of that error is a friend
of mine (and would later become a UNIX guy at Masscomp).  He said to me
about it once. he had checked with the DEC OS teams and thought it was an
'ok' because when they did it, none of the DEC OS used the RS04 for
permanent storage - the device had been designed to be a swapping device.

>From what I understand, the issue was actually short lived, but widely
known in the UNIX community.   He told me it the accident only occurred at
one site that he knew about (AT&T) and they made a quick change to have it
ask before it wiped it out and that verion of the tools was release to the
field quickly.  But he was personally wise to UNIX from them on (and it
later years would come to love become a UNIX a user although I don't think
he ever gave up the EDT macros we wrote for EMACS for him and the other
ex-VMS folks).

Clem

On Tue, May 16, 2017 at 9:20 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

>     > From: "Steve Johnson"
>
>     > a DEC repairperson showed up to do "preventive maintenance" and
> managed
>     > to clobber the nascent file system.
>     > Turns out DEC didn't have any permanent file systems on machines that
>     > small...
>
> A related story (possibly a different version of this one) which I read
> (can't
> remember where, now) was that he trashed the contents of the RS04
> fixed-head
> hard disk, because on DEC OS's, those were only used for swapping.
>
>         Noel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170516/8ce42c3e/attachment.html>

From ron at ronnatalie.com  Wed May 17 08:33:50 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Tue, 16 May 2017 18:33:50 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <d3bf73248162b9530b667787b7ac8dfbb59e3496@webmail.yaccman.com>
References: <CAJfiPzyatQQ7_+146ccs5+wDuNyb4fRQifMxeEdJKOYfuaZp2w@mail.gmail.com>
 <d3bf73248162b9530b667787b7ac8dfbb59e3496@webmail.yaccman.com>
Message-ID: <009101d2ce94$7de8dbe0$79ba93a0$@ronnatalie.com>

Our biggest UNIX vs. DEC OS problem was that UNIX set the system clock in GMT whereas the DEC OSes of the day used local time.

We got used to the time being 4/5 hours off after a DEC CE was there.

 

We actually contracted our maintenance out for the entire company.    We ended up going with GE.   We couldn’t convince the guys from DEC who bid that when we said there were certain critical components of the site that needed 24-hour response, that this meant, 24 hours in a row, not three successive 8 hour days.

 

We often were stuck using a DEC CE for initial system setup or warranty work.    We had a particularly bad one who would blow power supplies, screw up running systems, etc.    She was just simply incompetent.   The culmination of this was when she managed somehow to put herself across the AC line of a VAX over in one of our external buildings and ended up being taken a way in an ambulance.    Working off hours, we’d often set up the new machines and run diagnostic checks on them ahead of the CE showing up.    I got a testy CE show up and tell me that he “didn’t need me checking up on his work.”  (Hell, was the COTR and customer).   I told him I only had one word to say about that <insert the incompetent CE’s name>.   He beat a quick retreat saying that Nancy was a different story.

 

I was driving to work one morning at Christmas time, and one of our local radio stations was soliciting people to send in their sob stories about how bad a year they had and they would be given a special gift.   One story went on for a few minutes, and I hadn’t caught on until it got to the electrical shock at work part and I knew it was Nancy’s story.

 

Our standard joke was that the way you could tell a DEC CE with a flat tire was that he had to change all four before he found the problem.

 

Amusingly, working for the feds had some other interesting fiascos.    I got an amusing message from the security and facilities people one day.    I had to tell our CE.

 

ME:   Bill, I can’t let you in the machine room anymore.

BILL:   Why not 

ME:   You’re a fire hazard.

BILL:   How so?

ME:  You have soldering irons.

 

Of course, I was able to prevail on them that we’d keep an eye on the CE and stand by with fire suppression if we let him do his job.    The machine room in my building had no automatic halon system which was popular in those days.    What we had was a lot of large halon hand extinguishers.    The post fire department came out and set pan fires behind our building and let us practice putting them out with the halon.     I can’t imagine what the costs to the federal budget and the ozone layer were on that little activity.    Of course, I brought my turnout gear as I was a firefighter and paramedic at the time.    This led to another interesting call from the front office.

 

SEC:    You need to attend a CPR class.

ME:    I’ve already had one this year.   I’m a state certified paramedic.   I go through recurrent training every month.

SEC:   Well it is a requirement that you have a CPR card.

ME:   Why am I the only one in my office that this is a requirement for?

SEC:    It’s your job classification.

ME:   Because I’m an electrical engineer and the other guys are computer scientists?

SEC:   Yes.

ME:   Why?

SEC:  Because you work with electricity.

ME:   I work with digital logic.   Five volts.   Further, even if I was going to shock myself into cardiac arrest, I can’t do CPR on  myself.   You should make everybody else take CPR.

 

 

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170516/0b1f4efd/attachment.html>

From krewat at kilonet.net  Wed May 17 09:13:03 2017
From: krewat at kilonet.net (Arthur Krewat)
Date: Tue, 16 May 2017 19:13:03 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <009101d2ce94$7de8dbe0$79ba93a0$@ronnatalie.com>
References: <CAJfiPzyatQQ7_+146ccs5+wDuNyb4fRQifMxeEdJKOYfuaZp2w@mail.gmail.com>
 <d3bf73248162b9530b667787b7ac8dfbb59e3496@webmail.yaccman.com>
 <009101d2ce94$7de8dbe0$79ba93a0$@ronnatalie.com>
Message-ID: <14366930-f60d-023d-6591-6f45df26ff78@kilonet.net>

On 5/16/2017 6:33 PM, Ron Natalie wrote:
> Our standard joke was that the way you could tell a DEC CE with a flat 
> tire was that he had to change all four before he found the problem.

Conversely, at BOCES LIRICS in Dix Hills, NY, we had a field-service 
tech who seemed to know every single wire wrap in the PDP-10's. First 
the KA10 they had before I got there (but worked on in high school), and 
then the KS10's.

Maybe the VAX/PDP-11 crew were different :)

On the other hand, he left in his desk a complete list of KLINIK line 
phone numbers (modems connected in parallel with the CTY), and 
PASSWORDS, to just about every PDP-10 in the tri-state area (NY/NJ/CT).




From ron at ronnatalie.com  Wed May 17 09:18:29 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Tue, 16 May 2017 19:18:29 -0400
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <14366930-f60d-023d-6591-6f45df26ff78@kilonet.net>
References: <CAJfiPzyatQQ7_+146ccs5+wDuNyb4fRQifMxeEdJKOYfuaZp2w@mail.gmail.com>
 <d3bf73248162b9530b667787b7ac8dfbb59e3496@webmail.yaccman.com>
 <009101d2ce94$7de8dbe0$79ba93a0$@ronnatalie.com>
 <14366930-f60d-023d-6591-6f45df26ff78@kilonet.net>
Message-ID: <00e401d2ce9a$bacbf5a0$3063e0e0$@ronnatalie.com>

The wirewrap comment reminds me of another catastrophe.   BRL had one of the few operational Denelcor HEP supercomputers.    Amusingly the thing was front ended with an 11/34 and had an io processor with 32 Unibuses connected to it.    While we were still shaking the thing out at the factory (Mike Muuss shot his mouth off that we could get UNIX to run on the thing, and it turned out nobody had a better idea).    We'd come in at night (while the Denelcor employees had the machine in the day).    I'd regularly read the log about what happened that day.    Well, one day it read that someone had neatened up one of the system backplanes by taking the wires that were overly long and shortening them.    

Now this thing was all built out of 10800 ECL and those wires were "overly long" and stuffed into the backplane because the length was needed to control the signal propagation times.    I told Mike I had a bad feeling about this and the machine might not be usable.




From tim.newsham at gmail.com  Fri May 19 03:32:07 2017
From: tim.newsham at gmail.com (Tim Newsham)
Date: Thu, 18 May 2017 07:32:07 -1000
Subject: [TUHS] Old Unix vulnerabilities
In-Reply-To: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>
References: <alpine.BSF.2.20.1705140929150.67621@aneurin.horsfall.org>
Message-ID: <CAGSRWbgm4FWiO2yx7zeEFq-UO1Lt1KEmkxpKzhP7pFzo=PLWsQ@mail.gmail.com>

Here are some previously reported ones:

v1:
  http://minnie.tuhs.org/pipermail/unix-jun72/2008-May/000126.html
  http://minnie.tuhs.org/pipermail/unix-jun72/2008-May/000250.html

v7:
  http://seclists.org/bugtraq/2004/Jun/37


On Sat, May 13, 2017 at 1:34 PM, Dave Horsfall <dave at horsfall.org> wrote:

> OK, I'll kick it off.
>
> A beauty in V6 (and possibly V7) was discovered by the kiddies in Elec
> Eng; by sending a signal with an appropriately-crafted negative value (as
> determined from inspecting <user.h>) you could overwrite u.u_uid with
> zero...  Needless to say I scrambled to fix that one on my 11/40 network!
>
> --
> Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will
> suffer."
>



-- 
Tim Newsham | www.thenewsh.com/~newsham | @newshtwit | thenewsh.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170518/4c4b64b5/attachment.html>

From david at kdbarto.org  Sat May 20 00:31:54 2017
From: david at kdbarto.org (David)
Date: Fri, 19 May 2017 07:31:54 -0700
Subject: [TUHS] The evolution of Unix facilities and architecture
In-Reply-To: <mailman.1.1494986402.2329.tuhs@minnie.tuhs.org>
References: <mailman.1.1494986402.2329.tuhs@minnie.tuhs.org>
Message-ID: <C8EB9A79-2334-43D8-B7AC-CB2548B733E1@kdbarto.org>


At Celerity we were porting Unix to a new NCR chipset for our washing machine sized Workstation.
We had a VAX 750 as the development box and we cross compiled to the NCR box. We contracted
out the 750 maintenance to a 3rd party and had no problems for a couple of years. Then one day I
came in to work to find the VAX happy consuming power and doing nothing. Unix wasn’t running and
nothing I could do would bring it back. After about 2 hours I got my boss and we contacted the maintenance
company. They guy they sent did much what I’d done and then went around the back. He pushed on the
backplane of the machine and Lo, it started working. He then removed the pressure and it failed quite
immediately. Turns out the backplane had a broken trace in it. We had done no board swaps in many
months and the room had had no A/C faults of any kind.

The company got a new backplane and had it installed in 2 days. Being 3rd party we couldn’t get it
replaced any quicker. After that it worked like a champ.

Celerity eventually became part of Sun as Sun Supercomputer.

	David



From jnc at mercury.lcs.mit.edu  Sat May 20 01:15:18 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Fri, 19 May 2017 11:15:18 -0400 (EDT)
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
Message-ID: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>

I'm trying to find out if there is any existing Unix for the PDP-11
which supports TCP/IP on /40-/34-23 class machines (i.e. non-I+D
machines)?

Does 2.9 BSD with TCP/IP (assuming such a thing exists) fit on those
machines? (I know 2.9 does run on them, but I don't know about the TCP/IP
part.)


The reason I ask is that MIT did a TCP/IP for V6 which would run on them
(only incoming packet de-mux is in the kernel - the TCP is in with the
application, in the user process), which has recently been recovered from a
backup tape.

I'm trying to figure out if there is any use for it, as it would take some
work to make it usable (I'd have to write device drivers for available
Ethernet cards, and adapt an ARP implementation for it).

	Noel


From henry.r.bent at gmail.com  Sat May 20 02:29:54 2017
From: henry.r.bent at gmail.com (Henry Bent)
Date: Fri, 19 May 2017 12:29:54 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
Message-ID: <CAEdTPBd6vYKE-tP_-7UmDaqrcf+wtNu0W-PLLKQMsT14wikZ9A@mail.gmail.com>

There is a later patch release of 2.9BSD that includes TCP/IP support but I
have not had any luck getting a compiled kernel to boot. My recommendation
would be Ultrix 3.1 which does run on the smaller machines.

I think it is possible that 2.10BSD could be made to work but I have never
had any success booting it in SIMH. If anyone has booted it or has any
suggestions, I would be interested to hear it.

-Henry

On May 19, 2017 11:15 AM, "Noel Chiappa" <jnc at mercury.lcs.mit.edu> wrote:

I'm trying to find out if there is any existing Unix for the PDP-11
which supports TCP/IP on /40-/34-23 class machines (i.e. non-I+D
machines)?

Does 2.9 BSD with TCP/IP (assuming such a thing exists) fit on those
machines? (I know 2.9 does run on them, but I don't know about the TCP/IP
part.)


The reason I ask is that MIT did a TCP/IP for V6 which would run on them
(only incoming packet de-mux is in the kernel - the TCP is in with the
application, in the user process), which has recently been recovered from a
backup tape.

I'm trying to figure out if there is any use for it, as it would take some
work to make it usable (I'd have to write device drivers for available
Ethernet cards, and adapt an ARP implementation for it).

        Noel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170519/f37cd216/attachment.html>

From clemc at ccc.com  Sat May 20 03:39:41 2017
From: clemc at ccc.com (Clem Cole)
Date: Fri, 19 May 2017 13:39:41 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
Message-ID: <CAC20D2PhQwCiPuSNZBhJUng=ZoE-JgTPayvhf+UkNctNuZ3POg@mail.gmail.com>

My quick answer is yes, this sounds cool.

>From that time, that two attempts to network 40 class systems that I know
about were:


   -  As I have mentioned in the past, in the late 1970s CMU had developed
   an 8085 later 8086 based Multibus board that would eventually begat the
   Cisco AGS, called the CMU distributed Front-End.   It was an attempt to get
   IP connected to the 11/40 class systems, by splicing them into the 11's
   with an DR-11B or C (I've forgotten which).   But the primary protocol
   processing was done on the FE and not in the UNIX kernel.     Phil Karn
   might remember what happened to that stuff, I do not believe I still have
   it.
   - Shortly thereafter @ Tektronix had 3Com's UNET.  I was primarily
   running on the 11/70 and that was what we used to develop against the Cyber
   and VMS IP/TCP's we wrote.  We also had it on an 11/44 but both of those
   are 45 class.     We did try to run on an 11/60 - which is 40 class as you
   asked, and we did get it work as I recall after a fashion.   But we did not
   push it very hard;as we had built a front end that in the same vein as the
   CMU device, called the NIBB - Network Interface Black Box, which was a Z80
   that did the same sort of thing and the need was not there.

 I was under the impression the 2.X folks may have tried to do it later
with all the thunk code.  But that was all after my work.   Fred Canter at
DEC might have tried something in later versions of Ultrix/PDP11, we should
look at the sources in his stream had see what he did too.

Clem

On Fri, May 19, 2017 at 11:15 AM, Noel Chiappa <jnc at mercury.lcs.mit.edu>
wrote:

> I'm trying to find out if there is any existing Unix for the PDP-11
> which supports TCP/IP on /40-/34-23 class machines (i.e. non-I+D
> machines)?
>
> Does 2.9 BSD with TCP/IP (assuming such a thing exists) fit on those
> machines? (I know 2.9 does run on them, but I don't know about the TCP/IP
> part.)
>
>
> The reason I ask is that MIT did a TCP/IP for V6 which would run on them
> (only incoming packet de-mux is in the kernel - the TCP is in with the
> application, in the user process), which has recently been recovered from a
> backup tape.
>
> I'm trying to figure out if there is any use for it, as it would take some
> work to make it usable (I'd have to write device drivers for available
> Ethernet cards, and adapt an ARP implementation for it).
>
>         Noel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170519/56bea0f0/attachment.html>

From ron at ronnatalie.com  Sat May 20 05:04:09 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Fri, 19 May 2017 15:04:09 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAEdTPBd6vYKE-tP_-7UmDaqrcf+wtNu0W-PLLKQMsT14wikZ9A@mail.gmail.com>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAEdTPBd6vYKE-tP_-7UmDaqrcf+wtNu0W-PLLKQMsT14wikZ9A@mail.gmail.com>
Message-ID: <014301d2d0d2$b2b8f0e0$182ad2a0$@ronnatalie.com>

To my knowledge no.   We had already gone to code overlays on the 11/34’s just to get the pre-TCP unix working.    When we needed yet another segment to map mbufs (which we could do on the 11/70), there just weren’t any left.      It was then that I scarfed up all the 11/23, 24, 34’s that were available and turned them into routers.    

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170519/75140600/attachment.html>

From imp at bsdimp.com  Sat May 20 10:15:19 2017
From: imp at bsdimp.com (Warner Losh)
Date: Fri, 19 May 2017 18:15:19 -0600
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <014301d2d0d2$b2b8f0e0$182ad2a0$@ronnatalie.com>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAEdTPBd6vYKE-tP_-7UmDaqrcf+wtNu0W-PLLKQMsT14wikZ9A@mail.gmail.com>
 <014301d2d0d2$b2b8f0e0$182ad2a0$@ronnatalie.com>
Message-ID: <CANCZdfoCMEVG69=fHNHhxv=K=6xBe33kFWxYNUM+uSxKP0a2Ug@mail.gmail.com>

There's a copy of ultrix-11, which is v7 based with some 4.1BSD additions,
including sockets and a TCP/IP stack. Don't know if it runs on the smaller
PDP-11, but it looks like it might. The sources are in the TUHS archives.
Seems like the best low-end v7ish kernel with TCP/IP around, but I've not
used it extensively.

Warner

On Fri, May 19, 2017 at 1:04 PM, Ron Natalie <ron at ronnatalie.com> wrote:

> To my knowledge no.   We had already gone to code overlays on the 11/34’s
> just to get the pre-TCP unix working.    When we needed yet another segment
> to map mbufs (which we could do on the 11/70), there just weren’t any
> left.      It was then that I scarfed up all the 11/23, 24, 34’s that were
> available and turned them into routers.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170519/8b6f7875/attachment.html>

From imp at bsdimp.com  Sat May 20 10:21:01 2017
From: imp at bsdimp.com (Warner Losh)
Date: Fri, 19 May 2017 18:21:01 -0600
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CANCZdfoCMEVG69=fHNHhxv=K=6xBe33kFWxYNUM+uSxKP0a2Ug@mail.gmail.com>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAEdTPBd6vYKE-tP_-7UmDaqrcf+wtNu0W-PLLKQMsT14wikZ9A@mail.gmail.com>
 <014301d2d0d2$b2b8f0e0$182ad2a0$@ronnatalie.com>
 <CANCZdfoCMEVG69=fHNHhxv=K=6xBe33kFWxYNUM+uSxKP0a2Ug@mail.gmail.com>
Message-ID: <CANCZdfoibU4ftDtz_zkvb675-=Q9a7jy=vnugF-07it7=gJPRg@mail.gmail.com>

https://ia601901.us.archive.org/10/items/bitsavers_decpdp11ulLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf

Looks like it requires MMU, but not split I/D space as it lists the
following as compatible: M11, 11/23+, 11/24, 11/34, 11/40 and 11/60. It
does require 256kb of memory. See table 2, page 6 for details.

Warner



On Fri, May 19, 2017 at 6:15 PM, Warner Losh <imp at bsdimp.com> wrote:

> There's a copy of ultrix-11, which is v7 based with some 4.1BSD additions,
> including sockets and a TCP/IP stack. Don't know if it runs on the smaller
> PDP-11, but it looks like it might. The sources are in the TUHS archives.
> Seems like the best low-end v7ish kernel with TCP/IP around, but I've not
> used it extensively.
>
> Warner
>
> On Fri, May 19, 2017 at 1:04 PM, Ron Natalie <ron at ronnatalie.com> wrote:
>
>> To my knowledge no.   We had already gone to code overlays on the 11/34’s
>> just to get the pre-TCP unix working.    When we needed yet another segment
>> to map mbufs (which we could do on the 11/70), there just weren’t any
>> left.      It was then that I scarfed up all the 11/23, 24, 34’s that were
>> available and turned them into routers.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170519/a381ab39/attachment.html>

From bqt at update.uu.se  Sat May 20 20:46:34 2017
From: bqt at update.uu.se (Johnny Billquist)
Date: Sat, 20 May 2017 12:46:34 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
Message-ID: <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>

On 2017-05-20 04:00, Warner Losh <imp at bsdimp.com> wrote:

> https://ia601901.us.archive.org/10/items/bitsavers_decpdp11ulLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf
>
> Looks like it requires MMU, but not split I/D space as it lists the
> following as compatible: M11, 11/23+, 11/24, 11/34, 11/40 and 11/60. It
> does require 256kb of memory. See table 2, page 6 for details.


Uh...? Where do you see that there is any TCP/IP support in Ultrix-11? 
If any was done by someone else, there is no saying that it would be 
usable on a machine without split I/D. To be honest, I've never seen any 
mention of TCP/IP on any machine without split I/D space. I guess it 
could be done, but it would be a rather big headache...

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol


From henry.r.bent at gmail.com  Sun May 21 03:39:26 2017
From: henry.r.bent at gmail.com (Henry Bent)
Date: Sat, 20 May 2017 13:39:26 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
Message-ID: <CAEdTPBeoA6ivRXxSD9XKjhYEi5tqczBcUZ0==Wc-4Rj=GjuOGA@mail.gmail.com>

There is absolutely TCP/IP in Ultrix-11.  This is an emulated 11/23+:

--
hrb11# uname -a
ULTRIX-11 hrb11 3 0 PDP-11/23
hrb11# ifconfig qe0
qe0: 192.168.1.199 netmask ffffff00
flags=263<UP,BROADCAST,NOTRAILERS,RUNNING>
broadcast: 192.168.1.255
hrb11# netstat -rn
Routing tables
Destination     Gateway         Flags    Refcnt Use        Interface
74.42.148.138   192.168.1.1     UGH      0      0          qe0
192.168.1       192.168.1.199   U        1      0          qe0
default         192.168.1.1     UG       0      0          qe0
192.168.5       192.168.1.1     UG       0      0          qe0
hrb11# ftp 192.168.1.63
Connected to 192.168.1.63.
220 aelfric FTP server (Version 6.4/OpenBSD/Linux-ftpd-0.17) ready.
Name (192.168.1.63:root):
...
--

It has trouble talking to the outside world, probably not a big surprise,
but works just fine on my local network.  I am working on getting it
installed in a real 11/23 but the Ultrix RL02 driver doesn't seem to like
my Dilog emulation board.

-Henry

On 20 May 2017 at 06:46, Johnny Billquist <bqt at update.uu.se> wrote:

> On 2017-05-20 04:00, Warner Losh <imp at bsdimp.com> wrote:
>
> https://ia601901.us.archive.org/10/items/bitsavers_decpdp11u
>> lLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf
>>
>> Looks like it requires MMU, but not split I/D space as it lists the
>> following as compatible: M11, 11/23+, 11/24, 11/34, 11/40 and 11/60. It
>> does require 256kb of memory. See table 2, page 6 for details.
>>
>
>
> Uh...? Where do you see that there is any TCP/IP support in Ultrix-11? If
> any was done by someone else, there is no saying that it would be usable on
> a machine without split I/D. To be honest, I've never seen any mention of
> TCP/IP on any machine without split I/D space. I guess it could be done,
> but it would be a rather big headache...
>
>         Johnny
>
> --
> Johnny Billquist                  || "I'm on a bus
>                                   ||  on a psychedelic trip
> email: bqt at softjar.se             ||  Reading murder books
> pdp is alive!                     ||  tryin' to stay hip" - B. Idol
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170520/f571a6a9/attachment.html>

From imp at bsdimp.com  Sun May 21 04:18:30 2017
From: imp at bsdimp.com (Warner Losh)
Date: Sat, 20 May 2017 12:18:30 -0600
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
Message-ID: <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>

On Sat, May 20, 2017 at 4:46 AM, Johnny Billquist <bqt at update.uu.se> wrote:

> On 2017-05-20 04:00, Warner Losh <imp at bsdimp.com> wrote:
>
> https://ia601901.us.archive.org/10/items/bitsavers_decpdp11u
>> lLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf
>>
>> Looks like it requires MMU, but not split I/D space as it lists the
>> following as compatible: M11, 11/23+, 11/24, 11/34, 11/40 and 11/60. It
>> does require 256kb of memory. See table 2, page 6 for details.
>>
>
>
> Uh...? Where do you see that there is any TCP/IP support in Ultrix-11? If
> any was done by someone else, there is no saying that it would be usable on
> a machine without split I/D. To be honest, I've never seen any mention of
> TCP/IP on any machine without split I/D space. I guess it could be done,
> but it would be a rather big headache...


I read the sources to see the TCP/IP support was there (that's the bit
about adding Berkeley Sockets). I see nowhere that it's excluded for the
non I/D machines, but haven't tried it first hand. I got interested not
because of the PDP-11, but because I have an old Rainbow that recently
started running Venix (v7-based version) and was trolling around for some
way to do TCP/IP to it (though w/o readily available ethernet cards, I'm
not sure it is a viable project).

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170520/391c19a1/attachment.html>

From arnold at skeeve.com  Sun May 21 05:05:59 2017
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Sat, 20 May 2017 13:05:59 -0600
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
Message-ID: <201705201905.v4KJ5xcq005292@freefriends.org>

Warner Losh <imp at bsdimp.com> wrote:

> I read the sources to see the TCP/IP support was there (that's the bit
> about adding Berkeley Sockets). I see nowhere that it's excluded for the
> non I/D machines, but haven't tried it first hand. I got interested not
> because of the PDP-11, but because I have an old Rainbow that recently
> started running Venix (v7-based version) and was trolling around for some
> way to do TCP/IP to it (though w/o readily available ethernet cards, I'm
> not sure it is a viable project).

Boy is the memory going.  What was the TCP/IP implementation people
ran on DOS to do connections over serial lines?  Could that be found
and revived for such a system?

Thanks,

Arnold


From imp at bsdimp.com  Sun May 21 06:29:20 2017
From: imp at bsdimp.com (Warner Losh)
Date: Sat, 20 May 2017 14:29:20 -0600
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <201705201905.v4KJ5xcq005292@freefriends.org>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
 <201705201905.v4KJ5xcq005292@freefriends.org>
Message-ID: <CANCZdfoN6dhdUNfCTDfLTDMMY6X5R9BTbPWAy6ozUgZ0FvGGSg@mail.gmail.com>

On Sat, May 20, 2017 at 1:05 PM, <arnold at skeeve.com> wrote:

> Warner Losh <imp at bsdimp.com> wrote:
>
> > I read the sources to see the TCP/IP support was there (that's the bit
> > about adding Berkeley Sockets). I see nowhere that it's excluded for the
> > non I/D machines, but haven't tried it first hand. I got interested not
> > because of the PDP-11, but because I have an old Rainbow that recently
> > started running Venix (v7-based version) and was trolling around for some
> > way to do TCP/IP to it (though w/o readily available ethernet cards, I'm
> > not sure it is a viable project).
>
> Boy is the memory going.  What was the TCP/IP implementation people
> ran on DOS to do connections over serial lines?  Could that be found
> and revived for such a system?
>

That would be mTCP with CSLIP. And under DOS people have that running
already, but there's limitations and it's not designed for a Unix kernel....

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170520/41537f9a/attachment.html>

From pechter at gmail.com  Sun May 21 06:42:31 2017
From: pechter at gmail.com (William Pechter)
Date: Sat, 20 May 2017 16:42:31 -0400
Subject: [TUHS] Fwd: Re:  Unix with TCP/IP for small PDP-11s
Message-ID: <31e7f50d-2bd7-450b-8a25-1818d8dfbb85.maildroid@localhost>



-----Original Message-----
From: pechter at gmail.com
To: arnold at skeeve.com
Sent: Sat, 20 May 2017 16:41
Subject: Re: [TUHS] Unix with TCP/IP for small PDP-11s

Missed the reply all on the phone.  Phil Karn had KA9Q in the 80s...  It is mentioned on Wikipedia... Don't know much more.  PPP might be better than slip. 

Bill

-----Original Message-----
From: arnold at skeeve.com
To: pechter at gmail.com
Sent: Sat, 20 May 2017 16:12
Subject: Re: [TUHS] Unix with TCP/IP for small PDP-11s

Yes!  Do you want to follow up to the list please?

Thanks,

Arnold

William Pechter <pechter at gmail.com> wrote:

> KA9Q sound right? 
>
> -----Original Message-----
> From: arnold at skeeve.com
> To: imp at bsdimp.com, bqt at update.uu.se
> Cc: tuhs at minnie.tuhs.org
> Sent: Sat, 20 May 2017 15:06
> Subject: Re: [TUHS] Unix with TCP/IP for small PDP-11s
>
> Warner Losh <imp at bsdimp.com> wrote:
>
> > I read the sources to see the TCP/IP support was there (that's the bit
> > about adding Berkeley Sockets). I see nowhere that it's excluded for the
> > non I/D machines, but haven't tried it first hand. I got interested not
> > because of the PDP-11, but because I have an old Rainbow that recently
> > started running Venix (v7-based version) and was trolling around for some
> > way to do TCP/IP to it (though w/o readily available ethernet cards, I'm
> > not sure it is a viable project).
>
> Boy is the memory going.  What was the TCP/IP implementation people
> ran on DOS to do connections over serial lines?  Could that be found
> and revived for such a system?
>
> Thanks,
>
> Arnold


From henry.r.bent at gmail.com  Sun May 21 06:44:43 2017
From: henry.r.bent at gmail.com (Henry Bent)
Date: Sat, 20 May 2017 16:44:43 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <a242ca78-b895-299b-5988-2210d05119ae@update.uu.se>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CAEdTPBeoA6ivRXxSD9XKjhYEi5tqczBcUZ0==Wc-4Rj=GjuOGA@mail.gmail.com>
 <a242ca78-b895-299b-5988-2210d05119ae@update.uu.se>
Message-ID: <CAEdTPBfBx6zJtEgOdv27U9p5a=U=hy3VqiVQTJk3yAb=tA_1Ew@mail.gmail.com>

That was Ultrix-11 3.1, which I believe was the final version.  Full source
is available on tuhs.org.

-Henry

On 20 May 2017 at 16:40, Johnny Billquist <bqt at update.uu.se> wrote:

> On 2017-05-20 19:39, Henry Bent wrote:
>
>> There is absolutely TCP/IP in Ultrix-11.  This is an emulated 11/23+:
>>
>
> Cool. But obviously not the version the SPD was for then. Which version of
> Ultrix-11?
>
>
>         Johnny
>
> --
> Johnny Billquist                  || "I'm on a bus
>                                   ||  on a psychedelic trip
> email: bqt at softjar.se             ||  Reading murder books
> pdp is alive!                     ||  tryin' to stay hip" - B. Idol
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170520/1aeef558/attachment.html>

From bqt at update.uu.se  Sun May 21 06:40:03 2017
From: bqt at update.uu.se (Johnny Billquist)
Date: Sat, 20 May 2017 22:40:03 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAEdTPBeoA6ivRXxSD9XKjhYEi5tqczBcUZ0==Wc-4Rj=GjuOGA@mail.gmail.com>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CAEdTPBeoA6ivRXxSD9XKjhYEi5tqczBcUZ0==Wc-4Rj=GjuOGA@mail.gmail.com>
Message-ID: <a242ca78-b895-299b-5988-2210d05119ae@update.uu.se>

On 2017-05-20 19:39, Henry Bent wrote:
> There is absolutely TCP/IP in Ultrix-11.  This is an emulated 11/23+:

Cool. But obviously not the version the SPD was for then. Which version 
of Ultrix-11?

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol


From bqt at update.uu.se  Sun May 21 06:40:49 2017
From: bqt at update.uu.se (Johnny Billquist)
Date: Sat, 20 May 2017 22:40:49 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
Message-ID: <f788e1e9-3f55-cdb4-ac39-a7bb9992e62e@update.uu.se>

On 2017-05-20 20:18, Warner Losh wrote:
>
>
> On Sat, May 20, 2017 at 4:46 AM, Johnny Billquist <bqt at update.uu.se
> <mailto:bqt at update.uu.se>> wrote:
>
>     On 2017-05-20 04:00, Warner Losh <imp at bsdimp.com
>     <mailto:imp at bsdimp.com>> wrote:
>
>         https://ia601901.us.archive.org/10/items/bitsavers_decpdp11ulLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf
>         <https://ia601901.us.archive.org/10/items/bitsavers_decpdp11ulLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf>
>
>         Looks like it requires MMU, but not split I/D space as it lists the
>         following as compatible: M11, 11/23+, 11/24, 11/34, 11/40 and
>         11/60. It
>         does require 256kb of memory. See table 2, page 6 for details.
>
>
>
>     Uh...? Where do you see that there is any TCP/IP support in
>     Ultrix-11? If any was done by someone else, there is no saying that
>     it would be usable on a machine without split I/D. To be honest,
>     I've never seen any mention of TCP/IP on any machine without split
>     I/D space. I guess it could be done, but it would be a rather big
>     headache...
>
>
> I read the sources to see the TCP/IP support was there (that's the bit
> about adding Berkeley Sockets). I see nowhere that it's excluded for the
> non I/D machines, but haven't tried it first hand. I got interested not
> because of the PDP-11, but because I have an old Rainbow that recently
> started running Venix (v7-based version) and was trolling around for
> some way to do TCP/IP to it (though w/o readily available ethernet
> cards, I'm not sure it is a viable project).

You do know that the Rainbow is not a PDP-11, right?

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol


From imp at bsdimp.com  Sun May 21 07:05:29 2017
From: imp at bsdimp.com (Warner Losh)
Date: Sat, 20 May 2017 15:05:29 -0600
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <f788e1e9-3f55-cdb4-ac39-a7bb9992e62e@update.uu.se>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
 <f788e1e9-3f55-cdb4-ac39-a7bb9992e62e@update.uu.se>
Message-ID: <CANCZdfo-oaupKxY4yUWhL-dKwaBkTSTiz=JkJ8K3XfZq180yMQ@mail.gmail.com>

On Sat, May 20, 2017 at 2:40 PM, Johnny Billquist <bqt at update.uu.se> wrote:

> On 2017-05-20 20:18, Warner Losh wrote:
>
>>
>>
>> On Sat, May 20, 2017 at 4:46 AM, Johnny Billquist <bqt at update.uu.se
>> <mailto:bqt at update.uu.se>> wrote:
>>
>>     On 2017-05-20 04:00, Warner Losh <imp at bsdimp.com
>>     <mailto:imp at bsdimp.com>> wrote:
>>
>>         https://ia601901.us.archive.org/10/items/bitsavers_decpdp11u
>> lLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf
>>         <https://ia601901.us.archive.org/10/items/bitsavers_decpdp11
>> ulLTRIX112.0SPDSep84_870730/AE-X370C-TC_ULTRIX-11_2.0SPD_Sep84.pdf>
>>
>>         Looks like it requires MMU, but not split I/D space as it lists
>> the
>>         following as compatible: M11, 11/23+, 11/24, 11/34, 11/40 and
>>         11/60. It
>>         does require 256kb of memory. See table 2, page 6 for details.
>>
>>
>>
>>     Uh...? Where do you see that there is any TCP/IP support in
>>     Ultrix-11? If any was done by someone else, there is no saying that
>>     it would be usable on a machine without split I/D. To be honest,
>>     I've never seen any mention of TCP/IP on any machine without split
>>     I/D space. I guess it could be done, but it would be a rather big
>>     headache...
>>
>>
>> I read the sources to see the TCP/IP support was there (that's the bit
>> about adding Berkeley Sockets). I see nowhere that it's excluded for the
>> non I/D machines, but haven't tried it first hand. I got interested not
>> because of the PDP-11, but because I have an old Rainbow that recently
>> started running Venix (v7-based version) and was trolling around for
>> some way to do TCP/IP to it (though w/o readily available ethernet
>> cards, I'm not sure it is a viable project).
>>
>
> You do know that the Rainbow is not a PDP-11, right?


Only since '84 when I bought mine... They are both small memory boxes,
though, and something that can run in a 256kB machine should be able to run
on a 892kB machine...  I have bits of mch.c/low.s from Venix that I can
leverage for the port to the Ultrix kernel to the 8088 that's inside the
Rainbow it would be running on.

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170520/4fe0cc15/attachment.html>

From bqt at update.uu.se  Sun May 21 07:34:30 2017
From: bqt at update.uu.se (Johnny Billquist)
Date: Sat, 20 May 2017 23:34:30 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CANCZdfo-oaupKxY4yUWhL-dKwaBkTSTiz=JkJ8K3XfZq180yMQ@mail.gmail.com>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
 <f788e1e9-3f55-cdb4-ac39-a7bb9992e62e@update.uu.se>
 <CANCZdfo-oaupKxY4yUWhL-dKwaBkTSTiz=JkJ8K3XfZq180yMQ@mail.gmail.com>
Message-ID: <8f3662bc-f4f9-74c3-d648-15969230eacc@update.uu.se>

On 2017-05-20 23:05, Warner Losh wrote:
>
>
> On Sat, May 20, 2017 at 2:40 PM, Johnny Billquist <bqt at update.uu.se
> <mailto:bqt at update.uu.se>> wrote:
>
>     On 2017-05-20 20:18, Warner Losh wrote:
>
>         I read the sources to see the TCP/IP support was there (that's
>         the bit
>         about adding Berkeley Sockets). I see nowhere that it's excluded
>         for the
>         non I/D machines, but haven't tried it first hand. I got
>         interested not
>         because of the PDP-11, but because I have an old Rainbow that
>         recently
>         started running Venix (v7-based version) and was trolling around for
>         some way to do TCP/IP to it (though w/o readily available ethernet
>         cards, I'm not sure it is a viable project).
>
>
>     You do know that the Rainbow is not a PDP-11, right?
>
>
> Only since '84 when I bought mine... They are both small memory boxes,
> though, and something that can run in a 256kB machine should be able to
> run on a 892kB machine...  I have bits of mch.c/low.s from Venix that I
> can leverage for the port to the Ultrix kernel to the 8088 that's inside
> the Rainbow it would be running on.

Good. Just wanted to make sure you wouldn't get surprised/confused 
later. :-)
No offense intended.

That said, I suspect it is not that easy to move over, as this probably 
ties very closely in to how the hardware works, and more specifically 
the MMU. Thus, an 8088 have to do things rather differently than a PDP-11.

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol


From davida at pobox.com  Sun May 21 07:41:31 2017
From: davida at pobox.com (David Arnold)
Date: Sun, 21 May 2017 07:41:31 +1000
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <201705201905.v4KJ5xcq005292@freefriends.org>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
 <201705201905.v4KJ5xcq005292@freefriends.org>
Message-ID: <E818B631-118C-4AED-A8B1-F70B35A4DDAC@pobox.com>


> On 21 May 2017, at 05:05, arnold at skeeve.com wrote:
> 
> Warner Losh <imp at bsdimp.com> wrote:
> 
>> I read the sources to see the TCP/IP support was there (that's the bit
>> about adding Berkeley Sockets). I see nowhere that it's excluded for the
>> non I/D machines, but haven't tried it first hand. I got interested not
>> because of the PDP-11, but because I have an old Rainbow that recently
>> started running Venix (v7-based version) and was trolling around for some
>> way to do TCP/IP to it (though w/o readily available ethernet cards, I'm
>> not sure it is a viable project).
> 
> Boy is the memory going.  What was the TCP/IP implementation people
> ran on DOS to do connections over serial lines?  Could that be found
> and revived for such a system?

ka9q?



d


From fair-tuhs at netbsd.org  Sun May 21 07:59:06 2017
From: fair-tuhs at netbsd.org (Erik E. Fair)
Date: Sat, 20 May 2017 14:59:06 -0700
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <E818B631-118C-4AED-A8B1-F70B35A4DDAC@pobox.com>
References: <201705201905.v4KJ5xcq005292@freefriends.org>
Message-ID: <17526.1495317546@cesium.clock.org>

Also MIT PC/IP:

https://en.wikipedia.org/wiki/FTP_Software

	Erik Fair


From random832 at fastmail.com  Sun May 21 15:13:43 2017
From: random832 at fastmail.com (Random832)
Date: Sun, 21 May 2017 01:13:43 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <8f3662bc-f4f9-74c3-d648-15969230eacc@update.uu.se>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
 <f788e1e9-3f55-cdb4-ac39-a7bb9992e62e@update.uu.se>
 <CANCZdfo-oaupKxY4yUWhL-dKwaBkTSTiz=JkJ8K3XfZq180yMQ@mail.gmail.com>
 <8f3662bc-f4f9-74c3-d648-15969230eacc@update.uu.se>
Message-ID: <1495343623.794127.983422680.4D5F43AD@webmail.messagingengine.com>

On Sat, May 20, 2017, at 17:34, Johnny Billquist wrote:
> That said, I suspect it is not that easy to move over, as this probably 
> ties very closely in to how the hardware works, and more specifically 
> the MMU. Thus, an 8088 have to do things rather differently than a
> PDP-11.

Also, x86 is totally "split I/D" by any relevant definition, and has a
20-bit physical address space.


From ron at ronnatalie.com  Sun May 21 21:04:02 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Sun, 21 May 2017 07:04:02 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <1495343623.794127.983422680.4D5F43AD@webmail.messagingengine.com>
References: <mailman.1.1495245601.20449.tuhs@minnie.tuhs.org>
 <be785634-8e6b-7ede-c2eb-d8d4b475d176@update.uu.se>
 <CANCZdfrOt14kr-VkVTurcwG6g1Hg7oQYCyiOphJb_H=G9JqByw@mail.gmail.com>
 <f788e1e9-3f55-cdb4-ac39-a7bb9992e62e@update.uu.se>
 <CANCZdfo-oaupKxY4yUWhL-dKwaBkTSTiz=JkJ8K3XfZq180yMQ@mail.gmail.com>
 <8f3662bc-f4f9-74c3-d648-15969230eacc@update.uu.se>
 <1495343623.794127.983422680.4D5F43AD@webmail.messagingengine.com>
Message-ID: <020201d2d221$f55e08e0$e01a1aa0$@ronnatalie.com>

> Also, x86 is totally "split I/D" by any relevant definition, and has a 20-bit physical address space.

The physical address space isn't the issue.    The PDP-11's (well the later ones) had 22 bits of address space.

Even the amount of virtual address space wasn't the problem.   The problem was really the lack of segments    The PDP11, only had 8 for code and 8 for the combined stack/data (on the split I/D) machine.
While we were able to "thunk" in a code overlay system without any real hardware support, there's no way to do that practically for data.

As I stated earlier, using the BSD networking code, we ran out of the combined code/data registers when it came to having ot have one to map the mbufs.   I supposed you could have added more magic (and perhaps th is is what the later Ultrix did), to carefully unmap one of the data segements to get to the network data, but boy, we never figured out a good way to do that when we tried.    We retired all but the 11/70's (which we had three of) at the time as the Vaxes and other machines were beginning to roll in.    In fact, I did eventually also turn one of the 11/70's into a router using my LOS (little operating system...no time for sharing, uniprocessor system) gateway OS.    Each driver ran in it's own little program so it was a lot easier.




From dave at horsfall.org  Mon May 22 02:16:18 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Mon, 22 May 2017 02:16:18 +1000 (EST)
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAC20D2PhQwCiPuSNZBhJUng=ZoE-JgTPayvhf+UkNctNuZ3POg@mail.gmail.com>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAC20D2PhQwCiPuSNZBhJUng=ZoE-JgTPayvhf+UkNctNuZ3POg@mail.gmail.com>
Message-ID: <alpine.BSF.2.20.1705220154160.48990@aneurin.horsfall.org>

On Fri, 19 May 2017, Clem Cole wrote:

> We did try to run on an 11/60 - which is 40 class as you asked, and we 
> did get it work as I recall after a fashion.

Ah, the mighty 11/60; boy, but that takes me back.  We bought one to 
replace our ageing /40, and I was fascinated by its octal display (yes, I 
wrote an idle loop to shift its digits back and forth; there should be an 
article somewhere in the AUUG archives).

Story time: DEC were flogging the /60s real cheap here in Oz, because a 
deal with a major publisher (Murdoch?  Packer?  Who knows?) fell through 
at the last moment, and they had an entire warehouse to shift.

Trivia: the thing was micro-codeable, and I had dreams of implementing the 
Unix call/return sequences as single instructions, but I never did get a 
round tuit.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From clemc at ccc.com  Mon May 22 05:58:25 2017
From: clemc at ccc.com (Clem Cole)
Date: Sun, 21 May 2017 15:58:25 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <alpine.BSF.2.20.1705220154160.48990@aneurin.horsfall.org>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAC20D2PhQwCiPuSNZBhJUng=ZoE-JgTPayvhf+UkNctNuZ3POg@mail.gmail.com>
 <alpine.BSF.2.20.1705220154160.48990@aneurin.horsfall.org>
Message-ID: <CAC20D2N4i-kbLp6YyTyfntpNCzg36NaTUp8UHnJfzS4y=ejRtg@mail.gmail.com>

On Sun, May 21, 2017 at 12:16 PM, Dave Horsfall <dave at horsfall.org> wrote:

> Story time: DEC were flogging the /60s real cheap here in Oz, because a
> ​ ​
> deal with a major publisher (Murdoch?  Packer?  Who knows?) fell through
> ​ ​
> at the last moment, and they had an entire warehouse to shift.
>
​It was world wide.   The /60 was the fastest machine from release to
traditional product in DEC's history.​




>
> Trivia: the thing was micro-codeable, and I had dreams of implementing the
> ​ ​
> Unix call/return sequences as single instructions, but I never did get a
> ​ ​
> round tuit.
>
​Indeed.  @ Tek we had both the DEC tools and the CMU tools and the 11/40e
microcode from CMU (and the compiler that used csav/cret - probably still
do somewhere).   One problem was the microcode compiler from DEC ran on RSX
and CMU's ran on TOPS-10 of which I had neither.  ​

The /60 and /40 had similar but different base microengines and it was
going to take some work reimplementing the CMU extens for the 60.   I've
forgotten the details of what changed in the microengine, but something was
hugely different and enough that the CMU microcode would have been a model,
but would not work even if I could manage to get the tools together.
So, it was also on my todo list, but we got an 11/70 shortly after I pulled
together the pieces and we never got to it.

What I can tell you from the CMU experience is that it bought a small
amount of speed up, but the big thing code size reduction for some programs
and on 40 class machine -- anything like that helped.   But... the problem
was it made the binaries not portable.

We had to be very careful.   The EE production machines (which were 11/34
and later /34As) binaries of course could run on the CS systems, but the CS
machines with the /40e microcode, compiler et al, created binaries that
core dumped on the 34.   Worse yet, was accidently mixing libraries.

Ted and I talked about hacking the EE kernel to support a UUI like the FPs
did could take the CS binaries and run them.   But that project never
occurred - which was probably a shame.  Instead, Danny Klein and I got
really good at purging any tape or disk that passed through a CS system for
anything binary on it.

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170521/944d42bd/attachment.html>

From ron at ronnatalie.com  Mon May 22 06:57:32 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Sun, 21 May 2017 16:57:32 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAC20D2N4i-kbLp6YyTyfntpNCzg36NaTUp8UHnJfzS4y=ejRtg@mail.gmail.com>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAC20D2PhQwCiPuSNZBhJUng=ZoE-JgTPayvhf+UkNctNuZ3POg@mail.gmail.com>
 <alpine.BSF.2.20.1705220154160.48990@aneurin.horsfall.org>
 <CAC20D2N4i-kbLp6YyTyfntpNCzg36NaTUp8UHnJfzS4y=ejRtg@mail.gmail.com>
Message-ID: <024601d2d274$de4fb290$9aef17b0$@ronnatalie.com>

The 11/60’s main claim to fame if I recall was a writable control store.   Don’t know anybody who actually used that feature.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170521/bd053dd8/attachment.html>

From clemc at ccc.com  Mon May 22 07:26:42 2017
From: clemc at ccc.com (Clem Cole)
Date: Sun, 21 May 2017 17:26:42 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <024601d2d274$de4fb290$9aef17b0$@ronnatalie.com>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAC20D2PhQwCiPuSNZBhJUng=ZoE-JgTPayvhf+UkNctNuZ3POg@mail.gmail.com>
 <alpine.BSF.2.20.1705220154160.48990@aneurin.horsfall.org>
 <CAC20D2N4i-kbLp6YyTyfntpNCzg36NaTUp8UHnJfzS4y=ejRtg@mail.gmail.com>
 <024601d2d274$de4fb290$9aef17b0$@ronnatalie.com>
Message-ID: <CAC20D2MRO9TVKjFjACF63h9X1tDCV7T_0CN9YYzH7LhVpq3Diw@mail.gmail.com>

Actually, the 11/60's main claim to fame was it supposed to be a
'commercial' PDP-11 and was built for the small business market.  The WCS
was a side effect.

It was to built to run RSTS and RSX and had a commercial instruction set
exten *etc.*. Somebody had written a 'dentist office' package for it and a
'car dealership package' IIRC.    And was physically packaged a tad
differently than the other 11's as it was trying to be marketing to places
that might want to show it off instead of hiding it in a computer room.

The main competition at the time was the Burrough 1700 and the IBM System
38 and DEC was trying to find a system to compete in the market.   Now, the
B1700 is a very cool system and could swap its microcode depending on the
language running (it is also bit addressable). As I understand it from
talking to the 11/60s product manager many years later at a party, one of
the ideas was that the PDP 11/60 WCS an attempt to be able offer features
that might be needed to compete with it.

The the real truth was that issue in the original 11's had stored the ucode
in PROM and if an ECO went out the CPU boards had be swapped, so the chips
could be swapped at the factory.    Plus a number of places, like MIT and
CMU had been banging on DEC to make the ucoding of the system an available
option (CMU just built it's own the 11/40e that triple drip made for them).

Anyway, as the price of SRAM had fallen dramatically from when the 11 was
introduced, and the 11/60 was storing the primary ucode it, not PROM like
the other 45 and 40 class systems; WCS became a possible feature to expose.
  But, as I said, the problem was that the WCS tools were not very portable
and the microengine was (as I remember) not well enough documented.  Which
is why I don't think many people tried.

Also the 11/60 also had a unique feature we called the HCU instruction -
Halt and Confuse Microcode.  Unlike the 11/34 which had an Intel 8008 that
ran front (programmers) console, the 11/60's microcode loop read the LED's
and keypad.  But ... if the ucode took a dive, the only recourse was turn
off the system - as it would stop responding to anything (just like PC
years later ;-)  We never found what sequence caused it; but when I was
debugging the original 11/60 port of V6, I would run into a lot ;-)

IIRC - we realized on these pages that Dave Horsfall in OZ and I in Oregon
must have been doing similar work around the same time.  FWIW, my changes
made it back to AT&T via Ted and I believe eventually went out in the
System III release.   That said, I suspect if we were to compare our
patches I bet they are pretty similar.

Clem

On Sun, May 21, 2017 at 4:57 PM, Ron Natalie <ron at ronnatalie.com> wrote:

> The 11/60’s main claim to fame if I recall was a writable control store.
> Don’t know anybody who actually used that feature.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170521/873e6a94/attachment.html>

From pechter at gmail.com  Mon May 22 07:46:15 2017
From: pechter at gmail.com (William Pechter)
Date: Sun, 21 May 2017 17:46:15 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <024601d2d274$de4fb290$9aef17b0$@ronnatalie.com>
References: <20170519151518.8800D18C09D@mercury.lcs.mit.edu>
 <CAC20D2PhQwCiPuSNZBhJUng=ZoE-JgTPayvhf+UkNctNuZ3POg@mail.gmail.com>
 <alpine.BSF.2.20.1705220154160.48990@aneurin.horsfall.org>
 <CAC20D2N4i-kbLp6YyTyfntpNCzg36NaTUp8UHnJfzS4y=ejRtg@mail.gmail.com>
 <024601d2d274$de4fb290$9aef17b0$@ronnatalie.com>
Message-ID: <c69d68a4-3065-4aa6-be08-b92c7a392a0c.maildroid@localhost>

I believe Reuters turned them into faster PDP8s via Microcode changes... 

-----Original Message-----
From: Ron Natalie <ron at ronnatalie.com>
To: 'Clem Cole' <clemc at ccc.com>, 'Dave Horsfall' <dave at horsfall.org>
Cc: 'The Eunuchs Hysterical Society' <tuhs at tuhs.org>
Sent: Sun, 21 May 2017 16:58
Subject: Re: [TUHS] Unix with TCP/IP for small PDP-11s

The 11/60’s main claim to fame if I recall was a writable control store.   Don’t know anybody who actually used that feature.



From bqt at update.uu.se  Mon May 22 08:42:35 2017
From: bqt at update.uu.se (Johnny Billquist)
Date: Mon, 22 May 2017 00:42:35 +0200
Subject: [TUHS] 11/60 (was: Unix with TCP/IP for small PDP-11s)
In-Reply-To: <mailman.874.1495402035.3779.tuhs@minnie.tuhs.org>
References: <mailman.874.1495402035.3779.tuhs@minnie.tuhs.org>
Message-ID: <34efc530-b7be-ba8b-1762-756e60915ce5@update.uu.se>

On 2017-05-21 23:27, Clem Cole <clemc at ccc.com> wrote:

> Actually, the 11/60's main claim to fame was it supposed to be a
> 'commercial' PDP-11 and was built for the small business market.  The WCS
> was a side effect.

As you mentioned in another mail, it's main claim to fame was probably 
the short time it actually existed in the market. DEC seriously did some 
things wrong on it, such as only 18-bit address, and no split I/D space, 
at a time when pretty much any other PDP-11 was going there.

But it was also one of a couple of -11 models where you had the ability 
to write your own microcode. But I've never heard of anyone who had the 
WCS option. (But at one point, I was playing with four 11/60 machines in 
a computer club.)

I don't remember if it was you or someone else who said that there were 
several microcode bugs in the 11/60. It ran the DEC OSes, but there were 
some issues with Unix. I seem to also remember seeing some special code 
in the RSX kernel for some 11/60 oddity, but I would have to search 
through the code to remember exactly what that was about.

> It was to built to run RSTS and RSX and had a commercial instruction set
> exten *etc.*. Somebody had written a 'dentist office' package for it and a
> 'car dealership package' IIRC.    And was physically packaged a tad
> differently than the other 11's as it was trying to be marketing to places
> that might want to show it off instead of hiding it in a computer room.

Uh? I have not seen any plans ever mentioning CIS for the 11/60, but I 
guess it's possible it was considered at some point. But I can assure 
you no CIS ever existed for the 11/60.

The only machines that ever had the CIS option was the 11/44 and 
11/23,11/24.
But I have no idea how many machines ever actually had the option installed.

One nice detail of the 11/60 is that it had the FPP build in. But there 
was also an optional hardware FPP accelerator available.

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol


From wkt at tuhs.org  Mon May 22 11:15:05 2017
From: wkt at tuhs.org (Warren Toomey)
Date: Mon, 22 May 2017 11:15:05 +1000
Subject: [TUHS] KA9Q
In-Reply-To: <31e7f50d-2bd7-450b-8a25-1818d8dfbb85.maildroid@localhost>
References: <31e7f50d-2bd7-450b-8a25-1818d8dfbb85.maildroid@localhost>
Message-ID: <20170522011505.GA8612@minnie.tuhs.org>

On Sat, May 20, 2017 at 04:42:31PM -0400, William Pechter wrote:
> Missed the reply all on the phone.  Phil Karn had KA9Q in the 80s.

Minnie ran an FTP service on KA9Q on MS-DOS to start with. She had a 30M RLL
drive with a WD8003E NIC. I'm so glad she isn't still doing that :)

	Warren

(yes, off topic, mea culpa)


From b4 at gewt.net  Mon May 22 12:44:58 2017
From: b4 at gewt.net (Cory Smelosky)
Date: Sun, 21 May 2017 19:44:58 -0700
Subject: [TUHS] KA9Q
In-Reply-To: <20170522011505.GA8612@minnie.tuhs.org>
References: <31e7f50d-2bd7-450b-8a25-1818d8dfbb85.maildroid@localhost>
 <20170522011505.GA8612@minnie.tuhs.org>
Message-ID: <00B1142B-9DC4-4DC8-ABAC-ECD7F21F09E2@gewt.net>

I keep meaning to try similar for SysVR2

Sent from my iPhone

> On May 21, 2017, at 18:15, Warren Toomey <wkt at tuhs.org> wrote:
> 
>> On Sat, May 20, 2017 at 04:42:31PM -0400, William Pechter wrote:
>> Missed the reply all on the phone.  Phil Karn had KA9Q in the 80s.
> 
> Minnie ran an FTP service on KA9Q on MS-DOS to start with. She had a 30M RLL
> drive with a WD8003E NIC. I'm so glad she isn't still doing that :)
> 
>    Warren
> 
> (yes, off topic, mea culpa)



From dave at horsfall.org  Mon May 22 16:00:33 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Mon, 22 May 2017 16:00:33 +1000 (EST)
Subject: [TUHS] KA9Q
In-Reply-To: <20170522011505.GA8612@minnie.tuhs.org>
References: <31e7f50d-2bd7-450b-8a25-1818d8dfbb85.maildroid@localhost>
 <20170522011505.GA8612@minnie.tuhs.org>
Message-ID: <alpine.BSF.2.20.1705221556020.48990@aneurin.horsfall.org>

On Mon, 22 May 2017, Warren Toomey wrote:

> Minnie ran an FTP service on KA9Q on MS-DOS to start with. She had a 30M 
> RLL drive with a WD8003E NIC. I'm so glad she isn't still doing that :)

Ahh...  I remember Minnie well; those were the days...

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From grog at lemis.com  Mon May 22 16:50:10 2017
From: grog at lemis.com (Greg 'groggy' Lehey)
Date: Mon, 22 May 2017 16:50:10 +1000
Subject: [TUHS] KA9Q
In-Reply-To: <alpine.BSF.2.20.1705221556020.48990@aneurin.horsfall.org>
References: <31e7f50d-2bd7-450b-8a25-1818d8dfbb85.maildroid@localhost>
 <20170522011505.GA8612@minnie.tuhs.org>
 <alpine.BSF.2.20.1705221556020.48990@aneurin.horsfall.org>
Message-ID: <20170522065010.GC79144@eureka.lemis.com>

On Monday, 22 May 2017 at 16:00:33 +1000, Dave Horsfall wrote:
> Received: from minnie.tuhs.org (minnie.tuhs.org [45.79.103.53])
>        by www.lemis.com (Postfix) with ESMTP id 3D8211B72804
>
> On Mon, 22 May 2017, Warren Toomey wrote:
>
>> Minnie ran an FTP service on KA9Q on MS-DOS to start with. She had a 30M
>> RLL drive with a WD8003E NIC. I'm so glad she isn't still doing that :)
>
> Ahh...  I remember Minnie well; those were the days...

They still are, for a certain incarnation of minnie.

Greg
-- 
Sent from my desktop computer.
Finger grog at lemis.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed.  If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 163 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170522/3e929fee/attachment.sig>

From pnr at planet.nl  Mon May 22 19:28:19 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Mon, 22 May 2017 11:28:19 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
Message-ID: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>


I had somehow convinced myself that Ultrix-11 needed split I/D, but indeed it does not:

# file unix
unix:   (0450) pure overlay executable not stripped
# size unix
14784+(8192,8000,8064,8000,8064,8128,8000,7808,7936,7936,7680,7360,1344)+3524+13500 = 31808b = 076100b (111296 total text)

With only 16KB of permanent kernel there will be a lot of overlay switching. I'm not entirely sure why bss could not be 1KB smaller, enabling 8KB more of permanent kernel. The loss of performance from 2 disk buffers less really outweighed less overlay switching?

If I understand correctly, the network code continuously switches around segment 5 to access the right mbuf.

According to the notes in the TUHS archive (http://www.tuhs.org/Archive/Distributions/DEC/Ultrix-11/Fred-Ultrix3/setup-3.1.txt), running Ultrix-11 with networking on a 11/40 class machine is borderline workable:

"I have personally tested it on a 23+, 53 and 83.  I know it runs
fine on the 73.  The smaller machines (34, 40 etc) should work
akin to the 23, meaning using overlays and be very tight on RAM
for the drivers.  TCP/IP is a biiiiig load for those systems!"

There are two other routes to TCP/IP on a PDP11 without split I/D:
- 3COM's TCP/IP package (initially an overlay over V7, soon after also over 2BSD); I believe the source to this is lost.
- DCEC's adaptation of the Wingfield TCP/IP library, designed to work with V6. It is mostly a user space daemon, but requires some kernel enhancements. The Wingfield code is in the TUHS archive, but that version has a modified V6 kernel that also supports NCP networking and requires split I/D. If used with a minimally enhanced V6 kernel, it would easily fit in 64KB, without overlays.

Note that these last two options have very different API's and would not be so easy to work with.

Paul
 






From clemc at ccc.com  Tue May 23 00:09:30 2017
From: clemc at ccc.com (Clem Cole)
Date: Mon, 22 May 2017 10:09:30 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
References: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
Message-ID: <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>

On Mon, May 22, 2017 at 5:28 AM, Paul Ruizendaal <pnr at planet.nl> wrote:

> There are two other routes to TCP/IP on a PDP11 without split I/D:
> - 3COM's TCP/IP package (initially an overlay over V7, soon after also
> over 2BSD);
>
​It's called UNET​




> I believe the source to this is lost.
>
​So far that is true, there are a couple of leads including a couple of
unread tapes in my own basement.   The original authors and I have been in
touch.   Assuming we can find and recover it, there is still an open issue
of copyright.​




> ​.....​
>
>
> Note that these last two options have very different API's
>
​Indeed...​




> and would not be so easy to work with.
>
​Mumble ... it just means post-Joy sockets code does not 'just work.'  UNET
uses the Rand interface, which is similar to the interface MIT used for
Chaos and Chesson used for the ARPAnet code at UofI, so there are programs
that talk to it.   We had mail, ftp and telnet working just fine.​

​ Back in the day and IIRC we were playing with a port of the MIT SUPDUP
code but I do not remember if that ever worked on the small address
machines.​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170522/bf42b9fa/attachment.html>

From steve at quintile.net  Tue May 23 00:51:22 2017
From: steve at quintile.net (Steve Simon)
Date: Mon, 22 May 2017 15:51:22 +0100
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>
References: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
 <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>
Message-ID: <80EBBEAA-5774-4594-B69E-00FB1EE0F101@quintile.net>

hi Clem,

do you have pointers to any documentation on the rand/MIT network API?

-Steve




> On 22 May 2017, at 15:09, Clem Cole <clemc at ccc.com> wrote:
> 
> 
>> On Mon, May 22, 2017 at 5:28 AM, Paul Ruizendaal <pnr at planet.nl> wrote:
>> There are two other routes to TCP/IP on a PDP11 without split I/D:
>> - 3COM's TCP/IP package (initially an overlay over V7, soon after also over 2BSD);
> ​It's called UNET​
> 
> 
>  
>> I believe the source to this is lost.
> ​So far that is true, there are a couple of leads including a couple of unread tapes in my own basement.   The original authors and I have been in touch.   Assuming we can find and recover it, there is still an open issue of copyright.​ 
> 
>  
>> ​.....​
>> 
>> 
>> Note that these last two options have very different API's
> ​Indeed...​
> 
> 
>  
>> and would not be so easy to work with.
> ​Mumble ... it just means post-Joy sockets code does not 'just work.'  UNET uses the Rand interface, which is similar to the interface MIT used for Chaos and Chesson used for the ARPAnet code at UofI, so there are programs that talk to it.   We had mail, ftp and telnet working just fine.​ ​ Back in the day and IIRC we were playing with a port of the MIT SUPDUP code but I do not remember if that ever worked on the small address machines.​
>  
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170522/476191bd/attachment.html>

From clemc at ccc.com  Tue May 23 02:29:09 2017
From: clemc at ccc.com (Clem Cole)
Date: Mon, 22 May 2017 12:29:09 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <80EBBEAA-5774-4594-B69E-00FB1EE0F101@quintile.net>
References: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
 <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>
 <80EBBEAA-5774-4594-B69E-00FB1EE0F101@quintile.net>
Message-ID: <CAC20D2OXOwYnbwpv8O=E14j-PQRONJrKL6VEyvCc+Reeguy72Q@mail.gmail.com>

The Rand API is documented in a couple of papers that have been discussed
here previously.   I'll have to do a little googling to find them.   The
ChaosNET docs are the best MIT docs.


The API pretty much worked this way...  there was one simple kernel hook.
A small mod was done to the old nami routine (what modern UNIX calls
'lookup'), as a path was being parsed, the rest of the path was left in
place made available to other routines.

Thus you could open:  /dev/tcp/xxx  and the string xxx was available in
call that was implementing tcp and was then able to be passed xxx as a
parameter.  Hence, the traditional open/close/read/write was all that was
needed.

Joy codified a new API that was supposed to be more network centric and map
to different network protocols - the thinking being that
open/close/read/write were insufficient semantics for network operations.

Also, the only other issue at the time was that BSD's select(2) did not
exist, and the UNIX I/O were 100% synchronous.  So some other mechanism
(also discussed here) needed to be created to avoid blocking in the
application.  There were a couple of different schemes with V7's
multiplexor call was created by Chesson for DataKit (which was similar to
was used in the UofI code Arpanet code).   Rand, UNET & Chaos had something
else that gave the save async function, who's name I've forgotten at the
moment, but I believe Noel posted the code for same in the last year from
one of the MIT kernels - we had used it at CMU that we had gotten from Rand.

On Mon, May 22, 2017 at 10:51 AM, Steve Simon <steve at quintile.net> wrote:

> hi Clem,
>
> do you have pointers to any documentation on the rand/MIT network API?
>
> -Steve
>
>
>
>
> On 22 May 2017, at 15:09, Clem Cole <clemc at ccc.com> wrote:
>
>
> On Mon, May 22, 2017 at 5:28 AM, Paul Ruizendaal <pnr at planet.nl> wrote:
>
>> There are two other routes to TCP/IP on a PDP11 without split I/D:
>> - 3COM's TCP/IP package (initially an overlay over V7, soon after also
>> over 2BSD);
>>
> ​It's called UNET​
>
>
>
>
>> I believe the source to this is lost.
>>
> ​So far that is true, there are a couple of leads including a couple of
> unread tapes in my own basement.   The original authors and I have been in
> touch.   Assuming we can find and recover it, there is still an open issue
> of copyright.​
>
>
>
>
>> ​.....​
>>
>>
>> Note that these last two options have very different API's
>>
> ​Indeed...​
>
>
>
>
>> and would not be so easy to work with.
>>
> ​Mumble ... it just means post-Joy sockets code does not 'just work.'
>  UNET uses the Rand interface, which is similar to the interface MIT used
> for Chaos and Chesson used for the ARPAnet code at UofI, so there are
> programs that talk to it.   We had mail, ftp and telnet working just fine.​
>
> ​ Back in the day and IIRC we were playing with a port of the MIT SUPDUP
> code but I do not remember if that ever worked on the small address
> machines.​
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170522/b5457eb2/attachment.html>

From ron at ronnatalie.com  Tue May 23 02:35:34 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Mon, 22 May 2017 12:35:34 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAC20D2OXOwYnbwpv8O=E14j-PQRONJrKL6VEyvCc+Reeguy72Q@mail.gmail.com>
References: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
 <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>
 <80EBBEAA-5774-4594-B69E-00FB1EE0F101@quintile.net>
 <CAC20D2OXOwYnbwpv8O=E14j-PQRONJrKL6VEyvCc+Reeguy72Q@mail.gmail.com>
Message-ID: <003701d2d319$70919d40$51b4d7c0$@ronnatalie.com>

Gak, the days before select.    I remember writing some terminal programs where I had to fork and do the synchronous read in each direction separately.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170522/6f1762bc/attachment.html>

From pnr at planet.nl  Tue May 23 08:07:34 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Tue, 23 May 2017 00:07:34 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAC20D2OXOwYnbwpv8O=E14j-PQRONJrKL6VEyvCc+Reeguy72Q@mail.gmail.com>
References: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
 <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>
 <80EBBEAA-5774-4594-B69E-00FB1EE0F101@quintile.net>
 <CAC20D2OXOwYnbwpv8O=E14j-PQRONJrKL6VEyvCc+Reeguy72Q@mail.gmail.com>
Message-ID: <830AF76B-3E3C-4C76-9ED6-959AC38AF6EE@planet.nl>

Clem,

You've got me confused.

> The API pretty much worked this way...  there was one simple kernel hook.   A small mod was done to the old nami routine (what modern UNIX calls 'lookup'), as a path was being parsed, the rest of the path was left in place made available to other routines.

The UoI Arpanet code did not use the partial path parsing that you describe.
The namei() routine is unchanged (see http://www.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/ken/nami.c).

The code recognizes a network open, because the open() is of a special character file with major number 255;
the minor number would be the remote address (1974 arpanet used 8 bit network addresses - 6 bits IMP, 2 bits host).
If the 'mode' parameter to open() is 0, 1 or 2, then a NCP/telnet connection would be made to that host. If the mode parameter was anything else, it was a pointer to a data block with the full specification of the connection to be made. See:
http://www.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/ken/sys2.c - open1()
http://www.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/ncpk/nopcls.c - netopen()

The data block contained all the parameters that are set via calls to connect/bind/listen in the BSD sockets API. 

> There were a couple of different schemes with V7's multiplexor call was created by Chesson for DataKit (which was similar to was used in the UofI code Arpanet code).

You've lost me here.

Paul

On 22 May 2017, at 18:29 , Clem Cole wrote:

> The Rand API is documented in a couple of papers that have been discussed here previously.   I'll have to do a little googling to find them.   The ChaosNET docs are the best MIT docs.
> 
> 
> The API pretty much worked this way...  there was one simple kernel hook.   A small mod was done to the old nami routine (what modern UNIX calls 'lookup'), as a path was being parsed, the rest of the path was left in place made available to other routines.
> 
> Thus you could open:  /dev/tcp/xxx  and the string xxx was available in call that was implementing tcp and was then able to be passed xxx as a parameter.  Hence, the traditional open/close/read/write was all that was needed.
> 
> Joy codified a new API that was supposed to be more network centric and map to different network protocols - the thinking being that open/close/read/write were insufficient semantics for network operations.
> 
> Also, the only other issue at the time was that BSD's select(2) did not exist, and the UNIX I/O were 100% synchronous.  So some other mechanism (also discussed here) needed to be created to avoid blocking in the application.  There were a couple of different schemes with V7's multiplexor call was created by Chesson for DataKit (which was similar to was used in the UofI code Arpanet code).   Rand, UNET & Chaos had something else that gave the save async function, who's name I've forgotten at the moment, but I believe Noel posted the code for same in the last year from one of the MIT kernels - we had used it at CMU that we had gotten from Rand.
> 
> On Mon, May 22, 2017 at 10:51 AM, Steve Simon <steve at quintile.net> wrote:
> hi Clem,
> 
> do you have pointers to any documentation on the rand/MIT network API?
> 
> -Steve



From clemc at ccc.com  Tue May 23 09:25:34 2017
From: clemc at ccc.com (Clem Cole)
Date: Mon, 22 May 2017 19:25:34 -0400
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <830AF76B-3E3C-4C76-9ED6-959AC38AF6EE@planet.nl>
References: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
 <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>
 <80EBBEAA-5774-4594-B69E-00FB1EE0F101@quintile.net>
 <CAC20D2OXOwYnbwpv8O=E14j-PQRONJrKL6VEyvCc+Reeguy72Q@mail.gmail.com>
 <830AF76B-3E3C-4C76-9ED6-959AC38AF6EE@planet.nl>
Message-ID: <CAC20D2NP4C9V_0Rm93ka9zR572nFHYyJ-MU_+HeLbTws4rH+vw@mail.gmail.com>

We'll have to do some digging. And having too been part of too many, but
similar of different implementations. The Chaos code used the partial
parsing as did a few others.  IIRC: That's the same trick Apollo used with
Domain too.  I had thought I had remember that Chesson and the UofI folks
used the same trick with the Arpanet code, but I have not looked at it in
years and clearly as you point out he did not.

As for DataKit et al..  Greg Chesson was a grad student at UoI.  The UoI
folks did the original  V6 Arpanet for UNIX code and Greg was the one of
the primary developers same.  What I do not remember is who came first, the
Rand networking work for the UoI work.   Rand did the the ports (later
called named pipes) and a few other things pretty early.   But again all
those dates sort of mix together in the early middle 70s.  USENIX was not
yet publishing proceedings so it's hard to keep straight.   It pretty much
just email and memories of those of us that were sharing things at the time.


Anyway, when Greg graduated, he was hired by Ken when he finished and
developed DataKit at Bell labs.   One of the pieces of datakit that was
released as part of V7 was Greg's mpx(2) code - which was the multiplexer.
The legend is that on is way to UCB for his sabbatical, Ken was checking in
Greg and the infamous v6 patches tape was left behind at UoI.  Greg used to
deny all knowledge of same, but smiled all the time he said it.

Clem

On Mon, May 22, 2017 at 6:07 PM, Paul Ruizendaal <pnr at planet.nl> wrote:

> Clem,
>
> You've got me confused.
>
> > The API pretty much worked this way...  there was one simple kernel
> hook.   A small mod was done to the old nami routine (what modern UNIX
> calls 'lookup'), as a path was being parsed, the rest of the path was left
> in place made available to other routines.
>
> The UoI Arpanet code did not use the partial path parsing that you
> describe.
> The namei() routine is unchanged (see http://www.tuhs.org/cgi-bin/
> utree.pl?file=SRI-NOSC/ken/nami.c).
>
> The code recognizes a network open, because the open() is of a special
> character file with major number 255;
> the minor number would be the remote address (1974 arpanet used 8 bit
> network addresses - 6 bits IMP, 2 bits host).
> If the 'mode' parameter to open() is 0, 1 or 2, then a NCP/telnet
> connection would be made to that host. If the mode parameter was anything
> else, it was a pointer to a data block with the full specification of the
> connection to be made. See:
> http://www.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/ken/sys2.c - open1()
> http://www.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/ncpk/nopcls.c -
> netopen()
>
> The data block contained all the parameters that are set via calls to
> connect/bind/listen in the BSD sockets API.
>
> > There were a couple of different schemes with V7's multiplexor call was
> created by Chesson for DataKit (which was similar to was used in the UofI
> code Arpanet code).
>
> You've lost me here.
>
> Paul
>
> On 22 May 2017, at 18:29 , Clem Cole wrote:
>
> > The Rand API is documented in a couple of papers that have been
> discussed here previously.   I'll have to do a little googling to find
> them.   The ChaosNET docs are the best MIT docs.
> >
> >
> > The API pretty much worked this way...  there was one simple kernel
> hook.   A small mod was done to the old nami routine (what modern UNIX
> calls 'lookup'), as a path was being parsed, the rest of the path was left
> in place made available to other routines.
> >
> > Thus you could open:  /dev/tcp/xxx  and the string xxx was available in
> call that was implementing tcp and was then able to be passed xxx as a
> parameter.  Hence, the traditional open/close/read/write was all that was
> needed.
> >
> > Joy codified a new API that was supposed to be more network centric and
> map to different network protocols - the thinking being that
> open/close/read/write were insufficient semantics for network operations.
> >
> > Also, the only other issue at the time was that BSD's select(2) did not
> exist, and the UNIX I/O were 100% synchronous.  So some other mechanism
> (also discussed here) needed to be created to avoid blocking in the
> application.  There were a couple of different schemes with V7's
> multiplexor call was created by Chesson for DataKit (which was similar to
> was used in the UofI code Arpanet code).   Rand, UNET & Chaos had something
> else that gave the save async function, who's name I've forgotten at the
> moment, but I believe Noel posted the code for same in the last year from
> one of the MIT kernels - we had used it at CMU that we had gotten from Rand.
> >
> > On Mon, May 22, 2017 at 10:51 AM, Steve Simon <steve at quintile.net>
> wrote:
> > hi Clem,
> >
> > do you have pointers to any documentation on the rand/MIT network API?
> >
> > -Steve
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170522/e97f362d/attachment.html>

From pnr at planet.nl  Tue May 23 10:36:06 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Tue, 23 May 2017 02:36:06 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <CAC20D2NP4C9V_0Rm93ka9zR572nFHYyJ-MU_+HeLbTws4rH+vw@mail.gmail.com>
References: <35238F37-6324-4D40-A6C0-7957F2E22964@planet.nl>
 <CAC20D2Mi+iVqoMcHxoaOKwu00-A-V9OPWrAOVoCdH6c6MbKdyQ@mail.gmail.com>
 <80EBBEAA-5774-4594-B69E-00FB1EE0F101@quintile.net>
 <CAC20D2OXOwYnbwpv8O=E14j-PQRONJrKL6VEyvCc+Reeguy72Q@mail.gmail.com>
 <830AF76B-3E3C-4C76-9ED6-959AC38AF6EE@planet.nl>
 <CAC20D2NP4C9V_0Rm93ka9zR572nFHYyJ-MU_+HeLbTws4rH+vw@mail.gmail.com>
Message-ID: <B4993820-5C5E-491D-BEAD-1B2E1CBD66D7@planet.nl>


On 23 May 2017, at 1:25 , Clem Cole wrote:

> As for DataKit et al..  Greg Chesson was a grad student at UoI.  The UoI folks did the original  V6 Arpanet for UNIX code and Greg was the one of the primary developers same.  What I do not remember is who came first, the Rand networking work for the UoI work.   Rand did the the ports (later called named pipes) and a few other things pretty early.   But again all those dates sort of mix together in the early middle 70s.  USENIX was not yet publishing proceedings so it's hard to keep straight.   It pretty much just email and memories of those of us that were sharing things at the time.

These dates I can fill in:
- UoI Arpanet Unix was initially done between December 1974 and March 1975, building on the experience gained with the earlier ANTS I and II projects.
- Rand ports were done in 1977, under contract to the air force (report dated June '77).

> Anyway, when Greg graduated, he was hired by Ken when he finished and developed DataKit at Bell labs.   One of the pieces of datakit that was released as part of V7 was Greg's mpx(2) code - which was the multiplexer.

Interesting. I always thought that mpx files were driven by the work on the Blit terminal, the link to networking is new to me.

I'm looking into the history of Spider and early Datakit. Sandy Fraser was kind enough to send me the 1974 report on Spider and it already mentions actual usage for remote printing, remote login and a central 'file store'. Spider was an interesting bit of technology, essentially it linked a dozen or so computers over a 1.5Mb/s shared link (a daisy chained twisted pair cable) to a central router/switch. There was only ever one Spider router, but the design allowed for multiple routers to be connected over T1 long distance lines.

Does anybody know of surviving v5/v6/v7 code for Spider networking (e.g. the 'tiu' device driver, the 'nfs' file transfer package, etc.)?



From jnc at mercury.lcs.mit.edu  Tue May 23 11:14:52 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Mon, 22 May 2017 21:14:52 -0400 (EDT)
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
Message-ID: <20170523011452.99DD518C095@mercury.lcs.mit.edu>

    > From: Paul Ruizendaal

    > There are two other routes to TCP/IP on a PDP11 without split I/D:
    > ...
    > DCEC's adaptation of the Wingfield TCP/IP library, designed to work
    > with V6. It is mostly a user space daemon, but requires some kernel
    > enhancements.

I wonder what the performance would be like, since the TCP is in a user
process (a different one from the application), i.e. there's a process switch
every time the application goes to send or receive data. This wouldn't have
been such an issue when the code was written, since ARPANet-type networks
were not very fast, but with a better network, it would have been limiting.


    > From: Steve Simon

    > do you have pointers to any documentation on the rand/MIT network API?

There was no 'MIT' network API. He was talking about the CHAOSNet API. The
TCP/IP done in the CSR group at MIT used a totally different API.

The various Unix systems at MIT were pretty well out of touch with each other,
and did not exchange code. The only exceptions were the DSSR (later RTS) and
CSR groups in Tech Sq, who used pretty much the same system.

	Noel


From jnc at mercury.lcs.mit.edu  Tue May 23 11:33:41 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Mon, 22 May 2017 21:33:41 -0400 (EDT)
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
Message-ID: <20170523013341.CAA4118C095@mercury.lcs.mit.edu>

    > From: Clem Cole

    > So some other mechanism (also discussed here) needed to be created to
    > avoid blocking in the application.
    > ...
    > Rand, UNET & Chaos had something else that gave the save async function,
    > who's name I've forgotten at the moment

I don't think the RAND code had the non-blocking stuff; AFAICR, all it had was
named pipes (effectively). Jack Haverty at BBN defined and implemented two new
calls (IIRC, 'capac()' and 'await()') to do non-blocking I/O. The
documentation for that is in the 'BBN' branch at TUHS:

  http://minnie.tuhs.org/cgi-bin/utree.pl?file=BBN-V6/doc/ipc/await
  http://minnie.tuhs.org/cgi-bin/utree.pl?file=BBN-V6/doc/ipc/ipc

My memory might be incorrect, but I don't think it was asynchronous (i.e. a
process issued a read() or write(), and that returned right away, before the
I/O was actually done, and the system notified the process later when the I/O
actually completed).

I actually did implement asyn I/O for an early LAN device driver - and just to
make it fun, the device was a DMA device, and we didn't want the overhead of a
copy, so the DMA was direct to buffers in the process - i.e. 'raw' I/O. So
that required some major system tweaks, to keep the process from being swapped
out - or moved around - while the I/O was pending.
  
    > I believe Noel posted the code for same in the last year from one of the
    > MIT kernels

I found it on the dump of an MIT machine, but it was never run on any machine
at MIT - we just had the source in case we had any use fot it.

       Noel


From pnr at planet.nl  Tue May 23 21:35:02 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Tue, 23 May 2017 13:35:02 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
Message-ID: <12C21350-E009-46A8-B50F-76D2AA6A8CF7@planet.nl>


>> There are two other routes to TCP/IP on a PDP11 without split I/D:
>> ...
>> DCEC's adaptation of the Wingfield TCP/IP library, designed to work
>> with V6. It is mostly a user space daemon, but requires some kernel
>> enhancements.
> 
> I wonder what the performance would be like, since the TCP is in a user
> process (a different one from the application), i.e. there's a process switch
> every time the application goes to send or receive data. This wouldn't have
> been such an issue when the code was written, since ARPANet-type networks
> were not very fast, but with a better network, it would have been limiting.

IEN98 (http://www.rfc-editor.org/rfc/ien/ien98.txt, page 2) has the answer: about 10kb/s.

The DCEC version used shared memory instead of rand ports and was claimed to be
a bit more performant, but I have no number. I'd be surprised if it was twice as fast,
so perhaps 15kb/s.

Paul



From jnc at mercury.lcs.mit.edu  Tue May 23 23:43:51 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Tue, 23 May 2017 09:43:51 -0400 (EDT)
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
Message-ID: <20170523134351.3AFE118C09B@mercury.lcs.mit.edu>

    > From: Paul Ruizendaal

    > I'm looking into the history of Spider and early Datakit. Sandy Fraser
    > was kind enough to send me the 1974 report on Spider

Is that online? If not, any chances you can make it so?


    > Does anybody know of surviving v5/v6/v7 code for Spider networking (e.g.
    > the 'tiu' device driver, the 'nfs' file transfer package, etc.)?

You're in luck.

To start with, check out:

  http://minnie.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/dmr/oldstuff

which contains the drivers tiu.c, mpx.c - I'm not sure what other files there
are part of it?

I'm not at all clear how this stuff got there - someone at Bell must have just
dumped the contents of the 'dmr' directory, and sent it all off?


The PWB1-based MIT systems also have a lot of the Spider software (although it
was never used). It's a slightly different version than the one above: 'diff'
shows that 'tiu.c' is almost identical, but mpx.c has more significant
differences.

It also contains man pages, and sources for some (?) user programs; I have the
source and manpage for 'nfs'. What other names should I be looking for? (The
man page for 'nfs' doesn't list any other commands.) I'll put them up
momentarily.

In the meantime, I'll append the 'tiu' man page. There isn't one for mpx,
alas.

	Noel


--------


.th TIU IV 10/28/73
.sh NAME
tiu \*- Spider interface
.sh DESCRIPTION
Spider
is a fast digital switching network.
.it Tiu
is a directory which contains
files each referring to a Spider control
or data channel.
The file /dev/tiu/d\fIn\fR refers to data channel \fIn;\fR
likewise /dev/tiu/c\fIn\fR refers to control channel \fIn\fR.
.s3
The precise nature of the UNIX interface
is specified elsewhere.
.sh FILES
/dev/tiu/d?, /dev/tiu/c?
.sh BUGS




From pnr at planet.nl  Wed May 24 19:20:23 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Wed, 24 May 2017 11:20:23 +0200
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <mailman.1.1495591202.25149.tuhs@minnie.tuhs.org>
References: <mailman.1.1495591202.25149.tuhs@minnie.tuhs.org>
Message-ID: <104E0596-1A31-4F8D-BB0C-C50EA1868EED@planet.nl>

> 
>> I'm looking into the history of Spider and early Datakit. Sandy Fraser
>> was kind enough to send me the 1974 report on Spider
> 
> Is that online? If not, any chances you can make it so?

It is a paper copy, but I can make a scan for you.

> which contains the drivers tiu.c, mpx.c - I'm not sure what other files there
> are part of it?

I think tiu.c might be all. The TIU ("terminal access unit") was the network card,
so to speak (actually some 5 boards in a rack) and did a lot of the heavy lifting.

From the tiu.c file I understand that a DR11-B parallel I/O card was used on
the PDP side to connect to the TIU, and that access was structured as a block
device driver.

> I'm not at all clear how this stuff got there - someone at Bell must have just
> dumped the contents of the 'dmr' directory, and sent it all off?

Looks like it.

> The PWB1-based MIT systems also have a lot of the Spider software (although it
> was never used). It's a slightly different version than the one above: 'diff'
> shows that 'tiu.c' is almost identical, but mpx.c has more significant
> differences.
> 
> It also contains man pages, and sources for some (?) user programs; I have the
> source and manpage for 'nfs'. What other names should I be looking for? (The
> man page for 'nfs' doesn't list any other commands.) I'll put them up
> momentarily.

I think that in the lifespan of Spider (1972-1978) there were 3 main network
programs (basing myself on McIlroy's Unix Reader):
- 'nfs' an FTP-like program to copy files to/from a central File Store.
   I'm not sure whether the File Store was a Unix machine or something else.
- 'ufs' not sure what it was, but I think a telnet-like facility
- 'npr' a network printing program

A little surprising, but no reference to a Spider mail program in that document.

> In the meantime, I'll append the 'tiu' man page.

Thanks! It is from October 1973, which sounds right for Spider. I guess this
code is the first networking on Unix, predating the UoI work by about 18 months.




From jnc at mercury.lcs.mit.edu  Thu May 25 01:21:15 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Wed, 24 May 2017 11:21:15 -0400 (EDT)
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
Message-ID: <20170524152115.C90CE18C09B@mercury.lcs.mit.edu>

    > From: Paul Ruizendaal

    >>> the 1974 report on Spider

    >> Is that online? If not, any chances you can make it so?

    > It is a paper copy, but I can make a scan for you.

That makes it sounds like it might not be possible to put it online?
What's the exact title, so I can look and see if it's already online?
I'm pretty sure I've got a hardcopy of some Spider thing, but it would
probably take me a while to find it... ;-)


    > I think that in the lifespan of Spider (1972-1978) there were 3 main
    > network programs (basing myself on McIlroy's Unix Reader):
    > - 'nfs' an FTP-like program ...
    > - 'ufs' not sure what it was, but I think a telnet-like facility
    > - 'npr' a network printing program

OK, the only one I have is 'nfs'. Here's the source, and man page:

  http://ana-3.lcs.mit.edu/~jnc/tech/unix/s2/nfs.a
  http://ana-3.lcs.mit.edu/~jnc/tech/unix/man6/nfs.6

	Noel


From reed at reedmedia.net  Thu May 25 03:19:05 2017
From: reed at reedmedia.net (Jeremy C. Reed)
Date: Wed, 24 May 2017 12:19:05 -0500 (CDT)
Subject: [TUHS] Unix with TCP/IP for small PDP-11s
In-Reply-To: <20170524152115.C90CE18C09B@mercury.lcs.mit.edu>
References: <20170524152115.C90CE18C09B@mercury.lcs.mit.edu>
Message-ID: <alpine.NEB.2.20.1705241216310.28948@t1.m.reedmedia.net>

On Wed, 24 May 2017, Noel Chiappa wrote:

>     > From: Paul Ruizendaal
> 
>     >>> the 1974 report on Spider
> 
>     >> Is that online? If not, any chances you can make it so?
> 
>     > It is a paper copy, but I can make a scan for you.
> 
> That makes it sounds like it might not be possible to put it online?
> What's the exact title, so I can look and see if it's already online?
> I'm pretty sure I've got a hardcopy of some Spider thing, but it would
> probably take me a while to find it... ;-)

Maybe it is:

%T Spider \(em An Experimental Data Communications System
%Z ctr127
%A A. G. Fraser
%J Proc. IEEE Conf. on Communications
%P 21F
%O IEEE Cat. No. 74CH0859-9-CSCB.
%D June 1974

(as seen in usr/dict/papers/Rv7man)



From cubexyz at gmail.com  Thu May 25 10:23:07 2017
From: cubexyz at gmail.com (Mark Longridge)
Date: Wed, 24 May 2017 20:23:07 -0400
Subject: [TUHS] can't protect files from deletion v5,v6, or v7
Message-ID: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>

Ok, I just did an experiment with the rm command and the results surprised me.

On Unix v5 logged in as root I created a small test file then did
chmod 444 on it. Unfortunately it appears that mere users can still rm
the file and also directories are not safe from the rmdir command
(even directories set to mode 444).

This seems to be the case for v6 and v7 as well.

To be fair rm will prompt the user with: test1: 0100444 mode
but the user only has to type y and hit enter and the file is toast.

Is there no way to completely protect files from being deleted?

Mark


From chris at groessler.org  Thu May 25 10:28:13 2017
From: chris at groessler.org (Christian Groessler)
Date: Thu, 25 May 2017 02:28:13 +0200
Subject: [TUHS] can't protect files from deletion v5,v6, or v7
In-Reply-To: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>
References: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>
Message-ID: <c785aa10-f908-447b-165e-96707bf9ff0c@groessler.org>

On 05/25/17 02:23, Mark Longridge wrote:

> Is there no way to completely protect files from being deleted?

There is a way, write-protect the directory. What else?

regards,
chris



From crossd at gmail.com  Thu May 25 10:48:38 2017
From: crossd at gmail.com (Dan Cross)
Date: Wed, 24 May 2017 20:48:38 -0400
Subject: [TUHS] can't protect files from deletion v5,v6, or v7
In-Reply-To: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>
References: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>
Message-ID: <CAEoi9W59e5sZe2xLfTV2zT_PEuXJPdvym3R2J2qQ5H9YYWS1_g@mail.gmail.com>

On Wed, May 24, 2017 at 8:23 PM, Mark Longridge <cubexyz at gmail.com> wrote:

> Ok, I just did an experiment with the rm command and the results surprised
> me.
>
> On Unix v5 logged in as root I created a small test file then did
> chmod 444 on it. Unfortunately it appears that mere users can still rm
> the file and also directories are not safe from the rmdir command
> (even directories set to mode 444).
>
> This seems to be the case for v6 and v7 as well.
>
> To be fair rm will prompt the user with: test1: 0100444 mode
> but the user only has to type y and hit enter and the file is toast.
>
> Is there no way to completely protect files from being deleted?


Yes, these are the normal semantics, even on modern systems. If you want to
prevent a user from removing a file, remove the user's write permission to
the directory containing the file. Recall that a "file" in the removable
sense is really a directory entry that points to the inode that represents
the real file; to remove that, one must modify the directory to remove the
directory entry. The permissions on the file itself don't matter since
removal isn't an operation on the contents of the file; the only thing it
does to the actual file is update the link count in the inode.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170524/30b9b65b/attachment.html>

From ron at ronnatalie.com  Thu May 25 10:57:00 2017
From: ron at ronnatalie.com (Ron Natalie)
Date: Wed, 24 May 2017 20:57:00 -0400
Subject: [TUHS] can't protect files from deletion v5,v6, or v7
In-Reply-To: <CAEoi9W59e5sZe2xLfTV2zT_PEuXJPdvym3R2J2qQ5H9YYWS1_g@mail.gmail.com>
References: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>
 <CAEoi9W59e5sZe2xLfTV2zT_PEuXJPdvym3R2J2qQ5H9YYWS1_g@mail.gmail.com>
Message-ID: <021401d2d4f1$d1b89ef0$7529dcd0$@ronnatalie.com>

You have to understand that you can’t directly delete a file in UNIX.   It’s never worked that way.

What you can do is remove the directory reference (in UNIX terms, a link) to the file.   When the link count goes to zero, the inode (which embodies the storage and permissions, etc..) of the file then gets freed up (along with the data blocks comprising the file).

 

The inode permissions of the file have NEVER had any bearing on whether it can be unlinked or not.   The permission to unlink is that of the directory that contains it.   If the directory is writable, the file can be unlinked from that directory.   If that’s the last link, the file goes away.

 

RM in many versions checks to see if the file is read only and asks if you are sure, but that is an artificial safety net done by the rm program (which is not present in the unlink system call itself).

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170524/c41748ff/attachment.html>

From dave at horsfall.org  Thu May 25 11:36:02 2017
From: dave at horsfall.org (Dave Horsfall)
Date: Thu, 25 May 2017 11:36:02 +1000 (EST)
Subject: [TUHS] can't protect files from deletion v5,v6, or v7
In-Reply-To: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>
References: <CADxT5N6NWDJb-JMKk+PMbY9gQb2PsQpaKKO9XFoMuPQ===_sEA@mail.gmail.com>
Message-ID: <alpine.BSF.2.20.1705251132360.68852@aneurin.horsfall.org>

On Wed, 24 May 2017, Mark Longridge wrote:

> Ok, I just did an experiment with the rm command and the results 
> surprised me.

[...]

It's always been the case that you need write permission on the parent 
directory; after all, it's what you are actually updating.

> Is there no way to completely protect files from being deleted?

Don't make the parent directory writable...  And if your OS supports file 
attributes I believe there's an "immutable" flag.

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."


From pnr at planet.nl  Fri May 26 05:31:03 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Thu, 25 May 2017 21:31:03 +0200
Subject: [TUHS] Early Unix Spider network
In-Reply-To: <mailman.880.1495676169.3779.tuhs@minnie.tuhs.org>
References: <mailman.880.1495676169.3779.tuhs@minnie.tuhs.org>
Message-ID: <DC29453D-8C50-4A35-9373-7CD3840E756E@planet.nl>


>> It is a paper copy, but I can make a scan for you.
> 
> That makes it sounds like it might not be possible to put it online?
> What's the exact title, so I can look and see if it's already online?
> I'm pretty sure I've got a hardcopy of some Spider thing, but it would
> probably take me a while to find it... ;-)

The report I have is:
"SPIDER-a data communication experiment", Tech Report 23 , Bell Lab, 1974.
I did not find it online, but it may be out there somewhere.

I think it can be public now, but doing some checks.

> OK, the only one I have is 'nfs'. Here's the source, and man page:
> 
>  http://ana-3.lcs.mit.edu/~jnc/tech/unix/s2/nfs.a
>  http://ana-3.lcs.mit.edu/~jnc/tech/unix/man6/nfs.6

Many thanks! There is some puzzling stuff in there that I'd like to
figure out, but that is easier to discuss once the report is online.

Also, it turns out that the TIU driver was in Warren's repo all along:
http://minnie.tuhs.org/cgi-bin/utree.pl?file=V4/nsys/dmr/tdir/tiu.c
http://minnie.tuhs.org/cgi-bin/utree.pl?file=V4/man/man4/tiu.4
It's fun to read that the V4 man page says "The precise nature of the
UNIX interface has not been defined yet." and Noel's version says:
"The precise nature of the UNIX interface is defined elsewhere." (yet
the dates are the same!).

Some things are surprising (to me, at least):

First of all, opening a connection to the File Store is a single open on
data channel 1:
http://chiselapp.com/user/pnr/repository/Spider/artifact/854a591c0e7a3a54?ln=26
I would expected the code to first have sent a connection request to the
switch on control channel 1. Perhaps the File Store was an integral part
of the switch/router (a Tempo minicomputer 
ftp://bitsavers.informatik.uni-stuttgart.de/pdf/tempoComputers/TEMPO-1_ad_Nov69.pdf)
with channel 1 functionality hardwired.

Next, the code has a hackish form of non-blocking I/O:
http://chiselapp.com/user/pnr/repository/Spider/artifact/55ee75831bd98d6c?ln=35,37
I'm puzzled about the alarm() sys call. That did not exist in 1973 -- or did it
only exist in Bell Labs private builds?

The code calls snstat(), for instance here:
http://chiselapp.com/user/pnr/repository/Spider/artifact/55ee75831bd98d6c?ln=38
That seems to be a sys call to here:
http://chiselapp.com/user/pnr/repository/Spider/artifact/2c7d65073a7cb0a5?ln=474,524
Could that be the tiu sys call (#45) in the sysent.c table for V4-V6?











From pnr at planet.nl  Sat May 27 05:08:23 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Fri, 26 May 2017 21:08:23 +0200
Subject: [TUHS] Early Unix Spider network
Message-ID: <29939564-F3EC-461E-BA74-BBD4AB8FAA50@planet.nl>


Progressed a little further:

[1] The 'ufs' command was a variation on the 'nfs' command. The man page
that Noel provided for nfs includes the paragraph:

"There is a command /usr/usg/tom/ufs which transfers files to
the USG Unix systems.  The option letter 7 for the 11/70 or
4 for the 11/45 should be used.  Otherwise 'ufs' is similar to
'nfs'."

This means there must have been a Unix based File Store (server).
Does anybody have a suggestion who 'tom' at USG might have been?

[2] The V5 man pages in the archive have a man page for 'npr',
in section VI. It says:

NAME
	npr - print file on Spider line-printer

SYNOPSIS
	npr file …

DESCRIPTION
	Npr prints files on the line, printer in the Spider room,
	sending them over the Spider loop. If there are no arguments,
	the standard input is read and submitted. Thus npr may be used
	as a filter.

FILES
	/dev/tiu/d2 tiu to loop

It suggests that the printer was hooked up to the Spider switch and that
channel 2 was hardcoded to it.

[3] Upon closer inspection, the tiu.c driver is a character mode device,
the use of disk buffers and a strategy() routine had me confused.
It is just a reflection of the fact that it uses DMA hardware.

The code for tiu.c in NSYS/V4 is rather different from the code in
the SRI-NOSC tree: thinking on how to select channels seems to have
changed in between these two versions.

[4] Also I found the below post that mentions the snstat() call:
http://minnie.tuhs.org/pipermail/tuhs/2015-December/006286.html
The object code for snstat() is in libc.a in the dmr's V5 image.
Reconstructed, the source code is here:
http://chiselapp.com/user/pnr/repository/Spider/artifact/a93175746bd9f94f

In short, snstat() is a modified stty call, an evolution in the direction of
the later ioctl() system call.

No progress as yet on the early history of 'alarm()'.

Paul




From pnr at planet.nl  Sat May 27 21:20:39 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Sat, 27 May 2017 13:20:39 +0200
Subject: [TUHS] Early Unix Spider network / alarm()
Message-ID: <C36203F7-E4C2-4DEA-B5A9-CDEC3C39DC34@planet.nl>


Below what I've been able to find about alarm():

[1] Oldest alarm() code I can find is in PWB1, dated July 1977:
http://minnie.tuhs.org/cgi-bin/utree.pl?file=PWB1/sys/sys/os/sys4.c
http://minnie.tuhs.org/cgi-bin/utree.pl?file=PWB1/sys/sys/os/clock.c

Either alarm existed in V5 and V6, and was removed from distributions
(which seems unlikely to me), or is was added after V6 was released,
perhaps soon after. In the latter case the 'nfs' code that we have must
be later than 1974 (even though the man page is dated that way).
It could be from the 2nd half of 1975.

[2] Interestingly, the idea to implement sleep() in terms of alarm()
seems to originate in UoI network unix:
http://minnie.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/ken/sys2.c -- sslep()
This occurs in Oct 1977. In V7 this idea is taken to user space and
sleep() is no longer a system call.

[3] The UoI code has an instance of alarm() being used to break out of
a potentially stalled network call, so that usage seems to have
established itself early on.




From jnc at mercury.lcs.mit.edu  Sat May 27 23:52:59 2017
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Sat, 27 May 2017 09:52:59 -0400 (EDT)
Subject: [TUHS] Early Unix Spider network
Message-ID: <20170527135259.2EB4418C0A1@mercury.lcs.mit.edu>

    > From: Paul Ruizendaal

    >>> The report I have is: "SPIDER-a data communication experiment"
    >>> ...
    >>> I think it can be public now, but doing some checks.

OK, that would be great to have online. I _think_ the hardcopy I have
(somewhere! :-) is that report, but my memory should not be trusted.

The people working on TCP/IP did know of the Spider work (like they knew of
the Cambridge ring work), but it didn't really have any impact; it was a
totally different direction than the one we were going in.


    >>> it turns out that the TIU driver was in Warren's repo all along:

V4?! Wow. I'd have never guessed it went that far back.


    >>> The code calls snstat()

    >> The object code for snstat() is in libc.a in the dmr's V5 image.
    >> Reconstructed, the source code is here:
    >> ...
    >> In short, snstat() is a modified stty call

Yes, I looked and found the original source, appended below.


    >>> Could that be the tiu sys call (#45) in the sysent.c table for V4-V6?

I wonder if we'll ever be able to find a copy of the kernel code for that
tiu() system call. And I wonder what it did?


    > [1] Oldest alarm() code I can find is in PWB1
    > ...
    > Either alarm existed in V5 and V6 .. or is was added after V6 was
    > released, perhaps soon after. In the latter case the 'nfs' code that we
    > have must be later than 1974

Remember, that source came from the MIT system, which is a modified PWB1.
So it's not surprising it's using PWB1 system calls.

	Noel


--------

/ C interface to spider status call

.globl  _snstat
.globl  cerror

_snstat:
        mov     r5,-(sp)
        mov     sp,r5
        mov     4(r5),r0
        mov     6(r5),0f
        mov     8(r5),0f+2
        sys     stty; 0f
        bec     1f
        jmp     cerror
1:
        clr     r0
        mov     (sp)+,r5
        rts     pc

.data
0:      .=.+6



From pnr at planet.nl  Mon May 29 19:54:48 2017
From: pnr at planet.nl (Paul Ruizendaal)
Date: Mon, 29 May 2017 11:54:48 +0200
Subject: [TUHS] Early Unix Spider network
Message-ID: <C4A1635B-30E6-43BE-A573-0493286EFBBE@planet.nl>


> The people working on TCP/IP did know of the Spider work (like they knew of
> the Cambridge ring work), but it didn't really have any impact; it was a
> totally different direction than the one we were going in.

I'm aware of that, and I think it was the same the other way around. My
interest is tracing how the networking API of Unix developed in the very
early days, and that's were there is a link.

When I asked a few months back why Bell Labs did not jump onto the work
done at UoI, Doug observed that the lab's focus was on Datakit and that
triggered my interest.

>>>> it turns out that the TIU driver was in Warren's repo all along:
> 
> V4?! Wow. I'd have never guessed it went that far back.

My current understanding is that Spider development began in 1969 and
that it was first operational in 1972. By '73/'74 it connected a dozen
computers at Murray Hill and Unix had gained basic network programs.

From Sandy Fraser's "Origins of ATM" video lecture I understand that the
Spider learnings included that using a mini to simulate a switch/router
was too slow and too costly, and that doing flow control inside the network
induced avoidable complexity (I guess Fraser/Cerf/Pouzin all learned that
lesson around the same time). The follow-on, custom designed Datakit switch
was to correct these issues.

Work started in 1974 and I guess that prototypes may have been available
around 1978 (when Spider was apparently switched off at Murray Hill).
By 1981 a multi-site Datakit network connected various Bell labs and by
1983 Datakit was introduced as a commercial service.

As to the Spider network API, it currently seems that it was relatively
simple: it exposed the switch as a group of character mode devices, with
the user program responsible for doing all protocol work. Interestingly,
Spider used a high speed DMA based I/O board (DR11-B), whereas the
Datakit switch was apparently connected to a low speed polled I/O board
(DR11-C).

I did not find the Datakit device driver(s) in the V7 source tree (only a
few references in tty.h), so it is hard to be sure of anything. However,
it seems that in V7 the Datakit switch was used as "a fancy modem" so to
speak, supporting the uucp software stack.

There is source for a Datakit driver in the V8 tree, but I currently
have no time to study that (and perhaps it is beyond my scope anyway).

All input and corrections much appreciated.




