From clemc at ccc.com  Mon Jun  1 00:53:57 2020
From: clemc at ccc.com (Clem Cole)
Date: Sun, 31 May 2020 10:53:57 -0400
Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
In-Reply-To: <1jfNb2-7JV-00@marmaro.de>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
Message-ID: <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>

On Sun, May 31, 2020 at 9:02 AM markus schnalke <meillo at marmaro.de> wrote:

> Would you be so kind to explain a bit about the hm version of MH.
>
I can try ...

At the time V6 (and later V7) had a program called /bin/mail.  The
important thing about this program is that it was both a Mail Transport
(and delivery) Agent as well as a Mail User Interface.   The original
version only delivered mail locally.  When networking, like UUCP, was added
to UNIX, it was easy to build a scheme that could send a transfer a message
using UUCP and then fork /bin/mail on the proper system to deliver it.
IIRC it was in Seventh Edition that /bin/mail was extended to recognize
email addresses in the prefix form of:  host!user, strip the first host!
part and pass it to uucp.

I never used it on anything like Spider, so I have not idea what it did
internally and in fact, the prefix address form could have predated UUCP.
We would need someone like Doug, Steve or Ken to tell us how email worked
host to host, pre-UUCP.

I believe it was Bruce Borden and team developed the original Rand Message
Handler or MH.  I do not know who was the primary author, we need to ask
Bruce or one of the old Rand folks.  MH ran on 6th edition when it was
first released to rest of the USENIX community (at least I never saw it on
V5, but it's possible it went back further).   I just remember talking to
Bruce about it at any early USENIX.

MH used a new Mailbox (on-disk) scheme and formatted all messages in RFC733
form with addresses being flat an in the form: user at host. The new format,
used a line a control-As followed by a nl, before and after the message.
 The headers of course where RFC733 (type: value with a trailing nl) and
were separated from the body of the message by a single nl.  MH assumed a
traditional UNIX command line for the user interface and had a number of
programs that ran behind the scene for delivery.   This is important
because the original ARPAnet NCP used FTP to do mail transfer.  At least
one version could call the early ARPAnet subsystem to perform host to host
communications.   The key point is that MH separated the MTA and MUI.

A number of us ran MH at different places.  I don't remember if it was on a
USENIX tape or I had sent Bruce a tape @ Rand when I got a copy in the late
1970's (77-79 timeframe, I've forgotten).  I'm pretty sure Goble, as did
the Purdue crew ran it, as did Holmgrem, Greg Chesson at al, @ UofI and I
would not surprised if it was the mailer at Harvard, given the Harvard/Rand
connection in those days.

By the time of Seventh Edition of UNIX, the Mail Transport Agent (MTA) that
was part of the MH subsystem, could recognize ArpaNET address postfix and
had a hack in it, that allowed the 'user' part of the address to include
UUCP addresses and /bin/mail replacement knew to work like the AT&T mailer
and pass it off to UUCP.   So, at that point, life was good for those of us
in strickly ARPA and/or UUCP land.

BTW: At some point, the BBN TCP code was releases and a separate SMTPD was
included in the release.   I don't remember if it was the Rand folks or
someone else, but at some point, the MH delivery agent was updated to call
it or be called by it.  Similarly, by the late 1970s, when Bruce, Greg
Shaw, and Bob Metcalfe formed 3Com; Bruce and Greg brought MH with them and
added an SMTPD that they wrote for their commercial product, UNET.   This
was the smtpd, I ran on the Teklabs machine before I went off to UCB.

Meanwhile, as Mary Ann and I noted in earlier messages, Kurt Shoen's wrote
a different MTA called Mail(1ucb).   Unlike MH, Kurt continues to support
the original UNIX mailbox format (later named 'mbox' format).   The header
lines were in a specific UNIX style prose starting with the ACSII 'From'
and there are no special characters to demark the messages in the format,
so recovery can be fraught with error, there are the famous From-line
munging issues etc. (lots of details in other places start with
https://en.wikipedia.org/wiki/Mbox).

At some point in time, UCB built the 'Berk-net' (whose original code was
written by ABC/Google's Eric Schmidt).  BTW: Eric would have seen the
Spider Network in his summers at BTL.  The key thing with Berknet was
cheap.  It ran over 3 wire RS-232C between hosts at 9600 baud.   Like UUCP
was used to transfer files and email.    Like UUCP it used a
pre-fix addressing form: host:user ; but like RFC733 and unlike UUCP was
flat.

Where Mary Ann and I differ in our memories is who wrote the original
version of UCB's delivermail (8ucb) program.   We both agree that it is
possible it was Eric Schmidt, as the switch to using delivermail(8ucb) was
were Berknet was spliced into the email namespace.  I had thought Kurt
wrote it, Mary Ann thought it was Eric Allman.   We agree Eric Allman was
hacking on it for the Ing70.  For this response, it doesn't really matter
other than to try to get the history right, because it does not matter for
the Rand Mail subsystem.

Around, now I arrive at UCB.   I was not the only person that had used MH,
but I had the advantage of having 3 Vaxen 780 in the CAD lab.   The key
change we made at that point was to stop using the Rand MTA and make it use
delivermail.   There was hackery to allow the shared mbox to say, but in
each user directory it stored the messages separately, MH style so the MH
'ui' suite of tools 'just worked.'   [There were a number of arguments at
the time.  I remember having one with Sam.  He hated MH because 'of all the
small files and it chewed up inodes].

Anyway around this time, the curses library was created by Ken Arnold
(originally to support Rogue) by pulling the screen code out of vi and
using Mary Ann's termcap stuff.   A couple of us in the USENIX community
started playing with screen-based front ends to MH, including the folks at
Rand.  Numerous messages were exchanged, and a collaboration started (Rand
team should get 99% percent of the work, I knew how to make curses go).
 The comment in the main code was, 'Well it sure isn't MH' -- so it was
called HM.   I later brought it Masscomp, and it was the start of their
mailer.

The key with HM, is that it uses the MH backend for all the real work.   It
would fork scan, but save the output in a text file that could be
manipulated with the arrow keys.   The user interface is not unlike Gmail.
I only stopped running HM when I finally switched to Gmail a few years ago.

In fact, when I was a DEC, the precursor to Gmail, was called
Pachyderm which used the Altavista search.  I glued MH to Pachyderm for a
short period.  I even ported them both to FreeBSD to run at home in the
mid-1990s.  Then   I left DEC, the authors of Pachyderm headed for Google,
*etc*...  I got a chance to be an early Gmail/Google Apps tester for ccc.com
and I have not switched back since.

FWIW: It's been long enough, that the sources are no longer easy to find.

It's all very incestuous.    We all have seen other ideas and good ideas
seem to have a way of reappearing in different places.

Clem

Note the part of the story I left out was when Rand MH spun out to one of
the other Universities, UCI I think.  I was never much involved with that
team, so I can not tell you much about it and their code base.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200531/7abf97bc/attachment.htm>

From clemc at ccc.com  Mon Jun  1 02:05:47 2020
From: clemc at ccc.com (Clem Cole)
Date: Sun, 31 May 2020 12:05:47 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
Message-ID: <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>

On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal <pnr at planet.nl> wrote:

>  This behaviour seems to have continued into SysVR1, I’m not sure when
> EAGAIN came into use as a return value for this use case in the SysV
> lineage. Maybe with SysVR3 networking?

Actually, I'm pretty sure that was a product of the POSIX discussions.  BSD
already had networking an EWOULDBLOCK.   We had argued about EWOULDBLOCK a
great deal, we also were arguing about signal semantics.  I've forgotten
many of the details, Heinz may remember more than I do.  EAGAIN was created
as a compromise -- IIRC neither system had it yet.   SVR3 networking was
where it went into System V, although some of the AT&T representatives were
none too happy about it.



>
> In the Research lineage, the above SysIII approach does not seem to exist,
> although the V8 manual page for open() says under BUGS "It should be
> possible [...] to optionally call open without the possibility of hanging
> waiting for carrier on communication lines.” In the same location for V10
> it reads "It should be possible to call open without waiting for carrier on
> communication lines.”
>
> The July 1981 design proposals for 4.2BSD note that SysIII non-blocking
> files are a useful feature and should be included in the new system. In
> Jan/Feb 1982 this appears to be coded up, although not all affected files
> are under SCCS tracking at that point in time. Non-blocking behaviour is
> changed from the SysIII semantics, in that EWOULDBLOCK is returned instead
> of 0 when progress is not possible. The non-blocking behaviour is extended
> beyond TTY’s and pipes to sockets, with additional errors (such as
> EINPROGRESS). At this time EWOULDBLOCK is not the same error number as
> EGAIN.
>
My memory is that Keith was the BSD (CSRG) person at the POSIX meeting (he,
Jim McGinness of DEC, and I created PAX at one point as a compromise).   I
wish I could remember all of the details, but this was all argued at the
POSIX meetings.

As I said before the folks from AT&T just wanted to take the SVID and
rubber stamp it at the specification.  Part of it the problem was they
wanted to be free to do what things/make choices that the rest of us might
or might not like (for instance, they did not want the sockets interface).



>
> It would seem that the differences between the BSD and SysV lineages in
> this area persisted until around 2000 or so.
>
Yep - cause around then POSIX started to settle out and both systems began
to follow it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200531/9b67c7a9/attachment.htm>

From rich.salz at gmail.com  Mon Jun  1 02:25:47 2020
From: rich.salz at gmail.com (Richard Salz)
Date: Sun, 31 May 2020 12:25:47 -0400
Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
In-Reply-To: <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
Message-ID: <CAFH29toQesHQtrwQPw=eO4uiy3mw-PrvT3WcYaukBW9BP3hBWA@mail.gmail.com>

"I'll try"  Wow, that was quite some "try"  Kudo's!

> This is important because the original ARPAnet NCP used FTP to do mail
transfer.

If you look at RFC 765 (dated June 1980) , there are nearly a dozen
mail-related FTP commands. By RFC 959, it's successor five years later, all
of them were gone.

Another eary MTA was the MMDF, the Multichannel Memorandum Distribution
Facility, from U Delware.  BBN ran it.  One key point is that somehow when
used with MH you could get real-time address verification before sending,
"user rs at bbn.com doesn't exist." BBN ran MMDF for a long time because a key
exec liked/needed that feature, long after 4.2 and sendmail. (My group was
one of the first to run sendmail, which I liked because of the "R$" lines
in its CF files.  (Not really :) My Usenet/email gateway code had to
support MMDF but it was only ever used on bbn.com)

>  could recognize ArpaNET address postfix and had a hack in it, that
allowed the 'user' part of the address to include UUCP addresses

Originally ihnp4!mirror!rs at seismo.arpa, then ihnp4!mirror!rs at seismo.css.gov
and then rs%mirror.uucp at seismo.css.gov and then rs at mirror.tmc.com  Thanks
to Mary Ann and the "UUCP Mapping Project" for making much of that possible
and Peter Honeyman for Pathlias, and MX records for the last part. The
pre-MX styles received much scorn from Research, and if you search for
"that hideous name" you can find a paper on it. There's also a pathlias
paper, which Honey later said "was too good for Usenix" but he had nowhere
else to submit it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200531/5967afea/attachment.htm>

From imp at bsdimp.com  Mon Jun  1 02:46:37 2020
From: imp at bsdimp.com (Warner Losh)
Date: Sun, 31 May 2020 10:46:37 -0600
Subject: [TUHS] non-blocking IO
In-Reply-To: <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
Message-ID: <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>

Sorry to top post, but LSX or Miniunix had non blocking I/O as well. It was
in one of the documents that Clem scanned in the last year. It specifically
was an experiment into how to do it.

Warner

On Sun, May 31, 2020, 10:07 AM Clem Cole <clemc at ccc.com> wrote:

>
>
> On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal <pnr at planet.nl> wrote:
>
>>  This behaviour seems to have continued into SysVR1, I’m not sure when
>> EAGAIN came into use as a return value for this use case in the SysV
>> lineage. Maybe with SysVR3 networking?
>
> Actually, I'm pretty sure that was a product of the POSIX discussions.
> BSD already had networking an EWOULDBLOCK.   We had argued about
> EWOULDBLOCK a great deal, we also were arguing about signal semantics.
> I've forgotten many of the details, Heinz may remember more than I do.
> EAGAIN was created as a compromise -- IIRC neither system had it yet.
>  SVR3 networking was where it went into System V, although some of the AT&T
> representatives were none too happy about it.
>
>
>
>>
>> In the Research lineage, the above SysIII approach does not seem to
>> exist, although the V8 manual page for open() says under BUGS "It should be
>> possible [...] to optionally call open without the possibility of hanging
>> waiting for carrier on communication lines.” In the same location for V10
>> it reads "It should be possible to call open without waiting for carrier on
>> communication lines.”
>>
>> The July 1981 design proposals for 4.2BSD note that SysIII non-blocking
>> files are a useful feature and should be included in the new system. In
>> Jan/Feb 1982 this appears to be coded up, although not all affected files
>> are under SCCS tracking at that point in time. Non-blocking behaviour is
>> changed from the SysIII semantics, in that EWOULDBLOCK is returned instead
>> of 0 when progress is not possible. The non-blocking behaviour is extended
>> beyond TTY’s and pipes to sockets, with additional errors (such as
>> EINPROGRESS). At this time EWOULDBLOCK is not the same error number as
>> EGAIN.
>>
> My memory is that Keith was the BSD (CSRG) person at the POSIX meeting
> (he, Jim McGinness of DEC, and I created PAX at one point as a
> compromise).   I wish I could remember all of the details, but this was all
> argued at the POSIX meetings.
>
> As I said before the folks from AT&T just wanted to take the SVID and
> rubber stamp it at the specification.  Part of it the problem was they
> wanted to be free to do what things/make choices that the rest of us might
> or might not like (for instance, they did not want the sockets interface).
>
>
>
>>
>> It would seem that the differences between the BSD and SysV lineages in
>> this area persisted until around 2000 or so.
>>
> Yep - cause around then POSIX started to settle out and both systems began
> to follow it.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200531/9fda90b7/attachment.htm>

From lm at mcvoy.com  Mon Jun  1 02:50:50 2020
From: lm at mcvoy.com (Larry McVoy)
Date: Sun, 31 May 2020 09:50:50 -0700
Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
In-Reply-To: <CAFH29toQesHQtrwQPw=eO4uiy3mw-PrvT3WcYaukBW9BP3hBWA@mail.gmail.com>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
 <CAFH29toQesHQtrwQPw=eO4uiy3mw-PrvT3WcYaukBW9BP3hBWA@mail.gmail.com>
Message-ID: <20200531165050.GS22016@mcvoy.com>

On Sun, May 31, 2020 at 12:25:47PM -0400, Richard Salz wrote:
> Originally ihnp4!mirror!rs at seismo.arpa, then ihnp4!mirror!rs at seismo.css.gov
> and then rs%mirror.uucp at seismo.css.gov and then rs at mirror.tmc.com  Thanks
> to Mary Ann and the "UUCP Mapping Project" for making much of that possible
> and Peter Honeyman for Pathlias

My first real research paper was with Udi Manber and it was all about 
compressing the pathalias maps.  I had them on a 40MB disk on a Masscomp
that had 20 users.  The format was

seismo.css.gov	geowhiz!geophiz!uwisc!ihnp4!mirror

in other words

destination	path!to!get!there

I went to Udi and explained that all of our stuff started with
geowhiz!geophiz!uwisc
so that string was replicated over and over and it took too much
space.

Once Udi understood the problem he went "You have time best case,
space worst case, you can reverse them like so"

seismo	-> mirror
mirror	-> ihnp4
ihnp4	-> uwisc
uswisc	-> geophiz
geophiz	-> geowhiz

and you'll have time worst case and space best case.  I was like "oh,
yeah, you are right, cool, thanks" and got up to leave.  Not so fast,
says Udi, that's not "interesting".  What would be interesting is if
you could approximate time best case and space best case.

With that, I started down the only dynamic programming problem I have
ever solved.  Udi's idea was to break the graph at what he called
pivot points, where the pivot points as much as possible got rid of
the replicated strings.  He also wanted to limit lookups to just
2.

So I had to read in the maps and build a graph from them on my
creaky microvax with 4MB of ram.  Doing so lead me to the first
place I taught Udi about systems.  I was fgets() each line and 
mallocing a copy of it.  Then I sorted with qsort() and it took
overnight to get the sorted array.  Udi asked what sort?  Qsort.
Oh, that's garbage, use a radix sort, that will be better.  WTF
is a radix sort, look that up, implemented it, it was worse.

So I start poking around and somehow realized we were thrashing
VM, we didn't fit, and I have no idea how I figured out that
malloc was fragmenting memory but I did.  Probably figured out
that most of my program was backed by swap, not files, and that's
usually malloc or brk.

So I wrote my own malloc that grabbed memory in 300KB chunks (why
that size, I dunno, seemed right).  Then I allocated memory out
of those chunks.  Got rid of the fragmentation and we fit in
4MB.  qsort worked fine, took about 20 minutes (we didn't really
fit but we were way closer).

Udi was watching as I was doing all this, I think I was working
in his office, it was his microvax.  The fragmentation thing 
taught him that when Unix says it has virtual memory and you
can just use it, it only works well when you fit.

So the programming problem turned out that you calculated the 
space it work take for every possible pivot point and then at
the end, you took the set that gave you the minimum space.  We
wrote, heh, who am I kidding, Udi wrote, a paper about it.  I
was still green on writing papers, they were hard for me.  I told
Udi that and he stared at me blankly and said "Writing papers is
easy if you know the content.  You just make a good outline and
fill it in."  He's right, that's how you do it, and it is easy
if you know and can make the outline.

I went on to do a lot of work with/for him.  I wrote a user level
thread library that made me write swtch() in assembler for the VAX
(I believe I did a 68K version too but the source I have is VAX
only).  I "cheated" and did 99% of the work in C and popped down
the rabbit hole to assembler, did the nasty, and popped back out
as a different thread.  112 lines:

http://www.mcvoy.com/lm/T/src/Tasm.S

Funny Udi story, I was young and wild and drove too fast and got 
tickets.  So I got a radar detector that beeped when it saw the
radar gun.  Udi upgraded the microvax to some newer release of
X10 or X11 and the new xclock beeped on the hour.  Every time
it beeped, I jumped because it sounded just like the radar 
detector.  Udi asked about it, I explained, he laughed and said
"Larry, you're hacking too fast" each time I jumped :)

Good times.  I almost did a PhD under him but I didn't care for
the CS department at Tucson where he went and I liked industry
too much.

From ralph at inputplus.co.uk  Mon Jun  1 03:09:24 2020
From: ralph at inputplus.co.uk (Ralph Corderoy)
Date: Sun, 31 May 2020 18:09:24 +0100
Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
In-Reply-To: <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
Message-ID: <20200531170924.0609821F85@orac.inputplus.co.uk>

Hi Clem,

> I believe it was Bruce Borden and team developed the original Rand
> Message Handler or MH.  I do not know who was the primary author, we
> need to ask Bruce or one of the old Rand folks.

https://en.wikipedia.org/wiki/MH_Message_Handling_System#History says

    MH was proposed by R. Stockton Gaines and Norman Shapiro and
    developed by Bruce S. Borden and others at RAND Corporation.

and cites
https://www.rand.org/content/dam/rand/pubs/notes/2009/N3017.pdf

Norman Shapiro, or Norm, is still using nmh well into his eighties, nmh
is a descendant of MH, and pops up on the nmh-workers mailing list to
either fill in bits of history or ask for installation advice as the
Linux distro shifts under his feet.  He did visit Bell Labs, though in
'54.  :-)  Details at https://en.wikipedia.org/wiki/Norman_Shapiro

> The new format, used a line a control-As followed by a nl, before and
> after the message.

MH's mts.conf(5) had the ability to set mmdelim{1,2} to something other
than \ca\ca\ca\ca\n; no idea why.  And that magic marker would be
escaped if it occurred in the middle of a message, to \cb\ca\ca\ca\n.

> Note the part of the story I left out was when Rand MH spun out to one
> of the other Universities, UCI I think.

Wikipedia again:

    Subsequently development was taken over by Marshall T. Rose and John
    L. Romine working at University of California, Irvine.

    nmh... was forked [from that] by Richard Coleman while working at
    the Georgia Institute of Technology.  It incorporates the "LBL
    changes" made to MH in the late 80s by Van Jacobson, Mike Karels and
    Craig Leres.

MH also had support for UCI's bulletin boards called ‘BBoards’,
including on ZOTnet.  They were optionally available over POP3 and NNTP,
but only one or the other.  :-)  There was an MH shell of some kind
called msh, and a visual mh, vmh.

ZOTnet was a LAN for mail by Marshall Rose at UCI with gateways for
CSnet and ARPAnet.

    https://escholarship.org/content/qt4bm4k2vh/qt4bm4k2vh.pdf
    The ZOTnet: a local area mailing network, 1983

(I still use nmh and I'm using it for this reply.)

-- 
Cheers, Ralph.

From jon at fourwinds.com  Mon Jun  1 03:25:07 2020
From: jon at fourwinds.com (Jon Steinhart)
Date: Sun, 31 May 2020 10:25:07 -0700
Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
In-Reply-To: <20200531170924.0609821F85@orac.inputplus.co.uk>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
 <20200531170924.0609821F85@orac.inputplus.co.uk>
Message-ID: <202005311725.04VHP7bX3814695@darkstar.fourwinds.com>

Ralph Corderoy writes:
> ...
>
> (I still use nmh and I'm using it for this reply.)

+1 on that.  I also still use nmh.  To me, the set of shell commands
exemplifies the UNIX philosophy as I can easily "show | wc -l" or
whatever that is cumbersome with monolithic packages.  My minor
contributions were adding the attachment support and the hooks.  The
hooks allow other programs to be triggered behind the scenes when
nmh commands are run; I use it to keep an elasticsearch database of
my messages.  I have two additional command script to support thise:
gpick and gscan which are the equivalent of pick and scan except that
they search the elasticsearch database and pretty much return results
instantaneously.

Jon

From mah at mhorton.net  Mon Jun  1 03:28:56 2020
From: mah at mhorton.net (Mary Ann Horton)
Date: Sun, 31 May 2020 10:28:56 -0700
Subject: [TUHS] mh/hm, mmh
In-Reply-To: <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
Message-ID: <bd93f90e-c613-3ebf-749e-3c4ed734d34c@mhorton.net>

What a wonderful historical tour, Clem! Thank you!

On 5/31/20 7:53 AM, Clem Cole wrote:
> MH used a new Mailbox (on-disk) scheme and formatted all messages in 
> RFC733 form with addresses being flat an in the form: user at host. The 
> new format, used a line a control-As followed by a nl, before and 
> after the message.   The headers of course where RFC733 (type: value 
> with a trailing nl) and were separated from the body of the message by 
> a single nl.
MH's message/file format was also the inspiration for B news, which kept 
each Usenet message in a separate RFC733 format file. (The command line 
interface, however, was more like a feed, not separate shell commands.) 
No control-A separators, though, just separate files.
> At some point in time, UCB built the 'Berk-net' (whose original code 
> was written by ABC/Google's Eric Schmidt).  BTW: Eric would have seen 
> the Spider Network in his summers at BTL.  The key thing with Berknet 
> was cheap.  It ran over 3 wire RS-232C between hosts at 9600 baud.  
>  Like UUCP was used to transfer files and email.    Like UUCP it used 
> a pre-fix addressing form: host:user ; but like RFC733 and unlike UUCP 
> was flat.
>
> Where Mary Ann and I differ in our memories is who wrote the original 
> version of UCB's delivermail (8ucb) program.   We both agree that it 
> is possible it was Eric Schmidt, as the switch to using 
> delivermail(8ucb) was were Berknet was spliced into the email 
> namespace.  I had thought Kurt wrote it, Mary Ann thought it was Eric 
> Allman.   We agree Eric Allman was hacking on it for the Ing70.  For 
> this response, it doesn't really matter other than to try to get the 
> history right, because it does not matter for the Rand Mail subsystem.
To clarify, I was joking when I mentioned Eric Schmidt as a possible 
author of delivermail. (I should have inserted a smiley.) I wasn't 
directly involved, as Eric Allman was in the Ingres project on a 
different system, but I thought he (Eric Allman) wrote delivermail, and 
so does Wikipedia. I think Kurt Shoens wrote only the MTA, but i could 
be mistaken.
> Anyway around this time, the curses library was created by Ken Arnold 
> (originally to support Rogue) by pulling the screen code out of vi and 
> using Mary Ann's termcap stuff.

I didn't write termcap, Bill Joy did. I wrote terminfo, but that was 
later when I was at Bell Labs in 1982. Ken's curses used termcap, my 
1982 rewrite used terminfo.

     Mary Ann


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200531/08283591/attachment-0001.htm>

From imp at bsdimp.com  Mon Jun  1 03:43:46 2020
From: imp at bsdimp.com (Warner Losh)
Date: Sun, 31 May 2020 11:43:46 -0600
Subject: [TUHS] LSX on the PDP-11/03 (LSI-11)
In-Reply-To: <193c770d408ee131dbaa9a07dba6f068.squirrel@squirrelmail.tuffmail.net>
References: <CAD05_j2WM0z=oEz-woQLrB+rDH2Q3StoPSAQVa4zCLu1HdP2qA@mail.gmail.com>
 <193c770d408ee131dbaa9a07dba6f068.squirrel@squirrelmail.tuffmail.net>
Message-ID: <CANCZdfrpUd8JTAzf07xFu=j+TQyuXzzeHtO93BFv6YJMZGgp1w@mail.gmail.com>

On Sun, May 31, 2020 at 6:34 AM <ron at ronnatalie.com> wrote:

>
> > Anyway, I've tried compiling Hello World on LSX, and I get "1: External
> > definition syntax" error. Some help would be nice, but more generally, is
> > anyone on this list more than vaguely familiar with LSX, or 6th Edition
> > itself?
> >
> > void main () {
> >         printf("Hello World!");
> > }
> >
> > It seems that the 7th Edition was the beginning of the standard library
> in
> > C, and that this is missing in LSX. I'm not sure if printf is an
> intrinsic
> > function in (6th Edition) C, or if it's from a library.
> >
>
> First off, VOID MAIN is not legal in any standard version of C.  Even when
> the language allows implementation defined extensions to the main
> signature, it must still return int.
>
> If you have a later version of language supported, you have to define
> printf rather than allowing it default define as an int returning
> function.
>
> Add #include <stdio.h>
>

Or, alternatively, LSX is really old. The compiler there likely doesn't
support 'void'.

>From the lsx part of tuhs:

# cp /dev/tty8 addr.c
main() {
  int a;
  printf("main address: %u (0%o)\n", main, main);
  printf("stack address: %u (0%o)\n", &a, &a);
  printf("heap address: %u (0%o)\n", sbrk(0), sbrk(0));
}
^D
# cc addr.c
# a.out
main address: 16410 (040032)
stack address: 40934 (0117746)
heap address: 17920 (043000)

None of the disks appear to have a /usr/include. The only .h files look
like they were from the kernel.

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200531/317f6fc1/attachment.htm>

From tuhs at eric.allman.name  Mon Jun  1 04:56:07 2020
From: tuhs at eric.allman.name (Eric Allman)
Date: Sun, 31 May 2020 11:56:07 -0700
Subject: [TUHS] mh/hm, mmh
In-Reply-To: <bd93f90e-c613-3ebf-749e-3c4ed734d34c@mhorton.net>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
 <bd93f90e-c613-3ebf-749e-3c4ed734d34c@mhorton.net>
Message-ID: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com>

Ah, time for some "Email at Berkeley" history....

First the back story.  In the beginning, /bin/mail was the mail system. 
All mail was local to one machine so the distinction between MUA, MTA,
and MSA (Mail Storage Agent) didn't exist.  At this point there were no
options, at least at Berkeley, although I suspect that Bruce was working
on MH by then.

If my memory serves, networking at Berkeley started with BerkNET.  Clem
got it right: Eric Schmidt wrote it, and it was designed to be cheap. 
The main purpose was that Berkeley was getting more Unix machines and
carrying tapes was problematic.  It was purely batch (no interactive
sessions; file copy and email only) and ran over 9600 baud TTY lines. 
Eric hacked /bin/mail to do the appropriate calls if it saw a colon in
the address, so the flow of a typical message would be keyboard →
/bin/mail → BerkNET → /bin/mail → /var/mail.

In about 1976 the INGRES project at Berkeley got an ARPAnet connection. 
6th Edition was current then, and we got the software from (iirc) UC San
Diego.  It was kind of a mess, but it did work.  It was a VDH (Very
Distant Host) interface running at 9600 baud.  But the mail system it
included, MSG I think, was a world unto itself, and definitely did not
use /bin/mail.  So now we had two universes: to send something to both
someone on BerkNET and someone on ARPAnet you had to type the message
into two different MUAs.  But at first this was good enough.  But not
for long.

The problem was that to use the ARPAnet you had to be logged into our
PDP-11/70.  Back then the cool kids had "glass TTYs" (i.e., adm3a).  But
BerkNET didn't have interactive connections, so you had to actually have
a TTY line into our machine.  Two problems arose: first, INGRES was in a
different building (Cory Hall) than the rest of the department (Evans
Hall).  But that's OK, there were already wires running between the two
buildings that were probably inspired by BerkNET.  Second, our PDP-11
had a single DH-11 multiport interface with 16 ports.  Only two of those
ports were available, and new DH-11s cost on the order of $15,000, money
which we did not have.

But no problem.  At this point there was already a "plug board" in the
mail room in Evans that had ports to the half dozen or so machines that
were by then on campus and lines to each office.  To connect you would
plug in a wire between the machine you wanted to use and your office. 
Protocol was that you remove the cross connect after use, but that was
widely ignored.  Ing70 ports were quite frequently being pulled out
while in use and no one was happy.  Unfortunately since I was kind of
managing that machine, I was the one who got yelled at.

Also in that period Berkeley got connected into UUCP.  UUCP had also
done a hack to /bin/mail, so it was becoming quite the big mess.  I made
the observation that I could hack /bin/mail to forward messages with an
"@" sign in the address to our ARPAnet-connected machine, but at that
point the code was becoming unwieldy.  Enter delivermail, which was
essentially what we now call an MTA.  It was still a hack, but it was
easier to deal with because it didn't have MUA and MSA functionality.  I
changed /bin/mail to call delivermail unless it got a "-d" (deliver)
flag, so now the sequence for local mail was /bin/mail → delivermail →
/bin/mail -d → /var/mail.  And most importantly, I hacked the ARPAnet
code to use delivermail as well, so now Ing70 could forward mail from
BerkNET to ARPAnet and back again.  The fights over our two plug board
ports mostly dissipated and my life got better.

I'm pretty sure that at this point Kurt Schoens was working on Mail.  He
read /var/mail (at that point it had been migrated to /var/spool/mail)
directly, so there was no hiding of the mailbox format, so he had MUA +
a half of an MSA.  He converted Mail to use delivermail for outgoing
mail.  I don't recall the details of how MH got hooked in, but it did
have a separate program for sending (called "send" I think) that would
have been fairly easy to hack.  It still had "inc" (incorporate) to
bring mail from the /var/mail one-file-per-mailbox format into the MH
one-file-per-message format, so the three functions were not completely
broken out, but it was really the closest to that ideal model available
at the time, so far as I know.

That's the gist of that story.  Sendmail is a completely different
story, driven by the onset of the Internet, and I won't make this
already long message even longer by going into it here.

By the way, I have a 9-track tape sent to me by Marshall Rose containing
the November 1983 version of MH that UCI did (based on the Rand system),
complete with original versions of the documentation.  If anyone has any
way of reading it I'm happy to pass it on.

eric

From clemc at ccc.com  Mon Jun  1 05:49:07 2020
From: clemc at ccc.com (Clem Cole)
Date: Sun, 31 May 2020 15:49:07 -0400
Subject: [TUHS] mh/hm, mmh
In-Reply-To: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
 <bd93f90e-c613-3ebf-749e-3c4ed734d34c@mhorton.net>
 <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com>
Message-ID: <CAC20D2Pa-HS_ooP0Y3CBs9vjS-m7fFRpC-tyrEN5sEzgBFs9-Q@mail.gmail.com>

On Sun, May 31, 2020 at 3:38 PM Eric Allman <tuhs at eric.allman.name> wrote:

> but it did have a separate program for sending (called "send" I think)
> that would
> have been fairly easy to hack.  It still had "inc" (incorporate) to
> bring mail from the /var/mail one-file-per-mailbox format into the MH
> one-file-per-message format
>
Right, the hooks were in send(1mh) and inc(1mh).

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200531/68e7e9aa/attachment.htm>

From mah at mhorton.net  Mon Jun  1 07:31:30 2020
From: mah at mhorton.net (Mary Ann Horton)
Date: Sun, 31 May 2020 14:31:30 -0700
Subject: [TUHS] mh/hm, mmh
In-Reply-To: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
 <bd93f90e-c613-3ebf-749e-3c4ed734d34c@mhorton.net>
 <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com>
Message-ID: <e5f793d5-95b3-6848-8753-7fdb38be01d0@mhorton.net>

Wonderful! Thank you for the history, Eric!

     Mary Ann

On 5/31/20 11:56 AM, Eric Allman wrote:
> Ah, time for some "Email at Berkeley" history....
>

From robpike at gmail.com  Mon Jun  1 08:01:58 2020
From: robpike at gmail.com (Rob Pike)
Date: Mon, 1 Jun 2020 08:01:58 +1000
Subject: [TUHS] non-blocking IO
In-Reply-To: <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
Message-ID: <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>

>
> I’m not quite sure why the Research lineage did not include non-blocking
> behaviour, especially in view of the man page comments. Maybe it was seen
> as against the Unix philosophy, with select() offering sufficient mechanism
> to avoid blocking (with open() the hard corner case)?


That's it. Select was good enough for our purposes.

-rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200601/c8fb9aa9/attachment.htm>

From dave at horsfall.org  Mon Jun  1 13:32:56 2020
From: dave at horsfall.org (Dave Horsfall)
Date: Mon, 1 Jun 2020 13:32:56 +1000 (EST)
Subject: [TUHS] non-blocking IO
In-Reply-To: <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
 <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
Message-ID: <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>

On Mon, 1 Jun 2020, Rob Pike wrote:

>  I’m not quite sure why the Research lineage did not include
>  non-blocking behaviour, especially in view of the man page comments.
>  Maybe it was seen as against the Unix philosophy, with select()
>  offering sufficient mechanism to avoid blocking (with open() the hard
>  corner case)?
> 
> That's it. Select was good enough for our purposes.

After being dragged through both Berserkley and SysVile, I never did get 
the hang of poll()/select() etc,,,

-- Dave

From Caipenghui_c at 163.com  Mon Jun  1 16:32:01 2020
From: Caipenghui_c at 163.com (Caipenghui)
Date: Mon, 01 Jun 2020 14:32:01 +0800
Subject: [TUHS] mh/hm, mmh
In-Reply-To: <e5f793d5-95b3-6848-8753-7fdb38be01d0@mhorton.net>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
 <bd93f90e-c613-3ebf-749e-3c4ed734d34c@mhorton.net>
 <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com>
 <e5f793d5-95b3-6848-8753-7fdb38be01d0@mhorton.net>
Message-ID: <36190BD8-0A00-4729-9B81-2FF441552CF0@163.com>

it's beautiful.

于 2020年6月1日 GMT+08:00 上午5:31:30, Mary Ann Horton <mah at mhorton.net> 写到:
>Wonderful! Thank you for the history, Eric!
>
>     Mary Ann
>
>On 5/31/20 11:56 AM, Eric Allman wrote:
>> Ah, time for some "Email at Berkeley" history....
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200601/4233cea8/attachment.htm>

From meillo at marmaro.de  Mon Jun  1 17:14:06 2020
From: meillo at marmaro.de (markus schnalke)
Date: Mon, 01 Jun 2020 09:14:06 +0200
Subject: [TUHS] mh/hm, mmh
In-Reply-To: <e5f793d5-95b3-6848-8753-7fdb38be01d0@mhorton.net>
References: <1jeHk5-5LM-00@marmaro.de>
 <CAC20D2Pm9_u8BiExe+ELjt0jb4qnB-d-bUFbNCvpXJnDma68yQ@mail.gmail.com>
 <1jfNb2-7JV-00@marmaro.de>
 <CAC20D2P6_knffBX66XMb-SZf+om1AOooN36xaea+M0=Tb5weqg@mail.gmail.com>
 <bd93f90e-c613-3ebf-749e-3c4ed734d34c@mhorton.net>
 <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com>
 <e5f793d5-95b3-6848-8753-7fdb38be01d0@mhorton.net>
Message-ID: <1jfee7-82Z-00@marmaro.de>

Thanks to everyone who joined in. That was more than I had
expected. :-)


meillo


[2020-05-31 14:31] Mary Ann Horton <mah at mhorton.net>
>
> Wonderful! Thank you for the history, Eric!
> 
>      Mary Ann
> 
> On 5/31/20 11:56 AM, Eric Allman wrote:
> > Ah, time for some "Email at Berkeley" history....
> >
> 

From paul at rileyriot.com  Mon Jun  1 20:18:41 2020
From: paul at rileyriot.com (Paul Riley)
Date: Mon, 1 Jun 2020 18:18:41 +0800
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
Message-ID: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>

Team Unix,

Is there a Windows or Linux utility to create a disk image in any of the
above formats, from a local folder tree?

Paul

*Paul Riley*

Mo: +86 186 8227 8332
Email: paul at rileyriot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200601/be9a2d27/attachment.htm>

From ron at ronnatalie.com  Mon Jun  1 23:01:13 2020
From: ron at ronnatalie.com (Ronald Natalie)
Date: Mon, 1 Jun 2020 09:01:13 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
Message-ID: <C3CC67A4-1A8F-4FE6-8D15-ED85DF875C24@ronnatalie.com>

Eh?   An RL is just a cartridge disk drive, not a format.    It would depend on what the operating system you are using wanted to put on it.
Similarly the RX is just a floppy drive.    In my memory, there wasn’t much UNIX use of these drives.   The main reason the driver existed on the
VAX unices was to write the Console Floppy Disk in the 780 (which was in RT-11 format).

What are you trying to do?

> On Jun 1, 2020, at 6:18 AM, Paul Riley <paul at rileyriot.com> wrote:
> 
> Team Unix,
> 
> Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree?
> 
> Paul
> 
> Paul Riley
> 
> Mo: +86 186 8227 8332
> Email: paul at rileyriot.com <mailto:paul at rileyriot.com>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200601/c337eb59/attachment.htm>

From clemc at ccc.com  Mon Jun  1 23:04:28 2020
From: clemc at ccc.com (Clem Cole)
Date: Mon, 1 Jun 2020 09:04:28 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
Message-ID: <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>

On Mon, Jun 1, 2020 at 6:19 AM Paul Riley <paul at rileyriot.com> wrote:

> Is there a Windows or Linux utility to create a disk image in any of the
> above formats, from a local folder tree?
>
What I think you are asking, is there a utility for a modern OS that will
walk a local folder tree on my OS and create a new file whose structure is
that of the file system for OS <insert yours here>.

The issue is not the device as much as the OS and disk file layout.    As
far as UNIX (or simh at the OS level) is concerned, the disk is just a
linear array of bytes, addressed by blocks.  The physical format is not
seen by UNIX.

There are numerious utilities, as well as 'foreign file systems' that are
available.   For instance, many Unix's can write RT-11 and MS-DOS format
with standard utilities.   It really depends the OS.  That said,
if the target OS is modern enough to support NFS or Samba, the easiest way
might be export the file system from local system, and then running a
simulated OS, 'mount' the file system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200601/f7cc83ae/attachment.htm>

From lm at mcvoy.com  Tue Jun  2 00:58:02 2020
From: lm at mcvoy.com (Larry McVoy)
Date: Mon, 1 Jun 2020 07:58:02 -0700
Subject: [TUHS] non-blocking IO
In-Reply-To: <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
 <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
 <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>
Message-ID: <20200601145801.GE22016@mcvoy.com>

On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote:
> On Mon, 1 Jun 2020, Rob Pike wrote:
> 
> > I???m not quite sure why the Research lineage did not include
> > non-blocking behaviour, especially in view of the man page comments.
> > Maybe it was seen as against the Unix philosophy, with select()
> > offering sufficient mechanism to avoid blocking (with open() the hard
> > corner case)?
> >
> >That's it. Select was good enough for our purposes.
> 
> After being dragged through both Berserkley and SysVile, I never did get the
> hang of poll()/select() etc,,,

I'm sure you could, select is super handy, think a network server like
apache.

From pnr at planet.nl  Tue Jun  2 01:04:22 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Mon, 1 Jun 2020 17:04:22 +0200
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
Message-ID: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl>

> Team Unix, Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? Paul *Paul Riley*

It seems you are asking for two tools in the BK-UNIX toolbox, fsutil and dskutil
https://github.com/sergev/bkunix/tree/master/fsutil
https://github.com/sergev/bkunix/tree/master/diskutil

The first generates a Unix 6th edition file system from a local directory tree. The result is a binary file with the 512 byte disk blocks stored in sequence. Maybe this is what SIMH needs, I’m not into the details of SIMH.

The second appears to be a tool to take the a file generated by the fsutil tool and split that into sectors and tracks. I’m not familiar with this tool, but it looks like you might need something similar (I assume that you have some way to hook up a 8” drive to your PC?). Sector interleaving may be an issue to look out for.

When preparing a LSX system disk, you will need to think carefully about the layout:

Presumably the disk works with 128 byte sectors and 4 sectors are grouped together to create a 512 byte unix block. Check out the disk driver code for details:
https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/decfd.c

Block 0 contains bootstrap code which is loaded/called from the monitor rom (or from a short code sequence keyed in on a “blinkenpanel” console).

The filesystem itself starts at block 1 and runs up to a block N (you need to tell fsutil how big the filesystem needs to be).

After block N up to the end of the disk is space to contain 2 swapped out programs plus 1 block for the return code of the third (default LSX has a maximum of 3 processes). You have to figure out how many 512 blocks are on your floppy and subtract out the swap space to arrive at a figure for N.

In the LSX source code ’N’ is known as the define SWPLO, see param.h for details:
https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/param.h

In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h

Be careful with size units in the source code. Often sizes are expressed in words (2 bytes). Where memory is concerned it is often expressed in ‘clicks’, 64 bytes.





From pnr at planet.nl  Tue Jun  2 01:18:44 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Mon, 1 Jun 2020 17:18:44 +0200
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
Message-ID: <C55CB0CD-5E36-4F7E-A4AE-CF6002C59F0B@planet.nl>

> 
> In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h

Oops: 77 tracks by 26 sectors of 128 bytes, makes 250.25KB, so it should work as configured.






From heinz at osta.com  Tue Jun  2 02:41:03 2020
From: heinz at osta.com (Heinz Lycklama)
Date: Mon, 1 Jun 2020 09:41:03 -0700
Subject: [TUHS] LSX on the PDP-11/03 (LSI-11)
In-Reply-To: <CAD05_j2WM0z=oEz-woQLrB+rDH2Q3StoPSAQVa4zCLu1HdP2qA@mail.gmail.com>
References: <CAD05_j2WM0z=oEz-woQLrB+rDH2Q3StoPSAQVa4zCLu1HdP2qA@mail.gmail.com>
Message-ID: <b55c8031-6b0f-aeba-d86e-26d7bcd4992c@osta.com>

Paul, I'm surprised that copies of the LSX system are still around.
I'm assuming that you acquired only the binary code, and not the
source code. The LSX system source code could be compiled
on an LSX system running on floppy disks. I see you are getting
your C compiler issues addressed by others already.

Heinz Lycklama
Author of LSX at Bell Labs

On 5/30/2020 11:26 PM, Paul Riley wrote:
> I've managed to acquire a PDP-11/03 with twin floppy drives (Sykes 
> Datatronics RX01 or RX02 equivalents, not sure yet which).
>
> I've stumbled across LSX, and I have it running on SimH. I'm quite 
> inexperienced with Unix, but it's something I want to learn well, 
> having brushed against it at university in the '80s, and having played 
> with Linux somewhat.
>
> I have some interest in Forth, but I don't like the block system of 
> early forths such as FigForth, and I plan to create a new Forth based 
> on FigForth, but supporting external source files.
>
> Anyway, I've tried compiling Hello World on LSX, and I get "1: 
> External definition syntax" error. Some help would be nice, but more 
> generally, is anyone on this list more than vaguely familiar with LSX, 
> or 6th Edition itself?
>
> void main () {
>         printf("Hello World!");
> }
>
> It seems that the 7th Edition was the beginning of the standard 
> library in C, and that this is missing in LSX. I'm not sure if printf 
> is an intrinsic function in (6th Edition) C, or if it's from a library.
>
> My questions are a bit random, but looking to converse with others 
> with LSX experience.
>
> Paul
>
> *Paul Riley*
>
> Email: paul at rileyriot.com <mailto:paul at rileyriot.com>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200601/fab85f4e/attachment.htm>

From heinz at osta.com  Tue Jun  2 02:58:47 2020
From: heinz at osta.com (Heinz Lycklama)
Date: Mon, 1 Jun 2020 09:58:47 -0700
Subject: [TUHS] non-blocking IO
In-Reply-To: <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
Message-ID: <6aaca54f-83da-515f-e5b0-e3baa4560c44@osta.com>

I did  add a few new features to LSX to deal with contiguous files
and to handle asynchronous read/write's for real time applications.
They are described in the LSX paper in the 1978 BSTJ on the
UNIX Time-Sharing System.

Heinz

On 5/31/2020 9:46 AM, Warner Losh wrote:
> Sorry to top post, but LSX or Miniunix had non blocking I/O as well. 
> It was in one of the documents that Clem scanned in the last year. It 
> specifically was an experiment into how to do it.
>
> Warner
>
> On Sun, May 31, 2020, 10:07 AM Clem Cole <clemc at ccc.com 
> <mailto:clemc at ccc.com>> wrote:
>
>
>
>     On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal <pnr at planet.nl
>     <mailto:pnr at planet.nl>> wrote:
>
>          This behaviour seems to have continued into SysVR1, I’m not
>         sure when EAGAIN came into use as a return value for this use
>         case in the SysV lineage. Maybe with SysVR3 networking?
>
>     Actually, I'm pretty sure that was a product of the POSIX
>     discussions.  BSD already had networking an EWOULDBLOCK.   We had
>     argued about EWOULDBLOCK a great deal, we also were arguing about
>     signal semantics. I've forgotten many of the details, Heinz may
>     remember more than I do.  EAGAIN was created as a compromise --
>     IIRC neither system had it yet.   SVR3 networking was where it
>     went into System V, although some of the AT&T representatives were
>     none too happy about it.
>
>
>         In the Research lineage, the above SysIII approach does not
>         seem to exist, although the V8 manual page for open() says
>         under BUGS "It should be possible [...] to optionally call
>         open without the possibility of hanging waiting for carrier on
>         communication lines.” In the same location for V10 it reads
>         "It should be possible to call open without waiting for
>         carrier on communication lines.”
>
>         The July 1981 design proposals for 4.2BSD note that SysIII
>         non-blocking files are a useful feature and should be included
>         in the new system. In Jan/Feb 1982 this appears to be coded
>         up, although not all affected files are under SCCS tracking at
>         that point in time. Non-blocking behaviour is changed from the
>         SysIII semantics, in that EWOULDBLOCK is returned instead of 0
>         when progress is not possible. The non-blocking behaviour is
>         extended beyond TTY’s and pipes to sockets, with additional
>         errors (such as EINPROGRESS). At this time EWOULDBLOCK is not
>         the same error number as EGAIN.
>
>     My memory is that Keith was the BSD (CSRG) person at the POSIX
>     meeting (he, Jim McGinness of DEC, and I created PAX at one point
>     as a compromise).   I wish I could remember all of the details,
>     but this was all argued at the POSIX meetings.
>
>     As I said before the folks from AT&T just wanted to take the SVID
>     and rubber stamp it at the specification.  Part of it the problem
>     was they wanted to be free to do what things/make choices that the
>     rest of us might or might not like (for instance, they did not
>     want the sockets interface).
>
>
>         It would seem that the differences between the BSD and SysV
>         lineages in this area persisted until around 2000 or so.
>
>     Yep - cause around then POSIX started to settle out and both
>     systems began to follow it.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200601/3f4bd010/attachment.htm>

From paul at rileyriot.com  Tue Jun  2 08:35:21 2020
From: paul at rileyriot.com (Paul Riley)
Date: Tue, 2 Jun 2020 06:35:21 +0800
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <C3CC67A4-1A8F-4FE6-8D15-ED85DF875C24@ronnatalie.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <C3CC67A4-1A8F-4FE6-8D15-ED85DF875C24@ronnatalie.com>
Message-ID: <CAD05_j0y1hLdgfNL3u+sGMfug7pKdbeAXn24_Nb1Sqef88pt4Q@mail.gmail.com>

Ron,

Trying to move a large number of source files from my Windows machine to
the LSX simulation (or V6 simulation). This is a Unix forum, so I didn’t
explicitly say in a Unix format, but that was implied, sorry.

Paul

On Mon, 1 Jun 2020 at 9:01 pm, Ronald Natalie <ron at ronnatalie.com> wrote:

> Eh?   An RL is just a cartridge disk drive, not a format.    It would
> depend on what the operating system you are using wanted to put on it.
> Similarly the RX is just a floppy drive.    In my memory, there wasn’t
> much UNIX use of these drives.   The main reason the driver existed on the
> VAX unices was to write the Console Floppy Disk in the 780 (which was in
> RT-11 format).
>
> What are you trying to do?
>
> On Jun 1, 2020, at 6:18 AM, Paul Riley <paul at rileyriot.com> wrote:
>
> Team Unix,
>
> Is there a Windows or Linux utility to create a disk image in any of the
> above formats, from a local folder tree?
>
> Paul
>
> *Paul Riley*
>
> Mo: +86 186 8227 8332
> Email: paul at rileyriot.com
>
>
> --
*Paul Riley*

Mo: +86 186 8227 8332
Email: paul at rileyriot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/aa70e4fd/attachment.htm>

From paul at rileyriot.com  Tue Jun  2 08:45:07 2020
From: paul at rileyriot.com (Paul Riley)
Date: Tue, 2 Jun 2020 06:45:07 +0800
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
Message-ID: <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>

Clem,

Thanks for your help. You’ve correctly interpreted my question.

Is the disk image independent from the disk hardware? I’d assume that
different disks may have different block sizes etc, so the disk type may be
important.

The target system is LSX, a cut-down version of V6 designed to run on the
LSI-11. There are very few system utilities in the standard build (no mount
for example). The second floppy is permanently mounted at boot time. I’m
interested in making source file floppies on my modern system to use on the
LSX, so I want to be able to create an image file from a source folder tree.

Paul

On Mon, 1 Jun 2020 at 9:05 pm, Clem Cole <clemc at ccc.com> wrote:

>
>
> On Mon, Jun 1, 2020 at 6:19 AM Paul Riley <paul at rileyriot.com> wrote:
>
>> Is there a Windows or Linux utility to create a disk image in any of the
>> above formats, from a local folder tree?
>>
> What I think you are asking, is there a utility for a modern OS that will
> walk a local folder tree on my OS and create a new file whose structure is
> that of the file system for OS <insert yours here>.
>
> The issue is not the device as much as the OS and disk file layout.    As
> far as UNIX (or simh at the OS level) is concerned, the disk is just a
> linear array of bytes, addressed by blocks.  The physical format is not
> seen by UNIX.
>
> There are numerious utilities, as well as 'foreign file systems' that are
> available.   For instance, many Unix's can write RT-11 and MS-DOS format
> with standard utilities.   It really depends the OS.  That said,
> if the target OS is modern enough to support NFS or Samba, the easiest way
> might be export the file system from local system, and then running a
> simulated OS, 'mount' the file system.
>
-- 
*Paul Riley*

Mo: +86 186 8227 8332
Email: paul at rileyriot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/370be371/attachment.htm>

From paul at rileyriot.com  Tue Jun  2 08:54:28 2020
From: paul at rileyriot.com (Paul Riley)
Date: Tue, 2 Jun 2020 06:54:28 +0800
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl>
References: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl>
Message-ID: <CAD05_j3UwJSWPWc_O=h=RATq1NTHq2_zPy86nk5=e4Z33XKXMw@mail.gmail.com>

Paul,

Many thanks for your detailed reply, it’s much appreciated.

I’ll start with the Bkunix utility and see if SimH can use the output, then
move on to your other suggestions if not.

Paul

On Mon, 1 Jun 2020 at 11:04 pm, Paul Ruizendaal <pnr at planet.nl> wrote:

> > Team Unix, Is there a Windows or Linux utility to create a disk image in
> any of the above formats, from a local folder tree? Paul *Paul Riley*
>
> It seems you are asking for two tools in the BK-UNIX toolbox, fsutil and
> dskutil
> https://github.com/sergev/bkunix/tree/master/fsutil
> https://github.com/sergev/bkunix/tree/master/diskutil
>
> The first generates a Unix 6th edition file system from a local directory
> tree. The result is a binary file with the 512 byte disk blocks stored in
> sequence. Maybe this is what SIMH needs, I’m not into the details of SIMH.
>
> The second appears to be a tool to take the a file generated by the fsutil
> tool and split that into sectors and tracks. I’m not familiar with this
> tool, but it looks like you might need something similar (I assume that you
> have some way to hook up a 8” drive to your PC?). Sector interleaving may
> be an issue to look out for.
>
> When preparing a LSX system disk, you will need to think carefully about
> the layout:
>
> Presumably the disk works with 128 byte sectors and 4 sectors are grouped
> together to create a 512 byte unix block. Check out the disk driver code
> for details:
> https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/decfd.c
>
> Block 0 contains bootstrap code which is loaded/called from the monitor
> rom (or from a short code sequence keyed in on a “blinkenpanel” console).
>
> The filesystem itself starts at block 1 and runs up to a block N (you need
> to tell fsutil how big the filesystem needs to be).
>
> After block N up to the end of the disk is space to contain 2 swapped out
> programs plus 1 block for the return code of the third (default LSX has a
> maximum of 3 processes). You have to figure out how many 512 blocks are on
> your floppy and subtract out the swap space to arrive at a figure for N.
>
> In the LSX source code ’N’ is known as the define SWPLO, see param.h for
> details:
> https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/param.h
>
> In this file, 99 blocks are reserved for swap, corresponding to processes
> with 24KB memory; total disk size is defined as 500 blocks, 250KB - note
> that this slightly exceeds the 241KB offered by the standard IBM 77 track
> 26 sector formatting for 8” disks. Probably you will need to tweak the
> values in param.h
>
> Be careful with size units in the source code. Often sizes are expressed
> in words (2 bytes). Where memory is concerned it is often expressed in
> ‘clicks’, 64 bytes.
>
>
>
>
> --
*Paul Riley*

Mo: +86 186 8227 8332
Email: paul at rileyriot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/14102858/attachment.htm>

From jnc at mercury.lcs.mit.edu  Tue Jun  2 09:17:42 2020
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Mon,  1 Jun 2020 19:17:42 -0400 (EDT)
Subject: [TUHS] non-blocking IO
Message-ID: <20200601231742.CBAB118C073@mercury.lcs.mit.edu>

    > From: Paul Ruizendaal

    > This time looking into non-blocking file access.  (... right now my
    > scope is 'communication files' (tty's, pipes, network connections).
    > ...
    > First appearance of non-blocking behaviour seems to have been with
    > Chesson's multiplexed files ... in 1979.

At around that point in time (I don't have the very _earliest_ code, to get an
exact date, but the oldest traces I see [in mapalloc(), below] are from
September '78), the CSR group at MIT-LCS (which were the people in LCS doing
networking) was doing a lot with asynchronous I/O (when you're working below
the reliable stream level, you can't just do a blocking 'read' for a packet;
it pretty much has to be asynchronous). I was working in Unix V6 - we were
building an experimental 1Mbit/second ring - and there was work in Multics as
well.

I don't think the wider Unix community heard about the Unix work, but our
group regularly filed updates on our work for the 'Internet Monthly Reports',
which was distributed to the whole TCP/IP experimental community. If you can
find an archive of early issues (I'm too lazy to go look for one), we should
be in there (although our report will alsocover the Multics TCP/IP work, and
maybe some other stuff too).


There were two main generations of code; I don't recall the second one well,
and I'm too lazy to go look, but I can tell you off the top of my head a bit
about how the first one worked. Open/read/write all looked standard to the
user in the process (the latter two were oriented to packets, a bit like raw
disks being blocks); multiple operations could be queued in each
direction. (There was only one user allowed at a time for the network device;
no input demultiplexing.)

Whenever an I/O operation completed, the process was sent a signal. Since the
read/write call had long since returned, it had to do a getty() to get info
about that operation - the size of the packet, error indications, etc.

One complication was that for a variety of reasons (we wanted to avoid having
to copy data, and the interface did not have packet buffers) we did DMA
directly to/from the user's memory; this meant the process has to be locked
in place while I/O was pending.

(I didn't realize it at the time, but we dodged a bullet there; a comment
in xalloc(), which I only absorbed recently, explains the problem. More
here:

  https://gunkies.org/wiki/UNIX_V6_internals#exec()_and_pure-text_images

if anyone wants the gory details.)


That all (the queing, signals for I/O completion, locking the process to a
fixed location in memory while it continued to run) etc all worked well, as I
recall (although I guess it couldn't do an sbrk() while locked), but one
complication was the UNIBUS map on the -11/70.

The DSSR/RTS group at LCS wanted to have a ring interface, but their machine
was a /70 (ours, the one the driver was initially done on/for, was a /40), so
with DMA we had to use the UNIBUS map.

The stock V6 code had mapalloc(), (and mapfree(), both called on all DMA
operations), but... it allocated the whole map to whatever I/O operation asked
for the map. Clearly, if you're about to start a network input operation, and
wait a packet to show up, you don't want the disk controller to have to sit
and wait for for a packet to show up so _it_ can have the map.

Luckily, mapalloc() was called with a pointer to the buffer header (which had
all the info about the xfer), so I added a 'ubmap' array, and called the
existing malloc() on it, to allocate only a big enough chunk of the UNIBUS map
for the I/O operation defined by the header. Since there was 248KB of map
space, and the largest single DMA transfer possible in V6 was about 64KB
(maybe a little more, for a max-sized process with its 'user' block), there
was never a problem with contention for the map, and we didn't have to touch
any of the other drivers at all.

That was dandy, and only a couple of lines of extra code, but I somehow made a
math error in my changes, and as I recall I had to debug it with a printf() in
mapalloc(). I was not popular that day! Luckily, the error was quickly
obvious, a fix was applied, and we were on our way.

	 Noel


From ron at ronnatalie.com  Tue Jun  2 09:59:20 2020
From: ron at ronnatalie.com (Ronald Natalie)
Date: Mon, 1 Jun 2020 19:59:20 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
Message-ID: <F8D91F81-90C1-4744-BAE2-B7780E75C50F@ronnatalie.com>

As far as the early UNIXs go, any disk is collection of 512-byte blocks.     The filesystems either the early (what I’ll call V6) and the later (V7) don’t differ much.
The primary difference is that the later V7 had 16-bit uids and a provision for larger file systems/sizes.    The V6 file system was limited to 2^24 blocks while V7
did 2^32.

The 512 block size corresponded to the native sector size of all the DEC hardware except the RX which I think only had 128-byte sectors.    But again, we didn’t
do much with that other than write the standalone console disks for the 780 (in RT format) and I also used it to make “unix” file system disks for the BRL “LOS”
(little operating system…no time for sharing, uniprocessor system) that ran our internet routers and the IO hardware on the HEP supercomputer.



From jnc at mercury.lcs.mit.edu  Tue Jun  2 10:08:45 2020
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Mon,  1 Jun 2020 20:08:45 -0400 (EDT)
Subject: [TUHS] non-blocking IO
Message-ID: <20200602000845.CF26E18C073@mercury.lcs.mit.edu>

    > when you're working below the reliable stream level, you can't just do a
    > blocking 'read' for a packet; it pretty much has to be asynchronous

Oh, you should look at the early BBN TCP for V6 Unix - they would have faced
the same issue, with their TCP process. They did have the capac() call (which
kind of alleviates the need for non-blocking I/O), but that may have only been
available for ports/pipes; I'm not sure if the ARPANET device supported it.

(With the NCP as well, that did some amount of demultiplexing in the kernel,
and probably had buffering there, so, if so, in theory capac() could have been
done there. Of course, with the ARPANET link being only 100Kbit/sec maximum -
although only to a host on the same IMP - the overhead of copying buffered
data made kernel buffering more 'affordable'.)

     Noel

From jnc at mercury.lcs.mit.edu  Tue Jun  2 10:14:46 2020
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Mon,  1 Jun 2020 20:14:46 -0400 (EDT)
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
Message-ID: <20200602001446.359F618C073@mercury.lcs.mit.edu>

    > From: Ronald Natalie

    > The V6 file system was limited to 2^24 blocks

No, 2^16; from filsys.h:

        int     s_fsize;        /* size in blocks of entire volume */

and of course on an -11 an int is 16 bits.

Maybe you're thinking of the file size, which was 2^24 bytes (max).

      Noel

From ron at ronnatalie.com  Tue Jun  2 11:26:56 2020
From: ron at ronnatalie.com (Ronald Natalie)
Date: Mon, 1 Jun 2020 21:26:56 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <20200602001446.359F618C073@mercury.lcs.mit.edu>
References: <20200602001446.359F618C073@mercury.lcs.mit.edu>
Message-ID: <8C214FC8-A7F3-4DC1-9600-DA3CE357F041@ronnatalie.com>

You are right.   16 bit block numbers 24 bit file size for V6.


> On Jun 1, 2020, at 8:14 PM, Noel Chiappa <jnc at mercury.lcs.mit.edu> wrote:
> 
>> From: Ronald Natalie
> 
>> The V6 file system was limited to 2^24 blocks
> 
> No, 2^16; from filsys.h:
> 
>        int     s_fsize;        /* size in blocks of entire volume */
> 
> and of course on an -11 an int is 16 bits.
> 
> Maybe you're thinking of the file size, which was 2^24 bytes (max).
> 
>      Noel


From paul at rileyriot.com  Tue Jun  2 14:57:54 2020
From: paul at rileyriot.com (Paul Riley)
Date: Tue, 2 Jun 2020 12:57:54 +0800
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <F8D91F81-90C1-4744-BAE2-B7780E75C50F@ronnatalie.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
 <F8D91F81-90C1-4744-BAE2-B7780E75C50F@ronnatalie.com>
Message-ID: <CAD05_j3smD4Z5T_uX15vfNxTJn2wEWxmgc_33vFPFrAtLg04TQ@mail.gmail.com>

Thanks Ron, and all others.

Theres plenty of grist for the mill, time for me to grind it.

Paul

*Paul Riley*

Mo: +86 186 8227 8332
Email: paul at rileyriot.com



On Tue, 2 Jun 2020 at 07:59, Ronald Natalie <ron at ronnatalie.com> wrote:

> As far as the early UNIXs go, any disk is collection of 512-byte blocks.
>    The filesystems either the early (what I’ll call V6) and the later (V7)
> don’t differ much.
> The primary difference is that the later V7 had 16-bit uids and a
> provision for larger file systems/sizes.    The V6 file system was limited
> to 2^24 blocks while V7
> did 2^32.
>
> The 512 block size corresponded to the native sector size of all the DEC
> hardware except the RX which I think only had 128-byte sectors.    But
> again, we didn’t
> do much with that other than write the standalone console disks for the
> 780 (in RT format) and I also used it to make “unix” file system disks for
> the BRL “LOS”
> (little operating system…no time for sharing, uniprocessor system) that
> ran our internet routers and the IO hardware on the HEP supercomputer.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/b50787de/attachment.htm>

From pnr at planet.nl  Tue Jun  2 18:22:12 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Tue, 2 Jun 2020 10:22:12 +0200
Subject: [TUHS] non-blocking IO
Message-ID: <46A39BDD-A854-40EF-B82A-176F1AB7B92E@planet.nl>

>     > when you're working below the reliable stream level, you can't just do a > blocking 'read' for a packet; it pretty much has to be asynchronous
> Oh, you should look at the early BBN TCP for V6 Unix - they would have faced the same issue, with their TCP process. They did have the capac() call (which kind of alleviates the need for non-blocking I/O), but that may have only been available for ports/pipes; I'm not sure if the ARPANET device supported it.

I did. There is capac() support also for the IMP interface:
https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-V6/dmr/imp11a.c
(see bottom two functions)

BBN took the same approach as Research: with capac() or select() one can prevent blocking on read() and write().


From pnr at planet.nl  Tue Jun  2 19:34:54 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Tue, 2 Jun 2020 11:34:54 +0200
Subject: [TUHS] non-blocking IO (morphing to "The serial I/O loop")
Message-ID: <FE160071-81A7-4FEA-9637-A8DCDF793145@planet.nl>


> On 5/31/2020 9:46 AM, Warner Losh wrote:
> > Sorry to top post, but LSX or Miniunix had non blocking I/O as well. 
> > It was in one of the documents that Clem scanned in the last year. It 
> > specifically was an experiment into how to do it.
> >
> > Warner

> I did  add a few new features to LSX to deal with contiguous files
> and to handle asynchronous read/write's for real time applications.
> They are described in the LSX paper in the 1978 BSTJ on the
> UNIX Time-Sharing System.
> 
> Heinz

Thanks for highlighting this!

The realm here is async I/O to disk, my original scope was limited to “communication” files (tty’s, pipes, network connections). Still, I find it an interesting topic.

For others, the paper that you refer to can also be found here:
https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/UNIX_on_a_Microprocessor_19780322.pdf
If I read correctly, the async functionality was available only in the stand alone program version of LSX. Is that correct? In any case, the source code appears lost.

From another paper in that set, I get the impression that the async functionality for LSX builds on earlier work for a very early version of Unix:
https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-74-1352-1_Implementation_of_Large_Contiguous_Files_and_Asynchronous_IO_in_UNIX_19740104.pdf

- - -

When reading through the papers in that TUHS directory, something else caught my attention: early networking at the labs. For a while I have been puzzled by the “serial I/O loop” in use at the labs in the early seventies. Last Fall I found some 1970/1971 BSTJ papers about it, but there it stopped.

I see in the memo on Glance (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-3_GLANCE_Terminals_on_UNIX_Time-Sharing_19750303.pdf) that D.R. Weller continued to work on it up to 1973 at least and that it was integrated with Unix in some way. Is that correct? Did the two memo’s referred to (MM 70-1384-1 and TM 73-1356-8) survive?

Then the memo on satellite processors is very interesting (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-78-3114-2_A_Minicomputer_Satellite_Processor_System_19780322.pdf). This appears to show the serial I/O loop in use as late as 1978, with a very intriguing use case involving system call forwarding over the network.

Can you tell me more about the serial I/O loop and its use cases?





From clemc at ccc.com  Tue Jun  2 23:31:28 2020
From: clemc at ccc.com (Clem Cole)
Date: Tue, 2 Jun 2020 09:31:28 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
Message-ID: <CAC20D2OqxirooCOqVj_UKHoPEh97eJtvzja3hdV=gJ6OJcPVRw@mail.gmail.com>

Frankly, using a common tape format, and then a raw disk as the 'tape'
tends to be easy.  For instance,  if it's an older system like v6, there is
a binary for v6tar kicking around that was created as part of the v7
conversion utilities (or you can build it yourself with a little work - the
issue a few changes in some headers).   Put that binary on your v6 system.
 Then create a tar image on your modern system.   Tar uses a threaded ASCII
header and actually had a bug in it, that is exploited as an extension
mechanism and became a feature (I'll explain off line if need be).  So
modern tar's will produce a checksum that the original's will correctly
accept.  Note the modern tar can create >>file types<< that the old tar
will not understand, but it will just skip them.

Going the back works too, but is limited by the original's handling of
things like directories.  It is generally not a problem.

So, the result is that you can attach that 'tarball' as a raw disk on simh
and then read it with v6tar.

Another possible way to go it to try to get stp(1) to compile on the more
modern system [it's on the Harvard tape IIRC -- it was the first version of
tp in C -- earlier versions were in PDP-11 assembler).  But ...  since that
was written with the a pre-Typesetter C compiler, and has a PDP-11 binary
format knowledge in it and I think used Lesk's portable I/O package, so it
might be a little more difficult to get running on a more modern C.

On Mon, Jun 1, 2020 at 6:45 PM Paul Riley <paul at rileyriot.com> wrote:

> Clem,
>
> Thanks for your help. You’ve correctly interpreted my question.
>
> Is the disk image independent from the disk hardware? I’d assume that
> different disks may have different block sizes etc, so the disk type may be
> important.
>
> The target system is LSX, a cut-down version of V6 designed to run on the
> LSI-11. There are very few system utilities in the standard build (no mount
> for example). The second floppy is permanently mounted at boot time. I’m
> interested in making source file floppies on my modern system to use on the
> LSX, so I want to be able to create an image file from a source folder tree.
>
> Paul
>
> On Mon, 1 Jun 2020 at 9:05 pm, Clem Cole <clemc at ccc.com> wrote:
>
>>
>>
>> On Mon, Jun 1, 2020 at 6:19 AM Paul Riley <paul at rileyriot.com> wrote:
>>
>>> Is there a Windows or Linux utility to create a disk image in any of the
>>> above formats, from a local folder tree?
>>>
>> What I think you are asking, is there a utility for a modern OS that will
>> walk a local folder tree on my OS and create a new file whose structure is
>> that of the file system for OS <insert yours here>.
>>
>> The issue is not the device as much as the OS and disk file layout.    As
>> far as UNIX (or simh at the OS level) is concerned, the disk is just a
>> linear array of bytes, addressed by blocks.  The physical format is not
>> seen by UNIX.
>>
>> There are numerious utilities, as well as 'foreign file systems' that are
>> available.   For instance, many Unix's can write RT-11 and MS-DOS format
>> with standard utilities.   It really depends the OS.  That said,
>> if the target OS is modern enough to support NFS or Samba, the easiest
>> way might be export the file system from local system, and then running a
>> simulated OS, 'mount' the file system.
>>
> --
> *Paul Riley*
>
> Mo: +86 186 8227 8332
> Email: paul at rileyriot.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/9a2c6e4f/attachment.htm>

From clemc at ccc.com  Tue Jun  2 23:52:44 2020
From: clemc at ccc.com (Clem Cole)
Date: Tue, 2 Jun 2020 09:52:44 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
Message-ID: <CAC20D2MdxxoNONNbGvQABQfSktLEuoWtfWGS+ZfM35-JK2JSsg@mail.gmail.com>

On Mon, Jun 1, 2020 at 6:45 PM Paul Riley <paul at rileyriot.com> wrote:

> Is the disk image independent from the disk hardware? I’d assume that
> different disks may have different block sizes etc, so the disk type may be
> important.
>
A reasonable thought, but no, other than efficiency.  It's just an array of
bytes exposed to the OS.

The good news is that all the DEC systems used 512 byte fixed blocks
(Nothing like the funky Prime, Apollo or Alto tricks of putting some of the
file system metadata into the physical disk format -- if you ever wondered
why some of the original SCSI controllers supported a 1072 byte block size
- it was for Domain/Aegis - 1024 block + 48 byte meta data].



>
> The target system is LSX, a cut-down version of V6 designed to run on the
> LSI-11.
>
Everything should be able to be simulated using is 512-byte blocks.




> There are very few system utilities in the standard build (no mount for
> example). The second floppy is permanently mounted at boot time. I’m
> interested in making source file floppies on my modern system to use on the
> LSX, so I want to be able to create an image file from a source folder tree.
>

You'll have to check the docs, but I thought Heinz supported the RK05
[RKV11 controller].  That will give you 2.5M of 'raw' disk blocks, instead
of floppy.  Open it raw, and use it like mag tape.   If you don't have a
coming format, you can create something and then use dd(1) on the LSI-11
from the RKV11 -- a little clumsy but as a bootstrap, it should work fine.
The RL11 and RK711 controllers (RL01/02, RK07 disk) came later to the
PDP-11 (post 11/34 and 11/60). I think there was an RLV1[12] at some point,
but I don't think DEC even made an RKV71[12] -- however, I'm not up on
late-generation PDP-11 lore and they would have been much later than LSX so
driver support is doubtful.

Good luck,
Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/1bfd6256/attachment.htm>

From pnr at planet.nl  Wed Jun  3 00:19:26 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Tue, 2 Jun 2020 16:19:26 +0200
Subject: [TUHS] non-blocking IO
Message-ID: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>

> At around that point in time (I don't have the very _earliest_ code, to get an exact date, but the oldest traces I see [in mapalloc(), below] are from September '78), the CSR group at MIT-LCS (which were the people in LCS doing networking) was doing a lot with asynchronous I/O (when you're working below the reliable stream level, you can't just do a blocking 'read' for a packet; it pretty much has to be asynchronous). I was working in Unix V6 - we were building an experimental 1Mbit/second ring - and there was work in Multics as well.

> I don't think the wider Unix community heard about the Unix work, but our group regularly filed updates on our work for the 'Internet Monthly Reports', which was distributed to the whole TCP/IP experimental community. If you can find an archive of early issues (I'm too lazy to go look for one), we should be in there (although our report will alsocover the Multics TCP/IP work, and maybe some other stuff too).

Sounds very interesting!

Looked around a bit, but I did not find a source for the “Internet Monthly Reports” for the late 70’s (rfc-editor.org/museum/ has them for the 1990’s).

In the 1970’s era, it seems that NCP Unix went in another direction, using newly built message and event facilities to prevent blocking. This is described in "CAC Technical Memorandum No. 84, Illinois Inter-Process Communication Facility for Unix.” - but that document appears lost as well.

Ah, well, topics for another day.

From ron at ronnatalie.com  Wed Jun  3 01:44:00 2020
From: ron at ronnatalie.com (Ronald Natalie)
Date: Tue, 2 Jun 2020 11:44:00 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CAC20D2MdxxoNONNbGvQABQfSktLEuoWtfWGS+ZfM35-JK2JSsg@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
 <CAC20D2MdxxoNONNbGvQABQfSktLEuoWtfWGS+ZfM35-JK2JSsg@mail.gmail.com>
Message-ID: <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>



> On Jun 2, 2020, at 9:52 AM, Clem Cole <clemc at ccc.com> wrote:
> 
> 
> The good news is that all the DEC systems used 512 byte fixed blocks much later than LSX so driver support is doubtful. 
> 

Except on the RX’s.   The RX01 was 128 byte sectors and the later ones 256 and a odd interleaving strategy.    However, the boot block and the rest of the file systems (both UNIX and RT at least)
just aggregated the smaller (logical) sectors together to make a 512 byte one.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/103fa737/attachment.htm>

From imp at bsdimp.com  Wed Jun  3 02:26:54 2020
From: imp at bsdimp.com (Warner Losh)
Date: Tue, 2 Jun 2020 10:26:54 -0600
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
 <CAC20D2MdxxoNONNbGvQABQfSktLEuoWtfWGS+ZfM35-JK2JSsg@mail.gmail.com>
 <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>
Message-ID: <CANCZdfr=RWfK7W1BA-rarNpzDSVePJqujP4PW+HZY1H8t-hnrw@mail.gmail.com>

On Tue, Jun 2, 2020, 9:45 AM Ronald Natalie <ron at ronnatalie.com> wrote:

>
>
> On Jun 2, 2020, at 9:52 AM, Clem Cole <clemc at ccc.com> wrote:
>
>
> The good news is that all the DEC systems used 512 byte fixed blocks much
> later than LSX so driver support is doubtful.
>
>
> Except on the RX’s.   The RX01 was 128 byte sectors and the later ones 256
> and a odd interleaving strategy.    However, the boot block and the rest of
> the file systems (both UNIX and RT at least)
> just aggregated the smaller (logical) sectors together to make a 512 byte
> one.
>

RX50 also had weird interleaving... but it was quite the odd duck.

Warner

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/170aa748/attachment.htm>

From paul.winalski at gmail.com  Wed Jun  3 03:45:55 2020
From: paul.winalski at gmail.com (Paul Winalski)
Date: Tue, 2 Jun 2020 13:45:55 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
Message-ID: <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>

The operating systems that I cut my teeth on (OS/360, DOS/360,
VAX/VMS) all had basic I/O system calls that were non-blocking.
Blocking I/O calls were all built on top of that framework.  I thus
found it curious that Unix took the opposite tack, and non-blocking
I/O was an afterthought.

So I'm curious as to what the rationale was for Unix to have been
designed with basic I/O being blocking rather than asynchronous.
Especially that non-blocking I/O primitives were the norm for OSes in
those days.

-Paul W.

From arnold at skeeve.com  Wed Jun  3 03:59:05 2020
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Tue, 02 Jun 2020 11:59:05 -0600
Subject: [TUHS] non-blocking IO
In-Reply-To: <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
Message-ID: <202006021759.052Hx5Et022619@freefriends.org>

Paul Winalski <paul.winalski at gmail.com> wrote:

> So I'm curious as to what the rationale was for Unix to have been
> designed with basic I/O being blocking rather than asynchronous.

I don't doubt that it was "simplify, simplify, simplify". Async I/O
is much messier than Unix's read/write model. The Unix model was
simpler to design, simpler to code, simpler to get right, and undoubtedly
took much less OS code than an async model would have; on the PDP-11
that would have mattered.

Also, the early Unixs were on smaller -11s, not the /45 or /70 with
split I&D space and the ability to address lost more RAM.

My guess, anyway. :-)

Arnold

From clemc at ccc.com  Wed Jun  3 04:08:36 2020
From: clemc at ccc.com (Clem Cole)
Date: Tue, 2 Jun 2020 14:08:36 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
 <CAC20D2MdxxoNONNbGvQABQfSktLEuoWtfWGS+ZfM35-JK2JSsg@mail.gmail.com>
 <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>
Message-ID: <CAC20D2MkFcykqhA=K+Wr7Wkwg4wwNg=UsMSamGgvAn4_L4z+Yg@mail.gmail.com>

On Tue, Jun 2, 2020 at 11:44 AM Ronald Natalie <ron at ronnatalie.com> wrote:

> Except on the RX’s.   The RX01 was 128 byte sectors and the later ones 256
> and a odd interleaving strategy.    However, the boot block and the rest of
> the file systems (both UNIX and RT at least) just aggregated the smaller
> (logical) sectors together to make a 512 byte one.
>
Indeed, I was less than precise -- the key is that the SW treats them
everything as being in 512 byte 'hunks' no matter the underlying physical
formatting/interleaving actually is.

The point is that if you Paul is trying to move data from the two systems,
just treat the device as a serial byte stream of 512 byte blocks on both
sides and you you be fine.   That means if using dd, add conv=sync to fill
any write buffer to a complete multiple of 512.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/182a9323/attachment.htm>

From crossd at gmail.com  Wed Jun  3 04:23:59 2020
From: crossd at gmail.com (Dan Cross)
Date: Tue, 2 Jun 2020 14:23:59 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
Message-ID: <CAEoi9W7Y47oHyWzys6yWUnFk7Z-mRpMd13qz9zmOb-TGPTnE6w@mail.gmail.com>

On Tue, Jun 2, 2020 at 1:47 PM Paul Winalski <paul.winalski at gmail.com>
wrote:

> The operating systems that I cut my teeth on (OS/360, DOS/360,
> VAX/VMS) all had basic I/O system calls that were non-blocking.
> Blocking I/O calls were all built on top of that framework.  I thus
> found it curious that Unix took the opposite tack, and non-blocking
> I/O was an afterthought.
>
> So I'm curious as to what the rationale was for Unix to have been
> designed with basic I/O being blocking rather than asynchronous.
> Especially that non-blocking I/O primitives were the norm for OSes in
> those days.


Doug addressed this, albeit in an oblique manner, on this list back in
2015: https://minnie.tuhs.org/pipermail/tuhs/2015-September/007509.html

Quoting him:

"""
Unix was what the authors wanted for a productive computing environment,
not a bag of everything they thought somebody somewhere might want.
One objective, perhaps subliminal originally, was to make program
behavior easy to reason about. Thus pipes were accepted into research
Unix, but more general (and unruly) IPC mechanisms such as messages
and events never were.

The infrastructure had to be asynchronous. The whole point was to
surmount that difficult model and keep everyday programming simple.
User visibility of asynchrony was held to a minimum: fork(), signal(),
wait(). Signal() was there first and foremost to support SIGKILL; it
did not purport to provide a sound basis for asynchronous IPC.
The complexity of sigaction() is evidence that asynchrony remains
untamed 40 years on.
"""

My response at the time was to question whether asynchrony itself remains
untamed, as Doug put it, or if rather it has proved difficult to retrofit
asynchrony onto a system designed around fundamentally synchronous
primitives?

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/1b6fd400/attachment.htm>

From paul.winalski at gmail.com  Wed Jun  3 04:53:59 2020
From: paul.winalski at gmail.com (Paul Winalski)
Date: Tue, 2 Jun 2020 14:53:59 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <202006021759.052Hx5Et022619@freefriends.org>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
Message-ID: <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>

On 6/2/20, arnold at skeeve.com <arnold at skeeve.com> wrote:
> Paul Winalski <paul.winalski at gmail.com> wrote:
>
>> So I'm curious as to what the rationale was for Unix to have been
>> designed with basic I/O being blocking rather than asynchronous.
>
> Also, the early Unixs were on smaller -11s, not the /45 or /70 with
> split I&D space and the ability to address lost more RAM.

I first encountered DOS/360 on a System/360 model 25 with 48K of
memory.  This was a one-job-at-a-time batch system, but the I/O
primitive (EXCP--execute channel program) was asynchronous.  So I
don't think the small memory rationale really applies.

-Paul W.

From paul.winalski at gmail.com  Wed Jun  3 04:56:59 2020
From: paul.winalski at gmail.com (Paul Winalski)
Date: Tue, 2 Jun 2020 14:56:59 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <CAEoi9W7Y47oHyWzys6yWUnFk7Z-mRpMd13qz9zmOb-TGPTnE6w@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <CAEoi9W7Y47oHyWzys6yWUnFk7Z-mRpMd13qz9zmOb-TGPTnE6w@mail.gmail.com>
Message-ID: <CABH=_VSWNR1Zw-W=ZSJ9OO3E9e5dqtr_E=ohCsvqW-=C7zFebQ@mail.gmail.com>

On 6/2/20, Dan Cross <crossd at gmail.com> wrote:
>
> My response at the time was to question whether asynchrony itself remains
> untamed, as Doug put it, or if rather it has proved difficult to retrofit
> asynchrony onto a system designed around fundamentally synchronous
> primitives?

I think that's a very good question.  It's analogous to
record-oriented I/O vs. byte stream I/O.  It's easy to build
record-oriented I/O on top of a byte stream, but it's a real bear to
do it the other way around.  Similarly, it's easy to build synchronous
I/O on top of asynchronous I/O but the reverse ends up looking
contrived.

-Paul W.

From paul.winalski at gmail.com  Wed Jun  3 04:59:37 2020
From: paul.winalski at gmail.com (Paul Winalski)
Date: Tue, 2 Jun 2020 14:59:37 -0400
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
 <CAC20D2MdxxoNONNbGvQABQfSktLEuoWtfWGS+ZfM35-JK2JSsg@mail.gmail.com>
 <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>
Message-ID: <CABH=_VQnSnRASpeX84fwAosGyYi=zzxc4_o8KD9JYNoFWekCow@mail.gmail.com>

On 6/2/20, Ronald Natalie <ron at ronnatalie.com> wrote:
>
> Except on the RX’s.   The RX01 was 128 byte sectors and the later ones 256
> and a odd interleaving strategy.    However, the boot block and the rest of
> the file systems (both UNIX and RT at least)
> just aggregated the smaller (logical) sectors together to make a 512 byte
> one.

At least they all supported fixed-length sectors.  Was there ever a
Unix implementation that supported IBM-style CKD disk devices?

-Paul W.

From athornton at gmail.com  Wed Jun  3 05:08:23 2020
From: athornton at gmail.com (Adam Thornton)
Date: Tue, 2 Jun 2020 12:08:23 -0700
Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator
In-Reply-To: <CABH=_VQnSnRASpeX84fwAosGyYi=zzxc4_o8KD9JYNoFWekCow@mail.gmail.com>
References: <CAD05_j2At32G7kEF2CuTXBrw-ZUpjTUc_QP=YMLRa4OXRYRz7w@mail.gmail.com>
 <CAC20D2PpiYOHW8m=Pg0LgUifEV1iZ46z4ZUVrqAE8nFBqoMAqQ@mail.gmail.com>
 <CAD05_j2nL6vHY=ZdnSm5_pQ1_i4SH7xP9imMoX5jhx_Y3wYi0w@mail.gmail.com>
 <CAC20D2MdxxoNONNbGvQABQfSktLEuoWtfWGS+ZfM35-JK2JSsg@mail.gmail.com>
 <D5A39465-802B-4AE2-92DE-84D4BE27CB61@ronnatalie.com>
 <CABH=_VQnSnRASpeX84fwAosGyYi=zzxc4_o8KD9JYNoFWekCow@mail.gmail.com>
Message-ID: <3A17BE73-777A-4CF2-8D55-A5C802FC767D@gmail.com>



> On Jun 2, 2020, at 11:59 AM, Paul Winalski <paul.winalski at gmail.com> wrote:
> 
> On 6/2/20, Ronald Natalie <ron at ronnatalie.com> wrote:
>> 
>> Except on the RX’s.   The RX01 was 128 byte sectors and the later ones 256
>> and a odd interleaving strategy.    However, the boot block and the rest of
>> the file systems (both UNIX and RT at least)
>> just aggregated the smaller (logical) sectors together to make a 512 byte
>> one.
> 
> At least they all supported fixed-length sectors.  Was there ever a
> Unix implementation that supported IBM-style CKD disk devices?


If you count Linux, sure, Linux for the S/390 and zSeries.

If you don’t…well, for one brief shining moment before IBM squeezed too hard on the price and Oracle bought Sun instead, there was an OpenSolaris port to the zSeries, and it did support 3390 type devices.

I may have had something to do with that latter device driver.

Adam

From clemc at ccc.com  Wed Jun  3 05:18:16 2020
From: clemc at ccc.com (Clem Cole)
Date: Tue, 2 Jun 2020 15:18:16 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
Message-ID: <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>

On Tue, Jun 2, 2020 at 2:54 PM Paul Winalski <paul.winalski at gmail.com>
wrote:

> I first encountered DOS/360 on a System/360 model 25 with 48K of
> memory.  This was a one-job-at-a-time batch system, but the I/O
> primitive (EXCP--execute channel program) was asynchronous.  So I
> don't think the small memory rationale really applies.
>
Hrrmpt... it was single task.   Being asynchronous in the I/O and allowed
process asynchrony takes a lot more housekeeping.  Paul you know I agree
with you, it was always an issue with UNIX IMO.  The problem is how to do
it differently.

At Masscomp RRU, our solution was not to try to 'fix it' as much as add a
new scheme beside the synchronous one.  We added a general AST's mechanism
that anyone could use (very much like RSX and VMS in semantics), but left
signals alone.  We added new async calls, which were just implemented via
ioctl's (the universal hack) for the specific HW that supported it.   In
retrospect, that was an error, it should have been aread(2)/awrite(2) and
then added the completion routine/call back as the 4th & 5th parameters.
 Since Stellix was not Real-Time, we left them out.  [ tjt and I have
argued about that for years ].

So back to Doug/Dan's answer -- I think for a small system, like the
original PDP-7 and the PDP-11/20 putting the effort to make it multiprocess
and leaving the I/O synchronous definitely made it easier.   Particularly,
since what was created with the signal semantics.   To me, UNIX got way
more right than wrong.   I suspect if signals had been more like AST's, the
idea of being async might have gone farther.  But to use Dennis's like
about C being quirky, signals are very quirky.  wnj tried to 'fix' them and
frankly it just made matters worse.  And IMO, signaction(3) as Doug says,
is a nightmare.

I've generally been a fan of introducing a new idea separately as a
'stronger strain' and then seeing if people likely it.

FWIW: A couple of us did try to get AST's into the POSIX.4 (they were there
in a draft), but ultimately *.4 was rejected as 'not UNIX.'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/e230f73a/attachment-0001.htm>

From clemc at ccc.com  Wed Jun  3 05:23:20 2020
From: clemc at ccc.com (Clem Cole)
Date: Tue, 2 Jun 2020 15:23:20 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <CABH=_VSWNR1Zw-W=ZSJ9OO3E9e5dqtr_E=ohCsvqW-=C7zFebQ@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <CAEoi9W7Y47oHyWzys6yWUnFk7Z-mRpMd13qz9zmOb-TGPTnE6w@mail.gmail.com>
 <CABH=_VSWNR1Zw-W=ZSJ9OO3E9e5dqtr_E=ohCsvqW-=C7zFebQ@mail.gmail.com>
Message-ID: <CAC20D2Mttmiz1PDgOFYwF5AepZe5-p8myScsqBVSWfvCoY5oiQ@mail.gmail.com>

On Tue, Jun 2, 2020 at 2:58 PM Paul Winalski <paul.winalski at gmail.com>
wrote:

> I think that's a very good question.  It's analogous to
> record-oriented I/O vs. byte stream I/O.  It's easy to build
> record-oriented I/O on top of a byte stream, but it's a real bear to
> do it the other way around.  Similarly, it's easy to build synchronous
> I/O on top of asynchronous I/O but the reverse ends up looking contrived.
>
Which was exactly the point I tried to make in the POSIX.4 discussions, but
it does take more work in the basic housekeeping and you need a way to
handle events and completions that are priority based, queued, and a few
other details.   As Doug said, they stayed away from some features (like
messaging).   async I/O was one of them.

But as I said, Ken, Dennis and the rest of the crew did an amazing job with
very little.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/abdab5d3/attachment.htm>

From jnc at mercury.lcs.mit.edu  Wed Jun  3 06:13:34 2020
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Tue,  2 Jun 2020 16:13:34 -0400 (EDT)
Subject: [TUHS] non-blocking IO
Message-ID: <20200602201334.95D9718C079@mercury.lcs.mit.edu>

    > From: Paul Winalski

    > I'm curious as to what the rationale was for Unix to have been designed
    > with basic I/O being blocking rather than asynchronous.

It's a combination of two factors, I reckon. One, which is better depends a
lot on the type of thing you're trying to do. For many typical thing (e.g.
'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is
more complicated, and Unix was (well, back then at least) all about getting
the most bang for the least bucks.

More complicated things do sometimes benefit from asynchronous I/O, but
complicated things weren't Unix's 'target market'. E.g. even though pipes
post-date the I/O decision, they too are a better match to blocking I/O.


    > From: Arnold Skeeve

    > the early Unixs were on smaller -11s, not the /45 or /70 with split I&D
    > space and the ability to address lost more RAM.

Ahem. Lots more _core_. People keeep forgetting that we're looking at
decicions made at a time when each bit in main memory was stored in a
physically separate storage device, and having tons of memory was a dream of
the future.

E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total!
And that had to hold the resident OS, plus the application! It's no
surprise that Unix was so focused on small size - and as a corollary, on
high bang/buck ratio.

But even in his age of lighting one's cigars with gigabytes of main memory
(literally), small is still beautiful, because it's easier to understand, and
complexity is bad. So it's too bad Unix has lost that extreme parsimony.


    > From: Dan Cross 

    > question whether asynchrony itself remains untamed, as Doug put it, or
    > if rather it has proved difficult to retrofit asynchrony onto a system
    > designed around fundamentally synchronous primitives?

I'm not sure it's 'either or'; I reckon they are both true.


	Noel

From clemc at ccc.com  Wed Jun  3 06:43:25 2020
From: clemc at ccc.com (Clem Cole)
Date: Tue, 2 Jun 2020 16:43:25 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <20200602201334.95D9718C079@mercury.lcs.mit.edu>
References: <20200602201334.95D9718C079@mercury.lcs.mit.edu>
Message-ID: <CAC20D2PRThvBqMz_ApiQk1f36F1Yz4VSqf_BB1dnuC9HNxfOvQ@mail.gmail.com>

On Tue, Jun 2, 2020 at 4:14 PM Noel Chiappa <jnc at mercury.lcs.mit.edu> wrote:

> Ahem. Lots more _core_. People keeep forgetting that we're looking at
> decicions made at a time when each bit in main memory was stored in a
> physically separate storage device, and having tons of memory was a dream
> of
> the future.
>
Yeah -- that is something that forgotten.  There's a kit/hackday project to
make 32-byte core for an Arduino I did with some of my boy scouts doing
electronic MB a while back just to try to give them a feel what a 'bit'
was.    Similarly, there was a update of in late 1960's children's book
originally called 'A Million' it's now called: A Million Dots
<https://www.amazon.com/Million-Dots-Andrew-Clements/dp/0689858248/ref=sr_1_1?crid=2AX8H8L2EM0HL&dchild=1&keywords=a+million+dots+by+andrew+clements&qid=1591129965&sprefix=a+million+dots%2Caps%2C155&sr=8-1>
Each page has 10K dots.  The idea is to help young readers get a real feel
for what 'a million' means visually.


>
> E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total!
> And that had to hold the resident OS, plus the application! It's no
> surprise that Unix was so focused on small size - and as a corollary, on
> high bang/buck ratio.'

Amen -- I ran an 11/34 with 64K under V6 for about 3-6 months while we were
awaiting the 256K memory upgrade.


>
>
> But even in his age of lighting one's cigars with gigabytes of main memory
> (literally), small is still beautiful, because it's easier to understand,
> and
> complexity is bad. So it's too bad Unix has lost that extreme parsimony.
>
Yep -- I think we were discussing this last week WRT to cat -v/fmt et al.

I fear some people confuse 'progress' with 'feature creep.'   Just because
we can do something, does not mean we should.

As I said, I'm a real fan of async I/O and like Paul, feel that it is a
'better' primitive.  But I fully understand and accept, that given the
tradeoffs of the time, UNIX did really well and I much prefer what we got
than the alternative.  I'm happy we ended up with simply and just works.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/f7fe0efc/attachment.htm>

From stewart at serissa.com  Wed Jun  3 07:15:21 2020
From: stewart at serissa.com (Lawrence Stewart)
Date: Tue, 2 Jun 2020 17:15:21 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
Message-ID: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>

I remember working on getting Arpanet access on an 11/34 running V7 around 1978 or 1979.  (SU-ISL). We used an 11/23 as a front end to run NCP, using a variation of Rand’s code.  I wrote some sort of bisync driver for packet communications between the /23 and the /34, and I think added an IOCTL or some hack to ask if there was a message ready.  So a polling variety of non-blocking I/O :)

Meanwhile, on the Alto, surely an underpowered machine, the style for UI programming and asynch I/O was mostly event driven, much like libevent in more recent years.  I found that very easy to understand.

The main trajectory of PARC was lightweight threads calling synchronous I/O which really has to be counted as a Bad Idea not yet fully stamped out.

I’ve always thought select(2) was a pretty good thing.  In 1995 at Open Market we had a single-process web server that had no difficulties running 1000 to 1200 connections.  I think that was BSD, and later OSF-1.

-L


From rdm at cfcl.com  Wed Jun  3 08:14:10 2020
From: rdm at cfcl.com (Rich Morin)
Date: Tue, 2 Jun 2020 15:14:10 -0700
Subject: [TUHS] non-blocking IO
In-Reply-To: <CAC20D2PRThvBqMz_ApiQk1f36F1Yz4VSqf_BB1dnuC9HNxfOvQ@mail.gmail.com>
References: <20200602201334.95D9718C079@mercury.lcs.mit.edu>
 <CAC20D2PRThvBqMz_ApiQk1f36F1Yz4VSqf_BB1dnuC9HNxfOvQ@mail.gmail.com>
Message-ID: <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com>

Case study (read, war story :-) of non-blocking I/O...

Back in the mid-70's, I was doing scientific support programming for an X-Ray sky survey,
using Fortran-63 under DRUM SCOPE on a CDC 3800 system which NRL's Space Sciences Division
had (sigh) acquired and cobbled together from the US Government "excess property" list.

Most of the time, our satellite was used in "scan mode", spinning on an axis which pointed
at the Sun.  This let us collect data on a two degree wide strip of sky.  So, every six
months we scanned the entire X-ray sky.  The track was basically a helix, bent to match the
shape of the Earth's orbit.  Every so often, in order to look at a particularly interesting
area, the spinning would be stopped, so the instrument could be put into "point mode".

The challenge was to "bin" this data for further analysis.  We did this by loading sets of
7-track tapes onto a CDC 813 drive, then dumping it out into 36 2x10 degree bins.  This put
a quarter of the data onto a set of tapes.  Rinse, repeat for the other 3/4 of the data
(after skipping past the already-written data).  The result was 36 tapes, each holding an
8x10 degree bin (plus one extra tape for any point mode data). 

IIRC, we had five tape drives; my challenge was to keep them all as busy as possible, so as
to dump the data set expeditiously.  Because I had asynchronous I/O (mostly in the form of
BUFFER IN and BUFFER OUT commands), I was able to implement a simple but quite effective
polling loop.  The machine room was a bit of a madhouse, but the tapes were written about
as quickly as the hardware allowed.  Asynchronous I/O FTW...

https://en.wikipedia.org/wiki/United_States_Naval_Research_Laboratory#Space_sciences
https://en.wikipedia.org/wiki/High_Energy_Astronomy_Observatory_1#A1:_Large-Area_Sky_Survey_instrument
https://ub.fnwi.uva.nl/computermuseum/cdcdisk.html

-r


From jon at fourwinds.com  Wed Jun  3 08:03:58 2020
From: jon at fourwinds.com (Jon Steinhart)
Date: Tue, 02 Jun 2020 15:03:58 -0700
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
Message-ID: <202006022203.052M3wpa167150@darkstar.fourwinds.com>

Lawrence Stewart writes:
> I’ve always thought select(2) was a pretty good thing.  In 1995 at Open
> Market we had a single-process web server that had no difficulties
> running 1000 to 1200 connections.  I think that was BSD, and later OSF-1.

So select was a great thing until interactive graphics came along.  Just
got too messy.  I recall writing assembly language code to support threads
in the mid to late 1980s, and I remember reading some papers from Sun about
the same.  Everybody was doing it.  Anybody have enough of the history to
paint a picture of how we got from there to standardized threads?

Jon

From robpike at gmail.com  Wed Jun  3 09:05:39 2020
From: robpike at gmail.com (Rob Pike)
Date: Wed, 3 Jun 2020 09:05:39 +1000
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <202006022203.052M3wpa167150@darkstar.fourwinds.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <202006022203.052M3wpa167150@darkstar.fourwinds.com>
Message-ID: <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>

It's only messy if you make a messy job of it.

-rob


On Wed, Jun 3, 2020 at 8:36 AM Jon Steinhart <jon at fourwinds.com> wrote:

> Lawrence Stewart writes:
> > I’ve always thought select(2) was a pretty good thing.  In 1995 at Open
> > Market we had a single-process web server that had no difficulties
> > running 1000 to 1200 connections.  I think that was BSD, and later OSF-1.
>
> So select was a great thing until interactive graphics came along.  Just
> got too messy.  I recall writing assembly language code to support threads
> in the mid to late 1980s, and I remember reading some papers from Sun about
> the same.  Everybody was doing it.  Anybody have enough of the history to
> paint a picture of how we got from there to standardized threads?
>
> Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200603/522ac797/attachment.htm>

From robpike at gmail.com  Wed Jun  3 09:09:08 2020
From: robpike at gmail.com (Rob Pike)
Date: Wed, 3 Jun 2020 09:09:08 +1000
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <202006022203.052M3wpa167150@darkstar.fourwinds.com>
 <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
Message-ID: <CAKzdPgxH=Ub-EZLmNnRR4W75ipqjt6a_UmwG5nPqwUDRopxOGw@mail.gmail.com>

To be more forthcoming, there was a long period when the received wisdom
was that threads were unmanageable. But that was a category error: The
programming model in POSIX and others was what made it hard. The
fundamental concepts are a tiny step harder than trivial, and it was not
hard to build a programming model that matched them.

The first mistake people made was to split "thread" from "process". What
those words mean today is not what they meant a generation ago.

For me, it's like the mess sockets made of networking. Our programming
models could be so much simpler than what we are required to deal with.

-rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200603/9bdfb5ae/attachment.htm>

From jon at fourwinds.com  Wed Jun  3 09:12:22 2020
From: jon at fourwinds.com (Jon Steinhart)
Date: Tue, 02 Jun 2020 16:12:22 -0700
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <202006022203.052M3wpa167150@darkstar.fourwinds.com>
 <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
Message-ID: <202006022312.052NCMCR178629@darkstar.fourwinds.com>

Rob Pike writes:
>
> It's only messy if you make a messy job of it.

Maybe I should explain what I meant by messy.  To me, the issue with select is
that in many cases one has to either keep a lot of state somewhere, or live with
long response times while handlers complete.  Threads allowed handlers to be
written in the "normal" way with state kept on the stack.

From lm at mcvoy.com  Wed Jun  3 09:21:14 2020
From: lm at mcvoy.com (Larry McVoy)
Date: Tue, 2 Jun 2020 16:21:14 -0700
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <CAKzdPgxH=Ub-EZLmNnRR4W75ipqjt6a_UmwG5nPqwUDRopxOGw@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <202006022203.052M3wpa167150@darkstar.fourwinds.com>
 <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
 <CAKzdPgxH=Ub-EZLmNnRR4W75ipqjt6a_UmwG5nPqwUDRopxOGw@mail.gmail.com>
Message-ID: <20200602232114.GA22016@mcvoy.com>

On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote:
> The first mistake people made was to split "thread" from "process". What
> those words mean today is not what they meant a generation ago.

So I'm a fan of your quote, Rob, "If you think you need threads, your
processes are too fat".

Years ago, I asked Linus if he could fix the one flaw with that point
of view.  Which is, processes can't share page tables.  So N processes
will have N page tables, need an N way bigger TBL, to get the same
performance.

What I asked Linus to try and do is allow a process to share all or
part of its page table with another process.  If that could be done,
we could pretty do what you said, just use processes and if they want
to act like threads, just share the page table.

And I just looked, he seems to have done it.  man 2 clone on a
Linux box.  It's the swiss army knife of features, feels messy.

I don't know if CLONE_VM will share the page tables, I'll ask
Linus.

--lm

From robpike at gmail.com  Wed Jun  3 10:39:45 2020
From: robpike at gmail.com (Rob Pike)
Date: Wed, 3 Jun 2020 10:39:45 +1000
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <20200602232114.GA22016@mcvoy.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <202006022203.052M3wpa167150@darkstar.fourwinds.com>
 <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
 <CAKzdPgxH=Ub-EZLmNnRR4W75ipqjt6a_UmwG5nPqwUDRopxOGw@mail.gmail.com>
 <20200602232114.GA22016@mcvoy.com>
Message-ID: <CAKzdPgyAKQXa-GHK6C+-w=US8jAO_anB_t0nk4TVgFDzpiUfMA@mail.gmail.com>

See the description of rfork at http://man.cat-v.org/plan_9/2/fork.

-rob


On Wed, Jun 3, 2020 at 9:21 AM Larry McVoy <lm at mcvoy.com> wrote:

> On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote:
> > The first mistake people made was to split "thread" from "process". What
> > those words mean today is not what they meant a generation ago.
>
> So I'm a fan of your quote, Rob, "If you think you need threads, your
> processes are too fat".
>
> Years ago, I asked Linus if he could fix the one flaw with that point
> of view.  Which is, processes can't share page tables.  So N processes
> will have N page tables, need an N way bigger TBL, to get the same
> performance.
>
> What I asked Linus to try and do is allow a process to share all or
> part of its page table with another process.  If that could be done,
> we could pretty do what you said, just use processes and if they want
> to act like threads, just share the page table.
>
> And I just looked, he seems to have done it.  man 2 clone on a
> Linux box.  It's the swiss army knife of features, feels messy.
>
> I don't know if CLONE_VM will share the page tables, I'll ask
> Linus.
>
> --lm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200603/49f72587/attachment.htm>

From bakul at iitbombay.org  Wed Jun  3 10:43:18 2020
From: bakul at iitbombay.org (Bakul Shah)
Date: Tue, 2 Jun 2020 17:43:18 -0700
Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
Message-ID: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org>

> Would you be so kind to explain a bit about the hm version of MH.

Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my
Fortune box until I retired it. It provided a 2D interface. Later I
tried xmh but didn't like it.

The early history of MH is covered in some detail in Willis Ware's "RAND
and the Information Evolution" book from page 128 onward. hm & Guyton
get a paragraph on page 136.

https://www.rand.org/content/dam/rand/pubs/corporate_pubs/2008/RAND_CP537.pdf

From clemc at ccc.com  Wed Jun  3 10:49:46 2020
From: clemc at ccc.com (Clem Cole)
Date: Tue, 2 Jun 2020 20:49:46 -0400
Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c)
In-Reply-To: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org>
References: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org>
Message-ID: <CAC20D2PCXW65e-QsWDjB86KLBK1Bq=KxbBPn-RtH03EydvU_=g@mail.gmail.com>

On Tue, Jun 2, 2020 at 8:43 PM Bakul Shah <bakul at iitbombay.org> wrote:

> > Would you be so kind to explain a bit about the hm version of MH.
>
> Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my
> Fortune box until I retired it. It provided a 2D interface.
>
Yep, that would be the program ..   Jim did a great job with it.
It was built using a bunch of MH programs behind it, and was a curses front
end.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200602/b3c9ae02/attachment.htm>

From arnold at skeeve.com  Wed Jun  3 15:12:27 2020
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Tue, 02 Jun 2020 23:12:27 -0600
Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi
Message-ID: <202006030512.0535CRFC020727@freefriends.org>

Hi.

Does anyone have a mirror of the files that once upon a time were
at http://alf.uib.no/pub/vi? They were mirrored at
ftp://ftp.uu.net/pub/text-processing/vi also.

If so, can you please send me a tarball off list (or tell me where
I can download a copy from)?

Thanks!

Arnold

From heinz at osta.com  Wed Jun  3 15:41:16 2020
From: heinz at osta.com (Heinz Lycklama)
Date: Tue, 2 Jun 2020 22:41:16 -0700
Subject: [TUHS] non-blocking IO (morphing to "The serial I/O loop")
In-Reply-To: <FE160071-81A7-4FEA-9637-A8DCDF793145@planet.nl>
References: <FE160071-81A7-4FEA-9637-A8DCDF793145@planet.nl>
Message-ID: <3e482f51-2c2e-19a8-c9fd-f23c67f1df8c@osta.com>

See comments below.

Heinz

On 6/2/2020 2:34 AM, Paul Ruizendaal wrote:
>> On 5/31/2020 9:46 AM, Warner Losh wrote:
>>> Sorry to top post, but LSX or Miniunix had non blocking I/O as well.
>>> It was in one of the documents that Clem scanned in the last year. It
>>> specifically was an experiment into how to do it.
>>>
>>> Warner
>> I did  add a few new features to LSX to deal with contiguous files
>> and to handle asynchronous read/write's for real time applications.
>> They are described in the LSX paper in the 1978 BSTJ on the
>> UNIX Time-Sharing System.
>>
>> Heinz
> Thanks for highlighting this!
>
> The realm here is async I/O to disk, my original scope was limited to “communication” files (tty’s, pipes, network connections). Still, I find it an interesting topic.
>
> For others, the paper that you refer to can also be found here:
> https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/UNIX_on_a_Microprocessor_19780322.pdf
> If I read correctly, the async functionality was available only in the stand alone program version of LSX. Is that correct? In any case, the source code appears lost.
Any source code for LSX would be archived somewhere within
Nokia Bell Labs?
>  From another paper in that set, I get the impression that the async functionality for LSX builds on earlier work for a very early version of Unix:
> https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-74-1352-1_Implementation_of_Large_Contiguous_Files_and_Asynchronous_IO_in_UNIX_19740104.pdf
Yes, as this memo describes, I implemented a very simple version
of large contiguous files and asynchronous file I/O for a special
application. The MERT system developed a little later on, had a
completely integrated of implementation of contiguous files
and asynchronous file I/O.
> - - -
>
> When reading through the papers in that TUHS directory, something else caught my attention: early networking at the labs. For a while I have been puzzled by the “serial I/O loop” in use at the labs in the early seventies. Last Fall I found some 1970/1971 BSTJ papers about it, but there it stopped.
Jon Steinhart would have more information about the serial I/O loo.
> I see in the memo on Glance (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-3_GLANCE_Terminals_on_UNIX_Time-Sharing_19750303.pdf) that D.R. Weller continued to work on it up to 1973 at least and that it was integrated with Unix in some way. Is that correct? Did the two memo’s referred to (MM 70-1384-1 and TM 73-1356-8) survive?
I do not believe that the software for the Glance terminals was ever
integrated into the standard UNIX software distribution
> Then the memo on satellite processors is very interesting (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-78-3114-2_A_Minicomputer_Satellite_Processor_System_19780322.pdf). This appears to show the serial I/O loop in use as late as 1978, with a very intriguing use case involving system call forwarding over the network.
Carl Christensen and I developed this software to handle small
computer systems with disk storage, as described in the memo,
and in the BSTJ article. The satellite diskless computers could be attached
to a powerful central computer by means the serial I/O loop or
various standard DEC controllers. Each of the satellite computers
could run different tailored applications
> Can you tell me more about the serial I/O loop and its use cases?
I think Jon Steinhart has described this elsewhere?
>
>
>
>


From lars at nocrew.org  Wed Jun  3 15:38:29 2020
From: lars at nocrew.org (Lars Brinkhoff)
Date: Wed, 03 Jun 2020 05:38:29 +0000
Subject: [TUHS] Unix on the Arpanet
In-Reply-To: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> (Lawrence
 Stewart's message of "Tue, 2 Jun 2020 17:15:21 -0400")
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
Message-ID: <7wtuzs3e0a.fsf_-_@junk.nocrew.org>

Lawrence Stewart wrote:
> I remember working on getting Arpanet access on an 11/34 running V7
> around 1978 or 1979.  (SU-ISL). We used an 11/23 as a front end to run
> NCP, using a variation of Rand’s code.

Has this, or Rand's code, been preserved?

I'm only aware of one Arpanet NCP implementation for Unix that is
online, the one from University of Illinois.

From stewart at serissa.com  Wed Jun  3 22:23:33 2020
From: stewart at serissa.com (Lawrence Stewart)
Date: Wed, 3 Jun 2020 08:23:33 -0400
Subject: [TUHS] Unix on the Arpanet
In-Reply-To: <7wtuzs3e0a.fsf_-_@junk.nocrew.org>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <7wtuzs3e0a.fsf_-_@junk.nocrew.org>
Message-ID: <68231FF8-6E05-49AE-A33A-A22D5A05966A@serissa.com>


> On 2020, Jun 3, at 1:38 AM, Lars Brinkhoff <lars at nocrew.org> wrote:
> 
> Lawrence Stewart wrote:
>> I remember working on getting Arpanet access on an 11/34 running V7
>> around 1978 or 1979.  (SU-ISL). We used an 11/23 as a front end to run
>> NCP, using a variation of Rand’s code.
> 
> Has this, or Rand's code, been preserved?
> 
> I'm only aware of one Arpanet NCP implementation for Unix that is
> online, the one from University of Illinois.

Alas I do not know.  I may have some old emails from that era. I will check.




From clemc at ccc.com  Wed Jun  3 22:33:10 2020
From: clemc at ccc.com (Clem Cole)
Date: Wed, 3 Jun 2020 08:33:10 -0400
Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi
In-Reply-To: <202006030512.0535CRFC020727@freefriends.org>
References: <202006030512.0535CRFC020727@freefriends.org>
Message-ID: <CAC20D2PwVdNQzwbH9h11mufKOL6yMQERFi5HB9xEanaUQNBr0Q@mail.gmail.com>

I think so... I'll send you a URL off line.

Clem

On Wed, Jun 3, 2020 at 1:14 AM <arnold at skeeve.com> wrote:

> Hi.
>
> Does anyone have a mirror of the files that once upon a time were
> at http://alf.uib.no/pub/vi? They were mirrored at
> ftp://ftp.uu.net/pub/text-processing/vi also.
>
> If so, can you please send me a tarball off list (or tell me where
> I can download a copy from)?
>
> Thanks!
>
> Arnold
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200603/a2afce5e/attachment.htm>

From clemc at ccc.com  Wed Jun  3 22:49:10 2020
From: clemc at ccc.com (Clem Cole)
Date: Wed, 3 Jun 2020 08:49:10 -0400
Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi
In-Reply-To: <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com>
References: <202006030512.0535CRFC020727@freefriends.org>
 <CAC20D2PwVdNQzwbH9h11mufKOL6yMQERFi5HB9xEanaUQNBr0Q@mail.gmail.com>
 <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com>
Message-ID: <CAC20D2OEWqUXcEY5OZ0K6-6Js_KLj7UeLGvzcJob5i4GdX=r2g@mail.gmail.com>

It is too big to attach to email.

On Wed, Jun 3, 2020 at 8:39 AM Michael Stiller <mstiller at icloud.com> wrote:

> Hi Clem,
>
> why offline? Other people are also interested. :-)
>
> Cheers,
>
> Michael
>
>
> > On 3. Jun 2020, at 14:33, Clem Cole <clemc at ccc.com> wrote:
> >
> > I think so... I'll send you a URL off line.
> >
> > Clem
> >
> > On Wed, Jun 3, 2020 at 1:14 AM <arnold at skeeve.com> wrote:
> > Hi.
> >
> > Does anyone have a mirror of the files that once upon a time were
> > at http://alf.uib.no/pub/vi? They were mirrored at
> > ftp://ftp.uu.net/pub/text-processing/vi also.
> >
> > If so, can you please send me a tarball off list (or tell me where
> > I can download a copy from)?
> >
> > Thanks!
> >
> > Arnold
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200603/d1aa94d8/attachment.htm>

From paul.winalski at gmail.com  Thu Jun  4 02:31:57 2020
From: paul.winalski at gmail.com (Paul Winalski)
Date: Wed, 3 Jun 2020 12:31:57 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com>
References: <20200602201334.95D9718C079@mercury.lcs.mit.edu>
 <CAC20D2PRThvBqMz_ApiQk1f36F1Yz4VSqf_BB1dnuC9HNxfOvQ@mail.gmail.com>
 <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com>
Message-ID: <CABH=_VT4b+pv8jXk9vkV7V0wvuHLTX7VRNQnTCNbyaYoD-xgng@mail.gmail.com>

On 6/2/20, Rich Morin <rdm at cfcl.com> wrote:
>
> IIRC, we had five tape drives; my challenge was to keep them all as busy as
> possible, so as
> to dump the data set expeditiously.  Because I had asynchronous I/O (mostly
> in the form of
> BUFFER IN and BUFFER OUT commands), I was able to implement a simple but
> quite effective
> polling loop.  The machine room was a bit of a madhouse, but the tapes were
> written about
> as quickly as the hardware allowed.  Asynchronous I/O FTW...

With 9-track magnetic tape devices, reading and writing can't start
until the tape is up to speed.  Once up to speed the drive can read
and write records while keeping the tape moving at speed.  This is
called streaming.  If there's a pause in the read/write requests from
the CPU, time is lost as the drive stops and starts moving the tape.
It was essential that applications doing large amounts of tape I/O
keep up the I/O requests at a rate that allows streaming.
Asynchronous I/O with multi-buffering is a straightforward way to
accomplish this.  The IBM S/360 channel commands for tape devices
provided a mechanism for the tape control unit to send an interrupt to
the CPU when a read or write channel command completed.  This notified
the sequential access method (the user program I/O interface) when I/O
to each buffer had completed and the buffer was available for reuse.
OS/360's Sequential Access Method could read or write an entire tape
using a single SIO (start I/O) instruction, as long as no read or
write errors were encountered.

-Paul W.

From paul.winalski at gmail.com  Thu Jun  4 02:42:28 2020
From: paul.winalski at gmail.com (Paul Winalski)
Date: Wed, 3 Jun 2020 12:42:28 -0400
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <202006022312.052NCMCR178629@darkstar.fourwinds.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <202006022203.052M3wpa167150@darkstar.fourwinds.com>
 <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
 <202006022312.052NCMCR178629@darkstar.fourwinds.com>
Message-ID: <CABH=_VTtd8K74j66EMpaJKb93KYuWiO9m=o7qVa3v0VTdSF+CA@mail.gmail.com>

On 6/2/20, Jon Steinhart <jon at fourwinds.com> wrote:
>
> Maybe I should explain what I meant by messy.  To me, the issue with select is
> that in many cases one has to either keep a lot of state somewhere, or live with
> long response times while handlers complete.  Threads allowed handlers to be
> written in the "normal" way with state kept on the stack.
>
TOPS-10 had a nifty program called ptycon that would allow one to
control multiple login sessions, each on its own pseudo-terminal
device, from a single physical terminal.  In 1985 our software tools
group implemented a pseudo-terminal device driver for VMS, and I wrote
a program called ptycon-32.  It was multithreaded, with one thread of
control for each pseudo-terminal.  The problem is that VMS at the time
didn't have a threads package.  I had to roll my own threads by using
the asynchronous system traps (ASTs) delivered by completion of I/O to
the pseudo-terminals.  Talk about messy....

-Paul W.

From nobozo at gmail.com  Thu Jun  4 03:57:33 2020
From: nobozo at gmail.com (Jon Forrest)
Date: Wed, 3 Jun 2020 10:57:33 -0700
Subject: [TUHS] non-blocking IO - threads
In-Reply-To: <CABH=_VTtd8K74j66EMpaJKb93KYuWiO9m=o7qVa3v0VTdSF+CA@mail.gmail.com>
References: <F364B944-42D3-4AAA-88DB-6EBBFEDF9F21@planet.nl>
 <CABH=_VRjxL=p8f+ePVvBWKuQN3aFE-BW4aE9MAcjwkK-Mm1rkg@mail.gmail.com>
 <202006021759.052Hx5Et022619@freefriends.org>
 <CABH=_VQj_3qdK+Cu3t5hZPQFUny_TFPxR51Cfuj=Av4Jt3cYyQ@mail.gmail.com>
 <CAC20D2NT-KGP6hb0Zsd2uNpMJjRDxp3EKU2xC2WJW1U8Npawjw@mail.gmail.com>
 <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com>
 <202006022203.052M3wpa167150@darkstar.fourwinds.com>
 <CAKzdPgxQnFWf-E99zEyZJBB5z23ZUKwj-Dffe9y5if9R-w3Qpg@mail.gmail.com>
 <202006022312.052NCMCR178629@darkstar.fourwinds.com>
 <CABH=_VTtd8K74j66EMpaJKb93KYuWiO9m=o7qVa3v0VTdSF+CA@mail.gmail.com>
Message-ID: <332ca1eb-f862-b028-a2a2-e168399576a1@gmail.com>



On 6/3/2020 9:42 AM, Paul Winalski wrote:
> The problem is that VMS at the time
> didn't have a threads package.  I had to roll my own threads by using
> the asynchronous system traps (ASTs) delivered by completion of I/O to
> the pseudo-terminals.  Talk about messy....

That's also how the Sybase SQL Server worked on VMS at the same time.

Jon

From jpl.jpl at gmail.com  Thu Jun  4 05:19:40 2020
From: jpl.jpl at gmail.com (John P. Linderman)
Date: Wed, 3 Jun 2020 15:19:40 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <CABH=_VT4b+pv8jXk9vkV7V0wvuHLTX7VRNQnTCNbyaYoD-xgng@mail.gmail.com>
References: <20200602201334.95D9718C079@mercury.lcs.mit.edu>
 <CAC20D2PRThvBqMz_ApiQk1f36F1Yz4VSqf_BB1dnuC9HNxfOvQ@mail.gmail.com>
 <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com>
 <CABH=_VT4b+pv8jXk9vkV7V0wvuHLTX7VRNQnTCNbyaYoD-xgng@mail.gmail.com>
Message-ID: <CAC0cEp8C6VNckw7sXz=mC6kgMccFu_BgJjGzAC3BHcgt4YK3Fg@mail.gmail.com>

On a distantly related note, when I worked part time for the MIT
administration in the 60's, we'd do processing on the 7094 in the main comp
center, then bring a tape of results back to our office to do the printing
and card punching. For reasons that were never explained to me, the blocks
of print and punch data were combined on a single tape, and were
distinguished by being written with different parities. If you attempted to
read a block with the correct parity setting, the tape could stream. If you
guessed wrong, you had to stop, back up a block, change the parity, and try
again. So I wrote a simple 360 assembly program to keep track how often
blocks of each parity followed the observed parities of the previous 8
blocks. Essentially, 256 pairs of parity observations, indexed by the
previous 8 parity observations. Blocks of print and punch data tended to
fall into patterns that depended on what job was being run on the 7094, so
detecting those patterns and correctly anticipating the upcoming parity
made the tapes move much more smoothly. It was fun to watch the tape at the
start of a run. It was mostly just a coin-toss, so the tape was jerking
around fitfully. As the patterns started to emerge, the predictions got
better, the jerking got less and less common, and the tapes were streaming
most of the time. My introduction to learning algorithms.

On Wed, Jun 3, 2020 at 12:33 PM Paul Winalski <paul.winalski at gmail.com>
wrote:

> On 6/2/20, Rich Morin <rdm at cfcl.com> wrote:
> >
> > IIRC, we had five tape drives; my challenge was to keep them all as busy
> as
> > possible, so as
> > to dump the data set expeditiously.  Because I had asynchronous I/O
> (mostly
> > in the form of
> > BUFFER IN and BUFFER OUT commands), I was able to implement a simple but
> > quite effective
> > polling loop.  The machine room was a bit of a madhouse, but the tapes
> were
> > written about
> > as quickly as the hardware allowed.  Asynchronous I/O FTW...
>
> With 9-track magnetic tape devices, reading and writing can't start
> until the tape is up to speed.  Once up to speed the drive can read
> and write records while keeping the tape moving at speed.  This is
> called streaming.  If there's a pause in the read/write requests from
> the CPU, time is lost as the drive stops and starts moving the tape.
> It was essential that applications doing large amounts of tape I/O
> keep up the I/O requests at a rate that allows streaming.
> Asynchronous I/O with multi-buffering is a straightforward way to
> accomplish this.  The IBM S/360 channel commands for tape devices
> provided a mechanism for the tape control unit to send an interrupt to
> the CPU when a read or write channel command completed.  This notified
> the sequential access method (the user program I/O interface) when I/O
> to each buffer had completed and the buffer was available for reuse.
> OS/360's Sequential Access Method could read or write an entire tape
> using a single SIO (start I/O) instruction, as long as no read or
> write errors were encountered.
>
> -Paul W.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200603/46f29f1e/attachment.htm>

From gnu at toad.com  Thu Jun  4 11:24:52 2020
From: gnu at toad.com (John Gilmore)
Date: Wed, 03 Jun 2020 18:24:52 -0700
Subject: [TUHS] non-blocking IO: Simplicity has been subtituted for
 efficiency
In-Reply-To: <20200602201334.95D9718C079@mercury.lcs.mit.edu> 
References: <20200602201334.95D9718C079@mercury.lcs.mit.edu>
Message-ID: <8066.1591233892@hop.toad.com>

Noel Chiappa <jnc at mercury.lcs.mit.edu> wrote:

>     > I'm curious as to what the rationale was for Unix to have been designed
>     > with basic I/O being blocking rather than asynchronous.
> 
> It's a combination of two factors, I reckon. One, which is better depends a
> lot on the type of thing you're trying to do. For many typical thing (e.g.
> 'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is
> more complicated, and Unix was (well, back then at least) all about getting
> the most bang for the least bucks.

I just happened to open a binder today of old papers about UNIX(tm),
including Ken Thompson's "Unix Implementation" paper, which says at the
bottom of the introduction:

  "What is or is not implemented in the kernel represents both a great
  responsibiity and a great power.  It is a soap-box platform on "the
  way things should be done."  Even so, if "the way" is too radical, no
  one will follow it.  Every important decision was weighted carefully.
  Throughout, simplicity has been substituted for efficiency.  Complex
  algorithms are used only if their complexity can be localized."

	John
	

From imp at bsdimp.com  Thu Jun  4 13:38:43 2020
From: imp at bsdimp.com (Warner Losh)
Date: Wed, 3 Jun 2020 21:38:43 -0600
Subject: [TUHS] 2.11BSD pl 0 recovered (mostly) PRELIMINARY
Message-ID: <CANCZdfonmzQ+p=epaSLsccp6se6FRKy4gPmTnkmwa-itOeL78Q@mail.gmail.com>

OK. I've written a script to take the 2.11BSD pl 195 tape and reverse apply
all the patches that we have to get back to 2.11BSD original.

There's some problems. The biggest one is that ld.c was rewritten during
this series and what it replaced is lost. And the 2.10.1 ld.c isn't what's
in 2.11BSD and the patches to get from 2.10.1 to 2.11 don't seem to be out
there. And the other patches in the series make it clear that patches are
missing. 2.11BSD likely will work with 2.10.1's ld, so this isn't fatal.

There's 122 other files that I recovered from 2.10.1. Almost all of those
are a good guess, if not what's actually in 2.11. Many of these can be
verified via other means. Some of these can be snagged from comp.bugs.2bsd
(that's how as was recovered).

There's a number of small cosmetic changes made via copying, some to get
rid of redundant files, etc. I think these don't matter for the function of
the system, but are small differences from the actual tape that shipped.

I still need to try to still create a tape and try to compile. And forward
apply all the patches and create a git repo from it.

All things considered 99% of the files have been recovered at this point.

It's early days, but I've pushed this to github for comments.

http://github.com/bsdimp/mk211bsd

It just assumes you have the Tuhs archive (including the new Usenet
section) and that apout works on your computer. It works on FreeBSD for
sure. No clue about anything else...

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200603/b0e675e5/attachment.htm>

From arnold at skeeve.com  Thu Jun  4 16:27:10 2020
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Thu, 04 Jun 2020 00:27:10 -0600
Subject: [TUHS] non-blocking IO: Simplicity has been subtituted for
 efficiency
In-Reply-To: <8066.1591233892@hop.toad.com>
References: <20200602201334.95D9718C079@mercury.lcs.mit.edu>
 <8066.1591233892@hop.toad.com>
Message-ID: <202006040627.0546RAMW020424@freefriends.org>

John Gilmore <gnu at toad.com> wrote:

> I just happened to open a binder today of old papers about UNIX(tm),
> including Ken Thompson's "Unix Implementation" paper, which says at the
> bottom of the introduction:
>
>   "What is or is not implemented in the kernel represents both a great
>   responsibiity and a great power.  It is a soap-box platform on "the
>   way things should be done."  Even so, if "the way" is too radical, no
>   one will follow it.  Every important decision was weighted carefully.
>   Throughout, simplicity has been substituted for efficiency.  Complex
>   algorithms are used only if their complexity can be localized."

Words to live by.  I remember reading that paper early on in my
career and that this statement certainly influenced my subsequent
thinking about programming in general.

Arnold

From peter at rulingia.com  Thu Jun  4 19:04:36 2020
From: peter at rulingia.com (Peter Jeremy)
Date: Thu, 4 Jun 2020 19:04:36 +1000
Subject: [TUHS] non-blocking IO
In-Reply-To: <20200601145801.GE22016@mcvoy.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
 <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
 <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>
 <20200601145801.GE22016@mcvoy.com>
Message-ID: <20200604090436.GJ279@server.rulingia.com>

On 2020-Jun-01 07:58:02 -0700, Larry McVoy <lm at mcvoy.com> wrote:
>On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote:
>> On Mon, 1 Jun 2020, Rob Pike wrote:
>> 
>> > I???m not quite sure why the Research lineage did not include
>> > non-blocking behaviour, especially in view of the man page comments.
>> > Maybe it was seen as against the Unix philosophy, with select()
>> > offering sufficient mechanism to avoid blocking (with open() the hard
>> > corner case)?
>> >
>> >That's it. Select was good enough for our purposes.
>> 
>> After being dragged through both Berserkley and SysVile, I never did get the
>> hang of poll()/select() etc,,,
>
>I'm sure you could, select is super handy, think a network server like
>apache.

My view may be unpopular but I've always been disappointed that Unix
implemented blocking I/O only and then had to add various hacks to cover
up for the lack of asynchonous I/O.  It's trivial to build blocking I/O
operations on top of asynchonous I/O operations.  It's impossible to do
the opposite without additional functionality.

I also found it disappointing that poll()/select() only worked on TTY and
network operations.  HDDs are really slow compared to CPUs and it would be
really nice if a process could go and do something else whilst waiting for
a file to open.

-- 
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200604/ea72c2e5/attachment.sig>

From arnold at skeeve.com  Thu Jun  4 23:39:35 2020
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Thu, 04 Jun 2020 07:39:35 -0600
Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi
In-Reply-To: <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com>
References: <202006030512.0535CRFC020727@freefriends.org>
 <CAC20D2PwVdNQzwbH9h11mufKOL6yMQERFi5HB9xEanaUQNBr0Q@mail.gmail.com>
 <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com>
Message-ID: <202006041339.054DdZk7008288@freefriends.org>

Hi All.

I've put Clem's file up at http://www.skeeve.com/text-processing.tar.bz2.
It's a little over 16 meg and includes more than just the vi stuff.

Enjoy,

Arnold

Michael Stiller <mstiller at icloud.com> wrote:

> Hi Clem,
>
> why offline? Other people are also interested. :-)
>
> Cheers,
>
> Michael
>
>
> > On 3. Jun 2020, at 14:33, Clem Cole <clemc at ccc.com> wrote:
> > 
> > I think so... I'll send you a URL off line.
> > 
> > Clem
> > 
> > On Wed, Jun 3, 2020 at 1:14 AM <arnold at skeeve.com> wrote:
> > Hi.
> > 
> > Does anyone have a mirror of the files that once upon a time were
> > at http://alf.uib.no/pub/vi? They were mirrored at
> > ftp://ftp.uu.net/pub/text-processing/vi also.
> > 
> > If so, can you please send me a tarball off list (or tell me where
> > I can download a copy from)?
> > 
> > Thanks!
> > 
> > Arnold

From imp at bsdimp.com  Fri Jun  5 00:19:58 2020
From: imp at bsdimp.com (Warner Losh)
Date: Thu, 4 Jun 2020 08:19:58 -0600
Subject: [TUHS] non-blocking IO
In-Reply-To: <20200604090436.GJ279@server.rulingia.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
 <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
 <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>
 <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com>
Message-ID: <CANCZdfo289SQhA3tu5Z=RsDv3VrxgZ00r3M7mPNPCMEQiMvuQA@mail.gmail.com>

On Thu, Jun 4, 2020 at 3:06 AM Peter Jeremy <peter at rulingia.com> wrote:

> On 2020-Jun-01 07:58:02 -0700, Larry McVoy <lm at mcvoy.com> wrote:
> >On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote:
> >> On Mon, 1 Jun 2020, Rob Pike wrote:
> >>
> >> > I???m not quite sure why the Research lineage did not include
> >> > non-blocking behaviour, especially in view of the man page comments.
> >> > Maybe it was seen as against the Unix philosophy, with select()
> >> > offering sufficient mechanism to avoid blocking (with open() the hard
> >> > corner case)?
> >> >
> >> >That's it. Select was good enough for our purposes.
> >>
> >> After being dragged through both Berserkley and SysVile, I never did
> get the
> >> hang of poll()/select() etc,,,
> >
> >I'm sure you could, select is super handy, think a network server like
> >apache.
>
> My view may be unpopular but I've always been disappointed that Unix
> implemented blocking I/O only and then had to add various hacks to cover
> up for the lack of asynchonous I/O.  It's trivial to build blocking I/O
> operations on top of asynchonous I/O operations.  It's impossible to do
> the opposite without additional functionality.
>
> I also found it disappointing that poll()/select() only worked on TTY and
> network operations.  HDDs are really slow compared to CPUs and it would be
> really nice if a process could go and do something else whilst waiting for
> a file to open.
>

Lest anybody think this is a theoretical concern, Netflix has spent quite a
bit of effort to reduce the sources of latency in our system. The latency
for open doesn't happen often, due to caching, but when it does this causes
a hickup for nginx worker thread (since open is blocking). If you get
enough hickups, you wind up consuming all your worker threads and latency
for everybody suffers while waiting for these to complete (think flaky disk
that suddenly takes a really long time for each of its I/Os for one
example). We've hacked FreeBSD in various ways to reduce or eliminate this
delay.... In FreeBSD we have to translate from the pathname to a vnode, and
to do that we have to look up directories, indirect block tables, etc. All
these are surprising places that one could bottleneck at... And if you try
to access the vnode before all this is done, you'll wait for it (though in
the case of sendfile it doesn't matter since that's async and only affect
the one I/O)...

So I could see how having a async open could introduce a lot of hair into
the mix depending on how you do it. Without a robust callback/AST
mechanism, my brain is recoiling from the EALREADY errors in sockets for
things that are already in progress... reads and write are easy by
comparison :)  The kicker is that all of the kernel is callback driven. The
upper half queues the request and then sleeps until the lower half signals
it to wakeup. And that signal is often just a wakeup done from the
completion routine in the original request. All of that would be useful in
userland for high volume activity, none of it is exposed...

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200604/d957aa68/attachment.htm>

From dot at dotat.at  Fri Jun  5 02:34:54 2020
From: dot at dotat.at (Tony Finch)
Date: Thu, 4 Jun 2020 17:34:54 +0100
Subject: [TUHS] non-blocking IO
In-Reply-To: <CANCZdfo289SQhA3tu5Z=RsDv3VrxgZ00r3M7mPNPCMEQiMvuQA@mail.gmail.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
 <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
 <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>
 <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com>
 <CANCZdfo289SQhA3tu5Z=RsDv3VrxgZ00r3M7mPNPCMEQiMvuQA@mail.gmail.com>
Message-ID: <alpine.DEB.2.20.2006041730300.12007@grey.csi.cam.ac.uk>

Warner Losh <imp at bsdimp.com> wrote:
>
> So I could see how having a async open could introduce a lot of hair into
> the mix depending on how you do it.

Linux io_uring, for example.

I would be interested if anyone has a historical perspective on its design :-)

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/
Wight, Portland, Plymouth, Biscay, Fitzroy, Sole, Lundy, Fastnet: North or
northwest 4 to 6, occasionally 7 in Biscay. Slight or moderate at first except
in Biscay and Fitzroy, otherwise moderate or rough. Showers. Good,
occasionally moderate.

From lm at mcvoy.com  Fri Jun  5 02:50:11 2020
From: lm at mcvoy.com (Larry McVoy)
Date: Thu, 4 Jun 2020 09:50:11 -0700
Subject: [TUHS] non-blocking IO
In-Reply-To: <CANCZdfo289SQhA3tu5Z=RsDv3VrxgZ00r3M7mPNPCMEQiMvuQA@mail.gmail.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
 <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
 <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>
 <20200601145801.GE22016@mcvoy.com>
 <20200604090436.GJ279@server.rulingia.com>
 <CANCZdfo289SQhA3tu5Z=RsDv3VrxgZ00r3M7mPNPCMEQiMvuQA@mail.gmail.com>
Message-ID: <20200604165011.GC18437@mcvoy.com>

On Thu, Jun 04, 2020 at 08:19:58AM -0600, Warner Losh wrote:
> The kicker is that all of the kernel is callback driven. The
> upper half queues the request and then sleeps until the lower half signals
> it to wakeup. And that signal is often just a wakeup done from the
> completion routine in the original request. All of that would be useful in
> userland for high volume activity, none of it is exposed...

Yeah, I've often wondered why this stuff wasn't exposed.  We already have
signal handlers, seems like that maps.  

I tried to get the NFS guys at Sun to rethink the biod junk and do it like
UFS does, where it queues something and gets a callback.  I strongly suspect
that two processes, one to queue, one to handle callbacks, would be more
efficient and actually faster than the biod nonsense.

That's one of the arguments I lost unfortunately.

Warner, exposing that stuff in FreeBSD is not really that hard, I suspect.
Might be a fun project for a young kernel hacker with some old dude like
you or me or someone, watching over it and thinking about the API.

--lm

From crossd at gmail.com  Sat Jun  6 02:00:52 2020
From: crossd at gmail.com (Dan Cross)
Date: Fri, 5 Jun 2020 12:00:52 -0400
Subject: [TUHS] non-blocking IO
In-Reply-To: <20200604165011.GC18437@mcvoy.com>
References: <ADC32296-73FD-46D4-A3D6-4EA03A956103@planet.nl>
 <CAC20D2McV_i0d=m33McRVoPda8ZVLawaKJxndYDLbqAnHLE_Wg@mail.gmail.com>
 <CANCZdfrefhG42UEx=WXrVNvrxR0oBpT1u7oz6_PitjZDAbSALw@mail.gmail.com>
 <CAKzdPgyP0Vtkwh50hQQpFuFaztw9i+4fi2BVAzPhm8upJh4e_Q@mail.gmail.com>
 <alpine.BSF.2.21.9999.2006011328160.31987@aneurin.horsfall.org>
 <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com>
 <CANCZdfo289SQhA3tu5Z=RsDv3VrxgZ00r3M7mPNPCMEQiMvuQA@mail.gmail.com>
 <20200604165011.GC18437@mcvoy.com>
Message-ID: <CAEoi9W5svk9oATzkHQNrPOcFii6p7vbd+rYMFBopiqnvrXzMkQ@mail.gmail.com>

On Thu, Jun 4, 2020 at 12:51 PM Larry McVoy <lm at mcvoy.com> wrote:

> On Thu, Jun 04, 2020 at 08:19:58AM -0600, Warner Losh wrote:
> > The kicker is that all of the kernel is callback driven. The
> > upper half queues the request and then sleeps until the lower half
> signals
> > it to wakeup. And that signal is often just a wakeup done from the
> > completion routine in the original request. All of that would be useful
> in
> > userland for high volume activity, none of it is exposed...
>
> Yeah, I've often wondered why this stuff wasn't exposed.  We already have
> signal handlers, seems like that maps.
>

Was it Rob who said that signals were really just for SIGKILL? Here,
signals would be gang-pressed into service as a general IPC mechanism. In
fairness, they've mutated that way, but they didn't start out that way.
While I obviously wasn't there, the strong impression I get is that by the
time people were seriously thinking about async IO in Unix, the die
had already been cast for better or worse.


> I tried to get the NFS guys at Sun to rethink the biod junk and do it like
> UFS does, where it queues something and gets a callback.  I strongly
> suspect
> that two processes, one to queue, one to handle callbacks, would be more
> efficient and actually faster than the biod nonsense.
>
> That's one of the arguments I lost unfortunately.
>
> Warner, exposing that stuff in FreeBSD is not really that hard, I suspect.
> Might be a fun project for a young kernel hacker with some old dude like
> you or me or someone, watching over it and thinking about the API.
>

I'm going to actually disagree with you here, Larry. While I think a basic
mechanism wouldn't be THAT hard to implement, it wouldn't compose nicely
with the existing primitives. I suspect the edge cases would be really
thorny, particularly without a real AST abstraction. For instance, what
happens if you initiate an async IO operation, then block on a `read`?
Where does the callback happen? If on the same thread, The real challenge
isn't providing the operation, it's integrating it into the existing model.

As a counter-point to the idea that it's completely unruly, in Akaros this
was solved in the C library: all IO operations were fundamentally
asynchronous, but the C library provided blocking read(), write(), etc by
building those from the async primitives. It worked well, but Akaros had
something akin to an AST environment and fine-grain scheduling decisions
were made in userspace: in Akaros the unit of processor allocation is a CPU
core, not a thread, and support exists for determining the status of all
cores allocated to a process. There are edge cases (you can't roll-your-own
mutex, for example, and the basic threading library does a lot of heavy
lifting for you making it challenging to integrate into the runtime of a
language that doesn't use the same ABI), but by and large it worked. It was
also provided by a kernel that was a pretty radical departure from a
Unix-like kernel.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200605/0743f819/attachment.htm>

From dave at horsfall.org  Sat Jun  6 06:57:06 2020
From: dave at horsfall.org (Dave Horsfall)
Date: Sat, 6 Jun 2020 06:57:06 +1000 (EST)
Subject: [TUHS] History of popularity of C
In-Reply-To: <m1je5II-0036tPC@more.local>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <95e6e8de901c837a28b84e62556ba326@firemail.de> <m1je23J-0036tPC@more.local>
 <CAGg_6+OHu0QVqhcun=_+Dt69o4m9tT0YfsCPnu0n=j7Wbf+Ugg@mail.gmail.com>
 <m1je5II-0036tPC@more.local>
Message-ID: <alpine.BSF.2.21.9999.2006060630050.44790@aneurin.horsfall.org>

On Wed, 27 May 2020, Greg A. Woods wrote:

> Sadly most compilers, including GCC and Clang/LLVM will, at best, warn 
> (and warnings are only treated as errors by the most macho|wise); and 
> compilers only do that now because they've been getting flack from 
> developers whenever the optimizer does something unexpected.

Don't talk to me about optimisers...  That's not the code that I wrote! 
I've seen code simply disappear, because the "optimiser" though that it 
was cleverer than I was.

> The Linux kernel example I've referred to involved dereferencing a 
> pointer to do an assignment in a local variable definition, then a few 
> lines later testing if the pointer was NULL before using the local 
> variable.  Unoptimised the code will dereference a NULL pointer and load 
> junk from location zero into the variable (because it's kernel code), 
> then the NULL test will trigger and all will be good.  The optimizer 
> rips out the NULL check because "obviously" the programmer has assumed 
> the pointer is always a valid non-NULL pointer since they've explicitly 
> dereferenced it before checking it and they wouldn't want to waste even 
> a single jump-on-zero instruction checking it again.  (It's also quite 
> possible the code was written "correctly" at first, then someone mushed 
> all the variable initialisations up onto their definitions.)

Typical Penguin/OS behaviour...

> In any case there's now a GCC option:  -fno-delete-null-pointer-checks 
> (to go along with -fno-strict-aliasing and -fno-strict-overflow, and 
> -fno-strict-enums, all of which MUST be used, and sometimes 
> -fno-strict-volatile-bitfields too, on all legacy code that you don't 
> want to break)

I'm sure that there's a competition somewhere, to see who can come with 
GCC's -fmost-longest-and-most-obscure-option flags...

> It's even worse when you have to write bare-metal code that must 
> explictly dereference a NULL pointer (a not-so-real example:  you want 
> to use location zero in the CPU zero-page (e.g. on a 6502 or 6800, or 
> PDP-8, etc.) as a pointer) -- it is now impossible to do that in strict 
> Standard C even though trivially it "should just work" despite the silly 
> rules.  As far as I can tell it always did just work in "plain old" C.

I've programmed a PDP-8!  'Twas way back in high school, and I found a bug 
in my mentor's program; it controlled traffic lights...

> The crazy thing about modern optimizers is that they're way more 
> persistent and often somewhat more clever than your average programmer. 
> They follow all the paths.  They apply all the rules at every turn.

Optimisers...  Grrr...

-- Dave

From cym224 at gmail.com  Sat Jun  6 07:40:50 2020
From: cym224 at gmail.com (Nemo Nusquam)
Date: Fri, 5 Jun 2020 17:40:50 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <alpine.BSF.2.21.9999.2006060630050.44790@aneurin.horsfall.org>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <95e6e8de901c837a28b84e62556ba326@firemail.de> <m1je23J-0036tPC@more.local>
 <CAGg_6+OHu0QVqhcun=_+Dt69o4m9tT0YfsCPnu0n=j7Wbf+Ugg@mail.gmail.com>
 <m1je5II-0036tPC@more.local>
 <alpine.BSF.2.21.9999.2006060630050.44790@aneurin.horsfall.org>
Message-ID: <5f5d5c7f-ce33-d067-2a62-aa369175d9cb@gmail.com>

On 06/05/20 16:57, Dave Horsfall wrote (in part):
> [...]
> Optimisers...  Grrr...
Steve Johnson's position paper on optimising compilers may amuse you: 
https://dl.acm.org/doi/abs/10.1145/567532.567542

N.

>
> -- Dave


From rich.salz at gmail.com  Sat Jun  6 07:47:23 2020
From: rich.salz at gmail.com (Richard Salz)
Date: Fri, 5 Jun 2020 17:47:23 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <alpine.BSF.2.21.9999.2006060630050.44790@aneurin.horsfall.org>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <95e6e8de901c837a28b84e62556ba326@firemail.de> <m1je23J-0036tPC@more.local>
 <CAGg_6+OHu0QVqhcun=_+Dt69o4m9tT0YfsCPnu0n=j7Wbf+Ugg@mail.gmail.com>
 <m1je5II-0036tPC@more.local>
 <alpine.BSF.2.21.9999.2006060630050.44790@aneurin.horsfall.org>
Message-ID: <CAFH29tqJDCi7qvRpojbngmgOBh=ynfV5Lpmcb7k1QxiqCYZFKA@mail.gmail.com>

> I'm sure that there's a competition somewhere, to see who can come with
> GCC's -fmost-longest-and-most-obscure-option flags...
>

At least one of the GCC maintainers is German, so possibly.  Can clang keep
up? :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200605/67b42a2e/attachment.htm>

From bakul at iitbombay.org  Sat Jun  6 08:01:29 2020
From: bakul at iitbombay.org (Bakul Shah)
Date: Fri, 5 Jun 2020 15:01:29 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <CAFH29tqJDCi7qvRpojbngmgOBh=ynfV5Lpmcb7k1QxiqCYZFKA@mail.gmail.com>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <95e6e8de901c837a28b84e62556ba326@firemail.de> <m1je23J-0036tPC@more.local>
 <CAGg_6+OHu0QVqhcun=_+Dt69o4m9tT0YfsCPnu0n=j7Wbf+Ugg@mail.gmail.com>
 <m1je5II-0036tPC@more.local>
 <alpine.BSF.2.21.9999.2006060630050.44790@aneurin.horsfall.org>
 <CAFH29tqJDCi7qvRpojbngmgOBh=ynfV5Lpmcb7k1QxiqCYZFKA@mail.gmail.com>
Message-ID: <63245F94-E434-42D9-A294-8AE6971FE66D@iitbombay.org>

On Jun 5, 2020, at 2:47 PM, Richard Salz <rich.salz at gmail.com> wrote:
> 
> 
> | I'm sure that there's a competition somewhere, to see who can come with 
> | GCC's -fmost-longest-and-most-obscure-option flags...
> 
> At least one of the GCC maintainers is German, so possibly.  Can clang keep up? :) 

Clang has more than kept up!

clang:
-enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang<value>

gcc-9:
-print-sysroot-headers-suffix

Not counting gcc's
--help={common|optimizers|params|target|warnings|[^]{joined|separate|undocumented}}[,...].




From imp at bsdimp.com  Sat Jun  6 08:51:27 2020
From: imp at bsdimp.com (Warner Losh)
Date: Fri, 5 Jun 2020 16:51:27 -0600
Subject: [TUHS] My BSDcan talk
Message-ID: <CANCZdfpq8tiDYe2iVeFh1h0VMDK+4B=kXuGSJ3iNmtjbzHQT6Q@mail.gmail.com>

OK. Must be off my game... I forgot to tell people about my BSDcan talk
earlier today. It was streamed live, and will be online in a week or
three...

It's another similar to the last two. I've uploaded a version to youtube
until the conference has theirs ready. It's a private link, but should work
for anybody that has it. Now that I've given my talk it's cool to share
more widely...  https://www.youtube.com/watch?v=NRq8xEvFS_g

The link at the end is wrong. https://github.com/bsdimp/bsdcan2020-demos is
the proper link.

Please let me know what you think.

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200605/073f69ae/attachment.htm>

From pnr at planet.nl  Sat Jun  6 21:30:58 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Sat, 6 Jun 2020 13:30:58 +0200
Subject: [TUHS] Unix on Aprpanet
Message-ID: <CEA621AA-A2CF-40FD-89A0-7F62AD74CABD@planet.nl>

>> On 2020, Jun 3, at 1:38 AM, Lars Brinkhoff <lars at nocrew.org> wrote: 
>> 
>> Lawrence Stewart wrote: 
>>> I remember working on getting Arpanet access on an 11/34 running V7 
>>> around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run 
>>> NCP, using a variation of Rand’s code. I wrote some sort of bisync
>>> driver for packet communications between the /23 and the /34, and I
>>> think added an IOCTL or some hack to ask if there was a message ready.
>>> So a polling variety of non-blocking I/O :)
>> 
>> Has this, or Rand's code, been preserved? 
>> 
>> I'm only aware of one Arpanet NCP implementation for Unix that is 
>> online, the one from University of Illinois. 
> Alas I do not know. I may have some old emails from that era. I will check.

I have not so far come across an NCP Unix that was not based on the UoI code base.

At a time there was a “Rand Unix”, that combined the UoI code with a few other
extensions. A 1978 document describes the kernel as:

"UNIX version 6 with modifications some of which are:
- Accounting
- A system call to write end of files on pipes
- A system call which checks whether a process would sleep upon doing a read on either a pipe or tty
- Pseudo teletypes
- The network control program NCP written at the University of Illinois
- A new method of interprocess communication called ports. Ports are similar to pipes but have names and can be accessed by any process.”

The system call to write EOF on pipes is eofp() and the blocking read test is empty(). Both are available in the NCP Unix source code as included on the TUHS Unix Tree page. It could well be that the hack Lawrence refers to above was a modification to empty() to also cover his bisync driver.

I guess empty() is the first step on the path that includes capac()/await() and later select()/poll() as the preferred mechanism to prevent blocking on IPC files (incl. tty’s).



From jnc at mercury.lcs.mit.edu  Sat Jun  6 23:29:06 2020
From: jnc at mercury.lcs.mit.edu (Noel Chiappa)
Date: Sat,  6 Jun 2020 09:29:06 -0400 (EDT)
Subject: [TUHS] non-blocking IO
Message-ID: <20200606132906.B7FAB18C079@mercury.lcs.mit.edu>

    > From: Peter Jeremy <peter at rulingia.com>

    > My view may be unpopular but I've always been disappointed that Unix
    > implemented blocking I/O only and then had to add various hacks to cover
    > up for the lack of asynchonous I/O.  It's trivial to build blocking I/O
    > operations on top of asynchonous I/O operations.  It's impossible to do
    > the opposite without additional functionality.

Back when I started working on networks, I looked at other kinds of systems
to see what general lessons I could learn about the evolution of systems, which
might apply to the networks we were building. (I should have written all that up,
never did, sigh.)

One major one was that a system, when small, often collapses multiple needs
onto one machanism. Only as the system grows in size do scaling effects, etc
necessitate breaking them up into separate mechanisms. (There are some good
examples in file systems, for example.)

I/O is a perfect example of this; a small system can get away with only one
kind; it's only when the system grows that one benefits from having both
synchronous and asynchronous. Since the latter is more complicated, _both_ in
the system and in the applications which use it, it's no surprise that
synchronous was the pick.


The reasons why synchronous is simpler in applications have a nice
illustration in operating systems, which inevitably support both blocking
(i.e. implied process switching) and non-blocking 'operation initiation' and
'operation completed notification' mechanisms. (The 'timeout/callout'
mechanism is Unix is an example of the latter, albeit specialized to timers.)

Prior to the Master Control Program in the Burroughs B000 (there may be older
examples, but I don't know of them - I would be more than pleased to be
informed of any such, if there are), the technique of having a per-process
_kernel_ stack, and on a process block (and implied switch), switching stacks,
was not used. This idea was picked up for Jerry Saltzer's PhD thesis, used in
Multics, and then copied by almost every other OS since (including Unix).

The advantage is fairly obvious: if one is deep in some call stack, one can
just wait there until the thing one needs is done, and then resume without
having to work one's way back to that spot - which will inevitably be
complicated (perhaps more in the need to _return_ through all the places that
called down - although the code to handle a 'not yet' return through all those
places, after the initial call down, will not be inconsiderable either).


Exactly the same reasoning applies to blocking I/O; one can sit where one is,
waiting for the I/O to be done, without having to work one's way back there
later. (Examples are legion, e.g. in recursive descent parsers - and can make
the code _much_ simpler.)

It's only when one _can't_ wait for the I/O to complete (e.g. for a packet to
arrive - although others have mentioned other examples in this thread, such as
'having other stuff to do in the meanwhile') than having only blocking I/O
becomes a problem...

In cases where blocking would be better, one can always build a 'blocking' I/O
subsystem on top of asynchronous I/O primitives.

However, in a _tiny_ system (remember my -11/40 which ran Unix on a system
with _48KB_ of main memory _total_- i.e. OS and application together had to be
less than 48KB - no virtual memory on that machine :-), building blocking I/O
on top of asynchonous I/O, for those very few cases which need it, may not be
the best use of very limited space - although I agree that it's the way to go,
overall.

	Noel

From pnr at planet.nl  Sun Jun  7 00:55:50 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Sat, 6 Jun 2020 16:55:50 +0200
Subject: [TUHS] My BSDcan talk
Message-ID: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl>

> It's another similar to the last two. I've uploaded a version to youtube until the conference has theirs ready. It's a private link, but should work for anybody that has it. Now that I've given my talk it's cool to share more widely.
> The link at the end is wrong. https://github.com/bsdimp/bsdcan2020-demos is the proper link.
> Please let me know what you think.

Watched it & liked it a lot!

I have one nit-pick in the section on early networking: BBN's VAX TCP did not allow the ‘/dev/net/host’ syntax. That particular semantic comes from UoI’s NCP Unix, where the 8-bit host number was encoded in the minor number of character special file ‘host’ - but it did not carry through to the BBN code.

Other systems used something similar. The Chaos network code made namei() break when it recognised the Chaos driver and left the remainder of the path for the driver to fetch & parse. I’m also being told that Greg Chesson experimented with using the given name of a Datakit channel device as the connection string for the switch, but that this approach was abandoned early on.

In my view, exposing the host names through integration in the Unix file name space makes a lot of conceptual sense, but it unfortunately falls down on the practicalities, with the host name set being hard to enumerate (it is large, distributed and not stable - even back then).

A question mark is hard pin-pointing the start of Unix networking to V4 / 1974. Yes, that is the earliest evidence we currently have. However, Sandy Fraser says that Spider came into operation in 1972 and it must have connected to something. Maybe that something was a lab-bench test setup, but it could have been a computer - maybe even one running Unix.

There is another candidate for earliest Unix networking as well. The tech memo’s from Heinz Lycklama include one on the Glance terminal. That memo includes a section on the network used, referencing a 1973 report by D.R. Weller, "A High-Speed I/O Loop Communication System for the DEC PDP-11 Computer”. That computer appears to be an 11/45 running Unix and the loop is not Spider (nor the Pierce loop discussed in 1970/71 BSTJ). I have an off-list question outstanding to better understand this.



From clemc at ccc.com  Sun Jun  7 02:01:08 2020
From: clemc at ccc.com (Clem Cole)
Date: Sat, 6 Jun 2020 12:01:08 -0400
Subject: [TUHS] My BSDcan talk
In-Reply-To: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl>
References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl>
Message-ID: <CAC20D2M_VOM58EuCPnTzMdpshqUW6nQH+fKbj3F6gnZLHTqqdA@mail.gmail.com>

On Sat, Jun 6, 2020 at 10:56 AM Paul Ruizendaal <pnr at planet.nl> wrote:

> The Chaos network code made namei() break when it recognised the Chaos
> driver and left the remainder of the path for the driver to fetch & parse.
>
Yep - cute hack -- Domain (Aegis) did the same trick (same code author most
likely).  The biggest advantage is that open(2) can be used unchanged and
'parameters' are passed in ASCII to the lower code.

You mentioned that BBN Vax code did not use it.  I've forgotten than how
parameters like IP addresses were passed down?  I had thought that Rob had
is also in at least one version of the BBN code, as I remember talking to
him about it at one point; but I have frankly forgotten.   I'm pretty sure
that wnj thought it was an ugly solution; which is why UCB did not try it.

We played with this solution also in the early Masscomp networking code (I
had seen it in the Chaos driver from MIT); but by then the UCB code was
coming out and user code was using sockets, so we just kept with that
interface for code reuse reasons.

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200606/0c5f1c87/attachment.htm>

From erc at pobox.com  Sun Jun  7 06:49:36 2020
From: erc at pobox.com (Ed Carp)
Date: Sat, 6 Jun 2020 15:49:36 -0500
Subject: [TUHS] History of popularity of C
In-Reply-To: <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
Message-ID: <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>

On 5/27/20, Ronald Natalie <ron at ronnatalie.com> wrote:

> The large areas of undefined and unspecified behavior has always been an
> issue in C.   It was somewhat acceptable when you were using it as a direct
> replacement for assembler, but Java and many of other follow-ons endevaored to be more
> portable/rigourous.   Of course, you can write crap code in any language.

"It's not a bug, it's a feature"

C was written when the programmer had to be more rigorous instead of
just letting things slide and having the language do their thinking
for them. I remember being laughed at for using static arrays instead
of malloc() and friends, until people found out that safety-critical
systems were written the same way.

I have C code that was written 35 years ago that's still in
production. Back then, you had to be careful, and you actually had to
think about what you were writing.

We've gotten soft and lazy, and now we're paying for it.

From thomas.paulsen at firemail.de  Sun Jun  7 07:08:43 2020
From: thomas.paulsen at firemail.de (Thomas Paulsen)
Date: Sat, 06 Jun 2020 23:08:43 +0200
Subject: [TUHS] History of popularity of C
In-Reply-To: <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
Message-ID: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>

'C was written when the programmer had to be more rigorous instead of
just letting things slide and having the language do their thinking
for them. '
I fully subscribe to that.
Today the company owners have to pay a lot for programmers having the language do their thinking
for them. The memory hunger of the soa java business services of the company I worked prior to 
retirement, is sheer endless. Arnold once told that there is more demand for C developers
in Israel. I envy you



From lm at mcvoy.com  Sun Jun  7 07:13:28 2020
From: lm at mcvoy.com (Larry McVoy)
Date: Sat, 6 Jun 2020 14:13:28 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
References: <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
Message-ID: <20200606211328.GD1635@mcvoy.com>

I did one stint at a Java shop, Charles Schwab's web group.  No talent,
no architecture, no vision.  Lots of politics and back stabbing and
claiming credit for other people's work.  Totally toxic, hands down the
worst job I've ever had.  I lasted less than 6 months and am surprised
I made it that far.

On Sat, Jun 06, 2020 at 11:08:43PM +0200, Thomas Paulsen wrote:
> 'C was written when the programmer had to be more rigorous instead of
> just letting things slide and having the language do their thinking
> for them. '
> I fully subscribe to that.
> Today the company owners have to pay a lot for programmers having the language do their thinking
> for them. The memory hunger of the soa java business services of the company I worked prior to 
> retirement, is sheer endless. Arnold once told that there is more demand for C developers
> in Israel. I envy you
> 

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

From doug at cs.dartmouth.edu  Sun Jun  7 07:49:23 2020
From: doug at cs.dartmouth.edu (Doug McIlroy)
Date: Sat, 06 Jun 2020 17:49:23 -0400
Subject: [TUHS] History of popularity of C
Message-ID: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>

> Steve Johnson's position paper on optimising compilers may amuse you:
> https://dl.acm.org/doi/abs/10.1145/567532.567542

Indeed. This passage struck a particular chord:

"I contend that the class of applications that depend on, for example, loop
optimization and dead code elimination for their efficient solution is of
modest size, growing smaller, and often very susceptible to expression in
applicative languages where the optimization is built into the individual
applicative operators."

I don't know whether I saw that note at the time, but since then I've
come to believe, particularly in regard to C, that one case of dead-code
elmination should be guaranteed. That case is if(0), where 0 is the
value of a constant expression.

This guarantee would take the place of many--possibly even
most--ifdefs. Every ifdef is an ugly intrusion and a pain to read.
Syntactically it occurs at top level completely out of sync with the
indentation and flow of text. Conversion to if would be a big win.

Doug

From imp at bsdimp.com  Sun Jun  7 07:55:07 2020
From: imp at bsdimp.com (Warner Losh)
Date: Sat, 6 Jun 2020 15:55:07 -0600
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
Message-ID: <CANCZdfrnNhU4wLv=3gBfFBxaXBOg1yyaa6G_++LyzOOfWNxy+Q@mail.gmail.com>

On Sat, Jun 6, 2020 at 3:50 PM Doug McIlroy <doug at cs.dartmouth.edu> wrote:

> > Steve Johnson's position paper on optimising compilers may amuse you:
> > https://dl.acm.org/doi/abs/10.1145/567532.567542
>
> Indeed. This passage struck a particular chord:
>
> "I contend that the class of applications that depend on, for example, loop
> optimization and dead code elimination for their efficient solution is of
> modest size, growing smaller, and often very susceptible to expression in
> applicative languages where the optimization is built into the individual
> applicative operators."
>
> I don't know whether I saw that note at the time, but since then I've
> come to believe, particularly in regard to C, that one case of dead-code
> elmination should be guaranteed. That case is if(0), where 0 is the
> value of a constant expression.
>
> This guarantee would take the place of many--possibly even
> most--ifdefs. Every ifdef is an ugly intrusion and a pain to read.
> Syntactically it occurs at top level completely out of sync with the
> indentation and flow of text. Conversion to if would be a big win.
>

I'd love something like this to work, but the semantic interpretation would
need to also somehow be omitted, otherwise how do you replace

#ifdef AIX
ioct(fd, AIX_SPECIFIC_IOCTL, ...)
#endif

on a HUPX system that doesn't define AIX_SPECIFIC_IOCTL...

I remember hearing that BLISS could cope because it deferred the semantic
interpretation of the identifiers until after a round of dead code
elimination so it didn't need a pre-processor...

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200606/b560e7b0/attachment.htm>

From erc at pobox.com  Sun Jun  7 08:27:19 2020
From: erc at pobox.com (Ed Carp)
Date: Sat, 6 Jun 2020 17:27:19 -0500
Subject: [TUHS] History of popularity of C
In-Reply-To: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
Message-ID: <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>

"Arnold once told that there is more demand for C developers
in Israel. I envy you"

Maybe I ought to move to Israel. Sounds like they have more common
sense there.<div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br />
<table style="border-top: 1px solid #D3D4DE;">
	<tr>
        <td style="width: 55px; padding-top: 13px;"><a
href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon"
target="_blank"><img
src="https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif"
alt="" width="46" height="29" style="width: 46px; height: 29px;"
/></a></td>
		<td style="width: 470px; padding-top: 12px; color: #41424e;
font-size: 13px; font-family: Arial, Helvetica, sans-serif;
line-height: 18px;">Virus-free. <a
href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link"
target="_blank" style="color: #4453ea;">www.avast.com</a>
		</td>
	</tr>
</table><a href="#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1"
height="1"></a></div>

On 6/6/20, Thomas Paulsen <thomas.paulsen at firemail.de> wrote:
> 'C was written when the programmer had to be more rigorous instead of
> just letting things slide and having the language do their thinking
> for them. '
> I fully subscribe to that.
> Today the company owners have to pay a lot for programmers having the
> language do their thinking
> for them. The memory hunger of the soa java business services of the company
> I worked prior to
> retirement, is sheer endless. Arnold once told that there is more demand for
> C developers
> in Israel. I envy you
>
>
>

From coppero1237 at gmail.com  Sun Jun  7 09:14:55 2020
From: coppero1237 at gmail.com (Tyler Adams)
Date: Sun, 7 Jun 2020 02:14:55 +0300
Subject: [TUHS] History of popularity of C
In-Reply-To: <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
Message-ID: <CAEuQd1CP5JpJELw=H_HQY0K163mBxO5=E+tD+5TNKYgLdX_pNg@mail.gmail.com>

> "C was written when the programmer had to be more rigorous instead of
> just letting things slide and having the language do their thinking
> for them"

True, but a wise man once said "let the computer do the dirty work".

 Tyler


On Sun, Jun 7, 2020 at 1:28 AM Ed Carp <erc at pobox.com> wrote:

> "Arnold once told that there is more demand for C developers
> in Israel. I envy you"
>
> Maybe I ought to move to Israel. Sounds like they have more common
> sense there.<div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br />
> <table style="border-top: 1px solid #D3D4DE;">
>         <tr>
>         <td style="width: 55px; padding-top: 13px;"><a
> href="
> https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon
> "
> target="_blank"><img
> src="
> https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif
> "
> alt="" width="46" height="29" style="width: 46px; height: 29px;"
> /></a></td>
>                 <td style="width: 470px; padding-top: 12px; color: #41424e;
> font-size: 13px; font-family: Arial, Helvetica, sans-serif;
> line-height: 18px;">Virus-free. <a
> href="
> https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link
> "
> target="_blank" style="color: #4453ea;">www.avast.com</a>
>                 </td>
>         </tr>
> </table><a href="#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1"
> height="1"></a></div>
>
> On 6/6/20, Thomas Paulsen <thomas.paulsen at firemail.de> wrote:
> > 'C was written when the programmer had to be more rigorous instead of
> > just letting things slide and having the language do their thinking
> > for them. '
> > I fully subscribe to that.
> > Today the company owners have to pay a lot for programmers having the
> > language do their thinking
> > for them. The memory hunger of the soa java business services of the
> company
> > I worked prior to
> > retirement, is sheer endless. Arnold once told that there is more demand
> for
> > C developers
> > in Israel. I envy you
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200607/2c85b46c/attachment.htm>

From bakul at iitbombay.org  Sun Jun  7 09:31:42 2020
From: bakul at iitbombay.org (Bakul Shah)
Date: Sat, 6 Jun 2020 16:31:42 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
Message-ID: <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org>

On Jun 6, 2020, at 1:49 PM, Ed Carp <erc at pobox.com> wrote:
> 
> On 5/27/20, Ronald Natalie <ron at ronnatalie.com> wrote:
> 
>> The large areas of undefined and unspecified behavior has always been an
>> issue in C.   It was somewhat acceptable when you were using it as a direct
>> replacement for assembler, but Java and many of other follow-ons endevaored to be more
>> portable/rigourous.   Of course, you can write crap code in any language.
> 
> "It's not a bug, it's a feature"

A snippet of a recent comp.arch post by someone (the subject was C and safety):

	What you call "misfeatures", some other people call "features".  If you
	expect people to take you and your opinions seriously, you'll get on
	better if you stop mocking other opinions. I've written several times
	why undefined behaviour lets me write better and safer code, as well as
	more efficient code.  If you remain determinedly unconvinced, at least
	agree to disagree without sounding childish about it.

From woods at robohack.ca  Sun Jun  7 10:12:35 2020
From: woods at robohack.ca (Greg A. Woods)
Date: Sat, 06 Jun 2020 17:12:35 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org>
Message-ID: <m1jhivR-0036tPC@more.local>

At Sat, 6 Jun 2020 16:31:42 -0700, Bakul Shah <bakul at iitbombay.org> wrote:
Subject: Re: [TUHS] History of popularity of C
>
> On Jun 6, 2020, at 1:49 PM, Ed Carp <erc at pobox.com> wrote:
> >
> > On 5/27/20, Ronald Natalie <ron at ronnatalie.com> wrote:
> >
> >> The large areas of undefined and unspecified behavior has always
> >> been an issue in C.  It was somewhat acceptable when you were using
> >> it as a direct replacement for assembler, but Java and many of
> >> other follow-ons endevaored to be more portable/rigourous.  Of
> >> course, you can write crap code in any language.
> >
> > "It's not a bug, it's a feature"
>
> A snippet of a recent comp.arch post by someone (the subject was C and
> safety):
>
> 	What you call "misfeatures", some other people call "features".
> 	If you expect people to take you and your opinions seriously,
> 	you'll get on better if you stop mocking other opinions. I've
> 	written several times why undefined behaviour lets me write
> 	better and safer code, as well as more efficient code.  If you
> 	remain determinedly unconvinced, at least agree to disagree
> 	without sounding childish about it.

Heh.

W.r.t. efficiency, well undefined behaviour does allow the compiler to
turn their code, or anyone's else's code, into more "efficient" code if
they happen to (accidentally or otherwise) trip over undefined
behaviour.

However I don't think it can be argued in any valid way that "undefined
behaviour" can ever lead to "better and safer" code, in any way, or from
any viewing angle, whatsoever.

"Undefined behaviour" just means that the language definition is somehow
adversely compromised in such a way that it is impossible to prevent the
programmer from writing compilable and executable code that will always
produce some well defined behaviour in all standards-compliant
implementations.  I.e. the language allows that there are ways to write
syntactically correct code that cannot be guaranteed to do anything
particular whatsoever in _all_ standards-compatible implementations.

We can argue until the cows come home whether "undefined behaviour" is a
"necessary" part of the language definition (e.g. to keep the language
implementable, or backward-compatible, or whatever), but I don't see how
any valid argument can ever be made for it being a "good" and "useful"
thing from the perspective of a programmer using the language.

Undefined behaviours are black holes for which the language standard
offers no real guidance nor maps for safe passage other than the stern
warning to avoid them as best as possible.  Perhaps it is such
scare-mongering that the author above justifies as their influence to
write "better and safer" code, but that's no good argument for having
such pits of despair in the language definition in the first place.  If
we were arguing theology then I would say the bible we call the "C
Standard" is actually actively trying to trap its followers into
committing sins.

Luckily the real world of C is made of actual implementations, and they
are free to either offer definitions for how various (ab)uses of the
language will work, or to maintain the black holes of mystery that we
must try to avoid, or even sometimes to give us the choice in how they
will treat our code.  As programmers we should try to choose which
implementation(s) we use, and how we control _their_ behaviour, while at
the same time still doing our best to avoid giving them the rope to hang
us with.

--
					Greg A. Woods <gwoods at acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods at robohack.ca>
Planix, Inc. <woods at planix.com>     Avoncote Farms <woods at avoncote.ca>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP Digital Signature
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200606/c7ddac8d/attachment.sig>

From andreww591 at gmail.com  Sun Jun  7 15:15:42 2020
From: andreww591 at gmail.com (Andrew Warkentin)
Date: Sat, 6 Jun 2020 23:15:42 -0600
Subject: [TUHS] My BSDcan talk
In-Reply-To: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl>
References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl>
Message-ID: <CAD-qYGrPe_sYvaCPDCX=YkXBJEAZxx-q7rw6CZ8g=t4dFMu5RQ@mail.gmail.com>

On 6/6/20, Paul Ruizendaal <pnr at planet.nl> wrote:
>
> In my view, exposing the host names through integration in the Unix file
> name space makes a lot of conceptual sense, but it unfortunately falls down
> on the practicalities, with the host name set being hard to enumerate (it is
> large, distributed and not stable - even back then).
>
With a proper dynamic VFS architecture, there is no reason why a
resolver with a filesystem API has to bother supporting enumeration at
all. All it needs to be able to do is respond to open() and stat()
calls, returning ENOENT when resolution fails.

From arnold at skeeve.com  Sun Jun  7 15:57:46 2020
From: arnold at skeeve.com (arnold at skeeve.com)
Date: Sat, 06 Jun 2020 23:57:46 -0600
Subject: [TUHS] History of popularity of C
In-Reply-To: <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
Message-ID: <202006070557.0575vkwU011918@freefriends.org>

Ed Carp <erc at pobox.com> wrote:

> "Arnold once told that there is more demand for C developers
> in Israel. I envy you"

The market in Israel for software developers is VERY hot.

Based entirely on the emails I get from Linked-In about jobs that may
interest me, there's some C, but a lot more C++, both Windows and Linux.
Also a lot of Python.

> Maybe I ought to move to Israel.

Moving here isn't a trivial decision, especially if you don't speak
any Hebrew.  Off-topic. Sorry.

> Sounds like they have more common sense there.

We do, but the strong influence of western (US) culture is eroding it,
which is saddening and frustrating.  This definitely off-topic.

Arnold

From akosela at andykosela.com  Sun Jun  7 19:22:21 2020
From: akosela at andykosela.com (Andy Kosela)
Date: Sun, 7 Jun 2020 11:22:21 +0200
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006070557.0575vkwU011918@freefriends.org>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
Message-ID: <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>

On 6/7/20, arnold at skeeve.com <arnold at skeeve.com> wrote:
> Ed Carp <erc at pobox.com> wrote:
>
>> "Arnold once told that there is more demand for C developers
>> in Israel. I envy you"
>
> The market in Israel for software developers is VERY hot.
>
> Based entirely on the emails I get from Linked-In about jobs that may
> interest me, there's some C, but a lot more C++, both Windows and Linux.
> Also a lot of Python.

Seriously, is anyone still doing any real development in C besides
kernel programming and embedded world??  Maybe I was living under a
rock, but I always had an impression that the industry moved to C++ in
the late 90s and stayed with it ever since.

The last bastion of C was open source Linux/*BSD programming but I
remember the time when C was a truly universal programming language
used for _everything_ including games (e.g. Doom).  Maybe I just miss
the 90s.

--Andy

From erc at pobox.com  Sun Jun  7 19:39:28 2020
From: erc at pobox.com (Ed Carp)
Date: Sun, 7 Jun 2020 04:39:28 -0500
Subject: [TUHS] History of popularity of C
In-Reply-To: <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
Message-ID: <CACYmRNAyUvuorWGPRWr264KHmJuMqHHWs+eCCFd_RTd3nm2+-A@mail.gmail.com>

On 6/7/20, Andy Kosela <akosela at andykosela.com> wrote:

> Seriously, is anyone still doing any real development in C besides
> kernel programming and embedded world??  Maybe I was living under a
> rock, but I always had an impression that the industry moved to C++ in
> the late 90s and stayed with it ever since.

Absolutely. C++ isn't the panacea that it's made out to be, it's not
"superior" to C++, it's just C with other useful stuff bolted on, but
it didn't make C obsolete, not by a long shot. Some find the operator
overloading and classes to be a lot more confusing than just function
calls and such, without adding a lot to the language itself. Others,
of course, feel differently. It's more of a religious discussion than
anything, like debating the merits of vi vs. emacs - a pointless
discussion, since you're not going to change anyone else's mind
anyway.

Lots of C still being written out there. My most recent project was
writing a crypto library in C. Writing it was the hard part -
validating it was something else entirely.

One of the nicest things about C++ is that you can write your code
entirely in C and the C++ compiler will compile it, no problem.

From brantley at coraid.com  Sun Jun  7 20:02:23 2020
From: brantley at coraid.com (Brantley Coile)
Date: Sun, 7 Jun 2020 10:02:23 +0000
Subject: [TUHS] History of popularity of C
In-Reply-To: <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
Message-ID: <7563F472-96B8-4CF8-9176-404FB9761508@coraid.com>

This might register low on the useful information index, but I decided a few years back as VC Coraid was coming apart and New Coraid was being resurrected under the auspices of SouthSuite Inc., that I would have a C mono-culture and use the language for everything. Specifically no JavaScript and no Python.

Our website is written in C. Webpages generated on the server side using C. Our evolving ERP is written in C. Our subscription system is written in C. Our test systems are written in it. All work quite well.

The focus on C was for three main reasons. First, our products are infrastructure products and are meant to be simple, fast, and affordable. Forty-two years of experience plus C gives us the ability to squeeze all the performance possible from any hardware platform. We "see" the instructions our code will generate. C is a great choice for that. (As would have been Oberon, but that's another discussion.)

Second, if instead of having a set of complex languages, each with its own adherents, using a single language removes all the distracting and divisive language wars having multiple complex languages create. Little language like AWK and the shell script are fine. It's the more complex ones that divide people.

Lastly, a single, powerful, simple (on the other side of complexity) language that a single person can maintain is essential to our Software Atelier model of doing business. Like the workshops of the renaissance, we have to understand and work on all our own tools. I use Ken's C compiler under Plan 9. It weights in at a light, 20K lines of code.

As I said, I'm not sure how useful this data point is for you. Over the last thirty years I carefully chose my foot falls through the software swamp to avoid getting sucked under by one of the quagmires of complexity.

  Brantley


> On Jun 7, 2020, at 5:22 AM, Andy Kosela <akosela at andykosela.com> wrote:
> 
> On 6/7/20, arnold at skeeve.com <arnold at skeeve.com> wrote:
>> Ed Carp <erc at pobox.com> wrote:
>> 
>>> "Arnold once told that there is more demand for C developers
>>> in Israel. I envy you"
>> 
>> The market in Israel for software developers is VERY hot.
>> 
>> Based entirely on the emails I get from Linked-In about jobs that may
>> interest me, there's some C, but a lot more C++, both Windows and Linux.
>> Also a lot of Python.
> 
> Seriously, is anyone still doing any real development in C besides
> kernel programming and embedded world??  Maybe I was living under a
> rock, but I always had an impression that the industry moved to C++ in
> the late 90s and stayed with it ever since.
> 
> The last bastion of C was open source Linux/*BSD programming but I
> remember the time when C was a truly universal programming language
> used for _everything_ including games (e.g. Doom).  Maybe I just miss
> the 90s.
> 
> --Andy


From emu at e-bbes.com  Sun Jun  7 21:04:23 2020
From: emu at e-bbes.com (emanuel stiebler)
Date: Sun, 7 Jun 2020 07:04:23 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
References: <CAEuQd1B8gH-Lu22HKj9pn6JVXNVVYscAnL4TSVDY03k2ORy2qw@mail.gmail.com>
 <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
Message-ID: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com>

On 2020-06-06 16:49, Ed Carp wrote:

> C was written when the programmer had to be more rigorous instead of
> just letting things slide and having the language do their thinking
> for them. I remember being laughed at for using static arrays instead
> of malloc() and friends, until people found out that safety-critical
> systems were written the same way.

Still avoiding malloc and friends in safety critical systems, for a good
reason ...

From thomas.paulsen at firemail.de  Sun Jun  7 21:30:03 2020
From: thomas.paulsen at firemail.de (Thomas Paulsen)
Date: Sun, 07 Jun 2020 13:30:03 +0200
Subject: [TUHS] History of popularity of C
In-Reply-To: <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
References: <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
Message-ID: <5d6f1074119107ee81ff119c7a9463f6@firemail.de>


>Seriously, is anyone still doing any real development in C besides
>kernel programming and embedded world??  Maybe I was living under a
>rock, but I always had an impression that the industry moved to C++ in
>the late 90s and stayed with it ever since.

I repeat myself: C never was intended for writing business applications. It was made for OS 
development in the sense of a portable assembler replacement. It also has found itself as the 
main devel language for fundamental services like db engines etc.



From thomas.paulsen at firemail.de  Sun Jun  7 21:33:21 2020
From: thomas.paulsen at firemail.de (Thomas Paulsen)
Date: Sun, 07 Jun 2020 13:33:21 +0200
Subject: [TUHS] History of popularity of C
In-Reply-To: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com>
References: <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com>
Message-ID: <065f8b450e1884b5f4ab97d64d7ab05c@firemail.de>

>Still avoiding malloc and friends in safety critical systems, for a good reason ...

all modern languages allowing dynamical memory allocation one way or another. 




From pnr at planet.nl  Sun Jun  7 21:41:41 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Sun, 7 Jun 2020 13:41:41 +0200
Subject: [TUHS] My BSDcan talk
In-Reply-To: <CAC20D2M_VOM58EuCPnTzMdpshqUW6nQH+fKbj3F6gnZLHTqqdA@mail.gmail.com>
References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl>
 <CAC20D2M_VOM58EuCPnTzMdpshqUW6nQH+fKbj3F6gnZLHTqqdA@mail.gmail.com>
Message-ID: <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl>


> On 6 Jun 2020, at 18:01, Clem Cole <clemc at ccc.com> wrote:
> 
> You mentioned that BBN Vax code did not use it.  I've forgotten than how parameters like IP addresses were passed down?  I had thought that Rob had it also in at least one version of the BBN code, as I remember talking to him about it at one point; but I have frankly forgotten.

In the surviving source code, BBN VAX TCP re-purposed the ‘mode’ parameter of open() to be a pointer to a block with connection data (“struct con”). Details are here:
https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P



From pnr at planet.nl  Sun Jun  7 22:16:20 2020
From: pnr at planet.nl (Paul Ruizendaal)
Date: Sun, 7 Jun 2020 14:16:20 +0200
Subject: [TUHS] network name space (was: My BSDcon talk)
Message-ID: <42C43756-9800-41AC-88FF-4186510F83D4@planet.nl>


> On 6/6/20, Paul Ruizendaal <pnr at planet.nl
> > wrote:
> >
> > In my view, exposing the host names through integration in the Unix file
> > name space makes a lot of conceptual sense, but it unfortunately falls down
> > on the practicalities, with the host name set being hard to enumerate (it is
> > large, distributed and not stable - even back then).
> >
> With a proper dynamic VFS architecture, there is no reason why a
> resolver with a filesystem API has to bother supporting enumeration at
> all. All it needs to be able to do is respond to open() and stat()
> calls, returning ENOENT when resolution fails.

That is an intriguing thought. In Research Unix terms it would be a virtual directory that was not readable or writable, but still explorable (i.e. only the x bit set).

Maybe enumeration is only impractical for the networks that were designed to be ‘large’, such as Arpanet and Datakit. It would have been feasible for contemporary networks that were designed to be local only, such as Chaosnet or ARCnet.

A half-way house would be to only enumerate the local network and leaving everything else merely explorable. That is conceptually very messy, though.



From clemc at ccc.com  Mon Jun  8 01:26:45 2020
From: clemc at ccc.com (Clem Cole)
Date: Sun, 7 Jun 2020 11:26:45 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
Message-ID: <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>

On Sun, Jun 7, 2020 at 5:23 AM Andy Kosela <akosela at andykosela.com> wrote:

> Seriously, is anyone still doing any real development in C besides
> kernel programming and embedded world??  Maybe I was living under a
> rock, but I always had an impression that the industry moved to C++ in
> the late 90s and stayed with it ever since.
>
Hardly,  in my 45+ years, I have seen way more C projects than C++.  The C
projects tended to last longer, have a more profound impact and many are
still being developed.   I've been at Intel for the last qtr of my career and
without a doubt, C is the #1 language in use internally, with C++ probably
#2.   Most of our work is actually in user space, although obviously we do
a great deal of low-level work.  The Intel compilers are a mix of both
languages.  That said, as we move to more LLVM work (we actually have the
largest number of paid LLVM developers ).

For your amusement check out:  https://www.archer.ac.uk/status/codes/
Archer is a large HPC site in the UK.  Many supercomputer centers off
similar stats, but I often point to Archer because it easy to get an
understanding of what programming languages are used for codes that are
actually used in day-to-day production.

That said, I personally am the most excited about Go theses day, but I'm
also thinking Rust looks pretty interesting, but my experience with both
compared to C is extremely nominal.  Neither language is used for anything
in production in our world at this point.

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200607/26d3eeb9/attachment.htm>

From lm at mcvoy.com  Mon Jun  8 01:52:39 2020
From: lm at mcvoy.com (Larry McVoy)
Date: Sun, 7 Jun 2020 08:52:39 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
References: <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
 <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
Message-ID: <20200607155239.GL1635@mcvoy.com>

On Sun, Jun 07, 2020 at 11:26:45AM -0400, Clem Cole wrote:
> That said, I personally am the most excited about Go theses day, but I'm
> also thinking Rust looks pretty interesting, but my experience with both
> compared to C is extremely nominal.  Neither language is used for anything
> in production in our world at this point.

If I had to move to a modern language it would be Go.  I looked at Rust
and barfed.

But if I had the juice, I'd just evolve C.   Make a C+ that isn't object
oriented but does have some of the stuff that modern languages have.
I'd pretty much take http://little-lang.org and evolve C to that.

From bakul at iitbombay.org  Mon Jun  8 03:26:48 2020
From: bakul at iitbombay.org (Bakul Shah)
Date: Sun, 7 Jun 2020 10:26:48 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
 <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
Message-ID: <9FB6B5CB-A8A2-4EA7-8613-474C2B376FEC@iitbombay.org>

On Jun 7, 2020, at 8:26 AM, Clem Cole <clemc at ccc.com> wrote:
> 
> That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal.  Neither language is used for anything in production in our world at this point.

People who use Rust seem to really like it but so far I have not done
anything in it. I will take another look if I do any bare metal coding.
In contrast I use Go for almost all my own coding now. I am not entirely
happy with it but the ecosystem around it is great. And the really nice
thing about it is that not only I can cross-compile programs but also,
I believe it is the only compiled language (much more so than C) where
a lot of stuff just works on on plan9.

From bakul at iitbombay.org  Mon Jun  8 03:35:29 2020
From: bakul at iitbombay.org (Bakul Shah)
Date: Sun, 7 Jun 2020 10:35:29 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
 <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
Message-ID: <B9FA8D03-F88E-45AF-AD98-AFCDED0B316A@iitbombay.org>

On Jun 7, 2020, at 8:26 AM, Clem Cole <clemc at ccc.com> wrote:
> 
> That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal.  Neither language is used for anything in production in our world at this point.

People who use Rust seem to really like it but so far I have not done
anything in it. I will take another look if I do any bare metal coding.
In contrast I use Go for almost all my own coding now. I am not entirely
happy with it but the ecosystem around it is great. And the really nice
thing about it is that not only I can cross-compile programs but also,
I believe it is the only compiled language (much more so than C) where
a lot of stuff just works on on plan9.

From cym224 at gmail.com  Mon Jun  8 04:50:34 2020
From: cym224 at gmail.com (Nemo Nusquam)
Date: Sun, 7 Jun 2020 14:50:34 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au>
 <m1jcHQv-0036tPC@more.local>
 <CAC20D2NhWp8V88+7KFaRdYPtn=YrJBfWxUu9OM4bOu8Fp_7KEA@mail.gmail.com>
 <alpine.BSF.2.21.9999.2005261411560.79423@aneurin.horsfall.org>
 <CAC20D2O1QZamBa_7V7=QVKHy4BHdqNQmj1iuSJta=MpyuBjudw@mail.gmail.com>
 <m1jdfb4-0036tRC@more.local> <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de>
 <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
 <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
Message-ID: <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com>

On 06/07/20 11:26, Clem Cole wrote (in part):
> That said, I personally am the most excited about Go theses day, but 
> I'm also thinking Rust looks pretty interesting, but my experience 
> with both compared to C is extremely nominal.  Neither language is 
> used for anything in production in our world at this point.

They seem to be used in some worlds: https://blog.golang.org/10years and 
https://www.rust-lang.org/production

N.

From imp at bsdimp.com  Mon Jun  8 06:03:46 2020
From: imp at bsdimp.com (Warner Losh)
Date: Sun, 7 Jun 2020 14:03:46 -0600
Subject: [TUHS] My BSDcan talk
In-Reply-To: <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl>
References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl>
 <CAC20D2M_VOM58EuCPnTzMdpshqUW6nQH+fKbj3F6gnZLHTqqdA@mail.gmail.com>
 <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl>
Message-ID: <CANCZdfr=v_fBbWQ1FHcE5QRkerz_uJAzyXY+pwL=9xP-mJszBQ@mail.gmail.com>

On Sun, Jun 7, 2020 at 5:42 AM Paul Ruizendaal <pnr at planet.nl> wrote:

>
> > On 6 Jun 2020, at 18:01, Clem Cole <clemc at ccc.com> wrote:
> >
> > You mentioned that BBN Vax code did not use it.  I've forgotten than how
> parameters like IP addresses were passed down?  I had thought that Rob had
> it also in at least one version of the BBN code, as I remember talking to
> him about it at one point; but I have frankly forgotten.
>
> In the surviving source code, BBN VAX TCP re-purposed the ‘mode’ parameter
> of open() to be a pointer to a block with connection data (“struct con”).
> Details are here:
> https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P


That's... clever? Gross? I don't even know how I'd want to react to
this...  which likely means "too clever for its own good" :)

In any event, I've created a short, 2:00 correction to my talk. I'll have
the conference organizers tack onto the end before they upload it to
youtube.

https://www.youtube.com/watch?v=9uTjyfO6MM8

If people care :)

Thanks everybody for corrections...

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200607/16e8dfad/attachment.htm>

From torek at torek.net  Mon Jun  8 07:15:26 2020
From: torek at torek.net (Chris Torek)
Date: Sun, 07 Jun 2020 14:15:26 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: Your message of "Sun, 07 Jun 2020 14:50:34 -0400."
 <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com>
Message-ID: <202006072115.057LFV6v089953@elf.torek.net>

Yes, both Rust and Go are being used.

Rust has some advantages: like C++, it can compile to very fast
code that does not need a garbage collector but that provides
type-safety.  It also gives you thread-safety through clever
compiler analysis of who "owns" any given variable or data.

The ownership / lifetime-analysis / borrow-checker is quite
complicated and takes a lot of getting-used-to.  I have not
written anything "real" in Rust and had not had time to really
learn it (I was hoping to learn it for real and write code in
it the last few years, but that never actually happened).

Meanwhile, Go is actually a really nice language to use.  It has a
few quirks, but it gives you reasonably-fast-running code that
(because there is a garbage collector) does not require nearly as
much skull-sweat when figuring out who owns memory and how to make
sure it gets released appropriately.  Its built in channels and
goroutine support makes multi-threaded cod, and using all the CPUs
effectively, also much easier.

You pay (sometimes noticeably) for the GC, but the price is not
too bad in less time-critical situations.  The GC has a few short
stop-the-world points but since Go 1.6 or so, it's pretty smooth,
unlike what I remember from 1980s Lisp systems. :-)  (Note: I
started with Go 1.11, so I don't have a lot of history, nor that
much experience in it.  But I do like it.)

Both Go and Rust have build systems.  In Rust, this is a separate
front-end from the compiler proper (rustc): you run "cargo build",
for instance.  In Go, you run "go build", "go test", etc., to
build and invoke unit tests and so on.  Rust has generics (think
C++ templates, except sane) and Go lacks them although there's a
plan for them in Go 2.0.  Rust has, or at least had, rather bad
array support when I last looked at it: you could make an array
out of any type, but with only up to 32 elements.

Chris

From crossd at gmail.com  Mon Jun  8 08:16:36 2020
From: crossd at gmail.com (Dan Cross)
Date: Sun, 7 Jun 2020 18:16:36 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006072115.057LFV6v089953@elf.torek.net>
References: <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com>
 <202006072115.057LFV6v089953@elf.torek.net>
Message-ID: <CAEoi9W4FDH7ahPHNfEdA0kcMO-yQccF-r=hUCNANyKWMtoO3bA@mail.gmail.com>

On Sun, Jun 7, 2020 at 5:37 PM Chris Torek <torek at torek.net> wrote:

> Yes, both Rust and Go are being used.
>

Indeed.  The languages complement each other quite nicely. Disclaimer: I've
been programming in Rust for my day job for almost two years now and I sit
in the office next to the Go tech lead and in close proximity a number of
people working on Go.

Rust has some advantages: like C++, it can compile to very fast
> code that does not need a garbage collector but that provides
> type-safety.  It also gives you thread-safety through clever
> compiler analysis of who "owns" any given variable or data.
>

Safe Rust code is data race free, but not free of race conditions, let
alone thread safe in all ways.

The ownership / lifetime-analysis / borrow-checker is quite
> complicated and takes a lot of getting-used-to.


Once you get used to it, it doesn't feel that complex but yes, it takes a
while to get used to. When we switched our project to it (from C++, at my
behest) our tech lead quipped that Rust has a "near vertical" learning
curve. Suffice it to say we're all quite glad we switched but there was an
adjustment period.

Interestingly, we find it a selling point for attracting new engineers
regardless of the challenges in coming up to speed.

[snip]  Rust has, or at least had, rather bad
> array support when I last looked at it: you could make an array
> out of any type, but with only up to 32 elements.


I'm afraid this is incorrect.  Rust arrays are indexed by a `usize`, which
is basically whatever `size_t` would be in C. Rust arrays in general can be
essentially arbitrarily large (up to limitations imposed by the target
machine, of course).

However, Rust does not support dependent types, most certainly not for
arrays. In other words, an array's size is considered part of its type and
so when specializing traits on arrays, one must do so explicitly for each
supported array size. For practical reasons such implementations are often
limited to a relatively small number of distinct sizes; 32 is a believable
number. Perhaps that's what you're thinking of?

For example, the `Vec<T>` type was recently modified to implement the
`From` trait on arrays, but only on arrays up to size 32. This means that
one can essentially create a dynamic, growable vector seeded from an array,
but such source arrays are limited in how large they can be.

My current biggest frustrations are in assumptions made by the memory
allocator, the magical nature of `box` (the allocation primitive), and the
under-defined memory model.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200607/3982f393/attachment.htm>

From torek at torek.net  Mon Jun  8 08:56:45 2020
From: torek at torek.net (Chris Torek)
Date: Sun, 07 Jun 2020 15:56:45 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: Your message of "Sun, 07 Jun 2020 18:16:36 -0400."
 <CAEoi9W4FDH7ahPHNfEdA0kcMO-yQccF-r=hUCNANyKWMtoO3bA@mail.gmail.com>
Message-ID: <202006072256.057Muo1h090326@elf.torek.net>

>Safe Rust code is data race free, but not free of race
>conditions, let alone thread safe in all ways.

Er, yes.  I mainly wanted to contrast to Go, where you look for
race conditions by building with a flag that enables runtime
checking.  This can only detect races that actually occur,
and if there are paths that would still have races that didn't
occur on your test run, well...

>>[rust array limitations]

>I'm afraid this is incorrect.  Rust arrays are indexed by a
>`usize`, which is basically whatever `size_t` would be in C. Rust
>arrays in general can be essentially arbitrarily large (up to
>limitations imposed by the target machine, of course).

>However, Rust does not support dependent types, most certainly
>not for arrays.  In other words, an array's size is considered
>part of its type and so when specializing traits on arrays, one
>must do so explicitly for each supported array size.  For
>practical reasons such implementations are often limited to a
>relatively small number of distinct sizes; 32 is a believable
>number.  Perhaps that's what you're thinking of?

Ah, yes, that was it.  (As I said, I never really had time to
do anything "real" in Rust.)

Go's slices are nice to use, but slices (and maps, for that matter)
trip people up because they are headers that point to shared
spaces.

Chris

From wkt at tuhs.org  Mon Jun  8 09:14:34 2020
From: wkt at tuhs.org (Warren Toomey)
Date: Mon, 8 Jun 2020 09:14:34 +1000
Subject: [TUHS] Comparative languages
In-Reply-To: <202006072256.057Muo1h090326@elf.torek.net>
References: <CAEoi9W4FDH7ahPHNfEdA0kcMO-yQccF-r=hUCNANyKWMtoO3bA@mail.gmail.com>
 <202006072256.057Muo1h090326@elf.torek.net>
Message-ID: <20200607231434.GA13571@minnie.tuhs.org>

All, as the discussion on C has moved on to other languages, we might
move the followups to COFF.

Thanks, Warren

From bramwyllie at gmail.com  Mon Jun  8 10:24:38 2020
From: bramwyllie at gmail.com (Bram Wyllie)
Date: Sun, 7 Jun 2020 21:24:38 -0300
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006072256.057Muo1h090326@elf.torek.net>
References: <CAEoi9W4FDH7ahPHNfEdA0kcMO-yQccF-r=hUCNANyKWMtoO3bA@mail.gmail.com>
 <202006072256.057Muo1h090326@elf.torek.net>
Message-ID: <CAAYAaAPu=dKnXH82WmECenLNJKSnKNh6aqjFphOoc+NPWxuGhw@mail.gmail.com>

Dependent types aren't needed for sum types though, which is what you'd
normally use for an array that carries its size, correct?

On Sun, Jun 7, 2020 at 7:57 PM Chris Torek <torek at torek.net> wrote:

> >Safe Rust code is data race free, but not free of race
> >conditions, let alone thread safe in all ways.
>
> Er, yes.  I mainly wanted to contrast to Go, where you look for
> race conditions by building with a flag that enables runtime
> checking.  This can only detect races that actually occur,
> and if there are paths that would still have races that didn't
> occur on your test run, well...
>
> >>[rust array limitations]
>
> >I'm afraid this is incorrect.  Rust arrays are indexed by a
> >`usize`, which is basically whatever `size_t` would be in C. Rust
> >arrays in general can be essentially arbitrarily large (up to
> >limitations imposed by the target machine, of course).
>
> >However, Rust does not support dependent types, most certainly
> >not for arrays.  In other words, an array's size is considered
> >part of its type and so when specializing traits on arrays, one
> >must do so explicitly for each supported array size.  For
> >practical reasons such implementations are often limited to a
> >relatively small number of distinct sizes; 32 is a believable
> >number.  Perhaps that's what you're thinking of?
>
> Ah, yes, that was it.  (As I said, I never really had time to
> do anything "real" in Rust.)
>
> Go's slices are nice to use, but slices (and maps, for that matter)
> trip people up because they are headers that point to shared
> spaces.
>
> Chris
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200607/9f8592a5/attachment.htm>

From athornton at gmail.com  Mon Jun  8 11:02:18 2020
From: athornton at gmail.com (Adam Thornton)
Date: Sun, 7 Jun 2020 18:02:18 -0700
Subject: [TUHS] History of popularity of C
In-Reply-To: <20200607155239.GL1635@mcvoy.com>
References: <m1jdfb4-0036tRC@more.local>
 <d6bca9b8fc16b4fb5c34847a52aced59@firemail.de> <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
 <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
 <20200607155239.GL1635@mcvoy.com>
Message-ID: <D4429A85-9C28-4288-BF6F-2F6B9D94C945@gmail.com>



> On Jun 7, 2020, at 8:52 AM, Larry McVoy <lm at mcvoy.com> wrote:
> 
> On Sun, Jun 07, 2020 at 11:26:45AM -0400, Clem Cole wrote:
>> That said, I personally am the most excited about Go theses day, but I'm
>> also thinking Rust looks pretty interesting, but my experience with both
>> compared to C is extremely nominal.  Neither language is used for anything
>> in production in our world at this point.
> 
> If I had to move to a modern language it would be Go.  I looked at Rust
> and barfed.


Several years ago, this was a job talk I gave, based on my experience at the time developing a pretty nifty system that never found traction.

The new job (I got it!) doesn’t use Go, so I’ve grudgingly gone back to Python.  But I stand by most of what I wrote (although I am sure parts of it are outdated and wrong now).

The tl;dr is the title of the talk: 

https://athornton.github.io/go-it-mostly-doesnt-suck

I make the claim that Go *is* pretty much C with 35 years of lessons learned about what did and didn’t work in C, and 35 years of machine time getting cheaper and programmer time getting more expensive.

Adam

From lars at nocrew.org  Mon Jun  8 15:48:03 2020
From: lars at nocrew.org (Lars Brinkhoff)
Date: Mon, 08 Jun 2020 05:48:03 +0000
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006072115.057LFV6v089953@elf.torek.net> (Chris Torek's
 message of "Sun, 07 Jun 2020 14:15:26 -0700")
References: <202006072115.057LFV6v089953@elf.torek.net>
Message-ID: <7w4krmw1kc.fsf@junk.nocrew.org>

Chris Torek wrote:
> You pay (sometimes noticeably) for the GC, but the price is not
> too bad in less time-critical situations.  The GC has a few short
> stop-the-world points but since Go 1.6 or so, it's pretty smooth,
> unlike what I remember from 1980s Lisp systems. :-)

I'm guessing those 1980s Lisp systems would also be pretty smooth on
2020s hardware.

From thomas.paulsen at firemail.de  Mon Jun  8 18:04:41 2020
From: thomas.paulsen at firemail.de (Thomas Paulsen)
Date: Mon, 08 Jun 2020 10:04:41 +0200
Subject: [TUHS] History of popularity of C
In-Reply-To: <D4429A85-9C28-4288-BF6F-2F6B9D94C945@gmail.com>
References: <m1jdiBT-0036tRC@more.local>
 <B98F991C-667C-45D5-ADFD-BB71522FEA24@ronnatalie.com>
 <CACYmRNCOjopZm4YJEjBvCB6WGsV27SMkws-offmZ2VSR0gYDGw@mail.gmail.com>
 <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de>
 <CACYmRNCYidLpfXZ1VOQVwZg4c89+yP6388MSOKhaiSvtSxHtTQ@mail.gmail.com>
 <202006070557.0575vkwU011918@freefriends.org>
 <CALMnNGibSro7zv1t1f+zvxoKaaHMe1z9D1nQ85HwBnfa4psEng@mail.gmail.com>
 <CAC20D2NLY402O_0db0jcvEHTveXt1qiveYw1ir-tN2KsRvNB-g@mail.gmail.com>
 <20200607155239.GL1635@mcvoy.com>
 <D4429A85-9C28-4288-BF6F-2F6B9D94C945@gmail.com>
Message-ID: <7a76a39c7a0ceaa53a58a85fea3d6be2@firemail.de>

>I make the claim that Go *is* pretty much C with 35 years of lessons learned
>about what did and didn’t work in C, and 35 years of machine time getting
>cheaper and programmer time getting more expensive.
see Rob Pike 'Notes on Programming in C' freely available in the internet at: https://www.lysator.liu.se/c/pikestyle.html






From clemc at ccc.com  Mon Jun  8 23:10:37 2020
From: clemc at ccc.com (Clem Cole)
Date: Mon, 8 Jun 2020 09:10:37 -0400
Subject: [TUHS] UNESCO's Software Heritage Foundation
Message-ID: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>

I'm seeding this URL to TUHS as one would expect them to be interested in
the work from Warren and friends. FWIW: I tried to browse their archives
and was not impressed (I couldn't find anything).

https://www.softwareheritage.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200608/d8c96b1f/attachment.htm>

From dfawcus+lists-tuhs at employees.org  Mon Jun  8 23:56:19 2020
From: dfawcus+lists-tuhs at employees.org (Derek Fawcus)
Date: Mon, 8 Jun 2020 14:56:19 +0100
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
Message-ID: <20200608135619.GA22471@clarinet.employees.org>

On Sat, Jun 06, 2020 at 05:49:23PM -0400, Doug McIlroy wrote:
> Syntactically it occurs at top level completely out of sync with the
> indentation and flow of text. Conversion to if would be a big win.

Well, doesn't that rather depend upon how one indents it?
As I recall, that freedom came with ANSI C.

But yeah, I'd prefer the if(0) type of elimination.

DF

$ cc yes-no.c; ./a.out; cc -DYES yes-no.c; ./a.out; cat yes-no.c
No
Yes
#include <stdio.h>

int main(void)
{
	#ifdef YES
		puts("Yes");
	#else
		puts("No");
	#endif

	return 0;
}



From aek at bitsavers.org  Tue Jun  9 00:44:55 2020
From: aek at bitsavers.org (Al Kossow)
Date: Mon, 8 Jun 2020 07:44:55 -0700
Subject: [TUHS] UNESCO's Software Heritage Foundation
In-Reply-To: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
References: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
Message-ID: <e5b4f2cd-1162-62f3-8ee4-035640818a35@bitsavers.org>

On 6/8/20 6:10 AM, Clem Cole wrote:
> I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their 
> archives and was not impressed (I couldn't find anything).
> 
> https://www.softwareheritage.org/

Don't diss them too badly. They are good folks,
especially with how they are dealing with archiving and deduping almost-identical
source trees.

They stop by CHM every few years.





From clemc at ccc.com  Tue Jun  9 01:14:13 2020
From: clemc at ccc.com (Clem Cole)
Date: Mon, 8 Jun 2020 11:14:13 -0400
Subject: [TUHS] UNESCO's Software Heritage Foundation
In-Reply-To: <e5b4f2cd-1162-62f3-8ee4-035640818a35@bitsavers.org>
References: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
 <e5b4f2cd-1162-62f3-8ee4-035640818a35@bitsavers.org>
Message-ID: <CAC20D2M5DjObwhNdBu7pmwAsM9Wx=LARJV1toWS7CwFWJLnswA@mail.gmail.com>

Al, I'm not diss them or their efforts, just I wanted to know who they
were.
This is the first I had heard of them,  I am, however, dissing the web site
which I could not figure out how to use.

On Mon, Jun 8, 2020 at 10:51 AM Al Kossow <aek at bitsavers.org> wrote:

> On 6/8/20 6:10 AM, Clem Cole wrote:
> > I'm seeding this URL to TUHS as one would expect them to be interested
> in the work from Warren and friends. FWIW: I tried to browse their
> > archives and was not impressed (I couldn't find anything).
> >
> > https://www.softwareheritage.org/
>
> Don't diss them too badly. They are good folks,
> especially with how they are dealing with archiving and deduping
> almost-identical
> source trees.
>
> They stop by CHM every few years.
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200608/4d09bc86/attachment.htm>

From david at kdbarto.org  Tue Jun  9 00:58:30 2020
From: david at kdbarto.org (David Barto)
Date: Mon, 8 Jun 2020 07:58:30 -0700
Subject: [TUHS] UNESCO's Software Heritage Foundation
In-Reply-To: <e5b4f2cd-1162-62f3-8ee4-035640818a35@bitsavers.org>
References: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
 <e5b4f2cd-1162-62f3-8ee4-035640818a35@bitsavers.org>
Message-ID: <2A179FD5-E1DA-459A-8B06-8025F5238ACF@kdbarto.org>

As a quick look, I entered ‘unix’ to the search bar. 8^)

In the list was a UNIX_V6PP. A quick browse it looked like a (not very good IMO) c++ rewrite of the v6 kernel for the x86 processor.

	David

https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/lonway/UNIX_V6PP&path=UNIX_V6%2B%2B

> On Jun 8, 2020, at 7:44 AM, Al Kossow <aek at bitsavers.org> wrote:
> 
> On 6/8/20 6:10 AM, Clem Cole wrote:
>> I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their archives and was not impressed (I couldn't find anything).
>> https://www.softwareheritage.org/
> 
> Don't diss them too badly. They are good folks,
> especially with how they are dealing with archiving and deduping almost-identical
> source trees.
> 
> They stop by CHM every few years.
> 
> 
> 
> 


From rich.salz at gmail.com  Tue Jun  9 01:20:58 2020
From: rich.salz at gmail.com (Richard Salz)
Date: Mon, 8 Jun 2020 11:20:58 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <20200608135619.GA22471@clarinet.employees.org>
References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
 <20200608135619.GA22471@clarinet.employees.org>
Message-ID: <CAFH29tqo8J4G0R=3saw0742_nseTuyrNxKqtBucySuxVA9P1ig@mail.gmail.com>

> But yeah, I'd prefer the if(0) type of elimination.
>

Some parts of OpenSSL do this, and it seems to work well across a pretty
wide variety of platforms/toolchains.

A really important benefit is that you get code syntax-checked as opposed
to unused #ifdef branches rotting.  A minor benefit is you don't have ifdef
spaghetti around local variables, to avoid "xxx is not used" warnings.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200608/50d063a7/attachment.htm>

From crossd at gmail.com  Tue Jun  9 01:30:50 2020
From: crossd at gmail.com (Dan Cross)
Date: Mon, 8 Jun 2020 11:30:50 -0400
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
Message-ID: <CAEoi9W5KNncL=owWEM3JRcwUPL+HFzh7ieuJZ3hWGWUOT6KY3g@mail.gmail.com>

On Sat, Jun 6, 2020 at 5:50 PM Doug McIlroy <doug at cs.dartmouth.edu> wrote:

> > Steve Johnson's position paper on optimising compilers may amuse you:
> > https://dl.acm.org/doi/abs/10.1145/567532.567542
>
> Indeed. This passage struck a particular chord:
>
> "I contend that the class of applications that depend on, for example, loop
> optimization and dead code elimination for their efficient solution is of
> modest size, growing smaller, and often very susceptible to expression in
> applicative languages where the optimization is built into the individual
> applicative operators."
>
> I don't know whether I saw that note at the time, but since then I've
> come to believe, particularly in regard to C, that one case of dead-code
> elmination should be guaranteed. That case is if(0), where 0 is the
> value of a constant expression.
>
> This guarantee would take the place of many--possibly even
> most--ifdefs. Every ifdef is an ugly intrusion and a pain to read.
> Syntactically it occurs at top level completely out of sync with the
> indentation and flow of text. Conversion to if would be a big win.
>

This came up on this list back in 2017 and I posted something about this at
the time:

https://minnie.tuhs.org/pipermail/tuhs/2017-January/009325.html

If the language had been extended to support a "compile-type conditional"
syntax ("if #(LINUX)", for example) it would have obviated the need for
most preprocessor conditionals. As long as, syntactically, the compiler
could tokenize a block in absence of e.g. type definitions, dead-code
elimination could do the job.

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20200608/971f880f/attachment.htm>

From lars at nocrew.org  Tue Jun  9 02:06:00 2020
From: lars at nocrew.org (Lars Brinkhoff)
Date: Mon, 08 Jun 2020 16:06:00 +0000
Subject: [TUHS] UNESCO's Software Heritage Foundation
In-Reply-To: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
 (Clem Cole's message of "Mon, 8 Jun 2020 09:10:37 -0400")
References: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
Message-ID: <7wpna9v8yf.fsf@junk.nocrew.org>

Clem Cole wrote:
> I'm seeding this URL to TUHS as one would expect them to be interested
> in the work from Warren and friends. FWIW: I tried to browse their
> archives and was not impressed (I couldn't find anything).

They seem to have scraped GitHub, so they backed up everything I put
there.  Thanks guys.

From aek at bitsavers.org  Tue Jun  9 02:20:21 2020
From: aek at bitsavers.org (Al Kossow)
Date: Mon, 8 Jun 2020 09:20:21 -0700
Subject: [TUHS] UNESCO's Software Heritage Foundation
In-Reply-To: <7wpna9v8yf.fsf@junk.nocrew.org>
References: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
 <7wpna9v8yf.fsf@junk.nocrew.org>
Message-ID: <c45deb58-69b6-f3f4-8189-9015a39ffa80@bitsavers.org>

On 6/8/20 9:06 AM, Lars Brinkhoff wrote:
> Clem Cole wrote:
>> I'm seeding this URL to TUHS as one would expect them to be interested
>> in the work from Warren and friends. FWIW: I tried to browse their
>> archives and was not impressed (I couldn't find anything).
> 
> They seem to have scraped GitHub, so they backed up everything I put
> there.  Thanks guys.
> 

They are trying to archive and de-dup EVERYTHING that is open-source.

Unix is more difficult, because of the encumbrances.


https://www.softwareheritage.org/2019/06/06/announcing-the-software-heritage-graph-dataset/


From dot at dotat.at  Tue Jun  9 02:32:04 2020
From: dot at dotat.at (Tony Finch)
Date: Mon, 8 Jun 2020 17:32:04 +0100
Subject: [TUHS] History of popularity of C
In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU>
Message-ID: <alpine.DEB.2.20.2006081716410.590@grey.csi.cam.ac.uk>

Doug McIlroy <doug at cs.dartmouth.edu> wrote:

> Every ifdef is an ugly intrusion and a pain to read.

I maintain `unifdef`, which is not a complicated program to port but it
does require some platform-specific stuff on Windows (partly because of
the poor standards conformance of the Windows C library). There's a fun
rule that `unifdef` must contain no #ifdef (or #if etc.) so the
portability stuff is handled by the build scripts adjusting the include
path and list of object files. Not a very scalable solution :-)

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/
Biscay: North or northwest 3 or 4, occasionally 5 at first. Slight or
moderate, becoming slight later. Showers, thundery in east. Good, occasionally
moderate.

From cym224 at gmail.com  Tue Jun  9 04:21:18 2020
From: cym224 at gmail.com (Nemo Nusquam)
Date: Mon, 8 Jun 2020 14:21:18 -0400
Subject: [TUHS] UNESCO's Software Heritage Foundation
In-Reply-To: <CAC20D2M5DjObwhNdBu7pmwAsM9Wx=LARJV1toWS7CwFWJLnswA@mail.gmail.com>
References: <CAC20D2OaedZ1tDqr5C1hBb=F8iioWJGy-yYRDXSgk=NCLrjZag@mail.gmail.com>
 <e5b4f2cd-1162-62f3-8ee4-035640818a35@bitsavers.org>
 <CAC20D2M5DjObwhNdBu7pmwAsM9Wx=LARJV1toWS7CwFWJLnswA@mail.gmail.com>
Message-ID: <e4d9bb25-a541-3317-3a0b-ae9bf96f3ea5@gmail.com>

On 06/08/20 11:14, Clem Cole wrote:
> Al, I'm not diss them or their efforts, just I wanted to know who they 
> were.
> This is the first I had heard of them,  I am, however, dissing the web 
> site which I could not figure out how to use.
I have surfed there over the years but I have always found their UI 
nothing short of terrible. They have archived over 126 million projects 
but my search queries return little and their browsing i/f seems 
non-existant.

N.


