From clemc at ccc.com Mon Jun 1 00:53:57 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 31 May 2020 10:53:57 -0400 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: <1jfNb2-7JV-00@marmaro.de> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: On Sun, May 31, 2020 at 9:02 AM markus schnalke wrote: > Would you be so kind to explain a bit about the hm version of MH. > I can try ... At the time V6 (and later V7) had a program called /bin/mail. The important thing about this program is that it was both a Mail Transport (and delivery) Agent as well as a Mail User Interface. The original version only delivered mail locally. When networking, like UUCP, was added to UNIX, it was easy to build a scheme that could send a transfer a message using UUCP and then fork /bin/mail on the proper system to deliver it. IIRC it was in Seventh Edition that /bin/mail was extended to recognize email addresses in the prefix form of: host!user, strip the first host! part and pass it to uucp. I never used it on anything like Spider, so I have not idea what it did internally and in fact, the prefix address form could have predated UUCP. We would need someone like Doug, Steve or Ken to tell us how email worked host to host, pre-UUCP. I believe it was Bruce Borden and team developed the original Rand Message Handler or MH. I do not know who was the primary author, we need to ask Bruce or one of the old Rand folks. MH ran on 6th edition when it was first released to rest of the USENIX community (at least I never saw it on V5, but it's possible it went back further). I just remember talking to Bruce about it at any early USENIX. MH used a new Mailbox (on-disk) scheme and formatted all messages in RFC733 form with addresses being flat an in the form: user at host. The new format, used a line a control-As followed by a nl, before and after the message. The headers of course where RFC733 (type: value with a trailing nl) and were separated from the body of the message by a single nl. MH assumed a traditional UNIX command line for the user interface and had a number of programs that ran behind the scene for delivery. This is important because the original ARPAnet NCP used FTP to do mail transfer. At least one version could call the early ARPAnet subsystem to perform host to host communications. The key point is that MH separated the MTA and MUI. A number of us ran MH at different places. I don't remember if it was on a USENIX tape or I had sent Bruce a tape @ Rand when I got a copy in the late 1970's (77-79 timeframe, I've forgotten). I'm pretty sure Goble, as did the Purdue crew ran it, as did Holmgrem, Greg Chesson at al, @ UofI and I would not surprised if it was the mailer at Harvard, given the Harvard/Rand connection in those days. By the time of Seventh Edition of UNIX, the Mail Transport Agent (MTA) that was part of the MH subsystem, could recognize ArpaNET address postfix and had a hack in it, that allowed the 'user' part of the address to include UUCP addresses and /bin/mail replacement knew to work like the AT&T mailer and pass it off to UUCP. So, at that point, life was good for those of us in strickly ARPA and/or UUCP land. BTW: At some point, the BBN TCP code was releases and a separate SMTPD was included in the release. I don't remember if it was the Rand folks or someone else, but at some point, the MH delivery agent was updated to call it or be called by it. Similarly, by the late 1970s, when Bruce, Greg Shaw, and Bob Metcalfe formed 3Com; Bruce and Greg brought MH with them and added an SMTPD that they wrote for their commercial product, UNET. This was the smtpd, I ran on the Teklabs machine before I went off to UCB. Meanwhile, as Mary Ann and I noted in earlier messages, Kurt Shoen's wrote a different MTA called Mail(1ucb). Unlike MH, Kurt continues to support the original UNIX mailbox format (later named 'mbox' format). The header lines were in a specific UNIX style prose starting with the ACSII 'From' and there are no special characters to demark the messages in the format, so recovery can be fraught with error, there are the famous From-line munging issues etc. (lots of details in other places start with https://en.wikipedia.org/wiki/Mbox). At some point in time, UCB built the 'Berk-net' (whose original code was written by ABC/Google's Eric Schmidt). BTW: Eric would have seen the Spider Network in his summers at BTL. The key thing with Berknet was cheap. It ran over 3 wire RS-232C between hosts at 9600 baud. Like UUCP was used to transfer files and email. Like UUCP it used a pre-fix addressing form: host:user ; but like RFC733 and unlike UUCP was flat. Where Mary Ann and I differ in our memories is who wrote the original version of UCB's delivermail (8ucb) program. We both agree that it is possible it was Eric Schmidt, as the switch to using delivermail(8ucb) was were Berknet was spliced into the email namespace. I had thought Kurt wrote it, Mary Ann thought it was Eric Allman. We agree Eric Allman was hacking on it for the Ing70. For this response, it doesn't really matter other than to try to get the history right, because it does not matter for the Rand Mail subsystem. Around, now I arrive at UCB. I was not the only person that had used MH, but I had the advantage of having 3 Vaxen 780 in the CAD lab. The key change we made at that point was to stop using the Rand MTA and make it use delivermail. There was hackery to allow the shared mbox to say, but in each user directory it stored the messages separately, MH style so the MH 'ui' suite of tools 'just worked.' [There were a number of arguments at the time. I remember having one with Sam. He hated MH because 'of all the small files and it chewed up inodes]. Anyway around this time, the curses library was created by Ken Arnold (originally to support Rogue) by pulling the screen code out of vi and using Mary Ann's termcap stuff. A couple of us in the USENIX community started playing with screen-based front ends to MH, including the folks at Rand. Numerous messages were exchanged, and a collaboration started (Rand team should get 99% percent of the work, I knew how to make curses go). The comment in the main code was, 'Well it sure isn't MH' -- so it was called HM. I later brought it Masscomp, and it was the start of their mailer. The key with HM, is that it uses the MH backend for all the real work. It would fork scan, but save the output in a text file that could be manipulated with the arrow keys. The user interface is not unlike Gmail. I only stopped running HM when I finally switched to Gmail a few years ago. In fact, when I was a DEC, the precursor to Gmail, was called Pachyderm which used the Altavista search. I glued MH to Pachyderm for a short period. I even ported them both to FreeBSD to run at home in the mid-1990s. Then I left DEC, the authors of Pachyderm headed for Google, *etc*... I got a chance to be an early Gmail/Google Apps tester for ccc.com and I have not switched back since. FWIW: It's been long enough, that the sources are no longer easy to find. It's all very incestuous. We all have seen other ideas and good ideas seem to have a way of reappearing in different places. Clem Note the part of the story I left out was when Rand MH spun out to one of the other Universities, UCI I think. I was never much involved with that team, so I can not tell you much about it and their code base. -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Mon Jun 1 02:05:47 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 31 May 2020 12:05:47 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal wrote: > This behaviour seems to have continued into SysVR1, I’m not sure when > EAGAIN came into use as a return value for this use case in the SysV > lineage. Maybe with SysVR3 networking? Actually, I'm pretty sure that was a product of the POSIX discussions. BSD already had networking an EWOULDBLOCK. We had argued about EWOULDBLOCK a great deal, we also were arguing about signal semantics. I've forgotten many of the details, Heinz may remember more than I do. EAGAIN was created as a compromise -- IIRC neither system had it yet. SVR3 networking was where it went into System V, although some of the AT&T representatives were none too happy about it. > > In the Research lineage, the above SysIII approach does not seem to exist, > although the V8 manual page for open() says under BUGS "It should be > possible [...] to optionally call open without the possibility of hanging > waiting for carrier on communication lines.” In the same location for V10 > it reads "It should be possible to call open without waiting for carrier on > communication lines.” > > The July 1981 design proposals for 4.2BSD note that SysIII non-blocking > files are a useful feature and should be included in the new system. In > Jan/Feb 1982 this appears to be coded up, although not all affected files > are under SCCS tracking at that point in time. Non-blocking behaviour is > changed from the SysIII semantics, in that EWOULDBLOCK is returned instead > of 0 when progress is not possible. The non-blocking behaviour is extended > beyond TTY’s and pipes to sockets, with additional errors (such as > EINPROGRESS). At this time EWOULDBLOCK is not the same error number as > EGAIN. > My memory is that Keith was the BSD (CSRG) person at the POSIX meeting (he, Jim McGinness of DEC, and I created PAX at one point as a compromise). I wish I could remember all of the details, but this was all argued at the POSIX meetings. As I said before the folks from AT&T just wanted to take the SVID and rubber stamp it at the specification. Part of it the problem was they wanted to be free to do what things/make choices that the rest of us might or might not like (for instance, they did not want the sockets interface). > > It would seem that the differences between the BSD and SysV lineages in > this area persisted until around 2000 or so. > Yep - cause around then POSIX started to settle out and both systems began to follow it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.salz at gmail.com Mon Jun 1 02:25:47 2020 From: rich.salz at gmail.com (Richard Salz) Date: Sun, 31 May 2020 12:25:47 -0400 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: "I'll try" Wow, that was quite some "try" Kudo's! > This is important because the original ARPAnet NCP used FTP to do mail transfer. If you look at RFC 765 (dated June 1980) , there are nearly a dozen mail-related FTP commands. By RFC 959, it's successor five years later, all of them were gone. Another eary MTA was the MMDF, the Multichannel Memorandum Distribution Facility, from U Delware. BBN ran it. One key point is that somehow when used with MH you could get real-time address verification before sending, "user rs at bbn.com doesn't exist." BBN ran MMDF for a long time because a key exec liked/needed that feature, long after 4.2 and sendmail. (My group was one of the first to run sendmail, which I liked because of the "R$" lines in its CF files. (Not really :) My Usenet/email gateway code had to support MMDF but it was only ever used on bbn.com) > could recognize ArpaNET address postfix and had a hack in it, that allowed the 'user' part of the address to include UUCP addresses Originally ihnp4!mirror!rs at seismo.arpa, then ihnp4!mirror!rs at seismo.css.gov and then rs%mirror.uucp at seismo.css.gov and then rs at mirror.tmc.com Thanks to Mary Ann and the "UUCP Mapping Project" for making much of that possible and Peter Honeyman for Pathlias, and MX records for the last part. The pre-MX styles received much scorn from Research, and if you search for "that hideous name" you can find a paper on it. There's also a pathlias paper, which Honey later said "was too good for Usenix" but he had nowhere else to submit it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Mon Jun 1 02:46:37 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 31 May 2020 10:46:37 -0600 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: Sorry to top post, but LSX or Miniunix had non blocking I/O as well. It was in one of the documents that Clem scanned in the last year. It specifically was an experiment into how to do it. Warner On Sun, May 31, 2020, 10:07 AM Clem Cole wrote: > > > On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal wrote: > >> This behaviour seems to have continued into SysVR1, I’m not sure when >> EAGAIN came into use as a return value for this use case in the SysV >> lineage. Maybe with SysVR3 networking? > > Actually, I'm pretty sure that was a product of the POSIX discussions. > BSD already had networking an EWOULDBLOCK. We had argued about > EWOULDBLOCK a great deal, we also were arguing about signal semantics. > I've forgotten many of the details, Heinz may remember more than I do. > EAGAIN was created as a compromise -- IIRC neither system had it yet. > SVR3 networking was where it went into System V, although some of the AT&T > representatives were none too happy about it. > > > >> >> In the Research lineage, the above SysIII approach does not seem to >> exist, although the V8 manual page for open() says under BUGS "It should be >> possible [...] to optionally call open without the possibility of hanging >> waiting for carrier on communication lines.” In the same location for V10 >> it reads "It should be possible to call open without waiting for carrier on >> communication lines.” >> >> The July 1981 design proposals for 4.2BSD note that SysIII non-blocking >> files are a useful feature and should be included in the new system. In >> Jan/Feb 1982 this appears to be coded up, although not all affected files >> are under SCCS tracking at that point in time. Non-blocking behaviour is >> changed from the SysIII semantics, in that EWOULDBLOCK is returned instead >> of 0 when progress is not possible. The non-blocking behaviour is extended >> beyond TTY’s and pipes to sockets, with additional errors (such as >> EINPROGRESS). At this time EWOULDBLOCK is not the same error number as >> EGAIN. >> > My memory is that Keith was the BSD (CSRG) person at the POSIX meeting > (he, Jim McGinness of DEC, and I created PAX at one point as a > compromise). I wish I could remember all of the details, but this was all > argued at the POSIX meetings. > > As I said before the folks from AT&T just wanted to take the SVID and > rubber stamp it at the specification. Part of it the problem was they > wanted to be free to do what things/make choices that the rest of us might > or might not like (for instance, they did not want the sockets interface). > > > >> >> It would seem that the differences between the BSD and SysV lineages in >> this area persisted until around 2000 or so. >> > Yep - cause around then POSIX started to settle out and both systems began > to follow it. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Mon Jun 1 02:50:50 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 31 May 2020 09:50:50 -0700 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: <20200531165050.GS22016@mcvoy.com> On Sun, May 31, 2020 at 12:25:47PM -0400, Richard Salz wrote: > Originally ihnp4!mirror!rs at seismo.arpa, then ihnp4!mirror!rs at seismo.css.gov > and then rs%mirror.uucp at seismo.css.gov and then rs at mirror.tmc.com Thanks > to Mary Ann and the "UUCP Mapping Project" for making much of that possible > and Peter Honeyman for Pathlias My first real research paper was with Udi Manber and it was all about compressing the pathalias maps. I had them on a 40MB disk on a Masscomp that had 20 users. The format was seismo.css.gov geowhiz!geophiz!uwisc!ihnp4!mirror in other words destination path!to!get!there I went to Udi and explained that all of our stuff started with geowhiz!geophiz!uwisc so that string was replicated over and over and it took too much space. Once Udi understood the problem he went "You have time best case, space worst case, you can reverse them like so" seismo -> mirror mirror -> ihnp4 ihnp4 -> uwisc uswisc -> geophiz geophiz -> geowhiz and you'll have time worst case and space best case. I was like "oh, yeah, you are right, cool, thanks" and got up to leave. Not so fast, says Udi, that's not "interesting". What would be interesting is if you could approximate time best case and space best case. With that, I started down the only dynamic programming problem I have ever solved. Udi's idea was to break the graph at what he called pivot points, where the pivot points as much as possible got rid of the replicated strings. He also wanted to limit lookups to just 2. So I had to read in the maps and build a graph from them on my creaky microvax with 4MB of ram. Doing so lead me to the first place I taught Udi about systems. I was fgets() each line and mallocing a copy of it. Then I sorted with qsort() and it took overnight to get the sorted array. Udi asked what sort? Qsort. Oh, that's garbage, use a radix sort, that will be better. WTF is a radix sort, look that up, implemented it, it was worse. So I start poking around and somehow realized we were thrashing VM, we didn't fit, and I have no idea how I figured out that malloc was fragmenting memory but I did. Probably figured out that most of my program was backed by swap, not files, and that's usually malloc or brk. So I wrote my own malloc that grabbed memory in 300KB chunks (why that size, I dunno, seemed right). Then I allocated memory out of those chunks. Got rid of the fragmentation and we fit in 4MB. qsort worked fine, took about 20 minutes (we didn't really fit but we were way closer). Udi was watching as I was doing all this, I think I was working in his office, it was his microvax. The fragmentation thing taught him that when Unix says it has virtual memory and you can just use it, it only works well when you fit. So the programming problem turned out that you calculated the space it work take for every possible pivot point and then at the end, you took the set that gave you the minimum space. We wrote, heh, who am I kidding, Udi wrote, a paper about it. I was still green on writing papers, they were hard for me. I told Udi that and he stared at me blankly and said "Writing papers is easy if you know the content. You just make a good outline and fill it in." He's right, that's how you do it, and it is easy if you know and can make the outline. I went on to do a lot of work with/for him. I wrote a user level thread library that made me write swtch() in assembler for the VAX (I believe I did a 68K version too but the source I have is VAX only). I "cheated" and did 99% of the work in C and popped down the rabbit hole to assembler, did the nasty, and popped back out as a different thread. 112 lines: http://www.mcvoy.com/lm/T/src/Tasm.S Funny Udi story, I was young and wild and drove too fast and got tickets. So I got a radar detector that beeped when it saw the radar gun. Udi upgraded the microvax to some newer release of X10 or X11 and the new xclock beeped on the hour. Every time it beeped, I jumped because it sounded just like the radar detector. Udi asked about it, I explained, he laughed and said "Larry, you're hacking too fast" each time I jumped :) Good times. I almost did a PhD under him but I didn't care for the CS department at Tucson where he went and I liked industry too much. From ralph at inputplus.co.uk Mon Jun 1 03:09:24 2020 From: ralph at inputplus.co.uk (Ralph Corderoy) Date: Sun, 31 May 2020 18:09:24 +0100 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: <20200531170924.0609821F85@orac.inputplus.co.uk> Hi Clem, > I believe it was Bruce Borden and team developed the original Rand > Message Handler or MH. I do not know who was the primary author, we > need to ask Bruce or one of the old Rand folks. https://en.wikipedia.org/wiki/MH_Message_Handling_System#History says MH was proposed by R. Stockton Gaines and Norman Shapiro and developed by Bruce S. Borden and others at RAND Corporation. and cites https://www.rand.org/content/dam/rand/pubs/notes/2009/N3017.pdf Norman Shapiro, or Norm, is still using nmh well into his eighties, nmh is a descendant of MH, and pops up on the nmh-workers mailing list to either fill in bits of history or ask for installation advice as the Linux distro shifts under his feet. He did visit Bell Labs, though in '54. :-) Details at https://en.wikipedia.org/wiki/Norman_Shapiro > The new format, used a line a control-As followed by a nl, before and > after the message. MH's mts.conf(5) had the ability to set mmdelim{1,2} to something other than \ca\ca\ca\ca\n; no idea why. And that magic marker would be escaped if it occurred in the middle of a message, to \cb\ca\ca\ca\n. > Note the part of the story I left out was when Rand MH spun out to one > of the other Universities, UCI I think. Wikipedia again: Subsequently development was taken over by Marshall T. Rose and John L. Romine working at University of California, Irvine. nmh... was forked [from that] by Richard Coleman while working at the Georgia Institute of Technology. It incorporates the "LBL changes" made to MH in the late 80s by Van Jacobson, Mike Karels and Craig Leres. MH also had support for UCI's bulletin boards called ‘BBoards’, including on ZOTnet. They were optionally available over POP3 and NNTP, but only one or the other. :-) There was an MH shell of some kind called msh, and a visual mh, vmh. ZOTnet was a LAN for mail by Marshall Rose at UCI with gateways for CSnet and ARPAnet. https://escholarship.org/content/qt4bm4k2vh/qt4bm4k2vh.pdf The ZOTnet: a local area mailing network, 1983 (I still use nmh and I'm using it for this reply.) -- Cheers, Ralph. From jon at fourwinds.com Mon Jun 1 03:25:07 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Sun, 31 May 2020 10:25:07 -0700 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: <20200531170924.0609821F85@orac.inputplus.co.uk> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <20200531170924.0609821F85@orac.inputplus.co.uk> Message-ID: <202005311725.04VHP7bX3814695@darkstar.fourwinds.com> Ralph Corderoy writes: > ... > > (I still use nmh and I'm using it for this reply.) +1 on that. I also still use nmh. To me, the set of shell commands exemplifies the UNIX philosophy as I can easily "show | wc -l" or whatever that is cumbersome with monolithic packages. My minor contributions were adding the attachment support and the hooks. The hooks allow other programs to be triggered behind the scenes when nmh commands are run; I use it to keep an elasticsearch database of my messages. I have two additional command script to support thise: gpick and gscan which are the equivalent of pick and scan except that they search the elasticsearch database and pretty much return results instantaneously. Jon From mah at mhorton.net Mon Jun 1 03:28:56 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Sun, 31 May 2020 10:28:56 -0700 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: What a wonderful historical tour, Clem! Thank you! On 5/31/20 7:53 AM, Clem Cole wrote: > MH used a new Mailbox (on-disk) scheme and formatted all messages in > RFC733 form with addresses being flat an in the form: user at host. The > new format, used a line a control-As followed by a nl, before and > after the message.   The headers of course where RFC733 (type: value > with a trailing nl) and were separated from the body of the message by > a single nl. MH's message/file format was also the inspiration for B news, which kept each Usenet message in a separate RFC733 format file. (The command line interface, however, was more like a feed, not separate shell commands.) No control-A separators, though, just separate files. > At some point in time, UCB built the 'Berk-net' (whose original code > was written by ABC/Google's Eric Schmidt).  BTW: Eric would have seen > the Spider Network in his summers at BTL.  The key thing with Berknet > was cheap.  It ran over 3 wire RS-232C between hosts at 9600 baud.  >  Like UUCP was used to transfer files and email.    Like UUCP it used > a pre-fix addressing form: host:user ; but like RFC733 and unlike UUCP > was flat. > > Where Mary Ann and I differ in our memories is who wrote the original > version of UCB's delivermail (8ucb) program.   We both agree that it > is possible it was Eric Schmidt, as the switch to using > delivermail(8ucb) was were Berknet was spliced into the email > namespace.  I had thought Kurt wrote it, Mary Ann thought it was Eric > Allman.   We agree Eric Allman was hacking on it for the Ing70.  For > this response, it doesn't really matter other than to try to get the > history right, because it does not matter for the Rand Mail subsystem. To clarify, I was joking when I mentioned Eric Schmidt as a possible author of delivermail. (I should have inserted a smiley.) I wasn't directly involved, as Eric Allman was in the Ingres project on a different system, but I thought he (Eric Allman) wrote delivermail, and so does Wikipedia. I think Kurt Shoens wrote only the MTA, but i could be mistaken. > Anyway around this time, the curses library was created by Ken Arnold > (originally to support Rogue) by pulling the screen code out of vi and > using Mary Ann's termcap stuff. I didn't write termcap, Bill Joy did. I wrote terminfo, but that was later when I was at Bell Labs in 1982. Ken's curses used termcap, my 1982 rewrite used terminfo.     Mary Ann -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Mon Jun 1 03:43:46 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 31 May 2020 11:43:46 -0600 Subject: [TUHS] LSX on the PDP-11/03 (LSI-11) In-Reply-To: <193c770d408ee131dbaa9a07dba6f068.squirrel@squirrelmail.tuffmail.net> References: <193c770d408ee131dbaa9a07dba6f068.squirrel@squirrelmail.tuffmail.net> Message-ID: On Sun, May 31, 2020 at 6:34 AM wrote: > > > Anyway, I've tried compiling Hello World on LSX, and I get "1: External > > definition syntax" error. Some help would be nice, but more generally, is > > anyone on this list more than vaguely familiar with LSX, or 6th Edition > > itself? > > > > void main () { > > printf("Hello World!"); > > } > > > > It seems that the 7th Edition was the beginning of the standard library > in > > C, and that this is missing in LSX. I'm not sure if printf is an > intrinsic > > function in (6th Edition) C, or if it's from a library. > > > > First off, VOID MAIN is not legal in any standard version of C. Even when > the language allows implementation defined extensions to the main > signature, it must still return int. > > If you have a later version of language supported, you have to define > printf rather than allowing it default define as an int returning > function. > > Add #include > Or, alternatively, LSX is really old. The compiler there likely doesn't support 'void'. >From the lsx part of tuhs: # cp /dev/tty8 addr.c main() { int a; printf("main address: %u (0%o)\n", main, main); printf("stack address: %u (0%o)\n", &a, &a); printf("heap address: %u (0%o)\n", sbrk(0), sbrk(0)); } ^D # cc addr.c # a.out main address: 16410 (040032) stack address: 40934 (0117746) heap address: 17920 (043000) None of the disks appear to have a /usr/include. The only .h files look like they were from the kernel. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at eric.allman.name Mon Jun 1 04:56:07 2020 From: tuhs at eric.allman.name (Eric Allman) Date: Sun, 31 May 2020 11:56:07 -0700 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Ah, time for some "Email at Berkeley" history.... First the back story.  In the beginning, /bin/mail was the mail system.  All mail was local to one machine so the distinction between MUA, MTA, and MSA (Mail Storage Agent) didn't exist.  At this point there were no options, at least at Berkeley, although I suspect that Bruce was working on MH by then. If my memory serves, networking at Berkeley started with BerkNET.  Clem got it right: Eric Schmidt wrote it, and it was designed to be cheap.  The main purpose was that Berkeley was getting more Unix machines and carrying tapes was problematic.  It was purely batch (no interactive sessions; file copy and email only) and ran over 9600 baud TTY lines.  Eric hacked /bin/mail to do the appropriate calls if it saw a colon in the address, so the flow of a typical message would be keyboard → /bin/mail → BerkNET → /bin/mail → /var/mail. In about 1976 the INGRES project at Berkeley got an ARPAnet connection.  6th Edition was current then, and we got the software from (iirc) UC San Diego.  It was kind of a mess, but it did work.  It was a VDH (Very Distant Host) interface running at 9600 baud.  But the mail system it included, MSG I think, was a world unto itself, and definitely did not use /bin/mail.  So now we had two universes: to send something to both someone on BerkNET and someone on ARPAnet you had to type the message into two different MUAs.  But at first this was good enough.  But not for long. The problem was that to use the ARPAnet you had to be logged into our PDP-11/70.  Back then the cool kids had "glass TTYs" (i.e., adm3a).  But BerkNET didn't have interactive connections, so you had to actually have a TTY line into our machine.  Two problems arose: first, INGRES was in a different building (Cory Hall) than the rest of the department (Evans Hall).  But that's OK, there were already wires running between the two buildings that were probably inspired by BerkNET.  Second, our PDP-11 had a single DH-11 multiport interface with 16 ports.  Only two of those ports were available, and new DH-11s cost on the order of $15,000, money which we did not have. But no problem.  At this point there was already a "plug board" in the mail room in Evans that had ports to the half dozen or so machines that were by then on campus and lines to each office.  To connect you would plug in a wire between the machine you wanted to use and your office.  Protocol was that you remove the cross connect after use, but that was widely ignored.  Ing70 ports were quite frequently being pulled out while in use and no one was happy.  Unfortunately since I was kind of managing that machine, I was the one who got yelled at. Also in that period Berkeley got connected into UUCP.  UUCP had also done a hack to /bin/mail, so it was becoming quite the big mess.  I made the observation that I could hack /bin/mail to forward messages with an "@" sign in the address to our ARPAnet-connected machine, but at that point the code was becoming unwieldy.  Enter delivermail, which was essentially what we now call an MTA.  It was still a hack, but it was easier to deal with because it didn't have MUA and MSA functionality.  I changed /bin/mail to call delivermail unless it got a "-d" (deliver) flag, so now the sequence for local mail was /bin/mail → delivermail → /bin/mail -d → /var/mail.  And most importantly, I hacked the ARPAnet code to use delivermail as well, so now Ing70 could forward mail from BerkNET to ARPAnet and back again.  The fights over our two plug board ports mostly dissipated and my life got better. I'm pretty sure that at this point Kurt Schoens was working on Mail.  He read /var/mail (at that point it had been migrated to /var/spool/mail) directly, so there was no hiding of the mailbox format, so he had MUA + a half of an MSA.  He converted Mail to use delivermail for outgoing mail.  I don't recall the details of how MH got hooked in, but it did have a separate program for sending (called "send" I think) that would have been fairly easy to hack.  It still had "inc" (incorporate) to bring mail from the /var/mail one-file-per-mailbox format into the MH one-file-per-message format, so the three functions were not completely broken out, but it was really the closest to that ideal model available at the time, so far as I know. That's the gist of that story.  Sendmail is a completely different story, driven by the onset of the Internet, and I won't make this already long message even longer by going into it here. By the way, I have a 9-track tape sent to me by Marshall Rose containing the November 1983 version of MH that UCI did (based on the Rand system), complete with original versions of the documentation.  If anyone has any way of reading it I'm happy to pass it on. eric From clemc at ccc.com Mon Jun 1 05:49:07 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 31 May 2020 15:49:07 -0400 Subject: [TUHS] mh/hm, mmh In-Reply-To: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: On Sun, May 31, 2020 at 3:38 PM Eric Allman wrote: > but it did have a separate program for sending (called "send" I think) > that would > have been fairly easy to hack. It still had "inc" (incorporate) to > bring mail from the /var/mail one-file-per-mailbox format into the MH > one-file-per-message format > Right, the hooks were in send(1mh) and inc(1mh). Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From mah at mhorton.net Mon Jun 1 07:31:30 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Sun, 31 May 2020 14:31:30 -0700 Subject: [TUHS] mh/hm, mmh In-Reply-To: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: Wonderful! Thank you for the history, Eric!     Mary Ann On 5/31/20 11:56 AM, Eric Allman wrote: > Ah, time for some "Email at Berkeley" history.... > From robpike at gmail.com Mon Jun 1 08:01:58 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 1 Jun 2020 08:01:58 +1000 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: > > I’m not quite sure why the Research lineage did not include non-blocking > behaviour, especially in view of the man page comments. Maybe it was seen > as against the Unix philosophy, with select() offering sufficient mechanism > to avoid blocking (with open() the hard corner case)? That's it. Select was good enough for our purposes. -rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Mon Jun 1 13:32:56 2020 From: dave at horsfall.org (Dave Horsfall) Date: Mon, 1 Jun 2020 13:32:56 +1000 (EST) Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Mon, 1 Jun 2020, Rob Pike wrote: > I’m not quite sure why the Research lineage did not include > non-blocking behaviour, especially in view of the man page comments. > Maybe it was seen as against the Unix philosophy, with select() > offering sufficient mechanism to avoid blocking (with open() the hard > corner case)? > > That's it. Select was good enough for our purposes. After being dragged through both Berserkley and SysVile, I never did get the hang of poll()/select() etc,,, -- Dave From Caipenghui_c at 163.com Mon Jun 1 16:32:01 2020 From: Caipenghui_c at 163.com (Caipenghui) Date: Mon, 01 Jun 2020 14:32:01 +0800 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: <36190BD8-0A00-4729-9B81-2FF441552CF0@163.com> it's beautiful. 于 2020年6月1日 GMT+08:00 上午5:31:30, Mary Ann Horton 写到: >Wonderful! Thank you for the history, Eric! > >     Mary Ann > >On 5/31/20 11:56 AM, Eric Allman wrote: >> Ah, time for some "Email at Berkeley" history.... >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From meillo at marmaro.de Mon Jun 1 17:14:06 2020 From: meillo at marmaro.de (markus schnalke) Date: Mon, 01 Jun 2020 09:14:06 +0200 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: <1jfee7-82Z-00@marmaro.de> Thanks to everyone who joined in. That was more than I had expected. :-) meillo [2020-05-31 14:31] Mary Ann Horton > > Wonderful! Thank you for the history, Eric! > >     Mary Ann > > On 5/31/20 11:56 AM, Eric Allman wrote: > > Ah, time for some "Email at Berkeley" history.... > > > From paul at rileyriot.com Mon Jun 1 20:18:41 2020 From: paul at rileyriot.com (Paul Riley) Date: Mon, 1 Jun 2020 18:18:41 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: Team Unix, Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? Paul *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at ronnatalie.com Mon Jun 1 23:01:13 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Mon, 1 Jun 2020 09:01:13 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Eh? An RL is just a cartridge disk drive, not a format. It would depend on what the operating system you are using wanted to put on it. Similarly the RX is just a floppy drive. In my memory, there wasn’t much UNIX use of these drives. The main reason the driver existed on the VAX unices was to write the Console Floppy Disk in the 780 (which was in RT-11 format). What are you trying to do? > On Jun 1, 2020, at 6:18 AM, Paul Riley wrote: > > Team Unix, > > Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? > > Paul > > Paul Riley > > Mo: +86 186 8227 8332 > Email: paul at rileyriot.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Mon Jun 1 23:04:28 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 1 Jun 2020 09:04:28 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Mon, Jun 1, 2020 at 6:19 AM Paul Riley wrote: > Is there a Windows or Linux utility to create a disk image in any of the > above formats, from a local folder tree? > What I think you are asking, is there a utility for a modern OS that will walk a local folder tree on my OS and create a new file whose structure is that of the file system for OS . The issue is not the device as much as the OS and disk file layout. As far as UNIX (or simh at the OS level) is concerned, the disk is just a linear array of bytes, addressed by blocks. The physical format is not seen by UNIX. There are numerious utilities, as well as 'foreign file systems' that are available. For instance, many Unix's can write RT-11 and MS-DOS format with standard utilities. It really depends the OS. That said, if the target OS is modern enough to support NFS or Samba, the easiest way might be export the file system from local system, and then running a simulated OS, 'mount' the file system. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Tue Jun 2 00:58:02 2020 From: lm at mcvoy.com (Larry McVoy) Date: Mon, 1 Jun 2020 07:58:02 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: <20200601145801.GE22016@mcvoy.com> On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote: > On Mon, 1 Jun 2020, Rob Pike wrote: > > > I???m not quite sure why the Research lineage did not include > > non-blocking behaviour, especially in view of the man page comments. > > Maybe it was seen as against the Unix philosophy, with select() > > offering sufficient mechanism to avoid blocking (with open() the hard > > corner case)? > > > >That's it. Select was good enough for our purposes. > > After being dragged through both Berserkley and SysVile, I never did get the > hang of poll()/select() etc,,, I'm sure you could, select is super handy, think a network server like apache. From pnr at planet.nl Tue Jun 2 01:04:22 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 1 Jun 2020 17:04:22 +0200 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl> > Team Unix, Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? Paul *Paul Riley* It seems you are asking for two tools in the BK-UNIX toolbox, fsutil and dskutil https://github.com/sergev/bkunix/tree/master/fsutil https://github.com/sergev/bkunix/tree/master/diskutil The first generates a Unix 6th edition file system from a local directory tree. The result is a binary file with the 512 byte disk blocks stored in sequence. Maybe this is what SIMH needs, I’m not into the details of SIMH. The second appears to be a tool to take the a file generated by the fsutil tool and split that into sectors and tracks. I’m not familiar with this tool, but it looks like you might need something similar (I assume that you have some way to hook up a 8” drive to your PC?). Sector interleaving may be an issue to look out for. When preparing a LSX system disk, you will need to think carefully about the layout: Presumably the disk works with 128 byte sectors and 4 sectors are grouped together to create a 512 byte unix block. Check out the disk driver code for details: https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/decfd.c Block 0 contains bootstrap code which is loaded/called from the monitor rom (or from a short code sequence keyed in on a “blinkenpanel” console). The filesystem itself starts at block 1 and runs up to a block N (you need to tell fsutil how big the filesystem needs to be). After block N up to the end of the disk is space to contain 2 swapped out programs plus 1 block for the return code of the third (default LSX has a maximum of 3 processes). You have to figure out how many 512 blocks are on your floppy and subtract out the swap space to arrive at a figure for N. In the LSX source code ’N’ is known as the define SWPLO, see param.h for details: https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/param.h In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h Be careful with size units in the source code. Often sizes are expressed in words (2 bytes). Where memory is concerned it is often expressed in ‘clicks’, 64 bytes. From pnr at planet.nl Tue Jun 2 01:18:44 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 1 Jun 2020 17:18:44 +0200 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: > > In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h Oops: 77 tracks by 26 sectors of 128 bytes, makes 250.25KB, so it should work as configured. From heinz at osta.com Tue Jun 2 02:41:03 2020 From: heinz at osta.com (Heinz Lycklama) Date: Mon, 1 Jun 2020 09:41:03 -0700 Subject: [TUHS] LSX on the PDP-11/03 (LSI-11) In-Reply-To: References: Message-ID: Paul, I'm surprised that copies of the LSX system are still around. I'm assuming that you acquired only the binary code, and not the source code. The LSX system source code could be compiled on an LSX system running on floppy disks. I see you are getting your C compiler issues addressed by others already. Heinz Lycklama Author of LSX at Bell Labs On 5/30/2020 11:26 PM, Paul Riley wrote: > I've managed to acquire a PDP-11/03 with twin floppy drives (Sykes > Datatronics RX01 or RX02 equivalents, not sure yet which). > > I've stumbled across LSX, and I have it running on SimH. I'm quite > inexperienced with Unix, but it's something I want to learn well, > having brushed against it at university in the '80s, and having played > with Linux somewhat. > > I have some interest in Forth, but I don't like the block system of > early forths such as FigForth, and I plan to create a new Forth based > on FigForth, but supporting external source files. > > Anyway, I've tried compiling Hello World on LSX, and I get "1: > External definition syntax" error. Some help would be nice, but more > generally, is anyone on this list more than vaguely familiar with LSX, > or 6th Edition itself? > > void main () { >         printf("Hello World!"); > } > > It seems that the 7th Edition was the beginning of the standard > library in C, and that this is missing in LSX. I'm not sure if printf > is an intrinsic function in (6th Edition) C, or if it's from a library. > > My questions are a bit random, but looking to converse with others > with LSX experience. > > Paul > > *Paul Riley* > > Email: paul at rileyriot.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz at osta.com Tue Jun 2 02:58:47 2020 From: heinz at osta.com (Heinz Lycklama) Date: Mon, 1 Jun 2020 09:58:47 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: <6aaca54f-83da-515f-e5b0-e3baa4560c44@osta.com> I did  add a few new features to LSX to deal with contiguous files and to handle asynchronous read/write's for real time applications. They are described in the LSX paper in the 1978 BSTJ on the UNIX Time-Sharing System. Heinz On 5/31/2020 9:46 AM, Warner Losh wrote: > Sorry to top post, but LSX or Miniunix had non blocking I/O as well. > It was in one of the documents that Clem scanned in the last year. It > specifically was an experiment into how to do it. > > Warner > > On Sun, May 31, 2020, 10:07 AM Clem Cole > wrote: > > > > On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal > wrote: > >  This behaviour seems to have continued into SysVR1, I’m not > sure when EAGAIN came into use as a return value for this use > case in the SysV lineage. Maybe with SysVR3 networking? > > Actually, I'm pretty sure that was a product of the POSIX > discussions.  BSD already had networking an EWOULDBLOCK.   We had > argued about EWOULDBLOCK a great deal, we also were arguing about > signal semantics. I've forgotten many of the details, Heinz may > remember more than I do.  EAGAIN was created as a compromise -- > IIRC neither system had it yet.   SVR3 networking was where it > went into System V, although some of the AT&T representatives were > none too happy about it. > > > In the Research lineage, the above SysIII approach does not > seem to exist, although the V8 manual page for open() says > under BUGS "It should be possible [...] to optionally call > open without the possibility of hanging waiting for carrier on > communication lines.” In the same location for V10 it reads > "It should be possible to call open without waiting for > carrier on communication lines.” > > The July 1981 design proposals for 4.2BSD note that SysIII > non-blocking files are a useful feature and should be included > in the new system. In Jan/Feb 1982 this appears to be coded > up, although not all affected files are under SCCS tracking at > that point in time. Non-blocking behaviour is changed from the > SysIII semantics, in that EWOULDBLOCK is returned instead of 0 > when progress is not possible. The non-blocking behaviour is > extended beyond TTY’s and pipes to sockets, with additional > errors (such as EINPROGRESS). At this time EWOULDBLOCK is not > the same error number as EGAIN. > > My memory is that Keith was the BSD (CSRG) person at the POSIX > meeting (he, Jim McGinness of DEC, and I created PAX at one point > as a compromise).   I wish I could remember all of the details, > but this was all argued at the POSIX meetings. > > As I said before the folks from AT&T just wanted to take the SVID > and rubber stamp it at the specification.  Part of it the problem > was they wanted to be free to do what things/make choices that the > rest of us might or might not like (for instance, they did not > want the sockets interface). > > > It would seem that the differences between the BSD and SysV > lineages in this area persisted until around 2000 or so. > > Yep - cause around then POSIX started to settle out and both > systems began to follow it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at rileyriot.com Tue Jun 2 08:35:21 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 06:35:21 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Ron, Trying to move a large number of source files from my Windows machine to the LSX simulation (or V6 simulation). This is a Unix forum, so I didn’t explicitly say in a Unix format, but that was implied, sorry. Paul On Mon, 1 Jun 2020 at 9:01 pm, Ronald Natalie wrote: > Eh? An RL is just a cartridge disk drive, not a format. It would > depend on what the operating system you are using wanted to put on it. > Similarly the RX is just a floppy drive. In my memory, there wasn’t > much UNIX use of these drives. The main reason the driver existed on the > VAX unices was to write the Console Floppy Disk in the 780 (which was in > RT-11 format). > > What are you trying to do? > > On Jun 1, 2020, at 6:18 AM, Paul Riley wrote: > > Team Unix, > > Is there a Windows or Linux utility to create a disk image in any of the > above formats, from a local folder tree? > > Paul > > *Paul Riley* > > Mo: +86 186 8227 8332 > Email: paul at rileyriot.com > > > -- *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at rileyriot.com Tue Jun 2 08:45:07 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 06:45:07 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Clem, Thanks for your help. You’ve correctly interpreted my question. Is the disk image independent from the disk hardware? I’d assume that different disks may have different block sizes etc, so the disk type may be important. The target system is LSX, a cut-down version of V6 designed to run on the LSI-11. There are very few system utilities in the standard build (no mount for example). The second floppy is permanently mounted at boot time. I’m interested in making source file floppies on my modern system to use on the LSX, so I want to be able to create an image file from a source folder tree. Paul On Mon, 1 Jun 2020 at 9:05 pm, Clem Cole wrote: > > > On Mon, Jun 1, 2020 at 6:19 AM Paul Riley wrote: > >> Is there a Windows or Linux utility to create a disk image in any of the >> above formats, from a local folder tree? >> > What I think you are asking, is there a utility for a modern OS that will > walk a local folder tree on my OS and create a new file whose structure is > that of the file system for OS . > > The issue is not the device as much as the OS and disk file layout. As > far as UNIX (or simh at the OS level) is concerned, the disk is just a > linear array of bytes, addressed by blocks. The physical format is not > seen by UNIX. > > There are numerious utilities, as well as 'foreign file systems' that are > available. For instance, many Unix's can write RT-11 and MS-DOS format > with standard utilities. It really depends the OS. That said, > if the target OS is modern enough to support NFS or Samba, the easiest way > might be export the file system from local system, and then running a > simulated OS, 'mount' the file system. > -- *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at rileyriot.com Tue Jun 2 08:54:28 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 06:54:28 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl> References: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl> Message-ID: Paul, Many thanks for your detailed reply, it’s much appreciated. I’ll start with the Bkunix utility and see if SimH can use the output, then move on to your other suggestions if not. Paul On Mon, 1 Jun 2020 at 11:04 pm, Paul Ruizendaal wrote: > > Team Unix, Is there a Windows or Linux utility to create a disk image in > any of the above formats, from a local folder tree? Paul *Paul Riley* > > It seems you are asking for two tools in the BK-UNIX toolbox, fsutil and > dskutil > https://github.com/sergev/bkunix/tree/master/fsutil > https://github.com/sergev/bkunix/tree/master/diskutil > > The first generates a Unix 6th edition file system from a local directory > tree. The result is a binary file with the 512 byte disk blocks stored in > sequence. Maybe this is what SIMH needs, I’m not into the details of SIMH. > > The second appears to be a tool to take the a file generated by the fsutil > tool and split that into sectors and tracks. I’m not familiar with this > tool, but it looks like you might need something similar (I assume that you > have some way to hook up a 8” drive to your PC?). Sector interleaving may > be an issue to look out for. > > When preparing a LSX system disk, you will need to think carefully about > the layout: > > Presumably the disk works with 128 byte sectors and 4 sectors are grouped > together to create a 512 byte unix block. Check out the disk driver code > for details: > https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/decfd.c > > Block 0 contains bootstrap code which is loaded/called from the monitor > rom (or from a short code sequence keyed in on a “blinkenpanel” console). > > The filesystem itself starts at block 1 and runs up to a block N (you need > to tell fsutil how big the filesystem needs to be). > > After block N up to the end of the disk is space to contain 2 swapped out > programs plus 1 block for the return code of the third (default LSX has a > maximum of 3 processes). You have to figure out how many 512 blocks are on > your floppy and subtract out the swap space to arrive at a figure for N. > > In the LSX source code ’N’ is known as the define SWPLO, see param.h for > details: > https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/param.h > > In this file, 99 blocks are reserved for swap, corresponding to processes > with 24KB memory; total disk size is defined as 500 blocks, 250KB - note > that this slightly exceeds the 241KB offered by the standard IBM 77 track > 26 sector formatting for 8” disks. Probably you will need to tweak the > values in param.h > > Be careful with size units in the source code. Often sizes are expressed > in words (2 bytes). Where memory is concerned it is often expressed in > ‘clicks’, 64 bytes. > > > > > -- *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnc at mercury.lcs.mit.edu Tue Jun 2 09:17:42 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Mon, 1 Jun 2020 19:17:42 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200601231742.CBAB118C073@mercury.lcs.mit.edu> > From: Paul Ruizendaal > This time looking into non-blocking file access. (... right now my > scope is 'communication files' (tty's, pipes, network connections). > ... > First appearance of non-blocking behaviour seems to have been with > Chesson's multiplexed files ... in 1979. At around that point in time (I don't have the very _earliest_ code, to get an exact date, but the oldest traces I see [in mapalloc(), below] are from September '78), the CSR group at MIT-LCS (which were the people in LCS doing networking) was doing a lot with asynchronous I/O (when you're working below the reliable stream level, you can't just do a blocking 'read' for a packet; it pretty much has to be asynchronous). I was working in Unix V6 - we were building an experimental 1Mbit/second ring - and there was work in Multics as well. I don't think the wider Unix community heard about the Unix work, but our group regularly filed updates on our work for the 'Internet Monthly Reports', which was distributed to the whole TCP/IP experimental community. If you can find an archive of early issues (I'm too lazy to go look for one), we should be in there (although our report will alsocover the Multics TCP/IP work, and maybe some other stuff too). There were two main generations of code; I don't recall the second one well, and I'm too lazy to go look, but I can tell you off the top of my head a bit about how the first one worked. Open/read/write all looked standard to the user in the process (the latter two were oriented to packets, a bit like raw disks being blocks); multiple operations could be queued in each direction. (There was only one user allowed at a time for the network device; no input demultiplexing.) Whenever an I/O operation completed, the process was sent a signal. Since the read/write call had long since returned, it had to do a getty() to get info about that operation - the size of the packet, error indications, etc. One complication was that for a variety of reasons (we wanted to avoid having to copy data, and the interface did not have packet buffers) we did DMA directly to/from the user's memory; this meant the process has to be locked in place while I/O was pending. (I didn't realize it at the time, but we dodged a bullet there; a comment in xalloc(), which I only absorbed recently, explains the problem. More here: https://gunkies.org/wiki/UNIX_V6_internals#exec()_and_pure-text_images if anyone wants the gory details.) That all (the queing, signals for I/O completion, locking the process to a fixed location in memory while it continued to run) etc all worked well, as I recall (although I guess it couldn't do an sbrk() while locked), but one complication was the UNIBUS map on the -11/70. The DSSR/RTS group at LCS wanted to have a ring interface, but their machine was a /70 (ours, the one the driver was initially done on/for, was a /40), so with DMA we had to use the UNIBUS map. The stock V6 code had mapalloc(), (and mapfree(), both called on all DMA operations), but... it allocated the whole map to whatever I/O operation asked for the map. Clearly, if you're about to start a network input operation, and wait a packet to show up, you don't want the disk controller to have to sit and wait for for a packet to show up so _it_ can have the map. Luckily, mapalloc() was called with a pointer to the buffer header (which had all the info about the xfer), so I added a 'ubmap' array, and called the existing malloc() on it, to allocate only a big enough chunk of the UNIBUS map for the I/O operation defined by the header. Since there was 248KB of map space, and the largest single DMA transfer possible in V6 was about 64KB (maybe a little more, for a max-sized process with its 'user' block), there was never a problem with contention for the map, and we didn't have to touch any of the other drivers at all. That was dandy, and only a couple of lines of extra code, but I somehow made a math error in my changes, and as I recall I had to debug it with a printf() in mapalloc(). I was not popular that day! Luckily, the error was quickly obvious, a fix was applied, and we were on our way. Noel From ron at ronnatalie.com Tue Jun 2 09:59:20 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Mon, 1 Jun 2020 19:59:20 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: As far as the early UNIXs go, any disk is collection of 512-byte blocks. The filesystems either the early (what I’ll call V6) and the later (V7) don’t differ much. The primary difference is that the later V7 had 16-bit uids and a provision for larger file systems/sizes. The V6 file system was limited to 2^24 blocks while V7 did 2^32. The 512 block size corresponded to the native sector size of all the DEC hardware except the RX which I think only had 128-byte sectors. But again, we didn’t do much with that other than write the standalone console disks for the 780 (in RT format) and I also used it to make “unix” file system disks for the BRL “LOS” (little operating system…no time for sharing, uniprocessor system) that ran our internet routers and the IO hardware on the HEP supercomputer. From jnc at mercury.lcs.mit.edu Tue Jun 2 10:08:45 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Mon, 1 Jun 2020 20:08:45 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200602000845.CF26E18C073@mercury.lcs.mit.edu> > when you're working below the reliable stream level, you can't just do a > blocking 'read' for a packet; it pretty much has to be asynchronous Oh, you should look at the early BBN TCP for V6 Unix - they would have faced the same issue, with their TCP process. They did have the capac() call (which kind of alleviates the need for non-blocking I/O), but that may have only been available for ports/pipes; I'm not sure if the ARPANET device supported it. (With the NCP as well, that did some amount of demultiplexing in the kernel, and probably had buffering there, so, if so, in theory capac() could have been done there. Of course, with the ARPANET link being only 100Kbit/sec maximum - although only to a host on the same IMP - the overhead of copying buffered data made kernel buffering more 'affordable'.) Noel From jnc at mercury.lcs.mit.edu Tue Jun 2 10:14:46 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Mon, 1 Jun 2020 20:14:46 -0400 (EDT) Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: <20200602001446.359F618C073@mercury.lcs.mit.edu> > From: Ronald Natalie > The V6 file system was limited to 2^24 blocks No, 2^16; from filsys.h: int s_fsize; /* size in blocks of entire volume */ and of course on an -11 an int is 16 bits. Maybe you're thinking of the file size, which was 2^24 bytes (max). Noel From ron at ronnatalie.com Tue Jun 2 11:26:56 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Mon, 1 Jun 2020 21:26:56 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: <20200602001446.359F618C073@mercury.lcs.mit.edu> References: <20200602001446.359F618C073@mercury.lcs.mit.edu> Message-ID: <8C214FC8-A7F3-4DC1-9600-DA3CE357F041@ronnatalie.com> You are right. 16 bit block numbers 24 bit file size for V6. > On Jun 1, 2020, at 8:14 PM, Noel Chiappa wrote: > >> From: Ronald Natalie > >> The V6 file system was limited to 2^24 blocks > > No, 2^16; from filsys.h: > > int s_fsize; /* size in blocks of entire volume */ > > and of course on an -11 an int is 16 bits. > > Maybe you're thinking of the file size, which was 2^24 bytes (max). > > Noel From paul at rileyriot.com Tue Jun 2 14:57:54 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 12:57:54 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Thanks Ron, and all others. Theres plenty of grist for the mill, time for me to grind it. Paul *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com On Tue, 2 Jun 2020 at 07:59, Ronald Natalie wrote: > As far as the early UNIXs go, any disk is collection of 512-byte blocks. > The filesystems either the early (what I’ll call V6) and the later (V7) > don’t differ much. > The primary difference is that the later V7 had 16-bit uids and a > provision for larger file systems/sizes. The V6 file system was limited > to 2^24 blocks while V7 > did 2^32. > > The 512 block size corresponded to the native sector size of all the DEC > hardware except the RX which I think only had 128-byte sectors. But > again, we didn’t > do much with that other than write the standalone console disks for the > 780 (in RT format) and I also used it to make “unix” file system disks for > the BRL “LOS” > (little operating system…no time for sharing, uniprocessor system) that > ran our internet routers and the IO hardware on the HEP supercomputer. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Tue Jun 2 18:22:12 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 2 Jun 2020 10:22:12 +0200 Subject: [TUHS] non-blocking IO Message-ID: <46A39BDD-A854-40EF-B82A-176F1AB7B92E@planet.nl> > > when you're working below the reliable stream level, you can't just do a > blocking 'read' for a packet; it pretty much has to be asynchronous > Oh, you should look at the early BBN TCP for V6 Unix - they would have faced the same issue, with their TCP process. They did have the capac() call (which kind of alleviates the need for non-blocking I/O), but that may have only been available for ports/pipes; I'm not sure if the ARPANET device supported it. I did. There is capac() support also for the IMP interface: https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-V6/dmr/imp11a.c (see bottom two functions) BBN took the same approach as Research: with capac() or select() one can prevent blocking on read() and write(). From pnr at planet.nl Tue Jun 2 19:34:54 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 2 Jun 2020 11:34:54 +0200 Subject: [TUHS] non-blocking IO (morphing to "The serial I/O loop") Message-ID: > On 5/31/2020 9:46 AM, Warner Losh wrote: > > Sorry to top post, but LSX or Miniunix had non blocking I/O as well. > > It was in one of the documents that Clem scanned in the last year. It > > specifically was an experiment into how to do it. > > > > Warner > I did add a few new features to LSX to deal with contiguous files > and to handle asynchronous read/write's for real time applications. > They are described in the LSX paper in the 1978 BSTJ on the > UNIX Time-Sharing System. > > Heinz Thanks for highlighting this! The realm here is async I/O to disk, my original scope was limited to “communication” files (tty’s, pipes, network connections). Still, I find it an interesting topic. For others, the paper that you refer to can also be found here: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/UNIX_on_a_Microprocessor_19780322.pdf If I read correctly, the async functionality was available only in the stand alone program version of LSX. Is that correct? In any case, the source code appears lost. From another paper in that set, I get the impression that the async functionality for LSX builds on earlier work for a very early version of Unix: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-74-1352-1_Implementation_of_Large_Contiguous_Files_and_Asynchronous_IO_in_UNIX_19740104.pdf - - - When reading through the papers in that TUHS directory, something else caught my attention: early networking at the labs. For a while I have been puzzled by the “serial I/O loop” in use at the labs in the early seventies. Last Fall I found some 1970/1971 BSTJ papers about it, but there it stopped. I see in the memo on Glance (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-3_GLANCE_Terminals_on_UNIX_Time-Sharing_19750303.pdf) that D.R. Weller continued to work on it up to 1973 at least and that it was integrated with Unix in some way. Is that correct? Did the two memo’s referred to (MM 70-1384-1 and TM 73-1356-8) survive? Then the memo on satellite processors is very interesting (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-78-3114-2_A_Minicomputer_Satellite_Processor_System_19780322.pdf). This appears to show the serial I/O loop in use as late as 1978, with a very intriguing use case involving system call forwarding over the network. Can you tell me more about the serial I/O loop and its use cases? From clemc at ccc.com Tue Jun 2 23:31:28 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 09:31:28 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Frankly, using a common tape format, and then a raw disk as the 'tape' tends to be easy. For instance, if it's an older system like v6, there is a binary for v6tar kicking around that was created as part of the v7 conversion utilities (or you can build it yourself with a little work - the issue a few changes in some headers). Put that binary on your v6 system. Then create a tar image on your modern system. Tar uses a threaded ASCII header and actually had a bug in it, that is exploited as an extension mechanism and became a feature (I'll explain off line if need be). So modern tar's will produce a checksum that the original's will correctly accept. Note the modern tar can create >>file types<< that the old tar will not understand, but it will just skip them. Going the back works too, but is limited by the original's handling of things like directories. It is generally not a problem. So, the result is that you can attach that 'tarball' as a raw disk on simh and then read it with v6tar. Another possible way to go it to try to get stp(1) to compile on the more modern system [it's on the Harvard tape IIRC -- it was the first version of tp in C -- earlier versions were in PDP-11 assembler). But ... since that was written with the a pre-Typesetter C compiler, and has a PDP-11 binary format knowledge in it and I think used Lesk's portable I/O package, so it might be a little more difficult to get running on a more modern C. On Mon, Jun 1, 2020 at 6:45 PM Paul Riley wrote: > Clem, > > Thanks for your help. You’ve correctly interpreted my question. > > Is the disk image independent from the disk hardware? I’d assume that > different disks may have different block sizes etc, so the disk type may be > important. > > The target system is LSX, a cut-down version of V6 designed to run on the > LSI-11. There are very few system utilities in the standard build (no mount > for example). The second floppy is permanently mounted at boot time. I’m > interested in making source file floppies on my modern system to use on the > LSX, so I want to be able to create an image file from a source folder tree. > > Paul > > On Mon, 1 Jun 2020 at 9:05 pm, Clem Cole wrote: > >> >> >> On Mon, Jun 1, 2020 at 6:19 AM Paul Riley wrote: >> >>> Is there a Windows or Linux utility to create a disk image in any of the >>> above formats, from a local folder tree? >>> >> What I think you are asking, is there a utility for a modern OS that will >> walk a local folder tree on my OS and create a new file whose structure is >> that of the file system for OS . >> >> The issue is not the device as much as the OS and disk file layout. As >> far as UNIX (or simh at the OS level) is concerned, the disk is just a >> linear array of bytes, addressed by blocks. The physical format is not >> seen by UNIX. >> >> There are numerious utilities, as well as 'foreign file systems' that are >> available. For instance, many Unix's can write RT-11 and MS-DOS format >> with standard utilities. It really depends the OS. That said, >> if the target OS is modern enough to support NFS or Samba, the easiest >> way might be export the file system from local system, and then running a >> simulated OS, 'mount' the file system. >> > -- > *Paul Riley* > > Mo: +86 186 8227 8332 > Email: paul at rileyriot.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Tue Jun 2 23:52:44 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 09:52:44 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Mon, Jun 1, 2020 at 6:45 PM Paul Riley wrote: > Is the disk image independent from the disk hardware? I’d assume that > different disks may have different block sizes etc, so the disk type may be > important. > A reasonable thought, but no, other than efficiency. It's just an array of bytes exposed to the OS. The good news is that all the DEC systems used 512 byte fixed blocks (Nothing like the funky Prime, Apollo or Alto tricks of putting some of the file system metadata into the physical disk format -- if you ever wondered why some of the original SCSI controllers supported a 1072 byte block size - it was for Domain/Aegis - 1024 block + 48 byte meta data]. > > The target system is LSX, a cut-down version of V6 designed to run on the > LSI-11. > Everything should be able to be simulated using is 512-byte blocks. > There are very few system utilities in the standard build (no mount for > example). The second floppy is permanently mounted at boot time. I’m > interested in making source file floppies on my modern system to use on the > LSX, so I want to be able to create an image file from a source folder tree. > You'll have to check the docs, but I thought Heinz supported the RK05 [RKV11 controller]. That will give you 2.5M of 'raw' disk blocks, instead of floppy. Open it raw, and use it like mag tape. If you don't have a coming format, you can create something and then use dd(1) on the LSI-11 from the RKV11 -- a little clumsy but as a bootstrap, it should work fine. The RL11 and RK711 controllers (RL01/02, RK07 disk) came later to the PDP-11 (post 11/34 and 11/60). I think there was an RLV1[12] at some point, but I don't think DEC even made an RKV71[12] -- however, I'm not up on late-generation PDP-11 lore and they would have been much later than LSX so driver support is doubtful. Good luck, Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Wed Jun 3 00:19:26 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 2 Jun 2020 16:19:26 +0200 Subject: [TUHS] non-blocking IO Message-ID: > At around that point in time (I don't have the very _earliest_ code, to get an exact date, but the oldest traces I see [in mapalloc(), below] are from September '78), the CSR group at MIT-LCS (which were the people in LCS doing networking) was doing a lot with asynchronous I/O (when you're working below the reliable stream level, you can't just do a blocking 'read' for a packet; it pretty much has to be asynchronous). I was working in Unix V6 - we were building an experimental 1Mbit/second ring - and there was work in Multics as well. > I don't think the wider Unix community heard about the Unix work, but our group regularly filed updates on our work for the 'Internet Monthly Reports', which was distributed to the whole TCP/IP experimental community. If you can find an archive of early issues (I'm too lazy to go look for one), we should be in there (although our report will alsocover the Multics TCP/IP work, and maybe some other stuff too). Sounds very interesting! Looked around a bit, but I did not find a source for the “Internet Monthly Reports” for the late 70’s (rfc-editor.org/museum/ has them for the 1990’s). In the 1970’s era, it seems that NCP Unix went in another direction, using newly built message and event facilities to prevent blocking. This is described in "CAC Technical Memorandum No. 84, Illinois Inter-Process Communication Facility for Unix.” - but that document appears lost as well. Ah, well, topics for another day. From ron at ronnatalie.com Wed Jun 3 01:44:00 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Tue, 2 Jun 2020 11:44:00 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: > On Jun 2, 2020, at 9:52 AM, Clem Cole wrote: > > > The good news is that all the DEC systems used 512 byte fixed blocks much later than LSX so driver support is doubtful. > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 and a odd interleaving strategy. However, the boot block and the rest of the file systems (both UNIX and RT at least) just aggregated the smaller (logical) sectors together to make a 512 byte one. -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Wed Jun 3 02:26:54 2020 From: imp at bsdimp.com (Warner Losh) Date: Tue, 2 Jun 2020 10:26:54 -0600 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020, 9:45 AM Ronald Natalie wrote: > > > On Jun 2, 2020, at 9:52 AM, Clem Cole wrote: > > > The good news is that all the DEC systems used 512 byte fixed blocks much > later than LSX so driver support is doubtful. > > > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 > and a odd interleaving strategy. However, the boot block and the rest of > the file systems (both UNIX and RT at least) > just aggregated the smaller (logical) sectors together to make a 512 byte > one. > RX50 also had weird interleaving... but it was quite the odd duck. Warner > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Wed Jun 3 03:45:55 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 13:45:55 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: The operating systems that I cut my teeth on (OS/360, DOS/360, VAX/VMS) all had basic I/O system calls that were non-blocking. Blocking I/O calls were all built on top of that framework. I thus found it curious that Unix took the opposite tack, and non-blocking I/O was an afterthought. So I'm curious as to what the rationale was for Unix to have been designed with basic I/O being blocking rather than asynchronous. Especially that non-blocking I/O primitives were the norm for OSes in those days. -Paul W. From arnold at skeeve.com Wed Jun 3 03:59:05 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Tue, 02 Jun 2020 11:59:05 -0600 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: <202006021759.052Hx5Et022619@freefriends.org> Paul Winalski wrote: > So I'm curious as to what the rationale was for Unix to have been > designed with basic I/O being blocking rather than asynchronous. I don't doubt that it was "simplify, simplify, simplify". Async I/O is much messier than Unix's read/write model. The Unix model was simpler to design, simpler to code, simpler to get right, and undoubtedly took much less OS code than an async model would have; on the PDP-11 that would have mattered. Also, the early Unixs were on smaller -11s, not the /45 or /70 with split I&D space and the ability to address lost more RAM. My guess, anyway. :-) Arnold From clemc at ccc.com Wed Jun 3 04:08:36 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 14:08:36 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020 at 11:44 AM Ronald Natalie wrote: > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 > and a odd interleaving strategy. However, the boot block and the rest of > the file systems (both UNIX and RT at least) just aggregated the smaller > (logical) sectors together to make a 512 byte one. > Indeed, I was less than precise -- the key is that the SW treats them everything as being in 512 byte 'hunks' no matter the underlying physical formatting/interleaving actually is. The point is that if you Paul is trying to move data from the two systems, just treat the device as a serial byte stream of 512 byte blocks on both sides and you you be fine. That means if using dd, add conv=sync to fill any write buffer to a complete multiple of 512. -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Wed Jun 3 04:23:59 2020 From: crossd at gmail.com (Dan Cross) Date: Tue, 2 Jun 2020 14:23:59 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020 at 1:47 PM Paul Winalski wrote: > The operating systems that I cut my teeth on (OS/360, DOS/360, > VAX/VMS) all had basic I/O system calls that were non-blocking. > Blocking I/O calls were all built on top of that framework. I thus > found it curious that Unix took the opposite tack, and non-blocking > I/O was an afterthought. > > So I'm curious as to what the rationale was for Unix to have been > designed with basic I/O being blocking rather than asynchronous. > Especially that non-blocking I/O primitives were the norm for OSes in > those days. Doug addressed this, albeit in an oblique manner, on this list back in 2015: https://minnie.tuhs.org/pipermail/tuhs/2015-September/007509.html Quoting him: """ Unix was what the authors wanted for a productive computing environment, not a bag of everything they thought somebody somewhere might want. One objective, perhaps subliminal originally, was to make program behavior easy to reason about. Thus pipes were accepted into research Unix, but more general (and unruly) IPC mechanisms such as messages and events never were. The infrastructure had to be asynchronous. The whole point was to surmount that difficult model and keep everyday programming simple. User visibility of asynchrony was held to a minimum: fork(), signal(), wait(). Signal() was there first and foremost to support SIGKILL; it did not purport to provide a sound basis for asynchronous IPC. The complexity of sigaction() is evidence that asynchrony remains untamed 40 years on. """ My response at the time was to question whether asynchrony itself remains untamed, as Doug put it, or if rather it has proved difficult to retrofit asynchrony onto a system designed around fundamentally synchronous primitives? - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Wed Jun 3 04:53:59 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 14:53:59 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <202006021759.052Hx5Et022619@freefriends.org> References: <202006021759.052Hx5Et022619@freefriends.org> Message-ID: On 6/2/20, arnold at skeeve.com wrote: > Paul Winalski wrote: > >> So I'm curious as to what the rationale was for Unix to have been >> designed with basic I/O being blocking rather than asynchronous. > > Also, the early Unixs were on smaller -11s, not the /45 or /70 with > split I&D space and the ability to address lost more RAM. I first encountered DOS/360 on a System/360 model 25 with 48K of memory. This was a one-job-at-a-time batch system, but the I/O primitive (EXCP--execute channel program) was asynchronous. So I don't think the small memory rationale really applies. -Paul W. From paul.winalski at gmail.com Wed Jun 3 04:56:59 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 14:56:59 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On 6/2/20, Dan Cross wrote: > > My response at the time was to question whether asynchrony itself remains > untamed, as Doug put it, or if rather it has proved difficult to retrofit > asynchrony onto a system designed around fundamentally synchronous > primitives? I think that's a very good question. It's analogous to record-oriented I/O vs. byte stream I/O. It's easy to build record-oriented I/O on top of a byte stream, but it's a real bear to do it the other way around. Similarly, it's easy to build synchronous I/O on top of asynchronous I/O but the reverse ends up looking contrived. -Paul W. From paul.winalski at gmail.com Wed Jun 3 04:59:37 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 14:59:37 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On 6/2/20, Ronald Natalie wrote: > > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 > and a odd interleaving strategy. However, the boot block and the rest of > the file systems (both UNIX and RT at least) > just aggregated the smaller (logical) sectors together to make a 512 byte > one. At least they all supported fixed-length sectors. Was there ever a Unix implementation that supported IBM-style CKD disk devices? -Paul W. From athornton at gmail.com Wed Jun 3 05:08:23 2020 From: athornton at gmail.com (Adam Thornton) Date: Tue, 2 Jun 2020 12:08:23 -0700 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: <3A17BE73-777A-4CF2-8D55-A5C802FC767D@gmail.com> > On Jun 2, 2020, at 11:59 AM, Paul Winalski wrote: > > On 6/2/20, Ronald Natalie wrote: >> >> Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 >> and a odd interleaving strategy. However, the boot block and the rest of >> the file systems (both UNIX and RT at least) >> just aggregated the smaller (logical) sectors together to make a 512 byte >> one. > > At least they all supported fixed-length sectors. Was there ever a > Unix implementation that supported IBM-style CKD disk devices? If you count Linux, sure, Linux for the S/390 and zSeries. If you don’t…well, for one brief shining moment before IBM squeezed too hard on the price and Oracle bought Sun instead, there was an OpenSolaris port to the zSeries, and it did support 3390 type devices. I may have had something to do with that latter device driver. Adam From clemc at ccc.com Wed Jun 3 05:18:16 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 15:18:16 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> Message-ID: On Tue, Jun 2, 2020 at 2:54 PM Paul Winalski wrote: > I first encountered DOS/360 on a System/360 model 25 with 48K of > memory. This was a one-job-at-a-time batch system, but the I/O > primitive (EXCP--execute channel program) was asynchronous. So I > don't think the small memory rationale really applies. > Hrrmpt... it was single task. Being asynchronous in the I/O and allowed process asynchrony takes a lot more housekeeping. Paul you know I agree with you, it was always an issue with UNIX IMO. The problem is how to do it differently. At Masscomp RRU, our solution was not to try to 'fix it' as much as add a new scheme beside the synchronous one. We added a general AST's mechanism that anyone could use (very much like RSX and VMS in semantics), but left signals alone. We added new async calls, which were just implemented via ioctl's (the universal hack) for the specific HW that supported it. In retrospect, that was an error, it should have been aread(2)/awrite(2) and then added the completion routine/call back as the 4th & 5th parameters. Since Stellix was not Real-Time, we left them out. [ tjt and I have argued about that for years ]. So back to Doug/Dan's answer -- I think for a small system, like the original PDP-7 and the PDP-11/20 putting the effort to make it multiprocess and leaving the I/O synchronous definitely made it easier. Particularly, since what was created with the signal semantics. To me, UNIX got way more right than wrong. I suspect if signals had been more like AST's, the idea of being async might have gone farther. But to use Dennis's like about C being quirky, signals are very quirky. wnj tried to 'fix' them and frankly it just made matters worse. And IMO, signaction(3) as Doug says, is a nightmare. I've generally been a fan of introducing a new idea separately as a 'stronger strain' and then seeing if people likely it. FWIW: A couple of us did try to get AST's into the POSIX.4 (they were there in a draft), but ultimately *.4 was rejected as 'not UNIX.' -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Wed Jun 3 05:23:20 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 15:23:20 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020 at 2:58 PM Paul Winalski wrote: > I think that's a very good question. It's analogous to > record-oriented I/O vs. byte stream I/O. It's easy to build > record-oriented I/O on top of a byte stream, but it's a real bear to > do it the other way around. Similarly, it's easy to build synchronous > I/O on top of asynchronous I/O but the reverse ends up looking contrived. > Which was exactly the point I tried to make in the POSIX.4 discussions, but it does take more work in the basic housekeeping and you need a way to handle events and completions that are priority based, queued, and a few other details. As Doug said, they stayed away from some features (like messaging). async I/O was one of them. But as I said, Ken, Dennis and the rest of the crew did an amazing job with very little. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnc at mercury.lcs.mit.edu Wed Jun 3 06:13:34 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Tue, 2 Jun 2020 16:13:34 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200602201334.95D9718C079@mercury.lcs.mit.edu> > From: Paul Winalski > I'm curious as to what the rationale was for Unix to have been designed > with basic I/O being blocking rather than asynchronous. It's a combination of two factors, I reckon. One, which is better depends a lot on the type of thing you're trying to do. For many typical thing (e.g. 'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is more complicated, and Unix was (well, back then at least) all about getting the most bang for the least bucks. More complicated things do sometimes benefit from asynchronous I/O, but complicated things weren't Unix's 'target market'. E.g. even though pipes post-date the I/O decision, they too are a better match to blocking I/O. > From: Arnold Skeeve > the early Unixs were on smaller -11s, not the /45 or /70 with split I&D > space and the ability to address lost more RAM. Ahem. Lots more _core_. People keeep forgetting that we're looking at decicions made at a time when each bit in main memory was stored in a physically separate storage device, and having tons of memory was a dream of the future. E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total! And that had to hold the resident OS, plus the application! It's no surprise that Unix was so focused on small size - and as a corollary, on high bang/buck ratio. But even in his age of lighting one's cigars with gigabytes of main memory (literally), small is still beautiful, because it's easier to understand, and complexity is bad. So it's too bad Unix has lost that extreme parsimony. > From: Dan Cross > question whether asynchrony itself remains untamed, as Doug put it, or > if rather it has proved difficult to retrofit asynchrony onto a system > designed around fundamentally synchronous primitives? I'm not sure it's 'either or'; I reckon they are both true. Noel From clemc at ccc.com Wed Jun 3 06:43:25 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 16:43:25 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <20200602201334.95D9718C079@mercury.lcs.mit.edu> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> Message-ID: On Tue, Jun 2, 2020 at 4:14 PM Noel Chiappa wrote: > Ahem. Lots more _core_. People keeep forgetting that we're looking at > decicions made at a time when each bit in main memory was stored in a > physically separate storage device, and having tons of memory was a dream > of > the future. > Yeah -- that is something that forgotten. There's a kit/hackday project to make 32-byte core for an Arduino I did with some of my boy scouts doing electronic MB a while back just to try to give them a feel what a 'bit' was. Similarly, there was a update of in late 1960's children's book originally called 'A Million' it's now called: A Million Dots Each page has 10K dots. The idea is to help young readers get a real feel for what 'a million' means visually. > > E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total! > And that had to hold the resident OS, plus the application! It's no > surprise that Unix was so focused on small size - and as a corollary, on > high bang/buck ratio.' Amen -- I ran an 11/34 with 64K under V6 for about 3-6 months while we were awaiting the 256K memory upgrade. > > > But even in his age of lighting one's cigars with gigabytes of main memory > (literally), small is still beautiful, because it's easier to understand, > and > complexity is bad. So it's too bad Unix has lost that extreme parsimony. > Yep -- I think we were discussing this last week WRT to cat -v/fmt et al. I fear some people confuse 'progress' with 'feature creep.' Just because we can do something, does not mean we should. As I said, I'm a real fan of async I/O and like Paul, feel that it is a 'better' primitive. But I fully understand and accept, that given the tradeoffs of the time, UNIX did really well and I much prefer what we got than the alternative. I'm happy we ended up with simply and just works. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewart at serissa.com Wed Jun 3 07:15:21 2020 From: stewart at serissa.com (Lawrence Stewart) Date: Tue, 2 Jun 2020 17:15:21 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> Message-ID: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> I remember working on getting Arpanet access on an 11/34 running V7 around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run NCP, using a variation of Rand’s code. I wrote some sort of bisync driver for packet communications between the /23 and the /34, and I think added an IOCTL or some hack to ask if there was a message ready. So a polling variety of non-blocking I/O :) Meanwhile, on the Alto, surely an underpowered machine, the style for UI programming and asynch I/O was mostly event driven, much like libevent in more recent years. I found that very easy to understand. The main trajectory of PARC was lightweight threads calling synchronous I/O which really has to be counted as a Bad Idea not yet fully stamped out. I’ve always thought select(2) was a pretty good thing. In 1995 at Open Market we had a single-process web server that had no difficulties running 1000 to 1200 connections. I think that was BSD, and later OSF-1. -L From rdm at cfcl.com Wed Jun 3 08:14:10 2020 From: rdm at cfcl.com (Rich Morin) Date: Tue, 2 Jun 2020 15:14:10 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> Message-ID: <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> Case study (read, war story :-) of non-blocking I/O... Back in the mid-70's, I was doing scientific support programming for an X-Ray sky survey, using Fortran-63 under DRUM SCOPE on a CDC 3800 system which NRL's Space Sciences Division had (sigh) acquired and cobbled together from the US Government "excess property" list. Most of the time, our satellite was used in "scan mode", spinning on an axis which pointed at the Sun. This let us collect data on a two degree wide strip of sky. So, every six months we scanned the entire X-ray sky. The track was basically a helix, bent to match the shape of the Earth's orbit. Every so often, in order to look at a particularly interesting area, the spinning would be stopped, so the instrument could be put into "point mode". The challenge was to "bin" this data for further analysis. We did this by loading sets of 7-track tapes onto a CDC 813 drive, then dumping it out into 36 2x10 degree bins. This put a quarter of the data onto a set of tapes. Rinse, repeat for the other 3/4 of the data (after skipping past the already-written data). The result was 36 tapes, each holding an 8x10 degree bin (plus one extra tape for any point mode data). IIRC, we had five tape drives; my challenge was to keep them all as busy as possible, so as to dump the data set expeditiously. Because I had asynchronous I/O (mostly in the form of BUFFER IN and BUFFER OUT commands), I was able to implement a simple but quite effective polling loop. The machine room was a bit of a madhouse, but the tapes were written about as quickly as the hardware allowed. Asynchronous I/O FTW... https://en.wikipedia.org/wiki/United_States_Naval_Research_Laboratory#Space_sciences https://en.wikipedia.org/wiki/High_Energy_Astronomy_Observatory_1#A1:_Large-Area_Sky_Survey_instrument https://ub.fnwi.uva.nl/computermuseum/cdcdisk.html -r From jon at fourwinds.com Wed Jun 3 08:03:58 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Tue, 02 Jun 2020 15:03:58 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> Message-ID: <202006022203.052M3wpa167150@darkstar.fourwinds.com> Lawrence Stewart writes: > I’ve always thought select(2) was a pretty good thing. In 1995 at Open > Market we had a single-process web server that had no difficulties > running 1000 to 1200 connections. I think that was BSD, and later OSF-1. So select was a great thing until interactive graphics came along. Just got too messy. I recall writing assembly language code to support threads in the mid to late 1980s, and I remember reading some papers from Sun about the same. Everybody was doing it. Anybody have enough of the history to paint a picture of how we got from there to standardized threads? Jon From robpike at gmail.com Wed Jun 3 09:05:39 2020 From: robpike at gmail.com (Rob Pike) Date: Wed, 3 Jun 2020 09:05:39 +1000 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <202006022203.052M3wpa167150@darkstar.fourwinds.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: It's only messy if you make a messy job of it. -rob On Wed, Jun 3, 2020 at 8:36 AM Jon Steinhart wrote: > Lawrence Stewart writes: > > I’ve always thought select(2) was a pretty good thing. In 1995 at Open > > Market we had a single-process web server that had no difficulties > > running 1000 to 1200 connections. I think that was BSD, and later OSF-1. > > So select was a great thing until interactive graphics came along. Just > got too messy. I recall writing assembly language code to support threads > in the mid to late 1980s, and I remember reading some papers from Sun about > the same. Everybody was doing it. Anybody have enough of the history to > paint a picture of how we got from there to standardized threads? > > Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robpike at gmail.com Wed Jun 3 09:09:08 2020 From: robpike at gmail.com (Rob Pike) Date: Wed, 3 Jun 2020 09:09:08 +1000 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: To be more forthcoming, there was a long period when the received wisdom was that threads were unmanageable. But that was a category error: The programming model in POSIX and others was what made it hard. The fundamental concepts are a tiny step harder than trivial, and it was not hard to build a programming model that matched them. The first mistake people made was to split "thread" from "process". What those words mean today is not what they meant a generation ago. For me, it's like the mess sockets made of networking. Our programming models could be so much simpler than what we are required to deal with. -rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon at fourwinds.com Wed Jun 3 09:12:22 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Tue, 02 Jun 2020 16:12:22 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: <202006022312.052NCMCR178629@darkstar.fourwinds.com> Rob Pike writes: > > It's only messy if you make a messy job of it. Maybe I should explain what I meant by messy. To me, the issue with select is that in many cases one has to either keep a lot of state somewhere, or live with long response times while handlers complete. Threads allowed handlers to be written in the "normal" way with state kept on the stack. From lm at mcvoy.com Wed Jun 3 09:21:14 2020 From: lm at mcvoy.com (Larry McVoy) Date: Tue, 2 Jun 2020 16:21:14 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: <20200602232114.GA22016@mcvoy.com> On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote: > The first mistake people made was to split "thread" from "process". What > those words mean today is not what they meant a generation ago. So I'm a fan of your quote, Rob, "If you think you need threads, your processes are too fat". Years ago, I asked Linus if he could fix the one flaw with that point of view. Which is, processes can't share page tables. So N processes will have N page tables, need an N way bigger TBL, to get the same performance. What I asked Linus to try and do is allow a process to share all or part of its page table with another process. If that could be done, we could pretty do what you said, just use processes and if they want to act like threads, just share the page table. And I just looked, he seems to have done it. man 2 clone on a Linux box. It's the swiss army knife of features, feels messy. I don't know if CLONE_VM will share the page tables, I'll ask Linus. --lm From robpike at gmail.com Wed Jun 3 10:39:45 2020 From: robpike at gmail.com (Rob Pike) Date: Wed, 3 Jun 2020 10:39:45 +1000 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <20200602232114.GA22016@mcvoy.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> <20200602232114.GA22016@mcvoy.com> Message-ID: See the description of rfork at http://man.cat-v.org/plan_9/2/fork. -rob On Wed, Jun 3, 2020 at 9:21 AM Larry McVoy wrote: > On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote: > > The first mistake people made was to split "thread" from "process". What > > those words mean today is not what they meant a generation ago. > > So I'm a fan of your quote, Rob, "If you think you need threads, your > processes are too fat". > > Years ago, I asked Linus if he could fix the one flaw with that point > of view. Which is, processes can't share page tables. So N processes > will have N page tables, need an N way bigger TBL, to get the same > performance. > > What I asked Linus to try and do is allow a process to share all or > part of its page table with another process. If that could be done, > we could pretty do what you said, just use processes and if they want > to act like threads, just share the page table. > > And I just looked, he seems to have done it. man 2 clone on a > Linux box. It's the swiss army knife of features, feels messy. > > I don't know if CLONE_VM will share the page tables, I'll ask > Linus. > > --lm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Wed Jun 3 10:43:18 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Tue, 2 Jun 2020 17:43:18 -0700 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) Message-ID: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org> > Would you be so kind to explain a bit about the hm version of MH. Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my Fortune box until I retired it. It provided a 2D interface. Later I tried xmh but didn't like it. The early history of MH is covered in some detail in Willis Ware's "RAND and the Information Evolution" book from page 128 onward. hm & Guyton get a paragraph on page 136. https://www.rand.org/content/dam/rand/pubs/corporate_pubs/2008/RAND_CP537.pdf From clemc at ccc.com Wed Jun 3 10:49:46 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 20:49:46 -0400 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org> References: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org> Message-ID: On Tue, Jun 2, 2020 at 8:43 PM Bakul Shah wrote: > > Would you be so kind to explain a bit about the hm version of MH. > > Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my > Fortune box until I retired it. It provided a 2D interface. > Yep, that would be the program .. Jim did a great job with it. It was built using a bunch of MH programs behind it, and was a curses front end. -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Wed Jun 3 15:12:27 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Tue, 02 Jun 2020 23:12:27 -0600 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi Message-ID: <202006030512.0535CRFC020727@freefriends.org> Hi. Does anyone have a mirror of the files that once upon a time were at http://alf.uib.no/pub/vi? They were mirrored at ftp://ftp.uu.net/pub/text-processing/vi also. If so, can you please send me a tarball off list (or tell me where I can download a copy from)? Thanks! Arnold From heinz at osta.com Wed Jun 3 15:41:16 2020 From: heinz at osta.com (Heinz Lycklama) Date: Tue, 2 Jun 2020 22:41:16 -0700 Subject: [TUHS] non-blocking IO (morphing to "The serial I/O loop") In-Reply-To: References: Message-ID: <3e482f51-2c2e-19a8-c9fd-f23c67f1df8c@osta.com> See comments below. Heinz On 6/2/2020 2:34 AM, Paul Ruizendaal wrote: >> On 5/31/2020 9:46 AM, Warner Losh wrote: >>> Sorry to top post, but LSX or Miniunix had non blocking I/O as well. >>> It was in one of the documents that Clem scanned in the last year. It >>> specifically was an experiment into how to do it. >>> >>> Warner >> I did add a few new features to LSX to deal with contiguous files >> and to handle asynchronous read/write's for real time applications. >> They are described in the LSX paper in the 1978 BSTJ on the >> UNIX Time-Sharing System. >> >> Heinz > Thanks for highlighting this! > > The realm here is async I/O to disk, my original scope was limited to “communication” files (tty’s, pipes, network connections). Still, I find it an interesting topic. > > For others, the paper that you refer to can also be found here: > https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/UNIX_on_a_Microprocessor_19780322.pdf > If I read correctly, the async functionality was available only in the stand alone program version of LSX. Is that correct? In any case, the source code appears lost. Any source code for LSX would be archived somewhere within Nokia Bell Labs? > From another paper in that set, I get the impression that the async functionality for LSX builds on earlier work for a very early version of Unix: > https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-74-1352-1_Implementation_of_Large_Contiguous_Files_and_Asynchronous_IO_in_UNIX_19740104.pdf Yes, as this memo describes, I implemented a very simple version of large contiguous files and asynchronous file I/O for a special application. The MERT system developed a little later on, had a completely integrated of implementation of contiguous files and asynchronous file I/O. > - - - > > When reading through the papers in that TUHS directory, something else caught my attention: early networking at the labs. For a while I have been puzzled by the “serial I/O loop” in use at the labs in the early seventies. Last Fall I found some 1970/1971 BSTJ papers about it, but there it stopped. Jon Steinhart would have more information about the serial I/O loo. > I see in the memo on Glance (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-3_GLANCE_Terminals_on_UNIX_Time-Sharing_19750303.pdf) that D.R. Weller continued to work on it up to 1973 at least and that it was integrated with Unix in some way. Is that correct? Did the two memo’s referred to (MM 70-1384-1 and TM 73-1356-8) survive? I do not believe that the software for the Glance terminals was ever integrated into the standard UNIX software distribution > Then the memo on satellite processors is very interesting (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-78-3114-2_A_Minicomputer_Satellite_Processor_System_19780322.pdf). This appears to show the serial I/O loop in use as late as 1978, with a very intriguing use case involving system call forwarding over the network. Carl Christensen and I developed this software to handle small computer systems with disk storage, as described in the memo, and in the BSTJ article. The satellite diskless computers could be attached to a powerful central computer by means the serial I/O loop or various standard DEC controllers. Each of the satellite computers could run different tailored applications > Can you tell me more about the serial I/O loop and its use cases? I think Jon Steinhart has described this elsewhere? > > > > From lars at nocrew.org Wed Jun 3 15:38:29 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Wed, 03 Jun 2020 05:38:29 +0000 Subject: [TUHS] Unix on the Arpanet In-Reply-To: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> (Lawrence Stewart's message of "Tue, 2 Jun 2020 17:15:21 -0400") References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> Message-ID: <7wtuzs3e0a.fsf_-_@junk.nocrew.org> Lawrence Stewart wrote: > I remember working on getting Arpanet access on an 11/34 running V7 > around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run > NCP, using a variation of Rand’s code. Has this, or Rand's code, been preserved? I'm only aware of one Arpanet NCP implementation for Unix that is online, the one from University of Illinois. From stewart at serissa.com Wed Jun 3 22:23:33 2020 From: stewart at serissa.com (Lawrence Stewart) Date: Wed, 3 Jun 2020 08:23:33 -0400 Subject: [TUHS] Unix on the Arpanet In-Reply-To: <7wtuzs3e0a.fsf_-_@junk.nocrew.org> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <7wtuzs3e0a.fsf_-_@junk.nocrew.org> Message-ID: <68231FF8-6E05-49AE-A33A-A22D5A05966A@serissa.com> > On 2020, Jun 3, at 1:38 AM, Lars Brinkhoff wrote: > > Lawrence Stewart wrote: >> I remember working on getting Arpanet access on an 11/34 running V7 >> around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run >> NCP, using a variation of Rand’s code. > > Has this, or Rand's code, been preserved? > > I'm only aware of one Arpanet NCP implementation for Unix that is > online, the one from University of Illinois. Alas I do not know. I may have some old emails from that era. I will check. From clemc at ccc.com Wed Jun 3 22:33:10 2020 From: clemc at ccc.com (Clem Cole) Date: Wed, 3 Jun 2020 08:33:10 -0400 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi In-Reply-To: <202006030512.0535CRFC020727@freefriends.org> References: <202006030512.0535CRFC020727@freefriends.org> Message-ID: I think so... I'll send you a URL off line. Clem On Wed, Jun 3, 2020 at 1:14 AM wrote: > Hi. > > Does anyone have a mirror of the files that once upon a time were > at http://alf.uib.no/pub/vi? They were mirrored at > ftp://ftp.uu.net/pub/text-processing/vi also. > > If so, can you please send me a tarball off list (or tell me where > I can download a copy from)? > > Thanks! > > Arnold > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Wed Jun 3 22:49:10 2020 From: clemc at ccc.com (Clem Cole) Date: Wed, 3 Jun 2020 08:49:10 -0400 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi In-Reply-To: <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> References: <202006030512.0535CRFC020727@freefriends.org> <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> Message-ID: It is too big to attach to email. On Wed, Jun 3, 2020 at 8:39 AM Michael Stiller wrote: > Hi Clem, > > why offline? Other people are also interested. :-) > > Cheers, > > Michael > > > > On 3. Jun 2020, at 14:33, Clem Cole wrote: > > > > I think so... I'll send you a URL off line. > > > > Clem > > > > On Wed, Jun 3, 2020 at 1:14 AM wrote: > > Hi. > > > > Does anyone have a mirror of the files that once upon a time were > > at http://alf.uib.no/pub/vi? They were mirrored at > > ftp://ftp.uu.net/pub/text-processing/vi also. > > > > If so, can you please send me a tarball off list (or tell me where > > I can download a copy from)? > > > > Thanks! > > > > Arnold > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Thu Jun 4 02:31:57 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Wed, 3 Jun 2020 12:31:57 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> Message-ID: On 6/2/20, Rich Morin wrote: > > IIRC, we had five tape drives; my challenge was to keep them all as busy as > possible, so as > to dump the data set expeditiously. Because I had asynchronous I/O (mostly > in the form of > BUFFER IN and BUFFER OUT commands), I was able to implement a simple but > quite effective > polling loop. The machine room was a bit of a madhouse, but the tapes were > written about > as quickly as the hardware allowed. Asynchronous I/O FTW... With 9-track magnetic tape devices, reading and writing can't start until the tape is up to speed. Once up to speed the drive can read and write records while keeping the tape moving at speed. This is called streaming. If there's a pause in the read/write requests from the CPU, time is lost as the drive stops and starts moving the tape. It was essential that applications doing large amounts of tape I/O keep up the I/O requests at a rate that allows streaming. Asynchronous I/O with multi-buffering is a straightforward way to accomplish this. The IBM S/360 channel commands for tape devices provided a mechanism for the tape control unit to send an interrupt to the CPU when a read or write channel command completed. This notified the sequential access method (the user program I/O interface) when I/O to each buffer had completed and the buffer was available for reuse. OS/360's Sequential Access Method could read or write an entire tape using a single SIO (start I/O) instruction, as long as no read or write errors were encountered. -Paul W. From paul.winalski at gmail.com Thu Jun 4 02:42:28 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Wed, 3 Jun 2020 12:42:28 -0400 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <202006022312.052NCMCR178629@darkstar.fourwinds.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> <202006022312.052NCMCR178629@darkstar.fourwinds.com> Message-ID: On 6/2/20, Jon Steinhart wrote: > > Maybe I should explain what I meant by messy. To me, the issue with select is > that in many cases one has to either keep a lot of state somewhere, or live with > long response times while handlers complete. Threads allowed handlers to be > written in the "normal" way with state kept on the stack. > TOPS-10 had a nifty program called ptycon that would allow one to control multiple login sessions, each on its own pseudo-terminal device, from a single physical terminal. In 1985 our software tools group implemented a pseudo-terminal device driver for VMS, and I wrote a program called ptycon-32. It was multithreaded, with one thread of control for each pseudo-terminal. The problem is that VMS at the time didn't have a threads package. I had to roll my own threads by using the asynchronous system traps (ASTs) delivered by completion of I/O to the pseudo-terminals. Talk about messy.... -Paul W. From nobozo at gmail.com Thu Jun 4 03:57:33 2020 From: nobozo at gmail.com (Jon Forrest) Date: Wed, 3 Jun 2020 10:57:33 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> <202006022312.052NCMCR178629@darkstar.fourwinds.com> Message-ID: <332ca1eb-f862-b028-a2a2-e168399576a1@gmail.com> On 6/3/2020 9:42 AM, Paul Winalski wrote: > The problem is that VMS at the time > didn't have a threads package. I had to roll my own threads by using > the asynchronous system traps (ASTs) delivered by completion of I/O to > the pseudo-terminals. Talk about messy.... That's also how the Sybase SQL Server worked on VMS at the same time. Jon From jpl.jpl at gmail.com Thu Jun 4 05:19:40 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Wed, 3 Jun 2020 15:19:40 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> Message-ID: On a distantly related note, when I worked part time for the MIT administration in the 60's, we'd do processing on the 7094 in the main comp center, then bring a tape of results back to our office to do the printing and card punching. For reasons that were never explained to me, the blocks of print and punch data were combined on a single tape, and were distinguished by being written with different parities. If you attempted to read a block with the correct parity setting, the tape could stream. If you guessed wrong, you had to stop, back up a block, change the parity, and try again. So I wrote a simple 360 assembly program to keep track how often blocks of each parity followed the observed parities of the previous 8 blocks. Essentially, 256 pairs of parity observations, indexed by the previous 8 parity observations. Blocks of print and punch data tended to fall into patterns that depended on what job was being run on the 7094, so detecting those patterns and correctly anticipating the upcoming parity made the tapes move much more smoothly. It was fun to watch the tape at the start of a run. It was mostly just a coin-toss, so the tape was jerking around fitfully. As the patterns started to emerge, the predictions got better, the jerking got less and less common, and the tapes were streaming most of the time. My introduction to learning algorithms. On Wed, Jun 3, 2020 at 12:33 PM Paul Winalski wrote: > On 6/2/20, Rich Morin wrote: > > > > IIRC, we had five tape drives; my challenge was to keep them all as busy > as > > possible, so as > > to dump the data set expeditiously. Because I had asynchronous I/O > (mostly > > in the form of > > BUFFER IN and BUFFER OUT commands), I was able to implement a simple but > > quite effective > > polling loop. The machine room was a bit of a madhouse, but the tapes > were > > written about > > as quickly as the hardware allowed. Asynchronous I/O FTW... > > With 9-track magnetic tape devices, reading and writing can't start > until the tape is up to speed. Once up to speed the drive can read > and write records while keeping the tape moving at speed. This is > called streaming. If there's a pause in the read/write requests from > the CPU, time is lost as the drive stops and starts moving the tape. > It was essential that applications doing large amounts of tape I/O > keep up the I/O requests at a rate that allows streaming. > Asynchronous I/O with multi-buffering is a straightforward way to > accomplish this. The IBM S/360 channel commands for tape devices > provided a mechanism for the tape control unit to send an interrupt to > the CPU when a read or write channel command completed. This notified > the sequential access method (the user program I/O interface) when I/O > to each buffer had completed and the buffer was available for reuse. > OS/360's Sequential Access Method could read or write an entire tape > using a single SIO (start I/O) instruction, as long as no read or > write errors were encountered. > > -Paul W. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnu at toad.com Thu Jun 4 11:24:52 2020 From: gnu at toad.com (John Gilmore) Date: Wed, 03 Jun 2020 18:24:52 -0700 Subject: [TUHS] non-blocking IO: Simplicity has been subtituted for efficiency In-Reply-To: <20200602201334.95D9718C079@mercury.lcs.mit.edu> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> Message-ID: <8066.1591233892@hop.toad.com> Noel Chiappa wrote: > > I'm curious as to what the rationale was for Unix to have been designed > > with basic I/O being blocking rather than asynchronous. > > It's a combination of two factors, I reckon. One, which is better depends a > lot on the type of thing you're trying to do. For many typical thing (e.g. > 'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is > more complicated, and Unix was (well, back then at least) all about getting > the most bang for the least bucks. I just happened to open a binder today of old papers about UNIX(tm), including Ken Thompson's "Unix Implementation" paper, which says at the bottom of the introduction: "What is or is not implemented in the kernel represents both a great responsibiity and a great power. It is a soap-box platform on "the way things should be done." Even so, if "the way" is too radical, no one will follow it. Every important decision was weighted carefully. Throughout, simplicity has been substituted for efficiency. Complex algorithms are used only if their complexity can be localized." John From imp at bsdimp.com Thu Jun 4 13:38:43 2020 From: imp at bsdimp.com (Warner Losh) Date: Wed, 3 Jun 2020 21:38:43 -0600 Subject: [TUHS] 2.11BSD pl 0 recovered (mostly) PRELIMINARY Message-ID: OK. I've written a script to take the 2.11BSD pl 195 tape and reverse apply all the patches that we have to get back to 2.11BSD original. There's some problems. The biggest one is that ld.c was rewritten during this series and what it replaced is lost. And the 2.10.1 ld.c isn't what's in 2.11BSD and the patches to get from 2.10.1 to 2.11 don't seem to be out there. And the other patches in the series make it clear that patches are missing. 2.11BSD likely will work with 2.10.1's ld, so this isn't fatal. There's 122 other files that I recovered from 2.10.1. Almost all of those are a good guess, if not what's actually in 2.11. Many of these can be verified via other means. Some of these can be snagged from comp.bugs.2bsd (that's how as was recovered). There's a number of small cosmetic changes made via copying, some to get rid of redundant files, etc. I think these don't matter for the function of the system, but are small differences from the actual tape that shipped. I still need to try to still create a tape and try to compile. And forward apply all the patches and create a git repo from it. All things considered 99% of the files have been recovered at this point. It's early days, but I've pushed this to github for comments. http://github.com/bsdimp/mk211bsd It just assumes you have the Tuhs archive (including the new Usenet section) and that apout works on your computer. It works on FreeBSD for sure. No clue about anything else... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Thu Jun 4 16:27:10 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 04 Jun 2020 00:27:10 -0600 Subject: [TUHS] non-blocking IO: Simplicity has been subtituted for efficiency In-Reply-To: <8066.1591233892@hop.toad.com> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> <8066.1591233892@hop.toad.com> Message-ID: <202006040627.0546RAMW020424@freefriends.org> John Gilmore wrote: > I just happened to open a binder today of old papers about UNIX(tm), > including Ken Thompson's "Unix Implementation" paper, which says at the > bottom of the introduction: > > "What is or is not implemented in the kernel represents both a great > responsibiity and a great power. It is a soap-box platform on "the > way things should be done." Even so, if "the way" is too radical, no > one will follow it. Every important decision was weighted carefully. > Throughout, simplicity has been substituted for efficiency. Complex > algorithms are used only if their complexity can be localized." Words to live by. I remember reading that paper early on in my career and that this statement certainly influenced my subsequent thinking about programming in general. Arnold From peter at rulingia.com Thu Jun 4 19:04:36 2020 From: peter at rulingia.com (Peter Jeremy) Date: Thu, 4 Jun 2020 19:04:36 +1000 Subject: [TUHS] non-blocking IO In-Reply-To: <20200601145801.GE22016@mcvoy.com> References: <20200601145801.GE22016@mcvoy.com> Message-ID: <20200604090436.GJ279@server.rulingia.com> On 2020-Jun-01 07:58:02 -0700, Larry McVoy wrote: >On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote: >> On Mon, 1 Jun 2020, Rob Pike wrote: >> >> > I???m not quite sure why the Research lineage did not include >> > non-blocking behaviour, especially in view of the man page comments. >> > Maybe it was seen as against the Unix philosophy, with select() >> > offering sufficient mechanism to avoid blocking (with open() the hard >> > corner case)? >> > >> >That's it. Select was good enough for our purposes. >> >> After being dragged through both Berserkley and SysVile, I never did get the >> hang of poll()/select() etc,,, > >I'm sure you could, select is super handy, think a network server like >apache. My view may be unpopular but I've always been disappointed that Unix implemented blocking I/O only and then had to add various hacks to cover up for the lack of asynchonous I/O. It's trivial to build blocking I/O operations on top of asynchonous I/O operations. It's impossible to do the opposite without additional functionality. I also found it disappointing that poll()/select() only worked on TTY and network operations. HDDs are really slow compared to CPUs and it would be really nice if a process could go and do something else whilst waiting for a file to open. -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From arnold at skeeve.com Thu Jun 4 23:39:35 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 04 Jun 2020 07:39:35 -0600 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi In-Reply-To: <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> References: <202006030512.0535CRFC020727@freefriends.org> <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> Message-ID: <202006041339.054DdZk7008288@freefriends.org> Hi All. I've put Clem's file up at http://www.skeeve.com/text-processing.tar.bz2. It's a little over 16 meg and includes more than just the vi stuff. Enjoy, Arnold Michael Stiller wrote: > Hi Clem, > > why offline? Other people are also interested. :-) > > Cheers, > > Michael > > > > On 3. Jun 2020, at 14:33, Clem Cole wrote: > > > > I think so... I'll send you a URL off line. > > > > Clem > > > > On Wed, Jun 3, 2020 at 1:14 AM wrote: > > Hi. > > > > Does anyone have a mirror of the files that once upon a time were > > at http://alf.uib.no/pub/vi? They were mirrored at > > ftp://ftp.uu.net/pub/text-processing/vi also. > > > > If so, can you please send me a tarball off list (or tell me where > > I can download a copy from)? > > > > Thanks! > > > > Arnold From imp at bsdimp.com Fri Jun 5 00:19:58 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 4 Jun 2020 08:19:58 -0600 Subject: [TUHS] non-blocking IO In-Reply-To: <20200604090436.GJ279@server.rulingia.com> References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> Message-ID: On Thu, Jun 4, 2020 at 3:06 AM Peter Jeremy wrote: > On 2020-Jun-01 07:58:02 -0700, Larry McVoy wrote: > >On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote: > >> On Mon, 1 Jun 2020, Rob Pike wrote: > >> > >> > I???m not quite sure why the Research lineage did not include > >> > non-blocking behaviour, especially in view of the man page comments. > >> > Maybe it was seen as against the Unix philosophy, with select() > >> > offering sufficient mechanism to avoid blocking (with open() the hard > >> > corner case)? > >> > > >> >That's it. Select was good enough for our purposes. > >> > >> After being dragged through both Berserkley and SysVile, I never did > get the > >> hang of poll()/select() etc,,, > > > >I'm sure you could, select is super handy, think a network server like > >apache. > > My view may be unpopular but I've always been disappointed that Unix > implemented blocking I/O only and then had to add various hacks to cover > up for the lack of asynchonous I/O. It's trivial to build blocking I/O > operations on top of asynchonous I/O operations. It's impossible to do > the opposite without additional functionality. > > I also found it disappointing that poll()/select() only worked on TTY and > network operations. HDDs are really slow compared to CPUs and it would be > really nice if a process could go and do something else whilst waiting for > a file to open. > Lest anybody think this is a theoretical concern, Netflix has spent quite a bit of effort to reduce the sources of latency in our system. The latency for open doesn't happen often, due to caching, but when it does this causes a hickup for nginx worker thread (since open is blocking). If you get enough hickups, you wind up consuming all your worker threads and latency for everybody suffers while waiting for these to complete (think flaky disk that suddenly takes a really long time for each of its I/Os for one example). We've hacked FreeBSD in various ways to reduce or eliminate this delay.... In FreeBSD we have to translate from the pathname to a vnode, and to do that we have to look up directories, indirect block tables, etc. All these are surprising places that one could bottleneck at... And if you try to access the vnode before all this is done, you'll wait for it (though in the case of sendfile it doesn't matter since that's async and only affect the one I/O)... So I could see how having a async open could introduce a lot of hair into the mix depending on how you do it. Without a robust callback/AST mechanism, my brain is recoiling from the EALREADY errors in sockets for things that are already in progress... reads and write are easy by comparison :) The kicker is that all of the kernel is callback driven. The upper half queues the request and then sleeps until the lower half signals it to wakeup. And that signal is often just a wakeup done from the completion routine in the original request. All of that would be useful in userland for high volume activity, none of it is exposed... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From dot at dotat.at Fri Jun 5 02:34:54 2020 From: dot at dotat.at (Tony Finch) Date: Thu, 4 Jun 2020 17:34:54 +0100 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> Message-ID: Warner Losh wrote: > > So I could see how having a async open could introduce a lot of hair into > the mix depending on how you do it. Linux io_uring, for example. I would be interested if anyone has a historical perspective on its design :-) Tony. -- f.anthony.n.finch http://dotat.at/ Wight, Portland, Plymouth, Biscay, Fitzroy, Sole, Lundy, Fastnet: North or northwest 4 to 6, occasionally 7 in Biscay. Slight or moderate at first except in Biscay and Fitzroy, otherwise moderate or rough. Showers. Good, occasionally moderate. From lm at mcvoy.com Fri Jun 5 02:50:11 2020 From: lm at mcvoy.com (Larry McVoy) Date: Thu, 4 Jun 2020 09:50:11 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> Message-ID: <20200604165011.GC18437@mcvoy.com> On Thu, Jun 04, 2020 at 08:19:58AM -0600, Warner Losh wrote: > The kicker is that all of the kernel is callback driven. The > upper half queues the request and then sleeps until the lower half signals > it to wakeup. And that signal is often just a wakeup done from the > completion routine in the original request. All of that would be useful in > userland for high volume activity, none of it is exposed... Yeah, I've often wondered why this stuff wasn't exposed. We already have signal handlers, seems like that maps. I tried to get the NFS guys at Sun to rethink the biod junk and do it like UFS does, where it queues something and gets a callback. I strongly suspect that two processes, one to queue, one to handle callbacks, would be more efficient and actually faster than the biod nonsense. That's one of the arguments I lost unfortunately. Warner, exposing that stuff in FreeBSD is not really that hard, I suspect. Might be a fun project for a young kernel hacker with some old dude like you or me or someone, watching over it and thinking about the API. --lm From crossd at gmail.com Sat Jun 6 02:00:52 2020 From: crossd at gmail.com (Dan Cross) Date: Fri, 5 Jun 2020 12:00:52 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <20200604165011.GC18437@mcvoy.com> References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> <20200604165011.GC18437@mcvoy.com> Message-ID: On Thu, Jun 4, 2020 at 12:51 PM Larry McVoy wrote: > On Thu, Jun 04, 2020 at 08:19:58AM -0600, Warner Losh wrote: > > The kicker is that all of the kernel is callback driven. The > > upper half queues the request and then sleeps until the lower half > signals > > it to wakeup. And that signal is often just a wakeup done from the > > completion routine in the original request. All of that would be useful > in > > userland for high volume activity, none of it is exposed... > > Yeah, I've often wondered why this stuff wasn't exposed. We already have > signal handlers, seems like that maps. > Was it Rob who said that signals were really just for SIGKILL? Here, signals would be gang-pressed into service as a general IPC mechanism. In fairness, they've mutated that way, but they didn't start out that way. While I obviously wasn't there, the strong impression I get is that by the time people were seriously thinking about async IO in Unix, the die had already been cast for better or worse. > I tried to get the NFS guys at Sun to rethink the biod junk and do it like > UFS does, where it queues something and gets a callback. I strongly > suspect > that two processes, one to queue, one to handle callbacks, would be more > efficient and actually faster than the biod nonsense. > > That's one of the arguments I lost unfortunately. > > Warner, exposing that stuff in FreeBSD is not really that hard, I suspect. > Might be a fun project for a young kernel hacker with some old dude like > you or me or someone, watching over it and thinking about the API. > I'm going to actually disagree with you here, Larry. While I think a basic mechanism wouldn't be THAT hard to implement, it wouldn't compose nicely with the existing primitives. I suspect the edge cases would be really thorny, particularly without a real AST abstraction. For instance, what happens if you initiate an async IO operation, then block on a `read`? Where does the callback happen? If on the same thread, The real challenge isn't providing the operation, it's integrating it into the existing model. As a counter-point to the idea that it's completely unruly, in Akaros this was solved in the C library: all IO operations were fundamentally asynchronous, but the C library provided blocking read(), write(), etc by building those from the async primitives. It worked well, but Akaros had something akin to an AST environment and fine-grain scheduling decisions were made in userspace: in Akaros the unit of processor allocation is a CPU core, not a thread, and support exists for determining the status of all cores allocated to a process. There are edge cases (you can't roll-your-own mutex, for example, and the basic threading library does a lot of heavy lifting for you making it challenging to integrate into the runtime of a language that doesn't use the same ABI), but by and large it worked. It was also provided by a kernel that was a pretty radical departure from a Unix-like kernel. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Sat Jun 6 06:57:06 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 6 Jun 2020 06:57:06 +1000 (EST) Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: On Wed, 27 May 2020, Greg A. Woods wrote: > Sadly most compilers, including GCC and Clang/LLVM will, at best, warn > (and warnings are only treated as errors by the most macho|wise); and > compilers only do that now because they've been getting flack from > developers whenever the optimizer does something unexpected. Don't talk to me about optimisers... That's not the code that I wrote! I've seen code simply disappear, because the "optimiser" though that it was cleverer than I was. > The Linux kernel example I've referred to involved dereferencing a > pointer to do an assignment in a local variable definition, then a few > lines later testing if the pointer was NULL before using the local > variable. Unoptimised the code will dereference a NULL pointer and load > junk from location zero into the variable (because it's kernel code), > then the NULL test will trigger and all will be good. The optimizer > rips out the NULL check because "obviously" the programmer has assumed > the pointer is always a valid non-NULL pointer since they've explicitly > dereferenced it before checking it and they wouldn't want to waste even > a single jump-on-zero instruction checking it again. (It's also quite > possible the code was written "correctly" at first, then someone mushed > all the variable initialisations up onto their definitions.) Typical Penguin/OS behaviour... > In any case there's now a GCC option: -fno-delete-null-pointer-checks > (to go along with -fno-strict-aliasing and -fno-strict-overflow, and > -fno-strict-enums, all of which MUST be used, and sometimes > -fno-strict-volatile-bitfields too, on all legacy code that you don't > want to break) I'm sure that there's a competition somewhere, to see who can come with GCC's -fmost-longest-and-most-obscure-option flags... > It's even worse when you have to write bare-metal code that must > explictly dereference a NULL pointer (a not-so-real example: you want > to use location zero in the CPU zero-page (e.g. on a 6502 or 6800, or > PDP-8, etc.) as a pointer) -- it is now impossible to do that in strict > Standard C even though trivially it "should just work" despite the silly > rules. As far as I can tell it always did just work in "plain old" C. I've programmed a PDP-8! 'Twas way back in high school, and I found a bug in my mentor's program; it controlled traffic lights... > The crazy thing about modern optimizers is that they're way more > persistent and often somewhat more clever than your average programmer. > They follow all the paths. They apply all the rules at every turn. Optimisers... Grrr... -- Dave From cym224 at gmail.com Sat Jun 6 07:40:50 2020 From: cym224 at gmail.com (Nemo Nusquam) Date: Fri, 5 Jun 2020 17:40:50 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: <5f5d5c7f-ce33-d067-2a62-aa369175d9cb@gmail.com> On 06/05/20 16:57, Dave Horsfall wrote (in part): > [...] > Optimisers... Grrr... Steve Johnson's position paper on optimising compilers may amuse you: https://dl.acm.org/doi/abs/10.1145/567532.567542 N. > > -- Dave From rich.salz at gmail.com Sat Jun 6 07:47:23 2020 From: rich.salz at gmail.com (Richard Salz) Date: Fri, 5 Jun 2020 17:47:23 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: > I'm sure that there's a competition somewhere, to see who can come with > GCC's -fmost-longest-and-most-obscure-option flags... > At least one of the GCC maintainers is German, so possibly. Can clang keep up? :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Sat Jun 6 08:01:29 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Fri, 5 Jun 2020 15:01:29 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: <63245F94-E434-42D9-A294-8AE6971FE66D@iitbombay.org> On Jun 5, 2020, at 2:47 PM, Richard Salz wrote: > > > | I'm sure that there's a competition somewhere, to see who can come with > | GCC's -fmost-longest-and-most-obscure-option flags... > > At least one of the GCC maintainers is German, so possibly. Can clang keep up? :) Clang has more than kept up! clang: -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang gcc-9: -print-sysroot-headers-suffix Not counting gcc's --help={common|optimizers|params|target|warnings|[^]{joined|separate|undocumented}}[,...]. From imp at bsdimp.com Sat Jun 6 08:51:27 2020 From: imp at bsdimp.com (Warner Losh) Date: Fri, 5 Jun 2020 16:51:27 -0600 Subject: [TUHS] My BSDcan talk Message-ID: OK. Must be off my game... I forgot to tell people about my BSDcan talk earlier today. It was streamed live, and will be online in a week or three... It's another similar to the last two. I've uploaded a version to youtube until the conference has theirs ready. It's a private link, but should work for anybody that has it. Now that I've given my talk it's cool to share more widely... https://www.youtube.com/watch?v=NRq8xEvFS_g The link at the end is wrong. https://github.com/bsdimp/bsdcan2020-demos is the proper link. Please let me know what you think. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Sat Jun 6 21:30:58 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sat, 6 Jun 2020 13:30:58 +0200 Subject: [TUHS] Unix on Aprpanet Message-ID: >> On 2020, Jun 3, at 1:38 AM, Lars Brinkhoff wrote: >> >> Lawrence Stewart wrote: >>> I remember working on getting Arpanet access on an 11/34 running V7 >>> around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run >>> NCP, using a variation of Rand’s code. I wrote some sort of bisync >>> driver for packet communications between the /23 and the /34, and I >>> think added an IOCTL or some hack to ask if there was a message ready. >>> So a polling variety of non-blocking I/O :) >> >> Has this, or Rand's code, been preserved? >> >> I'm only aware of one Arpanet NCP implementation for Unix that is >> online, the one from University of Illinois. > Alas I do not know. I may have some old emails from that era. I will check. I have not so far come across an NCP Unix that was not based on the UoI code base. At a time there was a “Rand Unix”, that combined the UoI code with a few other extensions. A 1978 document describes the kernel as: "UNIX version 6 with modifications some of which are: - Accounting - A system call to write end of files on pipes - A system call which checks whether a process would sleep upon doing a read on either a pipe or tty - Pseudo teletypes - The network control program NCP written at the University of Illinois - A new method of interprocess communication called ports. Ports are similar to pipes but have names and can be accessed by any process.” The system call to write EOF on pipes is eofp() and the blocking read test is empty(). Both are available in the NCP Unix source code as included on the TUHS Unix Tree page. It could well be that the hack Lawrence refers to above was a modification to empty() to also cover his bisync driver. I guess empty() is the first step on the path that includes capac()/await() and later select()/poll() as the preferred mechanism to prevent blocking on IPC files (incl. tty’s). From jnc at mercury.lcs.mit.edu Sat Jun 6 23:29:06 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Sat, 6 Jun 2020 09:29:06 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200606132906.B7FAB18C079@mercury.lcs.mit.edu> > From: Peter Jeremy > My view may be unpopular but I've always been disappointed that Unix > implemented blocking I/O only and then had to add various hacks to cover > up for the lack of asynchonous I/O. It's trivial to build blocking I/O > operations on top of asynchonous I/O operations. It's impossible to do > the opposite without additional functionality. Back when I started working on networks, I looked at other kinds of systems to see what general lessons I could learn about the evolution of systems, which might apply to the networks we were building. (I should have written all that up, never did, sigh.) One major one was that a system, when small, often collapses multiple needs onto one machanism. Only as the system grows in size do scaling effects, etc necessitate breaking them up into separate mechanisms. (There are some good examples in file systems, for example.) I/O is a perfect example of this; a small system can get away with only one kind; it's only when the system grows that one benefits from having both synchronous and asynchronous. Since the latter is more complicated, _both_ in the system and in the applications which use it, it's no surprise that synchronous was the pick. The reasons why synchronous is simpler in applications have a nice illustration in operating systems, which inevitably support both blocking (i.e. implied process switching) and non-blocking 'operation initiation' and 'operation completed notification' mechanisms. (The 'timeout/callout' mechanism is Unix is an example of the latter, albeit specialized to timers.) Prior to the Master Control Program in the Burroughs B000 (there may be older examples, but I don't know of them - I would be more than pleased to be informed of any such, if there are), the technique of having a per-process _kernel_ stack, and on a process block (and implied switch), switching stacks, was not used. This idea was picked up for Jerry Saltzer's PhD thesis, used in Multics, and then copied by almost every other OS since (including Unix). The advantage is fairly obvious: if one is deep in some call stack, one can just wait there until the thing one needs is done, and then resume without having to work one's way back to that spot - which will inevitably be complicated (perhaps more in the need to _return_ through all the places that called down - although the code to handle a 'not yet' return through all those places, after the initial call down, will not be inconsiderable either). Exactly the same reasoning applies to blocking I/O; one can sit where one is, waiting for the I/O to be done, without having to work one's way back there later. (Examples are legion, e.g. in recursive descent parsers - and can make the code _much_ simpler.) It's only when one _can't_ wait for the I/O to complete (e.g. for a packet to arrive - although others have mentioned other examples in this thread, such as 'having other stuff to do in the meanwhile') than having only blocking I/O becomes a problem... In cases where blocking would be better, one can always build a 'blocking' I/O subsystem on top of asynchronous I/O primitives. However, in a _tiny_ system (remember my -11/40 which ran Unix on a system with _48KB_ of main memory _total_- i.e. OS and application together had to be less than 48KB - no virtual memory on that machine :-), building blocking I/O on top of asynchonous I/O, for those very few cases which need it, may not be the best use of very limited space - although I agree that it's the way to go, overall. Noel From pnr at planet.nl Sun Jun 7 00:55:50 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sat, 6 Jun 2020 16:55:50 +0200 Subject: [TUHS] My BSDcan talk Message-ID: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> > It's another similar to the last two. I've uploaded a version to youtube until the conference has theirs ready. It's a private link, but should work for anybody that has it. Now that I've given my talk it's cool to share more widely. > The link at the end is wrong. https://github.com/bsdimp/bsdcan2020-demos is the proper link. > Please let me know what you think. Watched it & liked it a lot! I have one nit-pick in the section on early networking: BBN's VAX TCP did not allow the ‘/dev/net/host’ syntax. That particular semantic comes from UoI’s NCP Unix, where the 8-bit host number was encoded in the minor number of character special file ‘host’ - but it did not carry through to the BBN code. Other systems used something similar. The Chaos network code made namei() break when it recognised the Chaos driver and left the remainder of the path for the driver to fetch & parse. I’m also being told that Greg Chesson experimented with using the given name of a Datakit channel device as the connection string for the switch, but that this approach was abandoned early on. In my view, exposing the host names through integration in the Unix file name space makes a lot of conceptual sense, but it unfortunately falls down on the practicalities, with the host name set being hard to enumerate (it is large, distributed and not stable - even back then). A question mark is hard pin-pointing the start of Unix networking to V4 / 1974. Yes, that is the earliest evidence we currently have. However, Sandy Fraser says that Spider came into operation in 1972 and it must have connected to something. Maybe that something was a lab-bench test setup, but it could have been a computer - maybe even one running Unix. There is another candidate for earliest Unix networking as well. The tech memo’s from Heinz Lycklama include one on the Glance terminal. That memo includes a section on the network used, referencing a 1973 report by D.R. Weller, "A High-Speed I/O Loop Communication System for the DEC PDP-11 Computer”. That computer appears to be an 11/45 running Unix and the loop is not Spider (nor the Pierce loop discussed in 1970/71 BSTJ). I have an off-list question outstanding to better understand this. From clemc at ccc.com Sun Jun 7 02:01:08 2020 From: clemc at ccc.com (Clem Cole) Date: Sat, 6 Jun 2020 12:01:08 -0400 Subject: [TUHS] My BSDcan talk In-Reply-To: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> Message-ID: On Sat, Jun 6, 2020 at 10:56 AM Paul Ruizendaal wrote: > The Chaos network code made namei() break when it recognised the Chaos > driver and left the remainder of the path for the driver to fetch & parse. > Yep - cute hack -- Domain (Aegis) did the same trick (same code author most likely). The biggest advantage is that open(2) can be used unchanged and 'parameters' are passed in ASCII to the lower code. You mentioned that BBN Vax code did not use it. I've forgotten than how parameters like IP addresses were passed down? I had thought that Rob had is also in at least one version of the BBN code, as I remember talking to him about it at one point; but I have frankly forgotten. I'm pretty sure that wnj thought it was an ugly solution; which is why UCB did not try it. We played with this solution also in the early Masscomp networking code (I had seen it in the Chaos driver from MIT); but by then the UCB code was coming out and user code was using sockets, so we just kept with that interface for code reuse reasons. Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From erc at pobox.com Sun Jun 7 06:49:36 2020 From: erc at pobox.com (Ed Carp) Date: Sat, 6 Jun 2020 15:49:36 -0500 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: On 5/27/20, Ronald Natalie wrote: > The large areas of undefined and unspecified behavior has always been an > issue in C. It was somewhat acceptable when you were using it as a direct > replacement for assembler, but Java and many of other follow-ons endevaored to be more > portable/rigourous. Of course, you can write crap code in any language. "It's not a bug, it's a feature" C was written when the programmer had to be more rigorous instead of just letting things slide and having the language do their thinking for them. I remember being laughed at for using static arrays instead of malloc() and friends, until people found out that safety-critical systems were written the same way. I have C code that was written 35 years ago that's still in production. Back then, you had to be careful, and you actually had to think about what you were writing. We've gotten soft and lazy, and now we're paying for it. From thomas.paulsen at firemail.de Sun Jun 7 07:08:43 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Sat, 06 Jun 2020 23:08:43 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> 'C was written when the programmer had to be more rigorous instead of just letting things slide and having the language do their thinking for them. ' I fully subscribe to that. Today the company owners have to pay a lot for programmers having the language do their thinking for them. The memory hunger of the soa java business services of the company I worked prior to retirement, is sheer endless. Arnold once told that there is more demand for C developers in Israel. I envy you From lm at mcvoy.com Sun Jun 7 07:13:28 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sat, 6 Jun 2020 14:13:28 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: <20200606211328.GD1635@mcvoy.com> I did one stint at a Java shop, Charles Schwab's web group. No talent, no architecture, no vision. Lots of politics and back stabbing and claiming credit for other people's work. Totally toxic, hands down the worst job I've ever had. I lasted less than 6 months and am surprised I made it that far. On Sat, Jun 06, 2020 at 11:08:43PM +0200, Thomas Paulsen wrote: > 'C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them. ' > I fully subscribe to that. > Today the company owners have to pay a lot for programmers having the language do their thinking > for them. The memory hunger of the soa java business services of the company I worked prior to > retirement, is sheer endless. Arnold once told that there is more demand for C developers > in Israel. I envy you > -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From doug at cs.dartmouth.edu Sun Jun 7 07:49:23 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Sat, 06 Jun 2020 17:49:23 -0400 Subject: [TUHS] History of popularity of C Message-ID: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> > Steve Johnson's position paper on optimising compilers may amuse you: > https://dl.acm.org/doi/abs/10.1145/567532.567542 Indeed. This passage struck a particular chord: "I contend that the class of applications that depend on, for example, loop optimization and dead code elimination for their efficient solution is of modest size, growing smaller, and often very susceptible to expression in applicative languages where the optimization is built into the individual applicative operators." I don't know whether I saw that note at the time, but since then I've come to believe, particularly in regard to C, that one case of dead-code elmination should be guaranteed. That case is if(0), where 0 is the value of a constant expression. This guarantee would take the place of many--possibly even most--ifdefs. Every ifdef is an ugly intrusion and a pain to read. Syntactically it occurs at top level completely out of sync with the indentation and flow of text. Conversion to if would be a big win. Doug From imp at bsdimp.com Sun Jun 7 07:55:07 2020 From: imp at bsdimp.com (Warner Losh) Date: Sat, 6 Jun 2020 15:55:07 -0600 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: On Sat, Jun 6, 2020 at 3:50 PM Doug McIlroy wrote: > > Steve Johnson's position paper on optimising compilers may amuse you: > > https://dl.acm.org/doi/abs/10.1145/567532.567542 > > Indeed. This passage struck a particular chord: > > "I contend that the class of applications that depend on, for example, loop > optimization and dead code elimination for their efficient solution is of > modest size, growing smaller, and often very susceptible to expression in > applicative languages where the optimization is built into the individual > applicative operators." > > I don't know whether I saw that note at the time, but since then I've > come to believe, particularly in regard to C, that one case of dead-code > elmination should be guaranteed. That case is if(0), where 0 is the > value of a constant expression. > > This guarantee would take the place of many--possibly even > most--ifdefs. Every ifdef is an ugly intrusion and a pain to read. > Syntactically it occurs at top level completely out of sync with the > indentation and flow of text. Conversion to if would be a big win. > I'd love something like this to work, but the semantic interpretation would need to also somehow be omitted, otherwise how do you replace #ifdef AIX ioct(fd, AIX_SPECIFIC_IOCTL, ...) #endif on a HUPX system that doesn't define AIX_SPECIFIC_IOCTL... I remember hearing that BLISS could cope because it deferred the semantic interpretation of the identifiers until after a round of dead code elimination so it didn't need a pre-processor... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From erc at pobox.com Sun Jun 7 08:27:19 2020 From: erc at pobox.com (Ed Carp) Date: Sat, 6 Jun 2020 17:27:19 -0500 Subject: [TUHS] History of popularity of C In-Reply-To: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: "Arnold once told that there is more demand for C developers in Israel. I envy you" Maybe I ought to move to Israel. Sounds like they have more common sense there.

Virus-free. www.avast.com
On 6/6/20, Thomas Paulsen wrote: > 'C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them. ' > I fully subscribe to that. > Today the company owners have to pay a lot for programmers having the > language do their thinking > for them. The memory hunger of the soa java business services of the company > I worked prior to > retirement, is sheer endless. Arnold once told that there is more demand for > C developers > in Israel. I envy you > > > From coppero1237 at gmail.com Sun Jun 7 09:14:55 2020 From: coppero1237 at gmail.com (Tyler Adams) Date: Sun, 7 Jun 2020 02:14:55 +0300 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: > "C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them" True, but a wise man once said "let the computer do the dirty work". Tyler On Sun, Jun 7, 2020 at 1:28 AM Ed Carp wrote: > "Arnold once told that there is more demand for C developers > in Israel. I envy you" > > Maybe I ought to move to Israel. Sounds like they have more common > sense there. > > On 6/6/20, Thomas Paulsen wrote: > > 'C was written when the programmer had to be more rigorous instead of > > just letting things slide and having the language do their thinking > > for them. ' > > I fully subscribe to that. > > Today the company owners have to pay a lot for programmers having the > > language do their thinking > > for them. The memory hunger of the soa java business services of the > company > > I worked prior to > > retirement, is sheer endless. Arnold once told that there is more demand > for > > C developers > > in Israel. I envy you > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Sun Jun 7 09:31:42 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sat, 6 Jun 2020 16:31:42 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org> On Jun 6, 2020, at 1:49 PM, Ed Carp wrote: > > On 5/27/20, Ronald Natalie wrote: > >> The large areas of undefined and unspecified behavior has always been an >> issue in C. It was somewhat acceptable when you were using it as a direct >> replacement for assembler, but Java and many of other follow-ons endevaored to be more >> portable/rigourous. Of course, you can write crap code in any language. > > "It's not a bug, it's a feature" A snippet of a recent comp.arch post by someone (the subject was C and safety): What you call "misfeatures", some other people call "features". If you expect people to take you and your opinions seriously, you'll get on better if you stop mocking other opinions. I've written several times why undefined behaviour lets me write better and safer code, as well as more efficient code. If you remain determinedly unconvinced, at least agree to disagree without sounding childish about it. From woods at robohack.ca Sun Jun 7 10:12:35 2020 From: woods at robohack.ca (Greg A. Woods) Date: Sat, 06 Jun 2020 17:12:35 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org> References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org> Message-ID: At Sat, 6 Jun 2020 16:31:42 -0700, Bakul Shah wrote: Subject: Re: [TUHS] History of popularity of C > > On Jun 6, 2020, at 1:49 PM, Ed Carp wrote: > > > > On 5/27/20, Ronald Natalie wrote: > > > >> The large areas of undefined and unspecified behavior has always > >> been an issue in C. It was somewhat acceptable when you were using > >> it as a direct replacement for assembler, but Java and many of > >> other follow-ons endevaored to be more portable/rigourous. Of > >> course, you can write crap code in any language. > > > > "It's not a bug, it's a feature" > > A snippet of a recent comp.arch post by someone (the subject was C and > safety): > > What you call "misfeatures", some other people call "features". > If you expect people to take you and your opinions seriously, > you'll get on better if you stop mocking other opinions. I've > written several times why undefined behaviour lets me write > better and safer code, as well as more efficient code. If you > remain determinedly unconvinced, at least agree to disagree > without sounding childish about it. Heh. W.r.t. efficiency, well undefined behaviour does allow the compiler to turn their code, or anyone's else's code, into more "efficient" code if they happen to (accidentally or otherwise) trip over undefined behaviour. However I don't think it can be argued in any valid way that "undefined behaviour" can ever lead to "better and safer" code, in any way, or from any viewing angle, whatsoever. "Undefined behaviour" just means that the language definition is somehow adversely compromised in such a way that it is impossible to prevent the programmer from writing compilable and executable code that will always produce some well defined behaviour in all standards-compliant implementations. I.e. the language allows that there are ways to write syntactically correct code that cannot be guaranteed to do anything particular whatsoever in _all_ standards-compatible implementations. We can argue until the cows come home whether "undefined behaviour" is a "necessary" part of the language definition (e.g. to keep the language implementable, or backward-compatible, or whatever), but I don't see how any valid argument can ever be made for it being a "good" and "useful" thing from the perspective of a programmer using the language. Undefined behaviours are black holes for which the language standard offers no real guidance nor maps for safe passage other than the stern warning to avoid them as best as possible. Perhaps it is such scare-mongering that the author above justifies as their influence to write "better and safer" code, but that's no good argument for having such pits of despair in the language definition in the first place. If we were arguing theology then I would say the bible we call the "C Standard" is actually actively trying to trap its followers into committing sins. Luckily the real world of C is made of actual implementations, and they are free to either offer definitions for how various (ab)uses of the language will work, or to maintain the black holes of mystery that we must try to avoid, or even sometimes to give us the choice in how they will treat our code. As programmers we should try to choose which implementation(s) we use, and how we control _their_ behaviour, while at the same time still doing our best to avoid giving them the rope to hang us with. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From andreww591 at gmail.com Sun Jun 7 15:15:42 2020 From: andreww591 at gmail.com (Andrew Warkentin) Date: Sat, 6 Jun 2020 23:15:42 -0600 Subject: [TUHS] My BSDcan talk In-Reply-To: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> Message-ID: On 6/6/20, Paul Ruizendaal wrote: > > In my view, exposing the host names through integration in the Unix file > name space makes a lot of conceptual sense, but it unfortunately falls down > on the practicalities, with the host name set being hard to enumerate (it is > large, distributed and not stable - even back then). > With a proper dynamic VFS architecture, there is no reason why a resolver with a filesystem API has to bother supporting enumeration at all. All it needs to be able to do is respond to open() and stat() calls, returning ENOENT when resolution fails. From arnold at skeeve.com Sun Jun 7 15:57:46 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sat, 06 Jun 2020 23:57:46 -0600 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: <202006070557.0575vkwU011918@freefriends.org> Ed Carp wrote: > "Arnold once told that there is more demand for C developers > in Israel. I envy you" The market in Israel for software developers is VERY hot. Based entirely on the emails I get from Linked-In about jobs that may interest me, there's some C, but a lot more C++, both Windows and Linux. Also a lot of Python. > Maybe I ought to move to Israel. Moving here isn't a trivial decision, especially if you don't speak any Hebrew. Off-topic. Sorry. > Sounds like they have more common sense there. We do, but the strong influence of western (US) culture is eroding it, which is saddening and frustrating. This definitely off-topic. Arnold From akosela at andykosela.com Sun Jun 7 19:22:21 2020 From: akosela at andykosela.com (Andy Kosela) Date: Sun, 7 Jun 2020 11:22:21 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: <202006070557.0575vkwU011918@freefriends.org> References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On 6/7/20, arnold at skeeve.com wrote: > Ed Carp wrote: > >> "Arnold once told that there is more demand for C developers >> in Israel. I envy you" > > The market in Israel for software developers is VERY hot. > > Based entirely on the emails I get from Linked-In about jobs that may > interest me, there's some C, but a lot more C++, both Windows and Linux. > Also a lot of Python. Seriously, is anyone still doing any real development in C besides kernel programming and embedded world?? Maybe I was living under a rock, but I always had an impression that the industry moved to C++ in the late 90s and stayed with it ever since. The last bastion of C was open source Linux/*BSD programming but I remember the time when C was a truly universal programming language used for _everything_ including games (e.g. Doom). Maybe I just miss the 90s. --Andy From erc at pobox.com Sun Jun 7 19:39:28 2020 From: erc at pobox.com (Ed Carp) Date: Sun, 7 Jun 2020 04:39:28 -0500 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On 6/7/20, Andy Kosela wrote: > Seriously, is anyone still doing any real development in C besides > kernel programming and embedded world?? Maybe I was living under a > rock, but I always had an impression that the industry moved to C++ in > the late 90s and stayed with it ever since. Absolutely. C++ isn't the panacea that it's made out to be, it's not "superior" to C++, it's just C with other useful stuff bolted on, but it didn't make C obsolete, not by a long shot. Some find the operator overloading and classes to be a lot more confusing than just function calls and such, without adding a lot to the language itself. Others, of course, feel differently. It's more of a religious discussion than anything, like debating the merits of vi vs. emacs - a pointless discussion, since you're not going to change anyone else's mind anyway. Lots of C still being written out there. My most recent project was writing a crypto library in C. Writing it was the hard part - validating it was something else entirely. One of the nicest things about C++ is that you can write your code entirely in C and the C++ compiler will compile it, no problem. From brantley at coraid.com Sun Jun 7 20:02:23 2020 From: brantley at coraid.com (Brantley Coile) Date: Sun, 7 Jun 2020 10:02:23 +0000 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <7563F472-96B8-4CF8-9176-404FB9761508@coraid.com> This might register low on the useful information index, but I decided a few years back as VC Coraid was coming apart and New Coraid was being resurrected under the auspices of SouthSuite Inc., that I would have a C mono-culture and use the language for everything. Specifically no JavaScript and no Python. Our website is written in C. Webpages generated on the server side using C. Our evolving ERP is written in C. Our subscription system is written in C. Our test systems are written in it. All work quite well. The focus on C was for three main reasons. First, our products are infrastructure products and are meant to be simple, fast, and affordable. Forty-two years of experience plus C gives us the ability to squeeze all the performance possible from any hardware platform. We "see" the instructions our code will generate. C is a great choice for that. (As would have been Oberon, but that's another discussion.) Second, if instead of having a set of complex languages, each with its own adherents, using a single language removes all the distracting and divisive language wars having multiple complex languages create. Little language like AWK and the shell script are fine. It's the more complex ones that divide people. Lastly, a single, powerful, simple (on the other side of complexity) language that a single person can maintain is essential to our Software Atelier model of doing business. Like the workshops of the renaissance, we have to understand and work on all our own tools. I use Ken's C compiler under Plan 9. It weights in at a light, 20K lines of code. As I said, I'm not sure how useful this data point is for you. Over the last thirty years I carefully chose my foot falls through the software swamp to avoid getting sucked under by one of the quagmires of complexity. Brantley > On Jun 7, 2020, at 5:22 AM, Andy Kosela wrote: > > On 6/7/20, arnold at skeeve.com wrote: >> Ed Carp wrote: >> >>> "Arnold once told that there is more demand for C developers >>> in Israel. I envy you" >> >> The market in Israel for software developers is VERY hot. >> >> Based entirely on the emails I get from Linked-In about jobs that may >> interest me, there's some C, but a lot more C++, both Windows and Linux. >> Also a lot of Python. > > Seriously, is anyone still doing any real development in C besides > kernel programming and embedded world?? Maybe I was living under a > rock, but I always had an impression that the industry moved to C++ in > the late 90s and stayed with it ever since. > > The last bastion of C was open source Linux/*BSD programming but I > remember the time when C was a truly universal programming language > used for _everything_ including games (e.g. Doom). Maybe I just miss > the 90s. > > --Andy From emu at e-bbes.com Sun Jun 7 21:04:23 2020 From: emu at e-bbes.com (emanuel stiebler) Date: Sun, 7 Jun 2020 07:04:23 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com> On 2020-06-06 16:49, Ed Carp wrote: > C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them. I remember being laughed at for using static arrays instead > of malloc() and friends, until people found out that safety-critical > systems were written the same way. Still avoiding malloc and friends in safety critical systems, for a good reason ... From thomas.paulsen at firemail.de Sun Jun 7 21:30:03 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Sun, 07 Jun 2020 13:30:03 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <5d6f1074119107ee81ff119c7a9463f6@firemail.de> >Seriously, is anyone still doing any real development in C besides >kernel programming and embedded world?? Maybe I was living under a >rock, but I always had an impression that the industry moved to C++ in >the late 90s and stayed with it ever since. I repeat myself: C never was intended for writing business applications. It was made for OS development in the sense of a portable assembler replacement. It also has found itself as the main devel language for fundamental services like db engines etc. From thomas.paulsen at firemail.de Sun Jun 7 21:33:21 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Sun, 07 Jun 2020 13:33:21 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com> References: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com> Message-ID: <065f8b450e1884b5f4ab97d64d7ab05c@firemail.de> >Still avoiding malloc and friends in safety critical systems, for a good reason ... all modern languages allowing dynamical memory allocation one way or another. From pnr at planet.nl Sun Jun 7 21:41:41 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sun, 7 Jun 2020 13:41:41 +0200 Subject: [TUHS] My BSDcan talk In-Reply-To: References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> Message-ID: <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl> > On 6 Jun 2020, at 18:01, Clem Cole wrote: > > You mentioned that BBN Vax code did not use it. I've forgotten than how parameters like IP addresses were passed down? I had thought that Rob had it also in at least one version of the BBN code, as I remember talking to him about it at one point; but I have frankly forgotten. In the surviving source code, BBN VAX TCP re-purposed the ‘mode’ parameter of open() to be a pointer to a block with connection data (“struct con”). Details are here: https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P From pnr at planet.nl Sun Jun 7 22:16:20 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sun, 7 Jun 2020 14:16:20 +0200 Subject: [TUHS] network name space (was: My BSDcon talk) Message-ID: <42C43756-9800-41AC-88FF-4186510F83D4@planet.nl> > On 6/6/20, Paul Ruizendaal > wrote: > > > > In my view, exposing the host names through integration in the Unix file > > name space makes a lot of conceptual sense, but it unfortunately falls down > > on the practicalities, with the host name set being hard to enumerate (it is > > large, distributed and not stable - even back then). > > > With a proper dynamic VFS architecture, there is no reason why a > resolver with a filesystem API has to bother supporting enumeration at > all. All it needs to be able to do is respond to open() and stat() > calls, returning ENOENT when resolution fails. That is an intriguing thought. In Research Unix terms it would be a virtual directory that was not readable or writable, but still explorable (i.e. only the x bit set). Maybe enumeration is only impractical for the networks that were designed to be ‘large’, such as Arpanet and Datakit. It would have been feasible for contemporary networks that were designed to be local only, such as Chaosnet or ARCnet. A half-way house would be to only enumerate the local network and leaving everything else merely explorable. That is conceptually very messy, though. From clemc at ccc.com Mon Jun 8 01:26:45 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 7 Jun 2020 11:26:45 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On Sun, Jun 7, 2020 at 5:23 AM Andy Kosela wrote: > Seriously, is anyone still doing any real development in C besides > kernel programming and embedded world?? Maybe I was living under a > rock, but I always had an impression that the industry moved to C++ in > the late 90s and stayed with it ever since. > Hardly, in my 45+ years, I have seen way more C projects than C++. The C projects tended to last longer, have a more profound impact and many are still being developed. I've been at Intel for the last qtr of my career and without a doubt, C is the #1 language in use internally, with C++ probably #2. Most of our work is actually in user space, although obviously we do a great deal of low-level work. The Intel compilers are a mix of both languages. That said, as we move to more LLVM work (we actually have the largest number of paid LLVM developers ). For your amusement check out: https://www.archer.ac.uk/status/codes/ Archer is a large HPC site in the UK. Many supercomputer centers off similar stats, but I often point to Archer because it easy to get an understanding of what programming languages are used for codes that are actually used in day-to-day production. That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal. Neither language is used for anything in production in our world at this point. Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Mon Jun 8 01:52:39 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 7 Jun 2020 08:52:39 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <20200607155239.GL1635@mcvoy.com> On Sun, Jun 07, 2020 at 11:26:45AM -0400, Clem Cole wrote: > That said, I personally am the most excited about Go theses day, but I'm > also thinking Rust looks pretty interesting, but my experience with both > compared to C is extremely nominal. Neither language is used for anything > in production in our world at this point. If I had to move to a modern language it would be Go. I looked at Rust and barfed. But if I had the juice, I'd just evolve C. Make a C+ that isn't object oriented but does have some of the stuff that modern languages have. I'd pretty much take http://little-lang.org and evolve C to that. From bakul at iitbombay.org Mon Jun 8 03:26:48 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sun, 7 Jun 2020 10:26:48 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <9FB6B5CB-A8A2-4EA7-8613-474C2B376FEC@iitbombay.org> On Jun 7, 2020, at 8:26 AM, Clem Cole wrote: > > That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal. Neither language is used for anything in production in our world at this point. People who use Rust seem to really like it but so far I have not done anything in it. I will take another look if I do any bare metal coding. In contrast I use Go for almost all my own coding now. I am not entirely happy with it but the ecosystem around it is great. And the really nice thing about it is that not only I can cross-compile programs but also, I believe it is the only compiled language (much more so than C) where a lot of stuff just works on on plan9. From bakul at iitbombay.org Mon Jun 8 03:35:29 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sun, 7 Jun 2020 10:35:29 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On Jun 7, 2020, at 8:26 AM, Clem Cole wrote: > > That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal. Neither language is used for anything in production in our world at this point. People who use Rust seem to really like it but so far I have not done anything in it. I will take another look if I do any bare metal coding. In contrast I use Go for almost all my own coding now. I am not entirely happy with it but the ecosystem around it is great. And the really nice thing about it is that not only I can cross-compile programs but also, I believe it is the only compiled language (much more so than C) where a lot of stuff just works on on plan9. From cym224 at gmail.com Mon Jun 8 04:50:34 2020 From: cym224 at gmail.com (Nemo Nusquam) Date: Sun, 7 Jun 2020 14:50:34 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com> On 06/07/20 11:26, Clem Cole wrote (in part): > That said, I personally am the most excited about Go theses day, but > I'm also thinking Rust looks pretty interesting, but my experience > with both compared to C is extremely nominal. Neither language is > used for anything in production in our world at this point. They seem to be used in some worlds: https://blog.golang.org/10years and https://www.rust-lang.org/production N. From imp at bsdimp.com Mon Jun 8 06:03:46 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 7 Jun 2020 14:03:46 -0600 Subject: [TUHS] My BSDcan talk In-Reply-To: <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl> References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl> Message-ID: On Sun, Jun 7, 2020 at 5:42 AM Paul Ruizendaal wrote: > > > On 6 Jun 2020, at 18:01, Clem Cole wrote: > > > > You mentioned that BBN Vax code did not use it. I've forgotten than how > parameters like IP addresses were passed down? I had thought that Rob had > it also in at least one version of the BBN code, as I remember talking to > him about it at one point; but I have frankly forgotten. > > In the surviving source code, BBN VAX TCP re-purposed the ‘mode’ parameter > of open() to be a pointer to a block with connection data (“struct con”). > Details are here: > https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P That's... clever? Gross? I don't even know how I'd want to react to this... which likely means "too clever for its own good" :) In any event, I've created a short, 2:00 correction to my talk. I'll have the conference organizers tack onto the end before they upload it to youtube. https://www.youtube.com/watch?v=9uTjyfO6MM8 If people care :) Thanks everybody for corrections... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From torek at torek.net Mon Jun 8 07:15:26 2020 From: torek at torek.net (Chris Torek) Date: Sun, 07 Jun 2020 14:15:26 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: Your message of "Sun, 07 Jun 2020 14:50:34 -0400." <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com> Message-ID: <202006072115.057LFV6v089953@elf.torek.net> Yes, both Rust and Go are being used. Rust has some advantages: like C++, it can compile to very fast code that does not need a garbage collector but that provides type-safety. It also gives you thread-safety through clever compiler analysis of who "owns" any given variable or data. The ownership / lifetime-analysis / borrow-checker is quite complicated and takes a lot of getting-used-to. I have not written anything "real" in Rust and had not had time to really learn it (I was hoping to learn it for real and write code in it the last few years, but that never actually happened). Meanwhile, Go is actually a really nice language to use. It has a few quirks, but it gives you reasonably-fast-running code that (because there is a garbage collector) does not require nearly as much skull-sweat when figuring out who owns memory and how to make sure it gets released appropriately. Its built in channels and goroutine support makes multi-threaded cod, and using all the CPUs effectively, also much easier. You pay (sometimes noticeably) for the GC, but the price is not too bad in less time-critical situations. The GC has a few short stop-the-world points but since Go 1.6 or so, it's pretty smooth, unlike what I remember from 1980s Lisp systems. :-) (Note: I started with Go 1.11, so I don't have a lot of history, nor that much experience in it. But I do like it.) Both Go and Rust have build systems. In Rust, this is a separate front-end from the compiler proper (rustc): you run "cargo build", for instance. In Go, you run "go build", "go test", etc., to build and invoke unit tests and so on. Rust has generics (think C++ templates, except sane) and Go lacks them although there's a plan for them in Go 2.0. Rust has, or at least had, rather bad array support when I last looked at it: you could make an array out of any type, but with only up to 32 elements. Chris From crossd at gmail.com Mon Jun 8 08:16:36 2020 From: crossd at gmail.com (Dan Cross) Date: Sun, 7 Jun 2020 18:16:36 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: <202006072115.057LFV6v089953@elf.torek.net> References: <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com> <202006072115.057LFV6v089953@elf.torek.net> Message-ID: On Sun, Jun 7, 2020 at 5:37 PM Chris Torek wrote: > Yes, both Rust and Go are being used. > Indeed. The languages complement each other quite nicely. Disclaimer: I've been programming in Rust for my day job for almost two years now and I sit in the office next to the Go tech lead and in close proximity a number of people working on Go. Rust has some advantages: like C++, it can compile to very fast > code that does not need a garbage collector but that provides > type-safety. It also gives you thread-safety through clever > compiler analysis of who "owns" any given variable or data. > Safe Rust code is data race free, but not free of race conditions, let alone thread safe in all ways. The ownership / lifetime-analysis / borrow-checker is quite > complicated and takes a lot of getting-used-to. Once you get used to it, it doesn't feel that complex but yes, it takes a while to get used to. When we switched our project to it (from C++, at my behest) our tech lead quipped that Rust has a "near vertical" learning curve. Suffice it to say we're all quite glad we switched but there was an adjustment period. Interestingly, we find it a selling point for attracting new engineers regardless of the challenges in coming up to speed. [snip] Rust has, or at least had, rather bad > array support when I last looked at it: you could make an array > out of any type, but with only up to 32 elements. I'm afraid this is incorrect. Rust arrays are indexed by a `usize`, which is basically whatever `size_t` would be in C. Rust arrays in general can be essentially arbitrarily large (up to limitations imposed by the target machine, of course). However, Rust does not support dependent types, most certainly not for arrays. In other words, an array's size is considered part of its type and so when specializing traits on arrays, one must do so explicitly for each supported array size. For practical reasons such implementations are often limited to a relatively small number of distinct sizes; 32 is a believable number. Perhaps that's what you're thinking of? For example, the `Vec` type was recently modified to implement the `From` trait on arrays, but only on arrays up to size 32. This means that one can essentially create a dynamic, growable vector seeded from an array, but such source arrays are limited in how large they can be. My current biggest frustrations are in assumptions made by the memory allocator, the magical nature of `box` (the allocation primitive), and the under-defined memory model. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From torek at torek.net Mon Jun 8 08:56:45 2020 From: torek at torek.net (Chris Torek) Date: Sun, 07 Jun 2020 15:56:45 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: Your message of "Sun, 07 Jun 2020 18:16:36 -0400." Message-ID: <202006072256.057Muo1h090326@elf.torek.net> >Safe Rust code is data race free, but not free of race >conditions, let alone thread safe in all ways. Er, yes. I mainly wanted to contrast to Go, where you look for race conditions by building with a flag that enables runtime checking. This can only detect races that actually occur, and if there are paths that would still have races that didn't occur on your test run, well... >>[rust array limitations] >I'm afraid this is incorrect. Rust arrays are indexed by a >`usize`, which is basically whatever `size_t` would be in C. Rust >arrays in general can be essentially arbitrarily large (up to >limitations imposed by the target machine, of course). >However, Rust does not support dependent types, most certainly >not for arrays. In other words, an array's size is considered >part of its type and so when specializing traits on arrays, one >must do so explicitly for each supported array size. For >practical reasons such implementations are often limited to a >relatively small number of distinct sizes; 32 is a believable >number. Perhaps that's what you're thinking of? Ah, yes, that was it. (As I said, I never really had time to do anything "real" in Rust.) Go's slices are nice to use, but slices (and maps, for that matter) trip people up because they are headers that point to shared spaces. Chris From wkt at tuhs.org Mon Jun 8 09:14:34 2020 From: wkt at tuhs.org (Warren Toomey) Date: Mon, 8 Jun 2020 09:14:34 +1000 Subject: [TUHS] Comparative languages In-Reply-To: <202006072256.057Muo1h090326@elf.torek.net> References: <202006072256.057Muo1h090326@elf.torek.net> Message-ID: <20200607231434.GA13571@minnie.tuhs.org> All, as the discussion on C has moved on to other languages, we might move the followups to COFF. Thanks, Warren From bramwyllie at gmail.com Mon Jun 8 10:24:38 2020 From: bramwyllie at gmail.com (Bram Wyllie) Date: Sun, 7 Jun 2020 21:24:38 -0300 Subject: [TUHS] History of popularity of C In-Reply-To: <202006072256.057Muo1h090326@elf.torek.net> References: <202006072256.057Muo1h090326@elf.torek.net> Message-ID: Dependent types aren't needed for sum types though, which is what you'd normally use for an array that carries its size, correct? On Sun, Jun 7, 2020 at 7:57 PM Chris Torek wrote: > >Safe Rust code is data race free, but not free of race > >conditions, let alone thread safe in all ways. > > Er, yes. I mainly wanted to contrast to Go, where you look for > race conditions by building with a flag that enables runtime > checking. This can only detect races that actually occur, > and if there are paths that would still have races that didn't > occur on your test run, well... > > >>[rust array limitations] > > >I'm afraid this is incorrect. Rust arrays are indexed by a > >`usize`, which is basically whatever `size_t` would be in C. Rust > >arrays in general can be essentially arbitrarily large (up to > >limitations imposed by the target machine, of course). > > >However, Rust does not support dependent types, most certainly > >not for arrays. In other words, an array's size is considered > >part of its type and so when specializing traits on arrays, one > >must do so explicitly for each supported array size. For > >practical reasons such implementations are often limited to a > >relatively small number of distinct sizes; 32 is a believable > >number. Perhaps that's what you're thinking of? > > Ah, yes, that was it. (As I said, I never really had time to > do anything "real" in Rust.) > > Go's slices are nice to use, but slices (and maps, for that matter) > trip people up because they are headers that point to shared > spaces. > > Chris > -------------- next part -------------- An HTML attachment was scrubbed... URL: From athornton at gmail.com Mon Jun 8 11:02:18 2020 From: athornton at gmail.com (Adam Thornton) Date: Sun, 7 Jun 2020 18:02:18 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: <20200607155239.GL1635@mcvoy.com> References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> <20200607155239.GL1635@mcvoy.com> Message-ID: > On Jun 7, 2020, at 8:52 AM, Larry McVoy wrote: > > On Sun, Jun 07, 2020 at 11:26:45AM -0400, Clem Cole wrote: >> That said, I personally am the most excited about Go theses day, but I'm >> also thinking Rust looks pretty interesting, but my experience with both >> compared to C is extremely nominal. Neither language is used for anything >> in production in our world at this point. > > If I had to move to a modern language it would be Go. I looked at Rust > and barfed. Several years ago, this was a job talk I gave, based on my experience at the time developing a pretty nifty system that never found traction. The new job (I got it!) doesn’t use Go, so I’ve grudgingly gone back to Python. But I stand by most of what I wrote (although I am sure parts of it are outdated and wrong now). The tl;dr is the title of the talk: https://athornton.github.io/go-it-mostly-doesnt-suck I make the claim that Go *is* pretty much C with 35 years of lessons learned about what did and didn’t work in C, and 35 years of machine time getting cheaper and programmer time getting more expensive. Adam From lars at nocrew.org Mon Jun 8 15:48:03 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Mon, 08 Jun 2020 05:48:03 +0000 Subject: [TUHS] History of popularity of C In-Reply-To: <202006072115.057LFV6v089953@elf.torek.net> (Chris Torek's message of "Sun, 07 Jun 2020 14:15:26 -0700") References: <202006072115.057LFV6v089953@elf.torek.net> Message-ID: <7w4krmw1kc.fsf@junk.nocrew.org> Chris Torek wrote: > You pay (sometimes noticeably) for the GC, but the price is not > too bad in less time-critical situations. The GC has a few short > stop-the-world points but since Go 1.6 or so, it's pretty smooth, > unlike what I remember from 1980s Lisp systems. :-) I'm guessing those 1980s Lisp systems would also be pretty smooth on 2020s hardware. From thomas.paulsen at firemail.de Mon Jun 8 18:04:41 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Mon, 08 Jun 2020 10:04:41 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> <20200607155239.GL1635@mcvoy.com> Message-ID: <7a76a39c7a0ceaa53a58a85fea3d6be2@firemail.de> >I make the claim that Go *is* pretty much C with 35 years of lessons learned >about what did and didn’t work in C, and 35 years of machine time getting >cheaper and programmer time getting more expensive. see Rob Pike 'Notes on Programming in C' freely available in the internet at: https://www.lysator.liu.se/c/pikestyle.html From clemc at ccc.com Mon Jun 8 23:10:37 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 8 Jun 2020 09:10:37 -0400 Subject: [TUHS] UNESCO's Software Heritage Foundation Message-ID: I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their archives and was not impressed (I couldn't find anything). https://www.softwareheritage.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dfawcus+lists-tuhs at employees.org Mon Jun 8 23:56:19 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Mon, 8 Jun 2020 14:56:19 +0100 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: <20200608135619.GA22471@clarinet.employees.org> On Sat, Jun 06, 2020 at 05:49:23PM -0400, Doug McIlroy wrote: > Syntactically it occurs at top level completely out of sync with the > indentation and flow of text. Conversion to if would be a big win. Well, doesn't that rather depend upon how one indents it? As I recall, that freedom came with ANSI C. But yeah, I'd prefer the if(0) type of elimination. DF $ cc yes-no.c; ./a.out; cc -DYES yes-no.c; ./a.out; cat yes-no.c No Yes #include int main(void) { #ifdef YES puts("Yes"); #else puts("No"); #endif return 0; } From aek at bitsavers.org Tue Jun 9 00:44:55 2020 From: aek at bitsavers.org (Al Kossow) Date: Mon, 8 Jun 2020 07:44:55 -0700 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: On 6/8/20 6:10 AM, Clem Cole wrote: > I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their > archives and was not impressed (I couldn't find anything). > > https://www.softwareheritage.org/ Don't diss them too badly. They are good folks, especially with how they are dealing with archiving and deduping almost-identical source trees. They stop by CHM every few years. From clemc at ccc.com Tue Jun 9 01:14:13 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 8 Jun 2020 11:14:13 -0400 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: Al, I'm not diss them or their efforts, just I wanted to know who they were. This is the first I had heard of them, I am, however, dissing the web site which I could not figure out how to use. On Mon, Jun 8, 2020 at 10:51 AM Al Kossow wrote: > On 6/8/20 6:10 AM, Clem Cole wrote: > > I'm seeding this URL to TUHS as one would expect them to be interested > in the work from Warren and friends. FWIW: I tried to browse their > > archives and was not impressed (I couldn't find anything). > > > > https://www.softwareheritage.org/ > > Don't diss them too badly. They are good folks, > especially with how they are dealing with archiving and deduping > almost-identical > source trees. > > They stop by CHM every few years. > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at kdbarto.org Tue Jun 9 00:58:30 2020 From: david at kdbarto.org (David Barto) Date: Mon, 8 Jun 2020 07:58:30 -0700 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: <2A179FD5-E1DA-459A-8B06-8025F5238ACF@kdbarto.org> As a quick look, I entered ‘unix’ to the search bar. 8^) In the list was a UNIX_V6PP. A quick browse it looked like a (not very good IMO) c++ rewrite of the v6 kernel for the x86 processor. David https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/lonway/UNIX_V6PP&path=UNIX_V6%2B%2B > On Jun 8, 2020, at 7:44 AM, Al Kossow wrote: > > On 6/8/20 6:10 AM, Clem Cole wrote: >> I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their archives and was not impressed (I couldn't find anything). >> https://www.softwareheritage.org/ > > Don't diss them too badly. They are good folks, > especially with how they are dealing with archiving and deduping almost-identical > source trees. > > They stop by CHM every few years. > > > > From rich.salz at gmail.com Tue Jun 9 01:20:58 2020 From: rich.salz at gmail.com (Richard Salz) Date: Mon, 8 Jun 2020 11:20:58 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: <20200608135619.GA22471@clarinet.employees.org> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> <20200608135619.GA22471@clarinet.employees.org> Message-ID: > But yeah, I'd prefer the if(0) type of elimination. > Some parts of OpenSSL do this, and it seems to work well across a pretty wide variety of platforms/toolchains. A really important benefit is that you get code syntax-checked as opposed to unused #ifdef branches rotting. A minor benefit is you don't have ifdef spaghetti around local variables, to avoid "xxx is not used" warnings. -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Tue Jun 9 01:30:50 2020 From: crossd at gmail.com (Dan Cross) Date: Mon, 8 Jun 2020 11:30:50 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: On Sat, Jun 6, 2020 at 5:50 PM Doug McIlroy wrote: > > Steve Johnson's position paper on optimising compilers may amuse you: > > https://dl.acm.org/doi/abs/10.1145/567532.567542 > > Indeed. This passage struck a particular chord: > > "I contend that the class of applications that depend on, for example, loop > optimization and dead code elimination for their efficient solution is of > modest size, growing smaller, and often very susceptible to expression in > applicative languages where the optimization is built into the individual > applicative operators." > > I don't know whether I saw that note at the time, but since then I've > come to believe, particularly in regard to C, that one case of dead-code > elmination should be guaranteed. That case is if(0), where 0 is the > value of a constant expression. > > This guarantee would take the place of many--possibly even > most--ifdefs. Every ifdef is an ugly intrusion and a pain to read. > Syntactically it occurs at top level completely out of sync with the > indentation and flow of text. Conversion to if would be a big win. > This came up on this list back in 2017 and I posted something about this at the time: https://minnie.tuhs.org/pipermail/tuhs/2017-January/009325.html If the language had been extended to support a "compile-type conditional" syntax ("if #(LINUX)", for example) it would have obviated the need for most preprocessor conditionals. As long as, syntactically, the compiler could tokenize a block in absence of e.g. type definitions, dead-code elimination could do the job. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Tue Jun 9 02:06:00 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Mon, 08 Jun 2020 16:06:00 +0000 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: (Clem Cole's message of "Mon, 8 Jun 2020 09:10:37 -0400") References: Message-ID: <7wpna9v8yf.fsf@junk.nocrew.org> Clem Cole wrote: > I'm seeding this URL to TUHS as one would expect them to be interested > in the work from Warren and friends. FWIW: I tried to browse their > archives and was not impressed (I couldn't find anything). They seem to have scraped GitHub, so they backed up everything I put there. Thanks guys. From aek at bitsavers.org Tue Jun 9 02:20:21 2020 From: aek at bitsavers.org (Al Kossow) Date: Mon, 8 Jun 2020 09:20:21 -0700 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: <7wpna9v8yf.fsf@junk.nocrew.org> References: <7wpna9v8yf.fsf@junk.nocrew.org> Message-ID: On 6/8/20 9:06 AM, Lars Brinkhoff wrote: > Clem Cole wrote: >> I'm seeding this URL to TUHS as one would expect them to be interested >> in the work from Warren and friends. FWIW: I tried to browse their >> archives and was not impressed (I couldn't find anything). > > They seem to have scraped GitHub, so they backed up everything I put > there. Thanks guys. > They are trying to archive and de-dup EVERYTHING that is open-source. Unix is more difficult, because of the encumbrances. https://www.softwareheritage.org/2019/06/06/announcing-the-software-heritage-graph-dataset/ From dot at dotat.at Tue Jun 9 02:32:04 2020 From: dot at dotat.at (Tony Finch) Date: Mon, 8 Jun 2020 17:32:04 +0100 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: Doug McIlroy wrote: > Every ifdef is an ugly intrusion and a pain to read. I maintain `unifdef`, which is not a complicated program to port but it does require some platform-specific stuff on Windows (partly because of the poor standards conformance of the Windows C library). There's a fun rule that `unifdef` must contain no #ifdef (or #if etc.) so the portability stuff is handled by the build scripts adjusting the include path and list of object files. Not a very scalable solution :-) Tony. -- f.anthony.n.finch http://dotat.at/ Biscay: North or northwest 3 or 4, occasionally 5 at first. Slight or moderate, becoming slight later. Showers, thundery in east. Good, occasionally moderate. From cym224 at gmail.com Tue Jun 9 04:21:18 2020 From: cym224 at gmail.com (Nemo Nusquam) Date: Mon, 8 Jun 2020 14:21:18 -0400 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: On 06/08/20 11:14, Clem Cole wrote: > Al, I'm not diss them or their efforts, just I wanted to know who they > were. > This is the first I had heard of them, I am, however, dissing the web > site which I could not figure out how to use. I have surfed there over the years but I have always found their UI nothing short of terrible. They have archived over 126 million projects but my search queries return little and their browsing i/f seems non-existant. N.