Subject: Disklabels arrive for 2.11BSD (#250 part 1 of 18)
Index:	sys,bin,usr.lib,(many more)/<many> 2.11BSD

Description:
	The moving the partitions tables out of the disk drivers and to
	a disklabel residing on the media has been on the wish list for
	many many years.

	Disklabels have finally arrived for 2.11BSD!

Repeat-By:
	Observation.  Also a reading of the setup and installation
	documentation for previous 2BSD releases (2.9, 2.10, 2.11)
	all have a paragraph similar to this present:

"It is possible to change the partitions by changing the code for the
table in the disk driver.  Since it's desirable to do this, these tables
really should be read off each pack...."

Fix:
	This is part 1 of 18.  Gather all parts before doing anything
	and read the instructions which are below.  The final part has
	a small collection of hints, tips and miscellaneous discussion.

	NOTE:   Only the MSCP and RL drivers implement labels at this time
		because those are the only disk types I have access to.  Other
		drivers will be modified when test sites with local expertise
		(or plane fare;-)) and a willingness to do testing volunteer.

	This is a _large_ kit, approximately 400kb of new files, patches
	and scripts (to assist in the installation).  

	It is always a good idea, but *especially* so in this case, to
	have a complete bootable backup of the system.  Previous large
	updates would, if things went awry, render a program unusable or
	unstable - if anything goes wrong in the installation of this
	update kit the system will likely be completely unbootable.

	Make a minimal backup now by:

		cd /sys/pdpstand
		make
		./maketape /dev/nrmt0 maketape.data
		dump 0uf /dev/nrmt0 /

	Substitute the appropriate device for /dev/nrmt0 if that is
	not the right device on your system.  Also be sure to print out
	a copy (if you do not have one at hand) of the setup/installation
	documents found in /usr/doc/2.10/setup.2.11 - they will come in
	handy if you need to use the backup just created and do not have
	the coldstart proceedure memorized.

	NOTE: Since the setup/installation documentation is updated as
	      part of this kit you should format and print the current
	      version now if you think you will need it.

	The above is a bootable tape that contains a dump of the root
	filesystem.  This is *usually* all that's needed to recover from
	a disaster.  You should also at this point backup /usr and any
	user files present. 

	Now on to the description of the update kit.  There are, as mentioned
	before, 18 parts to the kit.  Here is a brief description of the
	contents (each part contains details about its contents):

	   Part		Contents
	   ----		--------
	   1		Instructions.
	   2		sys/GENERIC patches
	   3		bin - the disklabel program
	   4		sys/conf changes
	   5		dev - changes to MAKEDEV
	   6		etc - restor, newfs, mkfs changes
	   7		sys/h - disklabel.h, disk.h, buf.h, ioctl.h changes
	   8		include - disktab.h removed
	   9		lib - disktab.c replaced, disklabel.c added
	   10		man - newfs, ra, rl man pages updated
	   11		mdec - ALL BOOT BLOCKS MODIFIED
	   12		pdp - conf.c, machdep2.c modified
	   13		pdpdist - *NEW* disktab file
	   14		pdpstand - standalone system _rewritten_ (/boot, usw.)
	   15		pdpuba - massive changes to MSCP and RL drivers to
				support disklabels.  All other disk drivers 
				lightly modified because buf.h changed.
	   16		sys - ufs_disksubr.c added (the label routines reside 
				here).  disk device open/close calls take 3
				args instead of 2 now.
	   17		usr.lib - dkcksum.c (disklabel checksum routine)
				added to libutil.a
	   18		doc - the setup and installation documents are updated

	Rest time - take a deep breath, grab a soda/bier/coffee, relax and
	let's go.

	Each part of the kit contains one or more of the following:

		1) a ".shar" file - this is a shar archive containing new 
		   files to be added to the system.

		2) a ".sh" file - this is a shell script to be run.  Typically
		   this script _removes_ (by renaming to a file with a ~ 
		   appended) files from the system.

		3) a ".patch" file - this is a 'patch' file to be run with
		   the command "patch -p0 < patchfile".
	
	Not all parts of the kit have all of the above.

	NOTE:  Some parts of the kit are almost trivial in size while others
		are quite large.  It made my life a lot easier to organize
		the kit around directories that were affected.

	How to go about the update
	--------------------------

	Step 0 - Preparation.
	======
	You have done a backup, right?

	Have you read this entire document?  It would be a good idea to
	do that.

	Do NOT compile or install _anything_ until requested to do so.
	The new kernel would be !extremely! unhappy when faced when a non-
	labeled disk.

	If you do not have a "generic" kernel installed in / as the
	file /genunix you may wish to create one now before applying
	any updates or patches.  See Step 11 ("Disaster Prevention").

	To create a generic kernel:

		cd /sys/GENERIC
		make
		mv unix /genunix
		make clean

	If anything goes wrong when rebooting the system (the kernel panics
	because it can't find the swap partition, and so on) you will need 
	the bootable tape made in the "Fix: " section above.  Having a
	'generic' (non-networking) kernel in the root partition is cheap
	insurance against disaster.

	Step 1 - Applying the patches.
	======

	Following the instructions in each part of the kit you should apply
	the patches, unshar the .shar files and run the scripts.

	[[[
		a long pause while the keyclicks sound and the disk whirls ...
	]]]

	There.  That was not so hard, was it?  Any failures?  There have
	been reports (from sites that possibly didn't keep the /sys/GENERIC
	files updated) that patches to /sys/GENERIC often don't work - if
	that is the case you can either email myself (I'll send new Makefiles)
	or simply "rm -r /sys/GENERIC" and then "cd /sys/conf; ./config GENERIC;
	cd ../GENERIC; make" (you'll have to adjust the overlay structure).

	If anything except the /sys/GENERIC patches fail then something is
	wrong and you need to fix it.  DO NOT proceed further unless you are
	know what to do - and I would appreciate a mail item detailing where
	I messed up.

	All the updates applied successfully?  Great!  There is an order
	in which things must be done now - it would not do at all to install
	a new kernel before building the utilities which place a label on the
	disk.

	Step 2 - Rebuilding the libraries.
	======

	First the object libraries 'libutil.a' and 'libc.a' need to be
	recompiled and installed:

		cd /usr/src/usr.lib/libutil
		make
		make install
		make clean

		cd /usr/src/lib/libc
		make
		make install
		make clean

	Step 3 - Installing new manpages.
	======

	Format the new or modified manpages.  These will come in handy
	during the next steps.  The 'disklabel' manpage is in the source
	directory for the disklabel program - that program is compiled
	and installed at this time.

		cd /usr/src/man/man8
		/usr/man/manroff newfs.8 > /usr/man/cat8/newfs.0
		cd ../man4
		/usr/man/manroff ra.4 > /usr/man/cat4/ra.0
		/usr/man/manroff rl.4 > /usr/man/cat4/rl.0
		cd /usr/src/bin/disklabel
		make
		make install
		make clean

	Step 4 - New disktab file.
	======

	Next the new disktab file is installed in /etc.

		cd /sys/pdpdist
		mv /etc/disktab /etc/disktab.nolabel
		cp disktab /etc/disktab
		chmod a+r /etc/disktab

	Initially the label on the disk must match the current partition
	sizes and offsets otherwise it will not be possible to access
	existing data on the disk.  

	IMPORTANT:  Now is not really the time to decide on a new and
		    better partition setup - if the label does not match
		    the existing partition setup then it will not be possible
		    to access existing data after the new kernel is
		    installed!  Far better, but more time consuming, to
		    complete the installation of the update first - then
		    do a backup and then edit the disklabel (using the
		    standalone program if either the 'a' or 'b' partitions
		    of the system drive are to be changed).

		    If it is desired to implement a new partition layout
		    then you must make certain that you do a complete
		    backup after the new kernel is installed but before
		    rebooting (and pray you do not have a power failure
		    while the backup is running).

	At this point double check those entries in /etc/disktab that
	you will be using when labeling the disks on the system.  You will
	need to know the starting offset and sizes of the partitions being
	used.  The RD54 entry is known to be correct but others may need
	a close look before being used.

	The previous version of the MSCP driver (pdpuba/ra.c~ after applying
	the patches) has the partition information in its tables.  The kernel
	when it booted printed out a message telling what type of partition
	table was being used.  That is the table to make sure matches the
	disktab entry you intend to use.

	NOTE: Only the MSCP and RL drivers implement labels at this time.

	      If you are installing this update on a SMD ('xp.c') disk it
	      will not hurt/damage anything to install labels on the drives.
	      Think of it as a "practice run" and a chance to spot errors
	      in /etc/disktab before they can do any harm.

	      The RL driver did not support partitions in the past.  If
	      you are currently using RL drives for data and wish to
	      preserve that data (and not have to reload it after making
	      new filesystems) then you should be sure to use a partition
	      which spans the entire disk (i.e. make rl0a into a 20480
	      sector partition on an RL02).

	An incorrect /etc/disktab entry will likely require that the backup 
	tapes created earlier be restored.

	SUMMARY:  What this step is all about is making sure that the
		  command:

			disklabel -r -w device type

		  (for example)

			disklabel -r -w ra0a ra81

		  will work correctly later on.  I used 'ra0' and 'ra81'
		  as examples - your type might be different.  What the
		  'disklabel' program will do is look up the "type" in
		  /etc/disktab and use the information there to create an
		  initial label.

		  The point of Step 4 is to make sure that the label reflects
		  the _current_ partition layout *exactly*.

		  If none of the existing entries in /etc/disktab match
		  your setup because the driver partition tables and the
		  disktab file have been modified locally then you have to
		  convert the old partition layout into the new style.

	Step 5 - New boot blocks.
	======

	Compile and install the new bootblocks.  It is safe to use
	these bootblocks with the old boot program and kernel as long
	as you are booting from unit 0 on the first controller.

		cd /sys/mdec
		make
		make install
		make clean

	Step 6 - Labeling the disks.
	======

	Now would be a very good time to print out or study very very
	carefully the disklabel manpage what was installed a couple steps
	ago.

	Writing the raw disk device requires that the system be in
	single user mode when the device is the root/system disk.  UNMOUNTED
	disks can be written without going into single user mode.

	The disklabel program that was compiled a short time ago can be run 
	using the '-r' option.  This option reads and writes the raw disk 
	rather than relying on the kernel's copy of the label.

	When 1) running on a kernel that does not have labels or 2) with a 
	device whose driver has not been updated to support labels or 3)
	when there is no initial label present the '-r' option needs to
	be used.

	NOTE: You will probably need to be in "single user" mode to perform
	      the labeling of the system drive.  This is because at 
	      securelevel 1 (the default) the kernel will not permit writing 
	      disk devices which are mounted (and the 'a' partition is always 
	      mounted by the kernel on /).

	NOTE:  YOU MUST install labels if you have MSCP or RL devices 
	       present!  Once a label aware kernel is present there are
	       no longer partition tables in the drivers!  The kernel will 
	       attempt to span the entire drive with the 'a' parition if 
	       it detects an absent or corrupted label.  This almost certainly 
	       will not be what you want, especially if there are partitions 
	       other than 'a' actually present.

	The -B option is used with 'disklabel' to install new boot blocks.
	Previously this was done by 'newfs' when a filesystem was created.

	In the example below the drive is an RD54 at unit 0, the system is 
	in single user mode and new bootblocks are to be installed.  

	To assign the label we type:

		disklabel -r -w -B ra0 rd54

	To display the label but not change anything:

		disklabel -r ra0

	To edit the label (normally not done, but you might want to change
	the pack ID):

		disklabel -r -e ra0

	This will invoke 'vi' (or the program given by the environment
	variable EDITOR) on a temporary file containing the formatted
	label.  Exiting the editor without saving the changes leaves the
	current label unchanged.

	IT IS TIME to assign an initial label to your system disk!

		disklabel -r -w -B ra0 xxxx

		(where xxxx is the drive type - RD54, RA81, ...)

	or

		disklabel -r -w -B rl0 rl02

	NOTE:  It is normal to see a warning error at this point.  This is
	       is because the existing kernel does not understand one of
	       the ioctl() calls that 'disklabel' is using.  'disklabel'
	       ignores the error, you should also.

	Then look at the label with:

		disklabel -r ra0

	If you see any errors then something is wrong.  You should see
	the disklabel displayed and the correct partition information
	at the end of the display.  ANYTHING else is an error.  The system
	is still bootable and runnable - write down exactly what did appear
	and mail the information to me.

	IMPORTANT:  Be sure to label *all* disks that are present in the
		    system.  If you have two RA81 drives then label them
		    both.  If there is an RL drive or two on the system
		    label them as well.

	Once the system disk has been labeled you can return to multiuser
	mode for the remainder of the installation by:

		umount -a
		^D

	Step 7 - New mkfs, newfs, restor.
	======

	The 'mkfs', 'newfs', and 'restor'  programs are installed next.

		cd /usr/src/etc
		make mkfs newfs
		install -s -m 755 mkfs newfs /etc
		cd restor
		make
		make install
		make clean

	NOTE:  'newfs' is now aware of labels and will not make a filesystem
	       on a device without a label.  For device drivers which do
	       not support labels there is the "-T type" option to 'newfs'.
	       The argument to "-T" is a disk type to be searched for in 
	       /etc/disktab.  Thus to make a file system on an RK07 it is
	       necessary to type:

			newfs -T rk07 /dev/rhk0h 

		'newfs' will also not create a filesystem on a partition 
		that is not "2.11BSD".

		The manpage has more details about the new options to 'newfs'.

	Step 8 - Rebuilding the kernel.
	======

	At long last the moment has arrived to build a new kernel.  (and you
	thought we'd never get around to it ;-)).

	Several new modules were added to the kernel, several other kernel
	modules grew a little bit and a couple of the drivers (RA and RL) 
	grew considerably in size.  All of these conspire to make existing
	kernel Makefiles unusable.

	The two methods available for updating existing kernel build
	directories are:

		1) simply remove the existing directory, cd into /sys/conf,
		   run ./config YOURKERNEL, and then rearrange the kernel
		   overlay structure until everything fits again.

		2) copy /sys/conf/Make.pdp, Make.sys, Make.pdpnet into
		   the kernel directory.  Then use the GENERIC/Makefile
		   patch (part 2, #251) as a guide and make similar 
		   changes to the Makefile in the kernel directory.

	Using old Makefiles or forgetting to copy Make.sys, Make.pdp,
	Make.pdpnet into each kernel directory will result in undefined
	symbols or 'make' complaining about not knowing how to make a file.

	It will probably take a couple tries to get the overlay structure
	adjusted and you may need to use an additional overlay.  The
	disklabel (ufs_disksubr.o) module is a good candidate to be in an 
	overlay by itself, it's large and infrequently called.

	The kernel is getting big (no surprise) - more and more modules
	are ending up in overlays with only the basic "core" modules
	remaining in the base segment.  Do not be surprised or alarmed
	if you need to expand into OV8 or OV9 (well, I'd be a little
	concerned about needing OV9, but not too much).

	This is also a good time to review and perhaps update the kernel
	config files.  Perhaps you want to include the INGRES lock driver
	now or adjust the number of disk buffers allocated, and so on.

	Make the kernel:

		cd /sys/YOURKERNEL
		make

	go out for dinner (or watch some TV).  It takes about 75 minutes
	+/- a few to make a kernel these days.

	DO NOT install the new kernel just yet, the new /boot program
	needs to be compiled and installed at the same time.

	Step 9 - Remaking RL device nodes.
	======

	NOTE:  This step should only be done *after* placing a label on
	       the RL disks.  
	       
	NOTE: This step was deferred until just before the system is rebooted
	      because after changing the device numbers in /dev it will no 
	      longer be possible to refer to the RL drives using the old kernel.

	The RL driver now supports partitions.  Previously only the
	/dev entry 'rl0h' was used and the unit number was stored in what
	is now the partition number portion of the minor device number.   
	The device entries in /dev for the RL device should be recreated now 
	with new minor device numbers.

		cd /dev
		rm rl* rrl*
		./MAKEDEV rl0

	NOTE:  IF you have 'rl' entries in /etc/fstab it may be
	       necessary to change them.  Where 'rl0h' was used before
	       it may be that 'rl0a' is the correct name to use now.

	       If you are using the entire RL as one partition "rl0h"
	       is still correct.

	Step 10 - Rebooting the system.
	=======

	Make the new boot program.

		cd /sys/pdpstand
		make

	For a few minutes let there be no power fails or crashes.  I'd hate
	to have to sort out the mess.

	If you have diskspace availabel you may wish to save the old
	kernel and networking images for a little while.  If (uh, I mean
	when;-)) the new kernel boots successfully and all your data is
	available then any old kernel images should be discarded.  To save
	the old images for now:

		mv /unix /ounix
		mv /netnix /onetnix

	Then install the new kernel:

		cd /sys/YOURKERNEL
		mv unix netnix /
		chmod 744 /unix /netnix

	Quickly the new boot:

		cd /sys/pdpstand
		cp boot /boot
		sync

	Ok - whew, made it.  Time to reboot.  You can either do:

		fastboot

	or
		
		kill 1
		halt
	and then go thru the boot ROMs

	Step 11 - Disaster recovery.
	=======

	If all went well (it did for me, but...) your system is now 
	running a label aware kernel for MSCP and RL devices.  You may
	now change the partition layout to your heart's content.  Doing
	so will of course mean doing full backups and restor sessions.

	If all did not go ok - damn!  You will have to reload from
	the backup tape made at the beginning of this and let me know
	what happened (in great detail).  The most likely problem is that
	the label did not get applied correctly.  You can use the bootable
	tape with the standalone disklabel program on it to recover from
	an incorrect label.  YOU WILL need to know the exact partition
	layout that should be present - but knowing that you can correct
	the label very easily with the standalone utility.

	Step 12 - Helpful hints, tips and miscellaneous ramblings.
	=======

	This is the less structured portion of the documentation.

	The disklabeling scheme was "borrowed" and adapted from the
	4.4-Lite CD.  Some things had to be changed, naturally, to fit
	in with the rest of the 2.11 system.  The concept of "filesystem
	fragments" doesn't exist for example in 2.11.  Also in 2.11
	the concept of "filesystem block" and "disk sector" are much more
	blurred together than 4.xBSD - this caused a lot of problems
	in 'disklabel' where the code that was being adapted kept wanting
	to make the "boot sector" 1kb long and things like that.

	The sector size _must_ be 512 bytes - this has been hard coded in
	to several places in the code to minimize the chances of scribbling
	filesystems.  Utter chaos would result if the disklabel program
	(or kernel) thought it had a superblock of 4kb or some crazy value.

	One difference between 2.11 and 4.x is in the choice of partition
	used when accessing the label.  This partition must start at 
	sector 0.  In 4.x the 'c' partition by convention spans the entire
	drive and it is the 'c' partition used to read/write the label.
	In 2.11 the convention is less well established with some drivers
	using 'h' to span the whole disk, other drivers use 'c' sometimes
	and 'g' at others.

	In the interests of compatibility it was decided to do as 4.3-Tahoe
	did and use 'a' for accessing the partition.  This is not a great
	hardship because the "root" partition has to start at the beginning
	of the disk anyways so that the bootblock code can find the kernel
	image.

	If you wish to create a partition layout more suited to your needs
	(a smaller swap partition for example) you will need to use the
	standalone disklabel utility which is loaded from tape.  The kernel
	does not permit changing the offset or size of an open partition
	and the 'a' and 'b' (root and swap) partitions are always open
	when the kernel is active.  Thus to change the offset or size of
	'b' and give the extra space to 'e' (for example) you need to

		BACK UP THE SYSTEM - you'll have to reload everything
		from tape because new filesystems will be created after
		the partition sizes have changed.

		boot the tape and load the standalone disklabel program.

		edit the label to be what you want.  The disklabel program
		will *warn* you about overlapping partitions but will not
		prohibit you from creating them.

		If you have altered the size of the 'a' partition you will
		need to run the standalone mkfs program, create a filesystem
		and then use the standalone 'restor' to load the dump of
		the root filesystem.

		Boot up single user and use the 'mkfs' program to create
		filesytems in the partitions you will be using.

		Edit/check /etc/fstab to make sure it is correct.

		Use 'restor' (or 'tar' if that was how you backed up /usr
		and/or the other parts of the system) to load the saved 
		filesystem.

		Bring the system up multiuser.