================================================================================
Note 27.0                  Using Messages with VAXELN                 No replies
FURILO::GIORGETTI                                   292 lines  21-AUG-1985 23:04
--------------------------------------------------------------------------------


      +---------------+                                    +-----------------+
      | d i g i t a l |                                    |  uNOTE # 027    |
      +---------------+                                    +-----------------+

                                                                   
      +----------------------------------------------------+-----------------+
      | Title: USING MESSAGES WITH VAXELN                  | Date: 19-JUN-85 |
      +----------------------------------------------------+-----------------+
      | Originator: Christopher DeMers                     | Page 1 of 5     |
      +----------------------------------------------------+-----------------+


      This  Micronote  discusses  how  VAXELN  uses  messages  for   inter-job
      communication.   Included  is  an  overview of the message architecture,
      benchmark data from MicroVAX II  configurations  and  a  sample  program
      illustrating the message handling technique.

      In a VAXELN application, every job(Pascal,C or Macro-32 program)  has  a
      unique  and protected virtual address space.  Within a single processor,
      the kernel separates each job's virtual  address  space  using  the  VAX
      memory  management  hardware.   Within  a  network,  each  job's virtual
      address space is separated by virtue of the fact that the jobs may exist
      in the memories of different computer systems.

      One of the principal reasons for dividing an application  into  separate
      jobs  is  to  aid  the migration and distribution of the jobs within the
      network.   In  order  for  the  jobs  to  work  together  on  the   same
      application, they must be able to share data, but the only way of moving
      data from the memory of one processor to another  in  a  network  is  by
      packaging  the  data  in a message and directing the network hardware to
      move the message to the destination memory.

      To make the network movement of data between jobs the  same  as  in  the
      non-network  case,  message  passing is the principal means of inter-job
      communication.  The kernel provides a number of facilities  to  make  an
      efficient and transparent means of communication.

      The VAXELN MESSAGE object describes a block of memory that can be  moved
      from  one job's virtual address space to another's.  The block of memory
      is called 'message data' and is allocated dynamically by the kernel from
      physically  contiguous, page-aligned blocks of memory.  A MESSAGE object
      and its  associated  message  data  are  both  created  by  calling  the
      CREATE_MESSAGE kernel service.

      Message data is mapped into a job's P0 virtual address space, so  it  is
      potentially accessible to all the processes in the job.  If a message is
      sent to a job on the sending job's local node, the kernel unmaps the






                                  211

uNOTE # 027
Page 2 of 5


      message data from the sending job's virtual address space and remaps  it
      into  the  receiver's  space.   Instead  of  moving the data, Page Table
      Entries are remapped, resulting in very fast message transmission  time.
      If  a  message  is  sent  to  a remote node, the kernel again unmaps the
      message data, but it remaps  it  into  the  appropriate  network  device
      driver  job  to  send  the  message  to  the remote system.  The reverse
      operations then cause the message data to be remapped in the  receiver's
      space.

      The code necessary to send a message local to  a  machine  or  over  the
      network  is  identical  in  both  cases.   A  port  name table parameter
      (NAME$LOCAL or NAME$UNIVERSAL) allows the programmer to define local  or
      remote  ports.  The kernel maintains the local name list, while the name
      table for network-wide names is maintained by the  name  server.   Names
      known  throughout  the  network are guaranteed to be unique.  Thus, jobs
      can initially be executed on one processor and later be segmented to run
      on multiple processors without modification of the program.

      Datagrams and Circuits
      ----------------------

      Messages can be used two ways to transmit data:

      - One process can obtain the value of a  port  anywhere  in  the  system
      (including  other  jobs) or in a different system running on a different
      Ethernet node.  The process can send the port a message  with  the  SEND
                                     datagram
      procedure.  This is called the datagram method.

      - Any two ports (usually in different jobs) can  be  bound  together  to
               circuit
      form  a  circuit.   In  this  case,  having established the circuit, the
      sending process has one port of its own bound  to  another  port,  which
      usually  is  in  a  different  job  or on a different network node.  The
      sender sends the message to its own port, and it is routed automatically
      to  the  other  port  in  the  circuit.   Processes can both send to and
      receive from a circuit port.

      In the datagram method, as well as in the circuit method, a process  can
      use  the  WAIT  procedures  to  wait for the receipt of a message on the
      specified port.

      The datagram method requires no connection sequence as in circuits,  but
      cannot guarantee that a message is actually received at the destination.
      However, it does guarantee that received messages are correct.

      At the cost of setting up and handling a  circuit  connection,  circuits
      offer several advantages over the basic datagram method:

        Guaranteed delivery
      - Guaranteed delivery.  The circuit method guarantees  that  either  the
      message  arrives  at the destination port regardless of its location, or
      that the sender is notified that the message could not be delivered.



                                  212

                                                           uNOTE # 027
                                                           Page 3 of 5


        Flow control
      - Flow control.  You can prevent a sending process from sending too many
      messages  to  a  slower  receiving process.  If a port is connected in a
      circuit and is full, the sender is put into the waiting state until  the
      port  is  no  longer full (you can override this default).  The implicit
      waiting performed by the SEND  procedure  evens  the  flow  of  messages
      between  the  transmitting  process  and  the  receiving process without
      having to explicitly program for the condition.   Furthermore,  circuits
      guarantee  the sequence of message delivery as no other unconnected port
      can send a message to a connected port.

        Segmentation
      - Segmentation.  Message can have any length (datagrams are  limited  to
      approx.   1500  bytes),  and, if the transmission is across the network,
      the network services will divide the message into segments of the proper
      length,  transmit  the  segments  and reassemble them at the destination
      node.

        User interface
      - User interface via Pascal I/O routines.  The  OPEN  procedure  permits
      you to 'open' a circuit as if it were a file and to use the I/O routines
      such as READ and WRITE to transmit messages.

      There is no performance penalty with circuits for  messages  transmitted
      over  the  same network node and very little over the network.  For full
      generality, programs should assume that the sending and  receiving  jobs
      may  be  distributed  on different nodes in a network.  Circuits are the
      preferred means of sending message in almost every practical case.

      Expedited Messages
      ------------------

      An expedited message bypasses the normal flow-control mechanism and  can
      be  sent  even  if  the receiving port already has its maximum number of
      messages.  The message is received by the port before  any  normal  data
      messages.   The  size of an expedited message is limited to a maximum of
      16 bytes.

      Monitoring Network Integrity
      ----------------------------

      Circuits can be used as a method for monitoring the status of a  network
      connection.   In  addition  to  the  circuit  used to send data, another
      process could establish a secondary circuit with a process in  a  remote
      node.   Neither  process  sends data, they only WAIT on their respective
      ports.  When a node fails,  the  circuit  is  broken  and  the  WAIT  is
      satisfied.   At  this point exception handling routines could be invoked
      to handle this condition.  This allows the programmer  to  separate  the
      error handling code from the message transmission code.







                                  213

uNOTE # 027
Page 4 of 5


      Connecting with VAX/VMS nodes
      -----------------------------

      The VAXELN procedure CONNECT_CIRCUIT can be used to request a connection
      with  a  VAX/VMS program on the same DECnet network.  Instead of using a
      port as the destination for the connection, a VMS  node  and  an  object
      name  (program  name  on  the VMS node) is used.  The program on the VMS
      node completes the connection by opening the "file" SYS$NET.

      A VAX/VMS node may also request a connection by specifying a  node  name
      and port name in the file portion of an OPEN statement.  The VAXELN node
      complete the connection by issuing an ACCEPT_CIRCUIT statement.

      Performance Data
      ----------------

      MicroVAX II, 5MB memory, connection via circuit
      All times in microseconds (us).
      Throughput in thousand bytes/second(KB/s).

                                             Delivery                
      Message Size|Create|Delete|  One  Machine |  Two Machines          
       (bytes)    | Time | Time |Time|Throughput| Time|Throughput
      ------------+------+------+----+----------+-----+----------
         0        |  131 |  144 |1094|   N/A    | 4479|   N/A 
       512        |  361 |  255 |1611|   318    | 5271|    97
      2048        |  450 |  274 |1885|  1087    | 8675|   236
      8192        |  481 |  394 |2139|  3830    |27538|   298

























                                  214

                                                           uNOTE # 027
                                                           Page 5 of 5


      PROGRAM msgsend(INPUT,OUTPUT);
      {++
      { This program connects a circuit to the global name 'msgrecv',
      { sends a message to the receiver and waits for a reply.
      {--}
      VAR  
           send_port,reply_port: PORT;
           msg,reply:            MESSAGE;
           msg_data_ptr:         ^VARYING_STRING(20);
           reply_data_ptr:       ^VARYING_STRING(10);

      BEGIN
        CREATE_PORT(send_port);
        CONNECT_CIRCUIT(send_port, destination_name := 'msgrecv');
        CREATE_MESSAGE(msg,msg_data_ptr);
        msg_data_ptr^ := 'message from msgsend';
        SEND(msg,send_port);
        WAIT_ANY(send_port); 
        RECEIVE(reply,reply_data_ptr,send_port);
        DELETE(reply);
        DISCONNECT_CIRCUIT(send_port);
      END; { program msgsend }


      PROGRAM msgrecv (INPUT,OUTPUT);
      {++
      { This module runs a receive program that accepts a circuit,receives 
      { a message and sends back a reply.
      {--}

      VAR  
           reply_port:     PORT;
           nam:            NAME;
           msg,reply:      MESSAGE;
           msg_data_ptr:   ^VARYING_STRING(20);
           reply_data_ptr: ^VARYING_STRING(20);

      BEGIN
        CREATE_PORT(reply_port);
        CREATE_NAME(nam,'msgrecv',reply_port,table := name$universal);
        INITIALIZATION_DONE;
        ACCEPT_CIRCUIT(reply_port);
        WAIT_ANY(reply_port);
        RECEIVE(msg,msg_data_ptr,reply_port);
        DELETE(msg);
        CREATE_MESSAGE(reply,reply_data_ptr);
        reply_data_ptr^ := 'reply';
        SEND(reply,reply_port);
      END; { program msgrecv }




                                  215

