Subject: Re: Problem with Sonic and rarp/bootp
To: None <lae@uniyar.ac.ru>
From: Ken Nakata <kenn@eden.rutgers.edu>
List: port-mac68k
Date: 03/05/1998 23:25:58
On Thu, 5 Mar 1998 21:16:46 +0300 (MSK),
"Andrey E. Lerman" <lae@uniyar.ac.ru> writes:
> On Wed, 4 Mar 1998, Mason Loring Bliss wrote:
> > On Wed, Mar 04, 1998 at 09:53:21PM +0300, Andrey E. Lerman wrote:
> > > sn0 at obio0 address 00:a0:40:1f:5c:11
> > > #comment             ^^^^^^^^^^^^^^^^^ this looks like hardware address of
> > > #this machine... But it is wrong! Real address is 00:05:02:F8:3A:88 and
> > > #BOOTP server configured to this address.
> > 
> > Um... Are you absolutely sure this isn't the address? If I were you, I'd
> > go through at least one iteration with the assumption that what NetBSD is
> > reporting is the correct address, with anything that needs to know about
> > the address set appropriately...
> 
> I configured bootpd to be more verbose and put more information to system
> logs. In first configuration (HW address showed by arp) I saw that bootpd
> cannot find 00:a0:40:1f:5c:11 in /etc/bootptab. Okey I rem-ed old config
> and inserted address reported by NetBSD. Now server successfully replied
> to request. But NetBSD still generates the same error.

I *know* why the address is different in MacOS and NetBSD.  Also, I
have a pretty good theory on the "Tx - timeout" error.  See below.

> I suppose that kernel determines HW address of the machine incorrectly and
> sends BOOTP packets with wrong address. Server replied to that address but
> network card do not receive these packets because it have different 
> address.

This is highly unlikely.  SONIC chip accepts whatever hardware address
it is programmed.  IOW, if the driver thinks its hardware address is
0:5:2:f8:3a:88 and program the chip accordingly, the SONIC chip will
also think its hardware address is that address, and accept packets
addressed to it.  So, SONIC missing packets because of hardware
address discripancy can't be the case.

Why NetBSD and MacOS use different addresses is documented in the
following comments at the beginning of function sn_get_addr() in
/sys/arch/mac68k/dev/if_sn.c:

	/*
	 * For reasons known only to Apple, MAC addresses in the ethernet
	 * PROM are stored in Token Ring (IEEE 802.5) format, that is
	 * with all of the bits in each byte reversed (canonical bit format).
	 * When the address is read out it must be reversed to ethernet format
	 * before use.
	 *
	 * Apple has been assigned OUI's 08:00:07 and 00:a0:40. All onboard
	 * ethernet addresses on 68K machines should be in one of these
	 * two ranges.
	 *
	 * Here is where it gets complicated.
	 *
	 * The PMac 7200, 7500, 8500, and 9500 accidentally had the PROM
	 * written in standard ethernet format. The MacOS accounted for this
	 * in these systems, and did not reverse the bytes. Some other
	 * networking utilities were not so forgiving, and got confused.
	 * "Some" of Apple's Nubus ethernet cards also had their bits
	 * burned in ethernet format.
	 *
	 * Apple petitioned the IEEE and was granted the 00:05:02 (bit reversal
	 * of 00:a0:40) as well. As of OpenTransport 1.1.1, Apple removed
	 * their workaround and now reverses the bits regardless of
	 * what kind of machine it is. So PMac systems and the affected
	 * Nubus cards now use 00:05:02, instead of the 00:a0:40 for which they
	 * were intended.
	 *
	 * See Apple Techinfo article TECHINFO-0020552, "OpenTransport 1.1.1
	 * and MacOS System 7.5.3 FAQ (10/96)" for more details.
	 */

In a nutshell: Apple goofed (again) and we suffer.

> I think the problem is on NetBSD kernel side and should be fixed.

Yes, the address difference is caused by the sn driver; it's written
in a way so that it bit-reverses the hardware address only when it's
in 8:0:7 range (the bit-reversal of 8:0:7, 10:0:e0, isn't a legal
address range for Apple hardware, so bit-reversal is necessary).  But
it's hard to say it is a bug.

After all, it uses a unique address officially assigned to Apple,
either with or without bit-reversal if the address is in 0:5:2 or
0:a0:40 range.  IOW, it doesn't break any protocol or law whether we
use a 0:5:2 address or a 0:a0:40 address.  And it's not always
possible for sn driver to use the same one MacOS driver is using,
since that depends on which MacOS driver (pre- or post-OT 1.1.1) you
are using.

Is it possible for you to configure the Bootp server such that it
looks like your Mac has two network interfaces on one network? (one
with the address in 0:a0:40 range, and another in 0:5:2 range)  That
should work around this problem.

Ok, we've still got a problem - the "Tx - timeout" problem is entirely
different matter.  I had the exact same problem when I tried to run
NetBSD on a IIci with Apple Ethernet NB TP.  I can't say for sure, but
I think what was happening there was the following:

Sn driver allocates a small chunk of *system memory* for communication
with SONIC chip, regardless of the presence or absence of on-board
RAM.  Now, the SONIC chip is supposed to read from and write to this
buffer, but the chip on the NB TP card can't seem to do that.  IOW,
the SONIC chip on board can't read outgoing packets from the buffer,
can't write incoming packets into the buffer, or can't even update the
status of any packet.  Hence timeouts.  I think the NB TP card is not
designed to be a bus master.

I modified sn driver so that it uses on-board RAM if the card is NB
TP, and got somewhat better results (never had a chance to finish the
debugging; the system was Rutgers Physics Department's for whom I no
longer work).  I think the same or something similar is happening on
your system.

What kind of Ethernet card you got?  We might need the output of Slots
program for the declaration ROM details.

Ken