Subject: re: L2CR_CONFIG in cpu.c
To: None <port-macppc@netbsd.org>
From: M L Riechers <mlr@rse.com>
List: port-macppc
Date: 01/05/2001 23:00:16
On Fri, 05 Jan 2001 00:09:56 +0900, Tsubai Masanari <tsubai@iri.co.jp>
advised:

>>As long as we're on the subject of kernels, does anyone know the use
>>of L2CR_CONFIG in cpu.c for macppc?
>
>Add, for example,
>
>options         L2CR_CONFIG="(L2SIZ_512K|L2CLK_20|L2RAM_PIPELINE_BURST)"
>(512K L2 cache, L2 clock is 1/2 of cpu clock, pipeline burst SRAM)
>
>to your kernel config.
>...

Very cool.

My Newer Tech G3 (500 mhz, 1 meg backside cache) backside cache runs
at 1/2 (250mhz), CPU speed, so the first part of the bit string built
as follows:

options         L2CR_CONFIG="(L2SIZ_1M|L2CLK_20)"

I took a flyer on the Newer Tech's access to the L2 SRAM;  I thought
PIPELINE_BURST sounded good, so I ended up with:

options         L2CR_CONFIG="(L2SIZ_1M|L2CLK_20|L2RAM_PIPELINE_BURST)"

Actually, this worked. dmesg:

"cpu0 at mainbus0: 1MB backside cache"

and proceded to boot.

Euphoria follows:  I then compiled a new netbsd kernel, and knocked my
socks off:

Compiling netbsd with a 604 @ ~132Mhz no L2 cache took about 35 minutes.
Compiling netbsd with a G3  @ ~500Mhz no L2 cache took about 15 minutes.
Compiling netbsd with a G3  @ ~500Mhz L2 1M cache took about  7 minutes.

5 times faster than with the 604!

I compared the outputs of compiles, and (except where you'd expect),
no differences!  Looks like it's sound.

Many thanks, Tsubai Masanari.  Thanks also for showing where to put
this value.

I hate to press, but is there any chance that some of this might make
it into the documentation?

On Wed, 3 Jan 2001 09:09:17 -0600, Donald Lee
<donlee_ppc@icompute.com> said:

> My spiffy new G3 MacPPC is also not enabling L2 cache, according to
> the syslog entries on boot.  Can
> anyone comment on how one would enable it?  (safely, of course. ;-> )
> 
> Should I do what M L Reichers suggests below?
> 
> -dgl-

Did you get this one going? Or is it the same unit as the PowerCenter
132 with the Sonnett upgrade card you refer to below?

On Fri, 5 Jan 2001 00:28:34 -0600, Donald Lee
<donlee_ppc@icompute.com> rejoined with:

> I was pleased to see the e-mail suggesting how to enable L2
> cache, but there is apparently more to it.
> 
> I'm running a PowerCenter 132 with a Sonnett 300Mhz/512Kb G3 card.
> 
> I tried several variations of options to make the backside cache work:
> 
> >options        L2CR_CONFIG="(L2SIZ_512K|L2CLK_20)"
> 
> Because I didn't know what kind of L2RAM I had, and because I misread the code
> at first, I tried this.  This did not enable the L2cache at all.
> This is because the code explicitly checks for L2CR_L2E (cache enable)?
> 

Yes, but not at this point.  According to the MPC750 RISC
Microprocessor User's Manual, Chapter 9, L2 Cache Interface Operation,
L2CR_L2E is actually "L2 Enable."  However, (in NetBSD-1.5.1_ALPHA, in
any case), cpu.c sets this as the last step in setting up the cache:

>       /*
>        * Configure L2 cache if not enabled.
>        */
>       if ((l2cr & L2CR_L2E) == 0 && l2cr_config != 0) {
>               ...
>               l2cr |= L2CR_L2E;
>               asm volatile ("mtspr 1017,%0" :: "r"(l2cr));
>       }

Ah, but.  With the setup below, you would have enabled the L2 cache
before cpu.c's clearing and initializing the cache.  This falls under
the general catagory of "results are undefined."  Your first
configuration should have worked, if your backside frequency is
150Mhz, and if your SRAM interface is "Flow-though (register-buffer)
synchronous burst SRAM."

>>options        L2CR_CONFIG="(L2CR_L2E|L2SIZ_512K|L2CLK_20)"
> 
> I tried this next, on the thought that it would enable the cache.  It did.
>
>>CPU: 750 (Revision 202)
>>total memory = 98304 KB
>>avail memory = 85280 KB
>>using 1254 buffers containing 5016 KB of memory
>>mainbus0 (root)
>>cpu0 at mainbus0: 512KB backside cache
>>trap type 300 at ff808ee0
>>Stopped in swapper at<C3>3<C3><C3><C3>3<C3>3<C3>3<C3><C3><9F>3<E5>3e 700 at 24b0
>>After
> (And that was it...)
> 
> Didn't boot, though.
> 

Yes.  It ended up in the Data Storage Exception Handler. It results
from either a data TLB error or data TLB miss.

There are two problems here.  If the backside cache is being operated
at a frequency other than its design frequency, or the RAM interface
is wrong.

#define  L2CLK_15               0x04000000 /*            / 1.5 */
							 ^
							 divide symbol

means the L2 cache is being run at 1/1.5 (1 divided by 1.5 = 2/3 =
.666 = two thirds) of the cpu core frequency.

#define  L2CLK_20               0x08000000 /*            / 2   */
							 ^
							 divide symbol

means the L2 cache is being run at 1/2 (1 divided by 2 = 1/2 = .5000 =
one half) of the cpu core frequency.

So, using your L2CLK_20 build symbol, your Sonnett 300Mhz/512Kb G3
card had best be expecting to run its backside cache at 150Mhz.

> 
>>options        L2CR_CONFIG="(L2CR_L2E|L2SIZ_512K|L2CLK_20|L2RAM_PIPELINE_BURST)"
> 
> Last, I tried this.
> That didn't boot either.  Same as the above.
> 

The other problem is how to interface to the SRAMs.  This will vary
according to the type of SRAM Sonnet used, or the interface to them.
I don't know much about this right now.  In any case, it's best just
to let Sonnet tell you what to use here, or read out, or find a value
that some Sonnet software sets up.

> There is obviously more to this than meets the eye.  Looks like
> no L2 cache is a pretty good option just now. ;->
> 
> -dgl-

Yup.

I believe I lucked out in selecting L2RAM_PIPELINE_BURST for my Newer
Tech card.  I suspect that if I chose any other value, I would get
what you got.  I poured through any and all documentation I could find
from Newer Tech, but found nothing pertinent.  I would try to reverse
engineer the chips on the card, but a nice, big, harry heat sink's in
the way.  Now that Newer Tech has met its great reward, hope is gone
to actually ask them.

Anyone know where to find this info, particularly for the Newer Tech
and Sonnet cards?  Discussion groups, tech info repositories?

Again, much obliged,

-Mike