Re: L2 and L3 caches on 7455 CPU card

[Monroe Williams <monroe%pobox.com@localhost>]

on 2/25/03 2:16 AM, Monroe Williams at monroe%pobox.com@localhost wrote:

[cross-posted to port-powerpc since the changes are in arch/powerpc]

Okay, I now have code that can enable the L3 cache on a 745x-class CPU.
This works similarly to the existing L2CR_CONFIG option that can be added to
a kernel config.  Lacking much imagination, I called the new option
L3CR_CONFIG.

The changes consist of some new code in cpu_config_l2cr() in
arch/powerpc/oea/cpu_subr.c, a bunch of L3CR definitions to
arch/powerpc/include/spr.h, and the addition of L3CR_CONFIG alongside
L2CR_CONFIG in arch/powerpc/conf/files.powerpc.  The patches are available
at:

http://users.easystreet.com/monroe/l3setup.patch

The system I've seen this work on is a Power Macintosh 7500 with one of
these:

http://sonnettech.com/product/crescendo_pci.html

Specifically the 700MHz G4 model.

The options I added to my kernel config are:

# Sonnet Crescendo/PCI G4/700 cache setup
options L2CR_CONFIG="(L2CR_L2E)"
options L3CR_CONFIG="(L3CLK_40|L3CKSP_4|L3PSP_0|L3RT_PB2_SRAM)"

which corresponds to a 1M L3 cache running at a 4:1 ratio.  (For the record,
this was derived by booting the machine under Mac OS X, installing the
Sonnet software and capturing a value of 0x8f020300 from L3CR.)  According
to the marketing specs on the upgrade, the cache should run at 200MHz (or
3.5:1), but I haven't tried that yet.

-----
NetBSD 1.6N (INDIRECT) #47: Tue Feb 25 22:58:49 PST 2003
    
monroe%xxxxxx.xxxxxxxx.xxx@localhost:/usr/local/netbsd/src/sys/arch/macppc/compile/IND
IRECT
total memory = 1024 MB
avail memory = 934 MB
using 3072 buffers containing 52528 KB of memory
mainbus0 (root)
cpu0 at mainbus0: 7450 (Revision 2.1), ID 0 (primary)
cpu0: HID0 8450c284<EMCP,TBEN,NAP,DPM,ICE,DCE,SPD,SGE,BHT>
cpu0: 700.00 MHz
cpu0: 256KB L2 cache, 1MB L3 backside cache at 4:1 ratio
bandit0 at mainbus0
...
-----

Could someone please review this patch?  If it passes inspection, what do I
need to do to get it committed?

Thanks,
-- monroe

> on 2/25/03 1:00 AM, John Klos at john%ziaspace.com@localhost wrote:
> 
>> Hi,
>> 
>> I just got my new Sonnet 700 MHz G4 upgrade for my 9600. It's nice - it
>> looks well built, runs very cool, and doesn't require special memory like
>> the NewerTech accelerators did.
>> 
>> The chip is a 7455 with 256k of CPU speed L2 cache; the board has one meg
>> of L3 cache running at 200 MHz (cpu/3.5).
>> 
>> So, next, I'd like to configure a kernel for this. On boot, it says:
>> 
>> cpu0 at mainbus0: 7450 (Revision 2.1), ID 0 (primary)
>> cpu0: HID0 8450c0a4<EMCP,TBEN,NAP,DPM,ICE,DCE,SGE,BTIC,BHT>
>> cpu0: 700.00 MHz
>> cpu0: L2 cache not enabled
>> 
>> Then I added the following lines to my config file:
>> options         L3CR_CONFIG="(L3CR_L3E)"
>> options         L2CR_CONFIG="(L2CR_L2E)"
>> 
>> which gave me:
>> 
>> cpu0 at mainbus0: 7450 (Revision 2.1), ID 0 (primary)
>> cpu0: HID0 8450c0a4<EMCP,TBEN,NAP,DPM,ICE,DCE,SGE,BTIC,BHT>
>> cpu0: 700.00 MHz
>> cpu0: 256KB L2 cache
>> 
>> Now, even though this board is significantly faster than the one it is
>> replacing, I'd really love to use the L3 cache, too. However, reading
>> spr.h implies that adding the option L3CR_L3SIZ should only be done for 2
>> meg L3 cache sizes, so I left that out (L3 size (0=1MB, 1=2MB)).
>> 
>> So does anyone know how I might go about getting both the L2 and L3 cache
>> to work at the same time? Also, where is the speed divisor set? I have a
>> feeling I should be bugging Matt...
> 
> I don't have a short answer, but I also recently bought one of these cards
> for my 7500, and I'd like to get it working.  I've already built a kernel
> that enables the L2 cache, and looked into the L3 issue a bit.
> 
> I think we'll have to start by finding out what value the card needs in
> L3CR, probably by booting into Mac OS and setting up the Sonnet software.
> (There are some settings in there that appear to depend strongly on the
> hardware design of the CPU card -- something about "clock sample points",
> whatever that means -- so I don't think we can just make something up.)
> I've tried this a couple of times with Mac OS 9, but the 9 utility I have
> that dumps the L2CR (which I modified to try and dump L3CR) just crashes.  I
> think the best bet will be to use XPostFacto to set up a Mac OS X
> installation and install the Sonnet software for X.  (I should be able to do
> this given a free evening or two.)  There are decent tools available that
> can read L3CR on Mac OS X.
> 
> Once we figure out what the proper value is, we will need to add code to set
> up L3CR.  According to the Motorola book on the 7450 family, this is
> non-trivial, but the procedure they give doesn't look _too_ much more
> complex than the existing code to set up L2CR (in
> arch/powerpc/oea/cpu_subr.c).
> 
> It doesn't appear that anyone has added many of the field definitions in
> L3CR to src/sys/arch/powerpc/include/spr.h.  The only ones there are the
> bits for L3CR_L3E and L3CR_L3SIZ, whereas the book specifies most of the
> bits in the register.  A little bit of time spent doing data entry should
> solve this problem.
> 
> For anyone following along at home, the book I'm referencing (MPC7450UM) is
> available as a PDF from the Motorola site here:
> 
> http://e-www.motorola.com/brdata/PDFDB/docs/MPC7450UM.pdf
> 
> The procedure for setting up the L3 cache is in section 3.7.3.1, and the
> bitfields in L3CR are described in section 2.1.5.5.2.

------------------------------------------------------------------------
Monroe Williams                                         
monroe%pobox.com@localhost