Subject: Re: Create permanent mapping from PCI bus to region of physical memory

From: David Hawkins <[email-masked]>
Date: Mon, 10 Apr 2006 20:52:06 -0700

> Not if you're tight on routing resouces and/or design time ;-)

Yep, understood.

> Dave, I appreciate your input but scatter/gather just isn't
> an option here for a variety of reasons.  Bandwidth/complexity/
> latency/time to design/time to debug/FPGA density all
> factor into this decision.  BTW we're talking 66/64 PCI-X;
> 33/32 PCI isn't nearly fast enough.

No problem.

> Nope, I disagree.  The MMU isn't involved here at all.  I'm
> talking about setting up the PIM (PCI Inbound Map) via
> a system call as opposed to just writing it directly.

The MMU is involved since its responsible for all memory,
i.e., Linux is told that all physical memory is under
its control. I haven't looked at the PCI inbound map,
but I would assume that it can be setup to say map all
of the incoming PCI addresses to all of the SDRAM
addresses, or some block thereof. So the FPGA can see physical
pages that Linux is busy swapping data into and out of.
What you are wanting to do is tell Linux to lay-off
managing a block of SDRAM, but make it visible to the
kernel as a memory area.

So, you need the kernel to setup page tables that map
to those physical addresses and you need to tell Linux
not to treat them like it does the rest of SDRAM.

So I believe the MMU/page tables are involved.

> I really can't go into detail about my application for a
> variety of reasons (both technical and legal) but suffice
> it to say that 16 MB is just the starting point.

No sweat, it never hurts to ask.

> I guess I'll dig a little deeper into the source to see
> where the kernel does the PIM mapping.  Re-architecting
> our app at this stage just isn't a practical consideration.

You always have the hack of reserving a block of memory
at the top of SDRAM, i.e., lie to Linux and tell it it
has less memory. It sounds like that might be the least
effort. Rubini has an example of how to do that.

Dave