My question is as follows:

Suppose I have a number of driver-allocated buffers. How can I map them
into a single contiguous user-space buffer. I guess I should build an
MDL that describes the physical pages mapped for these buffers and call
MmMapLockedPagesSpecifyCache(), but did someone do this in practice?
There seems to be no function to add several different pages to an MDL.

Thanks a lot in advance,
Greg.

Very bad idea. You expose kernel pool memory to user mode, which can cause
security leaks.
Why do such stange a thing?

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

<grishka@gmail.com> wrote in message
news:1154629702.353304.130590@i42g2000cwa.googlegr oups.com...

Even if you map them to a MDL you have not exported them to user space.
Your model is flawed, have the user space application pass the buffer to the
driver. Having the driver do the alloc always produces a copy.

--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply

<grishka@gmail.com> wrote in message
news:1154629702.353304.130590@i42g2000cwa.googlegr oups.com...

Don,

I know that this is an unusual practice, but, due to some severe
performance restrictions, I do have to expose kernel-space buffers to
the user mode. In short, these buffers are allocated by some other
device driver and I can't copy the data into another buffer because of
the associated performance penalty. Would it be a single buffer, it's
clear how to map it to the user space, but I'm talking about multiple
ones here.


Don Burn wrote:

Maxim,

I know that this is an unusual practice, but, due to some severe
performance restrictions, I do have to expose kernel-space buffers to
the user mode. In short, these buffers are allocated by some other
device driver and I can't copy the data into another buffer because of
the associated performance penalty. Would it be a single buffer, it's
clear how to map it to the user space, but I'm talking about multiple
ones here.



Maxim S. Shatskih wrote:

have you measured perf and seen that you have a perf problem or is this just
a guess based on potential bandwidth and w/out measurement?

--
Please do not send e-mail directly to this alias. this alias is for
newsgroup purposes only.
This posting is provided "AS IS" with no warranties, and confers no rights.


<grishka@gmail.com> wrote in message
news:1154637854.081643.210730@h48g2000cwc.googlegr oups.com...

Correct approach:

- allocate in user mode, pass the pending DeviceIoControl with METHOD_IN_DIRECT
to the driver. In the driver, map Irp->MdlAddress to kernel space and enjoy
your shared memory :-) do not forget to provide the cancel routine for the IRP.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Yes, this came out as the only solution after quite a bit of research.
Copying this amount of data eats up an unacceptable amount of CPU.

Doron Holan [MS] wrote:

OK, this is good, but my question was about mapping more than one
kernel-space buffer into ONE CONTIGUOUS user-mode virtual range. I know
that the MDL header is followed by an array of page indexes, but
there's no API to add separate pages to that array. Would it be OK if I
initialize an MDL, build this list manually and map it through
MmMap...()?

Maxim S. Shatskih wrote:

Then at least allocate in user mode and map in kernel mode, not vice versa.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

<grishka@gmail.com> wrote in message
news:1154688601.603603.57700@m73g2000cwd.googlegro ups.com...

Impossible. If you map several buffers - then MM provides no guarantees at all
about how they will be mapped to user.

Use one huge user-allocated buffer, chop it to parts and submit an overlapped
IOCTL for each part. This is the solution.

Surely not so.

Once more: mapping kernel pool memory to user space is a very bad idea.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

How many MB/s? What is your exact task?

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

1 GB/s. The data is delivered by a PCI device DMA into kernel buffers
allocated by another (opaque) driver. Copying the data into
IOCTL-supplied buffer utilizes CPU above the acceptable limit (about 5%
CPU usage on the most powerful machine I have, 20-30% on a typical
one). Once more, I can not supply my buffers for delivery. I can just
have pointers to those owned by the other driver.

Maxim S. Shatskih wrote:

Sorry but the opaque driver is bad news. If you are going to get
performance you need to fix that driver, any other schema by you is going to
result in a copy sooner or later.


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply



<grishka@gmail.com> wrote in message
news:1154693345.963482.31520@p79g2000cwp.googlegro ups.com...

Why such a strange architecture? Why "opaque drivers"? why not send the pending
IOCTLs from the app and run DMA over their MDLs?

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

If that was a good news, I wouldn't ask my question here
I do have quite a bit of experience in kernel mode and I'm aware of the
"right" and "wrong" ways of doing things. However, that's the task I
have and changing the other driver is definitely out of question.

Don Burn wrote:

OK, I think you're pointing me into a right direction.
If I use a DIRECT_IO IOCTL, the system allocates and locks down the
buffer for me.
Do you mean I can actually CHANGE the MDL in the IRP or even REPLACE it
completely? If so, splitting the user-mode-allocated buffer and mapping
each separate part into one kernel-allocated buffer would solve my
problem.

Maxim S. Shatskih wrote:

Then so is reasonable performance, anything you try will be an extreme hack
that will likely not be stable under stress. Since you have already said
the system is going to be stressed with large inputs since you care about
performance, you are in a lose or lose situation.

Note, trying to hack around with the MDL is a dead end, there is more to the
memory manager than that.


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply


<grishka@gmail.com> wrote in message
news:1154694149.595192.268790@m73g2000cwd.googlegr oups.com...

I agree with you about "extreme hacking", but remember that brilliant
things, such as RegMon by SysInternals wouldn't be possible had its
authors stick to "standard" methods only and avoid patching system
call pointer table.

Don Burn wrote:

No call table hacking was minor hacking compared to what you are looking at.
It might be easier to look at writing your own kernel.


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply


<grishka@gmail.com> wrote in message
news:1154694933.880099.60480@i42g2000cwa.googlegro ups.com...