Have this database in the kmode driver and write IOCTL accessors to it.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Well, you have the answer to your question: The interlocked operation,
implemented as a single instruction, will not execute until your exception
filter has returned STATUS_CONTINUE_EXECUTION and an arbitrary delay for
thread pre-emption, context switch etc.



As long as you have some mechanism to control access to your pages (call
sequence or locks), this will work just as well as is does on any other
datum. It is not inherently thread safe, but could be made so by VERY
careful implementation. It does not provide a lot of protection, and has
significant overhead, but is not all wrong.



As Maxim points out, the only way to achieve what you really want is to use
the CPU to enforce the code privilege level and write the code that modifies
the pages in KM. An IOCTL interface can also be more efficient then
continually changing the page protection in an exception filter.


BTW: I've made the unwarranted assumption that having multiple threads is
directly analogous to having multiple CPUs. As David Craig reminds me, this
is only true in UM on Windows (because of arbitrary pre-emptive thread
scheduling). It's been too long since I optimized for UP in KM!



"Hugo gleaves@hotmail.com>" <hugh<underbar> wrote in message
news:1C0DF457-3490-4C13-98C4-AEF269B76378@microsoft.com...

As long as it's not shipped it's OK. Then why anybody would want to pay for
that? I personally would rather be sure I won't buy some gadget that uses
such stuff...

"Pavel A." <pavel_a@NOwritemeNO.com> wrote in message
news:unxVxQKtIHA.1772@TK2MSFTNGP03.phx.gbl...

"Alexander Grigoriev" wrote:


Thanks for the disparaging and unhelpful remarks. Everything I have
discussed and used in the design of this software adheres to Microsoft
published documentation, every feature and mechanism is fully supported by
the operating system.

The limitations on the capabilities of the features are fully documented for
users.

Furthermore, the algorithms appear at this stage to be operating reliably,
even in SMP machines and fully delivering the required functionality.

The challenge to protect large amounts of memory mapped data from errant
user code, be it managed or unmanaged is non trivial.

Regards
Hugh

"Alexander Grigoriev" wrote:

never said this.

All data here is shared amongst multiple processes, the system allows
hundreds of gigabytes of memory-resident data to be accessed/manipulated by
multiple applications, web-services or websites.

It is not an exercise in futility, unless you can explain what weaknesses or
failings may befall such a system I see no grounds for your heated comments.

Hugh

"m" wrote:


Well I'd love to exploit kernel mode as the means of enforcing protection of
data pages, but this is not something we will be doing in the sort term.

Having said that, is it possible/feasible/good practice to use a kernel mode
"driver" purely for this purpose?

Bear in mind we want applications to be able to simply "load" the data into
their address space as is normal for memory mapping.

I have no idea how very large amounts of VM can be accomodated using this KM
idea, imagine that the system (on 64-bit Windows) could load (say) 100 GB.

It is worth stating that by design, there are system-only pages and
user-pages, that is all system structures appear in one set of mapped pages,
while only user-data appears in others (imagine that the first 100 pages are
system strucs, indexes etc and the next 500 pages are pure user data).

Maybe I will get a good book or two, any recommandations?

Thx
Hugh

Absolutely. Kernel mode driver with IOCTL accessors to it, or service process
with RPC/DCOM accessors to it.

Both are the preferred ways in Windows to maintain global per-machine or per
process-group data, on which lots of the OS's architectures are built.

You can map a copy of this data for reading only, user32 does this for the
global window table from win32k.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Why this much amount of resident data? Maybe SQL database is better for this
amount then memory-resident?

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Well, considering that your software will no doubt be deployed on a single
purpose server and not a desktop PC, using a KM component, nominally a
driver, should not distress you. MS does this for some of their server
products also (ISA, IAS, & Exchange all have, or have had, KM components).



As long as clients access the shared data through an API and not simply via
pointers & offsets, as is suggested by your original question, you should
not have a problem changing the API to use IOCTLs to transfer control to KM
and then do what you do now.


"Hugo gleaves@hotmail.com>" <hugh<underbar> wrote in message
news:417E376D-D4DF-4B38-9B1B-B26AE32DD0AA@microsoft.com...

This is wrong approach. If you want to protect shared data, share it by
other means, not by direct access. Isolate it in a separate process and use
interprocess communication.

"Hugo gleaves@hotmail.com>" <hugh<underbar> wrote in message
news:0D0DC88B-D816-42BE-A0A6-326355184DC0@microsoft.com...

"Hugo gleaves@hotmail.com>" <hugh<underbar> wrote in message
news:FF515DC5-E531-426A-9A10-74DB58F3ED1F@microsoft.com...
Also forgot to mention, in a massively multithreaded process running on a
multiprocessor, any change in virtual memory map can be VERY expensive,
because it involves interrupting other processors for TLB invalidation.

"Alexander Grigoriev" wrote:

"Hugo gleaves@hotmail.com>" <hugh<underbar> wrote in message
news:9DED99E6-05A3-4A01-BEE3-CD805775696E@microsoft.com...
It works well as IPC if you only need to do one way communication. For
example, the client process writes data to shared memory, the server reads
the data, creates a response and writes it to different window of shared
memory. And these shared memory windows are per client process.