The NT Insider

Transcription

The NT Insider

®
Page TM
1
The NT Insider
The only publication dedicated entirely to Windows® system software development
A publication by OSR Open Systems Resources, Inc. Not endorsed by or associated with Microsoft Corporation.
January — February 2009
Volume 16 Issue 1
Common Topics
Happy 2009?
Deferred Procedure Call Details
D
eferred Procedure Calls (DPCs) are
a commonly used feature of
Windows. Their uses are wide and
varied, but they are most commonly
used for what we typically refer to as
"ISR completion" and are the
underlying technology of timers in
Windows.
If they're so commonly used, then why
are we bothering to write an entire
article on them? Well, what we've
found is that most people don't really
understand
the
underlying
implementation details of how DPCs
work. And, as it turns out, a solid
understanding is important in choosing
the options available to you when
creating DPCs and is also a life saver in
some debug scenarios.
Introduction
This article is not meant to be a
comprehensive review of why or how
DPCs are used. It is assumed that the
reader already knows what a DPC is or,
even better, has used them in a driver. If
you do not fall into this category,
information at that level is readily
available on MSDN.
In addition, Threaded DPCs, which are a
special type of DPC available on
Windows Vista and later, will not be
covered in any detail.
As a basis of our discussion, let's briefly
review some basic DPC concepts.
A working definition of DPCs is that
they are a method by which a driver can
request a callback to an arbitrary thread
context at IRQL DISPATCH_LEVEL.
The DPC object itself is nothing more
than a data structure with a
LIST_ENTRY, a callback pointer, some
W
ell, that remains to seen for many
folks, regardless of the industry.
Profits are down, job losses are up,
belts are being tightened...all is not
right in the world.
For those of you who are not aware,
OSR provides job forums for both
those companies looking to hire
permanent,
experienced
driver
developers, and of course, those
looking to be hired by said companies.
Check out Driver Jobs in the left-hand
navigation menu at OSR Online
(www.osronline.com).
If you’re still gainfully employed,
great. If you’re not, rest assured The
NT Insider will still keep coming your
way.
(Continued on page 12)
Why Take An OSR Seminar?
Whether you want to learn kernel debugging techniques, or you crave in-depth kernel device driver or file systems knowhow, OSR has a specific solution tailored precisely to the type of training you need.
Who are your instructors? Make no mistake about it, when you take an OSR seminar, you’ll be learning from the best. First
of all, OSR isn’t a “training company.” We are consultants. Senior developers. Development leads on actual, cutting-edge
projects developing system software. As your instructors, they bring you real-world answers to questions that textbooks –
or other seminars – simply can ’t answer. We train more engineers in the design, development and debugging of Windows
kernel-mode software than any other company. Period.
Check out the latest seminar schedule on the back of this issue, or visit us on the web at www.osr.com. Interested in a
customized, on-site presentation for your team? Drop us an email ([email protected]) and we’ll work with you to custom
tailor one of our presentations for your needs. We can even schedule an extra day of consultancy to answer specific
questions relative to your current project!
Page 2
The NT Insider™
Published by
Inside This Issue:
OSR Open Systems Resources, Inc.
105 Route 101A, Suite 19
Amherst, New Hampshire USA 03031
(603) 595-6500
Common Topics — Deferred Procedure Call Details
1
http://www.osr.com
New WDK for Win7: TRY IT NOW!
3
Peter Pontificates: Computer Programming Has Ruined My Brain
4
Filtering File Systems — Ten Things You Should Know
6
Safety First — Using the Windows String Safe Functions
8
How to Fetch Checked Components for Vista/Server 2008
18
Who Was at the DDC? Well, We Were, But…
19
Consulting Partners
W. Anthony Mason
Peter G. Viscarola
Executive Editor
Daniel D. Root
Contributing Editors
Mark J. Cariddi
Scott J. Noone
OSR Associate Staff
Consultant At Large
Hector J. Rodriguez
Send Stuff To Us:
email: [email protected]
fax: (603) 595-6503
Single Issue Price: $15.00
The NT Insider is Copyright ©2007. All rights reserved.
No part of this work may be reproduced or used in any
form or by any means without the written permission of
OSR Open Systems Resources, Inc. (OSR).
We welcome both comments and unsolicited manuscripts
from our readers. We reserve the right to edit anything
submitted, and publish it at our exclusive option.
Stuff Our Lawyers Make Us Say
All trademarks mentioned in this publication are the
property of their respective owners. “OSR”, “The NT
Insider”, “OSR Online” and the OSR corporate logo are
trademarks or registered trademarks of OSR Open Systems
Resources, Inc.
We really try very hard to be sure that the information we
publish in The NT Insider is accurate. Sometimes we may
screw up. We’ll appreciate it if you call this to our
attention, if you do it gently.
OSR expressly disclaims any warranty for the material
presented herein. This material is presented “as is” without
warranty of any kind, either expressed or implied,
including, without limitation, the implied warranties of
merchantability or fitness for a particular purpose. The
entire risk arising from the use of this material remains with
you. OSR’s entire liability and your exclusive remedy shall
not exceed the price paid for this material. In no event shall
OSR or its suppliers be liable for any damages whatsoever.
It is the official policy of OSR Open Systems Resources,
Inc. to safeguard and protect as its own, the confidential
and proprietary information of its clients, partners, and
others. OSR will not knowingly divulge trade secret or
proprietary information of any party without prior written
permission. All information contained in The NT Insider
has been learned or deduced from public sources...often
using a lot of sweat and sometimes even a good deal of
ingenuity.
Neither OSR nor The NT Insider is in any way endorsed by
Microsoft Corporation. And we like it that way, thank you
very much.
OSR Seminar Schedule
Mini-filter driver development—Boston, MA
WDF driver development—Portland, OR
Kernel Debugging — Portland, OR
WDM driver development—Boston, MA
File systems development—Boston, MA
See Back Cover for Dates
Page 3
New WDK for Win7: TRY IT NOW
I
f you haven't done so, do make some time to install, examine, and actually play with the Beta version of the Win7
WDK. You can download it right now from Microsoft Connect (http://connect.microsoft.com). If you're not already
on the WDK Beta list, you can use invitation ID OSR-GMVB-T9HX to join (enter this invitation ID in the " Wer e
you invited to join Connect?" box on the right of the home page).
We know you're almost certainly busy doing work that's not related to Windows 7. So are we. But, here's the thing:
Your chance to influence what's in the Win7 WDK is now, not months from now when you've started to work on Win7
stuff.
You might be surprised at some of the things you find. Some good, some that may make you wonder. For example:




There's a new Office Automated Code Review tool that will automatically run PreFast on your
driver for you.
The Windows 2000 Build Environment has ceased to be. It is no longer. It has gone to meet
its maker.
There have been some changes to the SOURCES file syntax
The legacy file system filter driver samples are gone
These are just a few things that we noticed. Are these things good? Well, here at OSR we're not particularly happy
about the disappearance of the Win2K Build Environment, for example. But the point isn't what we think (we've
already provided the WDK team with our thoughts)... The point is what do you think about these things.? And what
will you notice that you like or dislike, that you find useful or problematic, about the Beta version of the Win7
WDK? If you don't install it, you won't know.
So, go get the WDK, install it, and give it a run-through. And provide feedback to the W DK team about your
experience. You complain that Microsoft doesn't listen to you? Well, we've heard the WDK team ask for feedback
repeatedly -- both publically and privately -- on the Win7 WDK Beta.
The best way to provide feedback is via the Win7 WDK feedback form on Microsoft Connect. This form is
specifically (and only) for WDK bugs -- not Win7 O/S bugs. The WDK team has told us that this is the "express
pipeline" directly to them for Win7 WDK Beta comments and bug reports. Alternatively, you can send feedback by
email to [email protected].
Design & Code Reviews
Have a great product design, but looking for extra security to validate internal operations before bringing it to your board
of directors? Or perhaps you’re in the late stages of development of your driver and are looking to have an expert pour
over the code to ensure stability and robustness before release to your client base.
A small investment in time and money in either service can “save face” in front of those who will be contributing to your
bottom line. OSR has worked with both startups and multi-national behemoths. Consider what a team of internals, device
driver and file system experts can do for you. Contact OSR Sales — [email protected].
Page 4
Peter Pontificates:
Computer Programming Has Ruined My Brain
J
ust a confirmation ladies and gentlemen: this is Alaska
Airlines flight 15 non-stop to Seattle. Please move down
the aisle to your seats as quickly as possible, so we can be
underway without delay. Please remember that the primary
storage location for your luggage is under the seat in front of
you. As this is a very fully flight…
I hadn't really been paying attention to the background blather,
but my mind just sort of locked-in on that last statement. Did
she say "a very full flight"? Huh? I turn to my seat mate, a
skinny dude with glasses trying very hard to make sure
everyone knows he is a medical doctor, because of his
superior manner, the non-stop use of his cell phone to talk
about his patients in a loud voice, and his constant waving of a
copy of Journal of Clinical Oncology.
Me: Did she say " ver y full flight" ?
Dr Boy: (looking over his Dolce and Gabbana
glasses) Excuse me?
Me: Ver y full. Did the flight attendant just say it
was a "very full flight"? Because, well I don't really
understand how that can be, given that "full" is a
boolean. It means "filled to capacity." The flight can
be FULL or NOT FULL. It can't be VERY FULL.
Dr Boy: Uh, uhm... I' m not sur e (waves Oncology
Journal)... Stewardess! (orders a scotch, while
ostentatiously making another phone call)
I can't stand stuff like this. Stuff like people acting highly
superior on planes (jet travel is soooo glamorous these days,
don't you think?) of course, but that's not what I specifically
mean in the context of this article. I mean booleans being
used as floating points. If full is a boolean, it can only be
TRUE or FALSE (or true or false if you're with me on that C#
thing from last year, but I digress). It can't have any other
value. So how could the plane be very full?
As I considered this, and noticed the stares I was still getting
from Dr Boy out of the corner of his eye, I started to worry
about my health just a bit. After I thought about it for a while,
I came to a realization: Computer programming has ruined my
brain.
I suppose I knew this even before I had the above described
revelation on the tarmac. It was just the other day that I was
looking for scheduled episodes of a particular TV show with
my DVR that I was driven into a fit of pique. I did "Search
for shows" and then "By title." I entered "House" (the show
title). The search results display page had three columns that
read as follows:
House
House
House
House
21 Jan
21 Jan
23 Jan
23 Jan
8:00PM
9:00PM
2:00AM
9:00PM
USAHD
USAHD
HDNet
USAHD
House
26 Jan 8:00PM
FoxHD
As an aside, please note that this was not a TiVo. TiVo is
great. TiVo is the highest form of its art. There's nothing
wrong with anything that TiVo does. There is probably not
even anything wrong with anything made in Alviso
California, in fact. I was using a different, low-rent, knockoff, ghetto, poseur TiVo wanna-be DVR. Obviously.
What drove me nuts -- insane, in fact -- about the results
display was that it showed the title of the show on each line...
but failed to show the episode name. I already knew the name
of the damn show. I was searching by title, right? Showing
the title was useless. My wife was sitting with me:
Me: Ar r gh. Why ar e they showing me the T itle
field?
Karen: Huh? It's the title of the show.
Me: I know that. But why ar e they showing it to
me? On the search results page. Why didn't they show
me the Episode name? I mean, heck... we've seen
"House's Head" like three times... but we missed
"Wilson's Heart."
Karen: Cuz that's what they show. So you'll know
the title. So…
Me: I searched by title. I k now the title. I'll tell
you why they show it. It's because the dev was too
lazy to code a separate page for each search result
type, that's why. If she had coded a search results
Learn KMDF from Experts
Why wouldn’t you? If you’ve got a new device
you need to support on Windows, you should
be considering the advantages of writing a
KMDF driver.
Hands on experience with labs that utilize
OSR’s USB FX2 device make learning easy and you get to walk away with the hardware!
Contact our seminar
[email protected].
coordinator
at
Page 5
Computer Programming...
(Continued from page 4)
page for "search by title" that showed the episode
name, and a separate search results page for "search
by actor" or whatever that showed the show title, that
would have been more work. Seriously, this is just a
bunch of HTML and how much time would it have
taken? Probably less time than it will now take me to
select each of the results from the search results page
and see what the episode title is.
Karen: You just click on it..
Me: I know I just click on it. But why was it written
this way? It's freakin' idiotic. If one of the devs who
works with me did that, they'd be fired. It's either
lazy or it's stupid…
Karen: I' m going to get a cup of tea. Let me know
when you're done talking to the dog about how the
TiVo works and there's something for us to watch.
Me: Ar r r gh... It's not a T iV o, it's a cheap-ass offbrand TiVo clone…
Karen: (fr om the kitchen now, with the teapot) uh
huh, yeah, TiVo, yup. Come here Toby!
She was familiar with this type of tirade. It was similar to the
one I loosed when I didn't like the fact that her new car
navigation system required you to go 7 menus deep in order to
set what I figured should be a common parameter. That tirade
contained a long soliloquy starting with "Who writes software
like this?" and ending with"...would be fired" and "... either
lazy or stupid."
accept the bad design choices.
The really awful
implementation decisions. The just plain dumb code.
I wish that I could report that the spongiform that is computer
programming had only affected my brain in the use of
consumer electronics products. Sadly, this is not the case. It's
affected just about everything I do. Recently, my dog got sick.
On one of my visits to the vet, I made him sit down and put
the various treatment options into "if... then... else..."
statements. He thought I was, shall we say, rather unusual . I
thought he had a very disorganized way of thinking. How
does somebody work in an office that doesn't even have a
white board! It's ridiculous. I wanted to find another vet -not really trusting anyone who doesn't have an immediate
grasp of C syntax or Expo markers -- but Karen talked me out
of it.
Anyhow, at least now I realized my malady. And as I sat on
the plane, getting ready to take off, I rolled the whole thing
over in my mind. Maybe it wasn't so bad to be infected by
computer programming. At least it kept me from speaking
loudly on my cell phone and waving an oncology journal at
the flight attendant. Hmmmm... I wonder who wrote the
software that does the avionics displays?
Peter Pontificates is a regular opinion column by OSR consulting
partner, Peter Viscarola. Peter doesn’t care if you agree or
disagree, but you do have the opportunity to see your comments
in a future issue. Send your own comments, rants, or distortions
of fact to: [email protected]
If I didn't know how to program, I don't think these things
would bother me. I'd probably just accept "that's the way that
the DVR works" and probably be amazed that the damn thing
knew when every episode of House was scheduled for the
next two weeks. I'd be blissful in my ignorance. But,
computer programming having ruined my brain, I couldn't
OSR Software Development—Experience, Expertise and Guarantee
In times like these, you can’t afford to hire a fly-by-night Windows driver developer. The money you think you’ll
save in hiring cheap help by-the-hour, will disappear once you realize this trial and error method of development
has turned your time and materials project into a lengthy “mopping up” exercise...long after your contract
programmer is gone.
Consider the advantages of working with OSR. If we can be of value-add to your project, we’ll tell you—if we
can’t we’ll tell you that too. You deserve (and should demand) definitive expertise. You shouldn't pay for
inexperienced devs to attempt to develop your solution. What you need is fixed-price solutions with guaranteed
results. Contact the OSR Sales team at [email protected] to discuss your next project.
Page 6
Filtering File Systems
Ten Things You Should Know
R
egardless of whether you are still supporting legacy file
system filter drivers, or maintaining or building new minifilter drivers, there are some common techniques (ok, there are
only nine…) you can use to improve the robustness of your
filter. Many of these are pragmatic – they are not things you
will read in the documentation, they come from experience
working with filters for many, many years.
Never Trust Buffers
Filter drivers are forced to endure a remarkably hostile
environment, both from their neighbors (other filter drivers) as
well as those that are inherently untrusted (any user mode
code). A single mistake in a file system filter driver can create
a potential “time bomb” waiting to happen.
User Buffers are the most commonly identified “risk factor”
when it comes to buffer management. Despite that, we still
routinely see filters that fail to take the two necessary steps for
protecting against user buffers. First, you must ensure that a
user buffer really is within the user portion of the address
space. This is performed by probing the address, normally
using ProbeForRead or ProbeForWrite.
Probing a user buffer only ensures it is in the user’s address
space, however. If it is invalid at that point in time the probe
will work, but this call does nothing to ensure the address
remains valid. Thus, the second key part of buffer handling is
to ensure any access to it is protected with a structured
exception handler. Microsoft provides their own custom
handler and it is invoked using special extensions
implemented by the C compiler used in the WDK
environment. This is done using __try and __except.
Both of these steps are essential to ensuring correct user buffer
handling.
In addition, file system filter drivers must deal with the
pragmatic issue that they may be handed invalid kernel data
structures. The harsh reality is that it is not uncommon for
other kernel components to pass along invalid structures.
Mini-filters are somewhat protected against this, since invalid
structures passed into filter manager will cause filter manager
to crash, not the mini-filter. Despite this, if you are the only
mini-filter on the system it won’t crash without your filter and
it will crash with your filter (and we all know that this means
it must be “your fault”).
Thus, what we generally suggest– as much as possible – is to
carefully validate your own data structures. While not
guaranteed, we’ve used this technique to detect erroneous
situations numerous times. In our experience it is better to
use “deprecated” functions like MmIsNonPagedSystem
AddressValid (which is of dubious value) to add even a
modest amount of robustness to our code. This is pragmatic –
if the system crashes in our driver, we will be blamed for the
crash. If we can instead return an error and it crashes in
another driver, it will become the problem of that other driver
writer. Remember, when someone types “!analyze –v”, it will
blame the first non-Microsoft driver on the stack. Hence, it’s
better to not be on the stack, at least from a support overhead
burden.
Validate Parameters
Sure, we all know that it isn’t necessary to validate buffers
that are passed to us by other kernel mode components, but
only Microsoft has the luxury of this type of magic thinking.
For the rest of us, it really is vital to keep in mind that the
system is not static and we are constantly forced to “figure
out” why things are not working.
Keep in mind when validating any information:


Buffers are not static. You do not own the only
reference to that buffer and they can (and do)
change. If you want to rely upon the contents of
buffers, you must capture that information (if it
changes inside a private captured buffer, it’s
called “memory corruption” and not a buffer
handling error!)
With Windows Vista, MDLs that arrive in your
filter can include “dummy pages”. These pages
can create potential confusion for any filter that
tries to modify the data (e.g., an encryption filter)
because that buffer’s content is not static. This
merely reinforces our observation before – if you
want static content, capture it and operate on it
separately from the original.
If you decide to switch around the MDLs in an I/O request,
don’t forget to also update the UserBuffer field (the file
systems rely upon this being “correct” – which means it must
be the same virtual address that is in the MDL passed).
Beware the Failure Cases
Over the years, we have consistently observed that developers
are, as a group, remarkably optimistic. This shows in their
coding, which tends to assume that everything is going to
work properly. As it turns out, however, this optimism is
unwarranted – and the source of frequent bugs. Further, as
your driver is more widely deployed, you will observe a
greater number of these failure scenarios.
Indeed, in our experience, even mature code bases will
experience “we’ve never seen this failure case before”
scenarios. Naturally, the fact that it is almost impossible to
test all of these failure scenarios further complicates writing
code to handle those cases. Nevertheless, it is important for
file system filter driver writers (both legacy and mini-filter
writers) to think about error cases and handle them.
Page 7
Filtering...
In general, we’ve noticed a few classes of such failures:



Allocation failures. These are quite common, but
something often overlooked is that handling
allocation failures at the point you need the
allocated object is frequently “too late” to handle
the error gracefully. Generally, we suggest moving
the allocation to an earlier point in the code –
keeping in mind that freeing an object won’t fail.
So, in a mini-filter (for example) we allocate our
data structure for our context tracking information
in the pre-create path, even though we won’t use it
until the post-create path. If we don’t need it, we
free it. In the pre-create path, however, we can
gracefully return an error. In post-create, we are
forced to try and cancel the file open (which is far
more complicated and subject to errors).
Connection failures.
While we’ll mention
redirectors “later”, we have noticed that timeouts
and disconnections are common enough failures
that just aren’t handled. If you use a service, you
have to consider what your driver will do when it
fails, not to mention how it fails. Nothing is worse
for a user than the frustration of having an
operation (in your filter) hang forever waiting on a
service that has entered an infinite loop.
Unwinding failures. These are perhaps the most
insidious of the error handling cases. You’ve done
your job and checked for a specific error case. But
you have already done operations that need to be
undone. Thus, your error handling code must then
attempt to undo something you’ve already done. It
could be as simple as changing the read-only bit on
a file, or as complicated as changing its ACL.
Some later step fails and you attempt to set it back
– but that attempt to unwind itself fails.
Of course, our point is not to provide some complete list, but
rather to point out that when writing code, and particularly
when code reviewing it, try looking through it and ignoring
the “everything works right” cases and look at the “everything
comes unraveled” cases. After all, the “everything works
right” case is by far the most likely case to be tested and fixed
quickly.
Avoid Changing File Sizes
The most difficult filters to build and maintain are those that
change the size of the underlying file and then try to mask this
behavior from the applications. There are very good reasons
for masking these changes from applications – after all,
applications rely upon the size information in the directory
entry being correct. For example the “verify” option for
xcopy (“/v”) actually looks at the size in the directory entry,
not in the file itself.
Exacerbating this issue is that directories can be surprisingly
large (we’ve received bug reports against directories with over
700,000 entries in a single directory – so large that the owner
of that large directory could not enumerate it with Explorer
and instead only enumerated it from the command line!) If
you have to actually open the file in order to retrieve the file
size, it will make directory enumeration dramatically slower.
In our experience, unless you implement caching, the cost of
“correcting directory entry sizes” has an amazing impact – it
effectively makes doing almost anything with high access
directories impossible (e.g., “\Windows”).
Underlying this is the fact that application programs are
sloppy – they use information that is demonstrably not
updated (NTFS, for example, only updates the size in the link
used, so this size from the directory entry trick can be spoofed
by using a link to change the file size).
We have seen people try to use “naming tricks” to resolve this
issue. Fair warning: whenever you think you have a clever
solution in a filter driver, ask yourself “and how will this work
when someone else implements a filter driver that plays the
same/similar trick.” If the answer is “it does not work” then
you don’t have a viable solution. Name prefix/suffix tricks
fall into this category.
The best thing to do is to never change size. If you must
change the size of the file, the next best trick is to do this only
on file systems that support both sparseness and alternate data
streams – then you can keep a dummy default stream
(“::$DATA” on NTFS) that is the correct size, with the
modified data in an alternate data stream. Unfortunately, that
often is not a viable solution because it is restricted to only
some file systems. In that case, be prepared for building a
very complicated filter driver.
FSD/MM/CC Interactions Are Complicated
We often observe that interactions in this business between
the components are horribly complicated (See Figure 1). File
system filter drivers sit in the midst of this complex
“File and Folder” Encryption
Several commercially shipping products are a
testament to the success of OSR’s most recent
development toolkit, the Data Modification Kit.
With the
encryption
codebase,
implement
file system
hassle of developing transparent file
solutions on the rise, why not work with a
a support team and a reputation to
your encryption or other data-modifying
solution?
Check out the DMK at www.osr.com, and/or contact
OSR to discuss the DMK and your needs to determine
if you can take advantage of this unique solution.
Phone: +1 603.595.6500 Email: [email protected]
Page 8
Safety First
Using the Windows String Safe Functions
U
nless you have had your head in a hole for the last couple
of years, you have undoubtedly heard about Microsoft
fixing security holes in Windows kernel mode code. Many
of the problems they encountered had to do with string
handling functions, for example strcat, strcpy, and their
related c-runtime library functions (CRT) . The weakness in
these functions is that no buffer lengths are specified, so it is
very easy for a user to copy a long string into a short buffer
and thus corrupt memory or worse, cause some unhandled
exception. Buffer overruns have even been used as security
attacks. To fix this class of problem Microsoft created a safer
set of functions called the "String Safe" Functions. These
relatively recent functions (XP SP1), to no one's surprise, add
parameters to describe the length of the string buffers, to
reduce the chance that buffers are overflowed
When Microsoft created the string safe functions they created
two sets of functions. One set uses byte counts to describe
buffer lengths, while the other set of functions uses character
counts to describe the buffer lengths. Byte Count related
functions always contain "Cb" (for count in bytes) in the
name, for example RtlStringCbCat. The character count
related functions always contain "Cch" (for count in
char acter s) in the name, for example RtlStringCchCat.
The type of function to use is entirely up to you.
Now, it is nice to know that Windows provides these
functions, and some of the advantages are probably already
evident to you. However, let's review some of the major
benefits to using these newer functions: Since you are
providing the correct size of the destination buffer this ensures
that the function will not write past the end of the input buffer.
All string safe functions guarantee that the output buffers are
null terminated, even if the operation truncates the intended
result.



All the string safe functions return an
NTSTATUS, with the only possible success code
being STATUS_SUCCESS (there are, of course,
several possible error codes).
Each string handling function has both a Cch and
Cb variant available for use.
Most functions have an extended "Ex" version
which provides extended functionality.
String Safe Character Count Functions
Table 1 lists the character count string safe functions
provided in "strsafe.h" and the c-runtime functions (CRT)
functions they are intended to replace.
String Safe Byte Count Functions
Table 2 lists the byte count string safe functions provided in
"strsafe.h" and the c-runtime functions (CRT) functions they
are intended to replace.
Unicode Strings
At this point I probably have given you the impression that the
string safe functions are only intended to replace c-runtime
functions. That is not true. The string safe functions include
many functions, of both Cch and Cb variants, to help you with
Unicode strings also. The list below contains the functions
provided:


RtlUnicodeStringXxxCatN,
RtlUnicodeStringXxxCatNEx
RtlUnicodeStringXxxCatStringN,
RtlUnicodeStringXxxCatStringNEx
String Safe Functions
CRT Functions Replaced
RtlStringCchCat,
RtlStringCchCatEx
RtlStringCchCatN,
RtlStringCchCatNEx
RtlStringCchCopy,
RtlStringCchCopyEx
RtlStringCchCopyN,
RtlStringCchCopyNEx
RtlStringCchGets,
RtlStringCchGetsEx
RtlStringCchPrintf,
RtlStringCchPrintFEx
RtlStringCchVPrintf,
RtlStringCchVPrintfEx
RtlStringCchLength
strcat,wcscat,lstrcat,strcat,StrCatBuff,_tcscat,_ftcscat
strncat,StrNCat
strcpy,wcscpy,lstrcpy,strcpy,_tcscpy,_ftcscpy
strncpy
gets,_getws_getts
sprintf,swprintf,wsprintf,wnsprintf,_stprintf,_snprintf, _snwprintf, _sntprintf
vsprintf,vswprintf,wvsprintf,wvnsprintf,_vstprintf,_vsnprintf, _vsnwprintf, _vsntprintf
Strlen
Table 1
Page 9
String Safe...
Cch version of the string safe functions (Figure 2)and the
other using the Cb version (Figure 3).
As you notice with the Cch and Cb versions of the function,
the main difference is that the size of the destination buffer is
specified in each call. This provides the string safe functions
with the ability to ensure that the destination buffer is not
overrun. In addition, the functions return a status so that we
can be sure to terminate function processing if an error occurs.


RtlUnicodeStringXxxCopyN,
RtlUnicodeStringXxxCopyNEx
RtlUnicodeStringXxxCopyStringN,
RtlUnicodeStringXxxCopyStringNEx
Building Your Code
Example
Let us assume for the moment that you need to write a
function to generate the full name of a file from an input full
directory path and the handle to a file. Furthermore, we will
also assume that we will never see a long path name, because
everyone must be like us and hate to have long file names.
Thus the function that you may write to handle this could be
as shown in Figure 1.
If you analyze this code, I am sure that you can see a great
many faults that could occur in this code. Most of them center
around overrunning the dirname buffer since we forgot to look
at the string lengths of the buffers we were concatenating
from.
A better solution to this function could be either
Figure 2 or Figure 3 (P. 10), one illustrating the use of the
In order to build your code correctly, you may want to know
about the compile time variables that you can define to allow
you to optimize how you use the string safe functions.
STRSAFE_LIB
If you want to use the string safe functions in library form
then define STRSAFE_LIB before including "strsafe.h" in
your code. In addition, you will need to insure that you link
your code against "$(SDK_LIB_PATH)\strsafe.lib".
Why
do this? Well, instead of including the string safe routines
that you use in your binary, this allows your driver to
dynamically bind to the string safe functions when it loads,
wchar* GenerateFullPath(wchar* fulldirPath,HANDLE File)
{
static wchar
dirname[100]; // assume you think all paths are small
wchar* fileName = NULL;
ASSERT(fulldirPath);
ASSERT(File);
fileName = GetFileName(File);
strcpy(dirname,fulldirPath);
strcat(dirname,L"\\");
strcat(dirname,fileName);
//
//
//
//
Get the name of the input file
copy the input dir name
append a \ to the name
concatenate the filename
return dirname;
// return the full path to the caller.
}
Figure 1
String Safe Functions
CRT Functions Replaced
StringCbCat, StringCbCatEx
strcat,wcscat,lstrcat,strcat,StrCatBuff,_tcscat,_ftcscat
StringCbCatN, StringCbCatNEx
strncat,StrNCat
StringCbCopy, StringCbCopyEx
strcpy,wcscpy,lstrcpy,strcpy,_tcscpy,_ftcscpy
StringCbCopyN, StringCbCopyNEx
strncpy
StringCbGets, StringCbGetsEx
gets,_getws_getts
StringCbPrintf, StringCbPrintFEx
sprintf,swprintf,wsprintf,wnsprintf,_stprintf,_snprintf,_snwprintf, _sntprintf
StringCbVPrintf, StringCbVPrintfEx
vsprintf,vswprintf,wvsprintf,wvnsprintf,_vstprintf,_vsnprintf, _vsnwprintf, _vsntprintf
StringCbLength
strlen
Table 2
Page 10
revision of the string safe functions will be compiled as part of
your driver code (i.e., the implementations of the routines will
be compiled as part of "strsafe.h"), and you do not need to link
against strsafe.lib
String Safe...
STRSAFE_NO_DEPRECATE
hopefully allowing you to be running against the latest most
improved implementations of these functions. This can be
both good and bad.
Example:
#define STRSAFE_LIB
#include <strsafe.h>
Normally when you include "strsafe.h" all older functions (cruntime functions) that are replaced by "strsafe.h" defined
functions are deprecated, i.e. you are given warnings that that
the functions are not to be used and you are urged to use their
string safe equivalents. If for some unknown reason you have
to use one of these functions, you should define
STRSAFE_NO_DEPRECATE. This definition will suppress
the warnings and allow you to compile your driver.
If you don't define STRSAFE_LIB in your code, the current
{
static wchar
dirname[MAX_PATH];
wchar*
fileName = NULL;
NTSTATUS
status;
ASSERT(File);
ASSERT(MAX_PATH <= STRSAFE_MAX_CCH);
// Get the name of the input file
status = RtlStringCchCopyW(dirname,MAX_PATH,fulldirPath);
if(!NT_SUCCESS(status) {
return NULL;
}
status = RtlStringCchCatW(dirname,MAX_PATH,L"\\");
return NULL;
}
status = RtlStringCchCatW(dirname,MAX_PATH,fileName);
return NULL;
}
return dirname;
}
Figure 2 — Cch String Safe Usage
{
static wchar
dirname[MAX_PATH];
wchar*
fileName = NULL;
NTSTATUS
status;
ASSERT(File);
ASSERT(MAX_PATH <= STRSAFE_MAX_CCH);
// Get the name of the input file
status = RtlStringCbCopyW(dirname,MAX_PATH*sizeof(WCHAR),fulldirPath);
return NULL;
}
status = RtlStringCbCatW(dirname,MAX_PATH*sizeof(WCHAR),L"\\");
return NULL;
}
status = RtlStringCbCatW(dirname,MAX_PATH*sizeof(WCHAR),fileName);
return NULL;
}
return dirname;
}
Figure 3 — Cb String Safe Usage
Page 11
String Safe...
Example:
Summary
#define STRSAFE_NO_CCH_FUNCTIONS
Example:
#define STRSAFE_NO_DEPRECATE
STRSAFE_NO_CB_FUNCTIONS
If for some reason you want to limit your code to only use
character count related string safe functions, then you should
define STRSAFE_NO_CB_FUNCTIONS.
This definition
will then limit your code to the use of the RtlStringCchxxx
functions.
While it may seem like a hassle to learn a new set of functions
to replace the old c-runtime functions that we have grown to
love, it really is in your best interest to change. The string
safe functions provided in "strsafe.h" provide you with a
robust set of functions which will make your driver safer from
buffer overruns and silly string mistakes that we all tend to
make.
Example:
#define STRSAFE_NO_CB_FUNCTIONS
STRSAFE_NO_CCH_FUNCTIONS
Similarly, the STRSAFE_NO_CCH_FUNCTIONS define
allows you to limit your code to only use byte count related
string functions. Using this definition will then limit your
code to use of the RtlStringCbxxx functions.
Corporate, Onsite Training
Save Money, Travel Hassles, Gain Customized Expert Instruction
If you’re not aware of the value and benefit of bringing OSR to your location for seminars, we’re obviously not
doing our job (blame Dan!). With tightened training budgets and/or travel restrictions combined with
increased travel costs, OSR’s private, on-site training solutions can be your way to convince management that
they can “have their cake and eat it too”.
We can:



Prepare and present a one-off, private, on-site seminar for your team to address a specific area of
deficiency or to prepare them for an upcoming project
Design and deliver a series of offerings with a roadmap catered to a new group of recent hires or within
an existing group.
Work with your internal training organization/HR department to offer monthly or quarterly seminars to
your division or engineering departments company-wide.
With any option, OSR’s seminar staff, in conjunction with the instructor, will work with you in advance to help
craft a training solution that will maximize technical value and ensure you’re getting the most from your
training budget.
To take advantage of our expertise in Windows internals, and in instructional design, contact an OSR seminar
consultant at +1.603.595.6500 or by email at [email protected].
Page 12
DPCs...
context for the callback, and a bit of control data:
typedef struct _KDPC {
UCHAR Type;
UCHAR Importance;
USHORT Number;
LIST_ENTRY DpcListEntry;
PKDEFERRED_ROUTINE DeferredRoutine;
PVOID DeferredContext;
PVOID SystemArgument1;
PVOID SystemArgument2;
__volatile PVOID DpcData;
} KDPC, *PKDPC, *PRKDPC;
You initialize a DPC Object with KeInitializeDpc and queue
the DPC Object with KeInsertQueueDpc. Drivers that use
DPCs to perform more extensive work than is appropriate for
an Interrupt Service Routine typically use the DPC Object
that's embedded in the Device Object, and cause this DPC
Object to be queued by calling the function IoRequestDpc
(which internally calls KeInsertQueueDpc). Once queued, at
some point in the future your DPC routine is invoked from an
arbitrary thread context at IRQL DISPATCH_LEVEL.
With that basic info in hand, we can now cover the gory
details of both the queuing and delivery mechanisms that are
used for DPCs. That will lead us to discussing what options
we have for controlling the behavior of DPCs and what impact
those options have.
DPC Queuing
As mentioned previously, DPCs are queued (directly or
indirectly) via the KeInsertQueueDpc DDI:
NTKERNELAPI
BOOLEAN
KeInsertQueueDpc (
__inout PRKDPC Dpc,
__in_opt PVOID SystemArgument1,
__in_opt PVOID SystemArgument2
);
DPCs are actually queued to a particular processor, which is
accomplished by linking the DPC Object into the DPC List
that’s located in the Processor Control Block (PRCB) of the
target processor. Determining the processor to which the DPC
Object is queued is fairly easy for the O/S. By default, the
DPC is queued to the processor from which
KeInsertQueueDpc is called (the " cur r ent pr ocessor " ).
However, a driver writer can indicate that a given processor be
used for a particular DPC object, using the function
KeSetTargetProcessorDpc.
Viewing the DPC List on a particular processor is easy using
WinDBG. While the DPC List is actually contained within the
PRCB, the PRCB is an extension of the Processor Control
Region (PCR). By viewing the PCR with the !pcr command
we can see any DPCs currently on the queue for that
processor:
0: kd> !pcr 0
KPCR for Processor 0 at ffdff000:
Major 1 Minor 1
NtTib.ExceptionList:
NtTib.StackBase:
NtTib.StackLimit:
NtTib.SubSystemTib:
NtTib.Version:
NtTib.UserPointer:
NtTib.SelfTib:
8054f624
805504f0
8054d700
00000000
00000000
00000000
00000000
SelfPcr:
Prcb:
Irql:
IRR:
IDR:
InterruptMode:
IDT:
GDT:
TSS:
ffdff000
ffdff120
00000000
00000000
ffffffff
00000000
8003f400
8003f000
80042000
CurrentThread: 8055ae40
NextThread: 81bc0a90
IdleThread: 8055ae40
DpcQueue: 0x8055b4a0 0x805015ae
[Normal] nt!KiTimerExpiration
0x81b690a4 0xf9806990
[Normal] atapi!IdePortCompletionDpc
0x818a12cc 0xf96c5ee0
[Normal] NDIS!ndisMDpcX
One aspect of DPCs to note is that once a DPC Object has
been queued to a processor, subsequent attempts to queue the
same DPC Object are ignored until the DPC Object has been
dequeued (by Windows for execution of its callback). This is
what the BOOLEAN return value of KeInsertQueueDpc
indicates: TRUE means that Windows queued the DPC to the
target processor and FALSE means that the DPC Object is
already queued to some processor. This makes sense from a
programming perspective, as the DPC data structure only has
a single LIST_ENTRY field and thus can only appear on a
single queue at a time.
What About Priority?
Where the DPC is placed on the target processor’s DPC List is
an interesting question. Whether a DPC Object is inserted at
the beggining or end of the target processor’s DPC List is one
aspect of the priority feature of DPCs. You can set the
importance of a given DPC Object by using the function
KeSetImportanceDpc. This DDI lets you indicate that the
DPC object is low, medium, or high importance. Also, in
Vista and later you can set the importance to “medium high.”
Low, medium, and medium high importance DPCs are placed
at the end of the DPC queue, while high importance DPCs are
placed at the front of the queue. You may ask yourself at this
point, "then what's the difference between low, medium, and
medium high?" We'll answer that question shortly.
The DISPATCH_LEVEL Software Interrupt
Once the DPC has been queued to the target processor, a
DISPATCH_LEVEL software interrupt is typically generated
on the processor. The choice of whether or not to request the
DISPATCH_LEVEL software interrupt when the DPC Object
is queued is largely based on four factors: the importance of
the DPC, the target processor of the DPC, the depth of the
DPC List on the target processor, and "drain rate" of the DPC
List on the target processor.
Page 13
DPCs...
If the target processor for the DPC Object is the current
processor, the DISPATCH_LEVEL software interrupt is
requested if the DPC Object is of any importance other than
low. For low importance DPCs, the software interrupt is only
requested if the O/S believes that the processor is not
servicing DPCs fast enough, either because the DPC queue
has become large or is not draining at a sufficiently fast rate. If
either of those are true, the interrupt is requested even if the
DPC is low importance.
If the target processor for the DPC Object is not the current
processor, the decision process is different. Because
requesting an interrupt on the other processor will involve a
costly Inter Processor Interrupt (IPI), the situations under
which it is requested are restricted. Prior to Vista, the IPI
request would only be made if the DPC was high importance
or if the DPC queue on the target processor had become too
deep. Vista added the medium high importance DPCs to the
check and went one step further to cut down the number of
IPIs by requiring the target processor to be idle for the
DISPATCH_LEVEL software interrupt to be requested (See
Table 1 for a high-level breakdown).
DPC Delivery
Once the DPC has been queued to the processor, at some point
it must be dequeued and the callback executed. Remember
that there were two scenarios that occurred after the DPC was
queued to the processor, either the DISPATCH_LEVEL
software interrupt was requested or it was not.
Delivery from the Software Interrupt Service
Routine
To keep things relatively simple, we’ll restrict our discussion
here to the case of queuing the DPC Object to the current
processor. Let's start with the case in which the IRQL
DISPATCH_LEVEL software interrupt was requested. At the
time KeInsertQueueDpc was called, there are two situations
the system could be in: The first would be running at an IRQL
<
DISPATCH_LEVEL,
in
which
case
the
DISPATCH_LEVEL interrupt
would be delivered
immediately. The second case would be if the current
processor is at IRQL >= DISPATCH_LEVEL, in which case
the interrupt would remain pending until the IRQL was about
to return to an IRQL < DISPATCH_LEVEL.
In either case, once the service routine for the
DISPATCH_LEVEL interrupt begins executing, it checks to
see if any DPCs are queued to the current processor. If the
DPC queue is not empty, Windows will loop and entirely
drain the DPC List before returning from the service routine.
Before draining the DPC List, Windows wants to ensure that
it has a fresh execution stack for the DPC routines to run on.
This will presumably cut down the incidents of stack
overflows in the case where the current stack does not have
much space remaining. Thus, every PRCB also contains a
pointer to a previously allocated DPC stack that Windows
switches to before calling any DPCs:
0: kd> dt nt!_KPRCB DpcStack
+0x868 DpcStack : Ptr32 Void
We can see evidence of the switch in the debugger if we set a
breakpoint in a DPC routine. Here we chose a DPC from the
ATAPI driver:
0: kd> bp atapi!IdePortCompletionDpc
0: kd> g
Breakpoint 1 hit
atapi!IdePortCompletionDpc:
f9806990 8bff
mov
edi,edi
0: kd> k
ChildEBP RetAddr
f9dc7fcc 80544e5f atapi!IdePortCompletionDpc
f9dc7ff4 805449cb nt!KiRetireDpcList+0x61
f9dc7ff8 f9a2b9e0 nt!KiDispatchInterrupt+0x2b
WARNING: Frame IP not in any known module. Following
frames may be wrong.
805449cb 00000000 0xf9a2b9e0
Notice the strange call stack - it seems to disappear after the
call to KiDispatchInterrupt. The problem is that WinDBG has
ceased to be able to unwind the stack due to the stack switch,
and the call stack that we see here is the call stack for the DPC
stack. If we try to match the EBP addresses shown with the
stack limits of the current stack we will see the discrepancy:
DPC Importance
Target Processor
DPC Inserted At…
DISPATCH_LEVEL Software Interrupt Generated?
Low
Current
END
NO
Low
Other than Current
END
Medium
Current
END
Medium
Other than Current
END
MediumHigh
Current
END
MediumHigh
Other than current
END
High
Current
HEAD
High
Other than current
HEAD
IPI Generated?
NO
YES
NO
YES
YES
(Vista: if target idle)
Table 1
YES
YES
(Vista: if target idle)
Page 14
DPCs...
stack. Because the idle loop uses so little thread stack itself,
there is not much use in going through the effort of swapping
stacks in this case.
Conclusion
0: kd> !thread
THREAD 81964770 Cid 028c.02b8 Teb: 7ffd8000
Win32Thread: e1873008 RUNNING on processor 0
IRP List:
8195b870: (0006,0190) Flags: 00000970 Mdl:
00000000
819128b0: (0006,0190) Flags: 00000970 Mdl:
00000000
Not impersonating
DeviceMap
e1001980
Owning Process
818d5978
Image:
csrss.exe
Attached Process
N/A
Image:
N/A
Wait Start TickCount
6779
Ticks: 0
Context Switch Count
4104
LargeStack
UserTime
00:00:00.000
KernelTime
00:00:00.265
Start Address 0x75b67cd7
Stack Init f9a2c000 Current f9a2ba58 Base f9a2c000
Limit f9a29000 Call 0
Hopefully this cleared up a few misconceptions about DPCs
and how they are handled by the system.
Note that the EBP addresses do not fall within the base and
limit of the current thread's stack. Using the techniques
outlined in last issue's Debugging Techniques: Take
One...Give One article (September-October 2008), you can
actually get the original call stack of the interrupted thread
back. This can be useful as it sometimes can provide key
information as to what kicked off a chain of events on that
processor.
Once the O/S has switched stacks, it begins dequeuing DPC
objects and then executing the call backs. Thus, while your
DPC routine is executing, it is no longer on any DPC queue
and may be queued again.
Delivery from the Idle Thread
But what about those low importance DPCs or targeted DPCs
that didn't request the DISPATCH_LEVEL software
interrupt? Who processes those? Well, there are actually two
ways in which they'll be processed. Either another DPC will
come along that will request the DISPATCH_LEVEL
interrupt and the DPC will be picked up on the subsequent
drain, or the idle loop will come along and notice that the
DPC queue is not empty.
Part of the idle loop's work is to check the DPC queue and
determine if it is empty or not. If it finds that the queue is not
empty, it begins draining the queue by dequeuing entries and
calling the callbacks. We can see this in a different call stack
but using the same DPC routine as the previous example:
Breakpoint 1 hit
atapi!IdePortCompletionDpc:
f9806990 8bff
mov
edi,edi
0: kd> k
ChildEBP RetAddr
80550428 80544e5f atapi!IdePortCompletionDpc
80550450 80544d44 nt!KiRetireDpcList+0x61
80550454 00000000 nt!KiIdleLoop+0x28
The difference here is that in this case the stack is not
switched, thus the DPCs actually executed on the idle thread's
Kernel Debugging for Windows
You’ve seen our articles where we delve into analyses of
various crash dumps or system hangs to determine root
cause. Want to learn the tools and techniques yourself?
Consider attendance at OSR’s Kernel Debugging for
Windows seminar. This hands-on training not only
provides fundamentals you need to debug Windows kernelmode software, but spends considerable time showing you
the valued techniques that are used and espoused
internally at OSR.
The next offering of this limited-attendance seminar is to
be held in Portland, Oregon from 6-10 April! Act soon, to
save your seat!
For more information, visit us at www.osr.com or call us at
+1.603.595.6500, or email us at [email protected].
Page 15
is that filters are often written to target NTFS first and only
once working against NTFS are they moved to other file
system.
Filtering...
environment and must be written to handle the ramifications
of those interactions.
The simplest way to handle these interactions is to simply
ignore them – in other words, don’t get involved in the data
flow through the system. That is definitely the simplest
solution. Many filters must become involved in this in at least
some fashion, however. Filters that are involved in I/O
operations need to understand these interactions.
Still, it is beyond the scope of this article to describe these
complexities. The point, however, is that in order to build a
robust file system filter driver, you really do need to
understand the file system.
Redirectors Do Not Behave Like Local File Systems
Some filters can restrict themselves simply to a single file
system (e.g,. NTFS) or perhaps to a class of file systems
(physical file systems) but many filters must handle the
network as well. Over the years, the pattern we have observed
Sadly, this approach fails horribly when it comes to the
network file systems. Filters that “work fine” on NTFS
frequently exhibit serious problems when moved to the
Windows redirector. Indeed, over the years, we’ve observed
that it is often better to start with the network case and then
move to NTFS – its generally a far less painful move than vice
versa, particularly because project management often assumes
that “once it works on NTFS we’re almost done.”
Some of the issues we have seen when it comes to filtering
network redirectors over the years include:


Naming. Nothing in the local file system arena
prepares you for the hideous complexity of naming in
the network. Effectively, file names may be aliased
due to hard links (which at least exist for physical file
systems) but also due to overlapping shares not to
mention special names sent by DFS to the redirector.
Context. On a local file system, it is rather easy to
track unique instances of a given stream (“stream” is
Figure 1—A Complex Environment to Exist In
Page 16
Filtering...

the NTFS terminology. On FAT, there is only a
single stream within a file, but in NTFS there can be
many streams within a single file). This is typically
done using the FsContext parameter of the file object.
Legacy filters may do this directly (via a lookup table
keyed from this value) or indirectly (via a “filter
context” that is attached to a list within the FsContext
structure). Mini-filters rely upon Filter Manager;
Filter Manager in turn relies upon filter contexts.
The Windows redirector, however, does not
necessarily use the same context even for two file
objects pointing to the same file. This works
correctly for them, but it violates this basic
assumption in many filter drivers.
Further
complicating this, the network file systems may
coalesce file objects together – changing their
FsContext values when it detects two instances of the
same file. Filters, once again, must be able to handle
this particular case.
Relative open “issue”. IoCreateFileSpecifyDevice
ObjectHint (in at least some still in-use versions) has
an issue when performing a relative open over the
Windows redirector.
In our experience, it is
necessary to always construct “fully qualified path
names” and send these into this API – or any API
(e.g., FltCreateFile and its progeny) that relies upon
it.
Let’s not lose sight of the fact that there are also two other
network redirectors that ship with at least one version of
Windows (“in the box”) – WebDAV (the http protocol for
“distributed authoring and versioning”) and NFS (which is
shipped as part of the Vista and Server 2008 distributions,
although versions were previously available as “Services for
Unix”).
Maybe You Need to Support Systems Older Than
Windows 7
As difficult as it is for those in Redmond to believe, there are
still a few stragglers left using Windows versions prior to
Windows 7 (please don’t miss the sarcasm, though the point is
still valid). There are various reasons for this, but the biggest
driver really is the cost associated with upgrading the systems.
Particularly when those systems are performing their primary
function properly, there is no compelling reason. People that
do not need transactions, bit locker, resizable thumbnails,
fancy graphical features and the like and instead are focused
on executing custom software solutions, running embedded
devices, etc. are very resistant to upgrading their systems.
In spite of this resistance, they do expect their anti-virus
products, quota managers, content control managers, backup
components, encryption products, etc. to work on their version
of Windows. Microsoft does not receive revenue from old
versions of Windows, but those of us building filter drivers
receive revenue from building products for those older
versions. This creates a constant tension between the general
development community (that needs support for any version
that is still in use by their customer base) and the desires of
Microsoft (which is to move the platform forward so their
customers will buy an upgrade).
The harsh reality is that we have a huge number of versions of
Windows to support. Microsoft wants everyone to move to
mini-filters (and has clearly stated they will enforce this at
some point, which seems as if it will just force the legacy
support to go “underground”). Each new platform requires
additional resources to test and fix bugs – there are always
changes in behavior, often for very specialized usage cases
(transactions come to mind) that can directly impact on your
own development (transactions matter to filters because filters
see the “forward” operations but see no I/O if the transaction
is aborted – even though the state of the underlying system
changes).
Testing is Difficult
We can give you a basic list of the “tools of the trade” but the
bottom line is that no matter how much testing you perform,
you will never be able to cover every case. In all fairness,
debugging a problem that manifests itself via an application –
perhaps a common one like “Explorer” (common but still
amazingly frustrating) or perhaps something far less common,
like “Avid Newscutter Nitris DX” – is always going to be
rather challenging to debug because it often requires that you
determine why the application fails – you know it is related to
something you are doing, but you don’t know what it is.
Microsoft does not provide us with a conformance test. The
closest thing we have (the ifskit tests) have mysterious failure
modes and are often as inscrutable to understand as any failing
application. Indeed, we generally do not investigate “SEV2”
issues because we’ve found they generally are of the “oh, we
don’t work the same way that NTFS does in this case”
category.
Another tool we often point out to people is the “FileTest”
utility. It allows you to exercise uncommon operations like
“open by file ID” that can be difficult to simulate with an offthe-shelf program, but need to work properly within your filter
driver.
I/O Stress can be useful for finding issues in your I/O
handling paths. Driver Verifier is a must – if you aren’t using
it during development and part of your testing, you aren’t
doing a good job. However, you must also test without Driver
Verifier enabled. We’ve even seen cases where Driver
Verifier has tests that only trigger on single processor
computers!
At OSR, we have our own testing framework – it’s a scripting
language that is built against the native API and allows us to
trivially write scripts that can test the native windows API. If
you restrict yourself to the Win32 API, your tests will
definitely be deficient, since it is a subset of the functionality
in the native API.
Finally, don’t forget to test via the network – but this time,
with the file SERVER running over your filter. Again, you
will see different behavior.
For example, the
SL_INDEX_SPECIFIED option that is used for directory
enumeration is difficult to trigger –but does show up with
Page 17
Filtering...
some versions of the Windows file server software. Naturally,
this is exercised when you access files over the network. Oh,
and lest you missed this fact, you now (beginning with Server
2003) need to test between machines – loopback mode has
been optimized out so that it bypasses the normal client/server
model (allegedly there is a registry parameter to disable this
functionality, but usually we find it easier to simply tell test
organizations to use two computers, since that’s going to be
the most common configuration “in the field”).
Don’t Forget Plugfest
For anyone working with file system filter drivers should
always plan on attending the periodic 'Plugfest' that Microsoft
sponsors to aid interoperability. While the opportunity to do
direct developer-to-developer interop testing is invaluable, the
contacts - both Microsoft and third party vendors - are often
more valuable. The events are sponsored by the file systems
filtering team at Microsoft and are done directly at the
Microsoft main campus. These events are publicized in a
variety of ways, including via OSR's well regarded NTFSD
mailing list, a highly technical discussion group talking about
the development of file systems and file system filter drivers
for Windows."
Conclusions
File system filter drivers are the most complicated type of
filter driver to write on Windows; deceptively, they span a
very broad range, from “simplistic” (a surprising number of
examples fall into this category) to “more complicated than a
file system driver”. There are many reasons for this and
we’ve tried to touch on some of them. But if you are building
a file system filter driver because you think it will be
“simpler” than a file system driver, you are likely to be
unpleasantly surprised.
If you can achieve your functionality without a filter driver,
we would suggest doing it in that way. A real filter driver is
truly a challenge to build.
File Systems Development Kit
For more than ten years, OSR’s File System Development
Kit has been used to develop stable, highly functional,
performant and commercially successful file system
solutions in the Windows enterprise.
Local, remote, embedded, pseudo, layered—whatever type
of file system your project requires, the OSR FSDK has
been put to the test.
What more can you ask for but a hardened solution, that is
maintained and fully-supported by industry-recognized
experts?
We provide a straight-forward interface for you to develop
against, while managing the complex interactions with
Windows. This means you can utilize your engineering
resources to implement your product’s value-add, without
having to worry about the myriad of edge cases that you
don’t even know about.
Want to know more? Check out the interface document
available for the FSDK toolkit at www.osr.com. Call or email
us with questions. Send your requirements for us to review
under NDA.
Remember we’re here to help — were not in the business
of licensing one of our toolkits to someone with a, “good
luck and have a nice day”. We are here to make our clients
successful.
If this technology and OSR’s reputation is something you
think you can take advantage of, we look forward to hearing
from you.
Contact us at [email protected].
Page 18
How to Fetch...
Checked Components for Vista/Server 2008
OSR would like to thank Mark Roddy for this contribution,
originally posted to the OSR-hosted NTDEV mailing list.
Just because this was such a royal PITA for what used to be
such a trivial task, I thought I would share how I wasted an
entire day with the rest of you, and perhaps spare some
unfortunate the grief I went through.
Task: obtain the checked version of an inbox driver for Vista
or later OS release in order to install that driver for debugging
purposes.
Estimated time to complete: one to two hours, based on prior
experience with other OS platforms. Most of that time would
be spent downloading an OS image if a DVD or CD were not
available.
and installed, you now have to puzzle over which three little
files Don was referring to. Read the awful WAIK docs. Zero
in on imagex and the clutter of command line options that go
with it. That would be one file of the requisite three. Then
infer that wimfltr.sys and wimfltr.inf are the other two useful
files. Note that the WAIK does not contain any reference I can
find regarding how one ought
to install wimfltr.sys, although it does note that it is required
every now and then.
5. Decide that imagex rivals bcdedit in command line option
fatigue. Google how other people are actually using this thing.
End up here: http://www.windows-now.com/blogs/robert/
archive/2007/11/04/mounting-wim-images-from-windowsexplorer.aspx
and from there here: http://www.autoitscript.com/gimagex/.
6. Download gimagex from that last link.
Actual time to complete: approximately eight hours and one
headache.
7. Install wimfltr (right click on the inf file, select install.) No
need to reboot!
1. Search MSDN for a complete checked distribution of the
OS (Vista SP1 in this case). MSDN, which had a frightfully
bad archive catalog system, now has managed to reimplement that as a hideously and frightfully bad catalog
system. As far as I can tell, without an MSDN subscription
you cannot find a post-RTM version of the checked
distribution legitimately. Note that if you just want the hal and
kernel, they are in the WDK.
8. Use gimagex to mount \sources\install.wim from the
checked debug build.
2. Download the image - approximately 2.5GB. Take a nap.
Run some errands.
Now all I have to do is remember what it was I was doing to
begin with that made me decide that using a debug inbox
driver would be a good idea.
3. Mount the image either by burning it to DVD or using any
of the useful image mounting programs out there. Spend
(waste) quite a while searching the image for drivers and
come up with nothing. Google this puzzle and discover of
course that Don Burn has been here before and noted that now
you need to download the WAIK:
Microsoft definitely changed things. What you need to do is
download the Windows Automated Installation Kit at http://
www.microsoft.com/downloads/details.aspx?
familyid=C7D4BC6D-15F3-4284-9123679830D629F2&displaylang=en.
This provides a file system that will allow you to open the
files on the DVD's and extract the individual files you need.
Unfortunately the kit is almost 1GB and you need three small
files
from
it.
http://www.tech-archive.net/Archive/
Development/microsoft.public.win32.programmer.
kernel/2007-09/msg00073.html.
4. Download the WAIK - another hour of my life lost. I did
have a nice camembert on rye sandwich. Once downloaded
9. Browse around like you did back in the good old XP/W2K3
days and find the driver images you were looking for to begin
with and copy them somewhere safe. Heck, while you are
there, copy all the drivers from the image to someplace
reasonable.
Page 19
Who Was at the DDC?
Well, We Were, But...
N
fall?
othing like hanging around in building 33 on the
Microsoft Redmond campus waiting for...um, the sky to
Anyway, this is where we found ourselves during a break
from presentations at the Windows Driver Development
Conference (DDC) when we ran into a few colleagues from
the driver development community. Fortunately, someone
even had a camera to capture the Kodak moment!
To the right (left to right) you’ll see: OSR’s Peter Viscarola;
DDK MVP Maxim Shatskih; Jake Oshins, Microsoft Hyper-V
I/O Architect.
Below, Peter and Jake are joined (left to right) by DDK MVP
Martin O’Brien; OSR’s Scott Noone; OSR’s Mark Cariddi.
A hearty thanks to Maxim for the pics, and to these folks and
others who hung with us for some couch time during the
conference.
Page 20
New OSR Seminar Schedule!
Seminar
Developing File System Mini-Filters (Lab)
Writing WDF Drivers (Lab)
Kernel Debugging (Lab)
Writing WDM Drivers (Lab)
Developing File Systems
Dates
Location
23-27 February
Boston, MA
30 March—3 April
Portland, OR
6-10 April
Portland, OR
27 April—1 May
Boston, MA
4-7 May
Boston, MA
Course outlines, pricing, and how to register at: www.osr.com/schedule
The NT Insider™ is a subscription-based publication
OSR Open Systems Resources, Inc.
105 State Route 101A, Suite 19
Amherst, New Hampshire 03031 USA
(603)595-6500 ♦ Fax (603)595-6503
RETURN SERVICE REQUESTED
PRESORTED
FIRST CLASS MAIL
U.S. POSTAGE
PAID
PERMIT # 456
MANCHESTER, NH
First Class

The NT Insider

Transcription

Similar documents

OSR Flyer

The Ohio State Reformatory

Decreasing Dialysis Patient/Provider Conflict

AzureWave ARR5B95 Us..

dpc newsletter - Dubai Pharmacy College

RZR623 Fixing Wrong USB Devices in Windows 7

Teen Driver Contract

Instruction Manual

Hydro-Filter™ - Phoenix Truckmounts