Anyone outside of MSFT see DMF coming?

https://github.com/Microsoft/DMF

I find this to be a delightful surprise. I think this is another *big* step along the path WDF started. I hope it’s all that it appears to be. This is the first I’ve heard of it, and that’s the most surprising part (to me).

Phil B

Not speaking for LogRhythm

[https://logrhythm.com/images/icons/email-icons/logrhythm-logo-dots.png]http:</http:>

Philip D Barila

Senior Software Engineer

720.881.5364 (w)

LogRhythm.comhttp:</http:>

[https://logrhythm.com/images/icons/email-icons/linkedin-icon-logrhythm.png]https:

[https://logrhythm.com/images/icons/email-icons/twitter-icon-logrhythm.png]https:

[https://logrhythm.com/images/icons/email-icons/facebook-icon-logrhythm.png]https:</https:>

[https://logrhythm.com/images/icons/email-icons/blog-icon-logrhythm.png]https:</https:></https:></https:>

Well, I think it’s really very nice for… and very generous of… and commendable that… … and valuable that… Microsoft have provided us this extensive set of WDF infrastructure code. It really is a great gift.

After spending some significant time (literally days) reading and working through the DMF code, I am sorry to say that I am not QUITE as enthused about DMF as you apparently are.

It is, effectively, an additional abstraction… an overlay framework that exists above the level of WDF and augments it. I guess that’s OK… if that’s what you want. But it’s a LOT of extra code and infrastructure to grok when you’re just trying to fix a bug. You now have to understand WDF *and* DMF just to get to your own code.

It’s also effectively unsupported. There’s no “contract” by MSFT to maintain, fix, or update DMF for future versions of Windows. Sure: The source code is there, and DMF is buildable from source. That also opens the door for folks forking the sources, so that when you go to company X, THEY have a “DMF” that’s different from the DMF at company Z.

These sorts of infrastructural components are a great idea when they live within a particular company or a particular group. The owners of the infrastructure are “right there” – you can ask them questions, they can make fixes and/or adaptations of the infrastructure for you when you need it. Part of your team’s culture is the use of the infrastructure.

Outside of that original environment? I’m not convinced.

Sure, once learned, a super-structural framework like DMF might make it “fast and easier” for folks to put together a driver. But let’s bear in mind that more than 50% of the effort required by ANY piece of software comes in the maintenance. I am unconvinced that loosing a large super-structure like DMF on the world creates net value.

It is *extremely* generous of the Surface Team to share their code with the world. DMF is a great gift to the community. And it’s a WONDERFUL demonstration of Microsoft’s new openness.

But it’s really not the solution that the Community most needs. I have *long* advocated for more work to be done on WDF. The full dream for WDF has never been “completed” IMHO. As wonderful as WDF is, it seriously needs a WDF Round Two to make it truly great. SOME of the more important work has come to us in the form of Class Extensions. But WDF Core still could use with some evolution to smooth out the lumps and bumps that have become evident as we’ve used WDF over the years.

This needs a longer article. Look for it in the next issue of The NT Insider.

Peter
OSR
@OSRDrivers

It is an open source project on github. So support is ‘open source project
on github’ support.

Mark Roddy

On Fri, Aug 17, 2018 at 12:29 PM xxxxx@osr.com
wrote:

>


>
> Well, I think it’s really very nice for… and very generous of… and
> commendable that… … and valuable that… Microsoft have provided us
> this extensive set of WDF infrastructure code. It really is a great gift.
>
> After spending some significant time (literally days) reading and working
> through the DMF code, I am sorry to say that I am not QUITE as enthused
> about DMF as you apparently are.
>
> It is, effectively, an additional abstraction… an overlay framework that
> exists above the level of WDF and augments it. I guess that’s OK… if
> that’s what you want. But it’s a LOT of extra code and infrastructure to
> grok when you’re just trying to fix a bug. You now have to understand WDF
> and DMF just to get to your own code.
>
> It’s also effectively unsupported. There’s no “contract” by MSFT to
> maintain, fix, or update DMF for future versions of Windows. Sure: The
> source code is there, and DMF is buildable from source. That also opens
> the door for folks forking the sources, so that when you go to company X,
> THEY have a “DMF” that’s different from the DMF at company Z.
>
> These sorts of infrastructural components are a great idea when they live
> within a particular company or a particular group. The owners of the
> infrastructure are “right there” – you can ask them questions, they can
> make fixes and/or adaptations of the infrastructure for you when you need
> it. Part of your team’s culture is the use of the infrastructure.
>
> Outside of that original environment? I’m not convinced.
>
> Sure, once learned, a super-structural framework like DMF might make it
> “fast and easier” for folks to put together a driver. But let’s bear in
> mind that more than 50% of the effort required by ANY piece of software
> comes in the maintenance. I am unconvinced that loosing a large
> super-structure like DMF on the world creates net value.
>
> It is extremely generous of the Surface Team to share their code with
> the world. DMF is a great gift to the community. And it’s a WONDERFUL
> demonstration of Microsoft’s new openness.
>
> But it’s really not the solution that the Community most needs. I have
> long advocated for more work to be done on WDF. The full dream for WDF
> has never been “completed” IMHO. As wonderful as WDF is, it seriously
> needs a WDF Round Two to make it truly great. SOME of the more important
> work has come to us in the form of Class Extensions. But WDF Core still
> could use with some evolution to smooth out the lumps and bumps that have
> become evident as we’ve used WDF over the years.
>
> This needs a longer article. Look for it in the next issue of The NT
> Insider.
>
> Peter
> OSR
> @OSRDrivers
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

> I am unconvinced that loosing a large super-structure like DMF on the world creates net value.

I have nothing of value to add (I’ve only read the DMF announcement, nothing more), but I just wanted to thank you for using “loosing” correctly! I reflexively cringe every time I see that word on the Internet.

Precisely. Which, when it comes to a kernel-mode driver framework, is – in my not so humble opinion – a profoundly bad option.

Peter
OSR
@OSRDrivers

Despite having declared open source to be open sores at one point in time,
I’ve managed to accept that this is the way things are done now, and the
results are acceptable.

Mark Roddy

On Mon, Aug 20, 2018 at 10:31 AM xxxxx@osr.com
wrote:

>


>
> Precisely. Which, when it comes to a kernel-mode driver framework, is –
> in my not so humble opinion – a profoundly bad option.
>
> Peter
> OSR
> @OSRDrivers
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev&gt;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

I’ve never much developed “the serenity to accept those things I cannot change”.

I’m not trying to argue, believe it or not, I’m trying to see another point of view.

This solution is about 66K lines long (that’s a quick count) in something like 125 different files.

My questions:

a) If MSFT don’t ever update this (I’m not saying they will or they won’t… just assuming they won’t)… It’s “acceptable” that The Community should be responsible for validating and fixing 66K lines of semi-complex code?

b) If the people go off and fork this… that’s reasonable? I mention this because making the source available and buildable as a primary mechanism for support specifically facilitates this.

This doesn’t feel to ME like a great plan, or (frankly) one that’s even remotely desirable.

Peter
OSR
@OSRDrivers

From Peter’s email:

[quote]
This solution is about 66K lines long (that’s a quick count) in something like 125 different files.

That’s rather small in scope compared to many OSS projects and are well-maintained. It would be incumbent upon MS to be the gatekeepers in deciding what updates to allow to merge back into mainline.

[quote]
a) If MSFT don’t ever update this (I’m not saying they will or they won’t… just assuming they won’t)… It’s “acceptable” that The Community should be responsible for validating and fixing 66K lines of semi-complex code?

Small scope here in the grand scheme of things.

[quote]
b) If the people go off and fork this… that’s reasonable? I mention this because making the source available and buildable as a primary mechanism for support specifically facilitates this.

This doesn’t feel to ME like a great plan, or (frankly) one that’s even remotely desirable.

You fork it, you bought it. Those who have worked OSS in the past understand the dangers of obtaining and using forked-up code from untrusted sources. If MS maintains control over the repository, they can gain a legion of free SW developers and support to do almost all the work for them. They just have to approve/reject submissions.

It’s worked quite well for projects many orders of magnitude greater than yet another driver framework. (YADF?)

I just wish MS would open-source Windows Embedded Compact since they’ve decided to abandon all of us who made our livings with that OS. It’s still the best general-purpose RTOS (I know, oxymoron) out there.

Just my observations from making a living in both worlds.

Greg

> I just wanted to thank you for using “loosing” correctly! I reflexively cringe every time

I see that word on the Internet.

What about the “masterpieces” like, for example, “diffusing the situation” - you may encounter them not only in the comments sections. In fact, even in this NG archives you may find some threads where Truly Yours is accused of trying to “bate” people, and these statements are made by those for whom English is supposedly a native language…

Anton Bassov

> a) If MSFT don’t ever update this (I’m not saying they will or they won’t… just assuming

they won’t)… It’s “acceptable” that The Community should be responsible for validating
and fixing 66K lines of semi-complex code?

Well, I would say that the whole situation has been perfectly described by Mr.Roddy’s
“open source project on github” statement. There is nothing that obliges either them or the community to provide any maintenance/support for it. If they choose to do so they can do it either completely on their own, or with some help from the community. In the latter case they will have to appoint some gatekeeper(s) who decide(s) upon the changes that are allowed to be merged into the tree.

If the people go off and fork this…

Well, this is entirely possible, but it turns into a totally independent project at this point . As they say, go and use it at your own risk…

that’s reasonable?

Well, it depends…

You should realise that “forked” and "supported by the community and/or /third-party company"does not necessarily imply “of sub par quality”, although sometimes it may, indeed, be the case. Everything depends on the gatekeeper(s) of the particular project and on their requirements to the code that may be accepted to the tree. Just look at the major open-source OSes - you can find some examples of just a legendary quality/stability(i.e. OpenBSD)

The only thing I can assure you of is that that we are more than likely to hear the “exciting” things (probably, even in capitals) from the usual suspects in this NG about those who decide to use this code in their projects (OK, I shut up - otherwise I may experience with my own back( or below) the full power of the new platform’s troll-management functionality)…

Anton Bassov

> “diffusing the situation”

This is, in fact, perfectly good English. Am I missing something?

is accused of trying to “bate” people

and THIS is a simple misspelling of “bate” for “bait”… which would make this an otherwise perfectly proper English language phrase. One can hardly blame people for a quick misspelling… this is the Internet, and MY iPad keyboard misspells things all the time. Again, what am I missing?

This is a philosophical debate, and largely depends on what “par” for quality means to those who are debating. But I’ll grant that in SOME cases, an open source project that’s supported by the community COULD not entirely suck. I will save you (and the entire Community) my standard dissertation on the topics of devs “scratching your own itch” and not “doing the dirty and unglamorous work that needs done” on a given project. I’m sure you can imagine what it is. Just insert all that here.

And we have, to this point, exactly zero indication from Microsoft how they’ll be proceeding in this area. Who’s reviewing PRs, what are the criteria for approving them, etc.

MY issues are:

a) For the average dev, who writes one or two drivers *ever* while they work for a given company, I don’t see how something like DMF works to their advantage.

You COULD argue, that once a given dev has learned DMF they could go from company to company throughout their career efficiently writing DMF drivers. Maybe. That assumes DMF is maintained and continues to exist for many years going forward. But it also leads to my second issue.

b) I am not convinced that a driver written using DMF is easier (or even, as easy) to maintain as one written using base WDF and the necessary custom code alone. If the dev above, who learns DMF and writes one driver using it at company A, and another two drivers using it at company B… the devs who come after that DMF-using dev now have to ALSO spend the time to learn DMF and debug through DMF and track the status of the DMF project. And they need to do this on their own. And every time there’s a bug, they have to determine whether the bug is in their device-specific code or in DMF … or, at the very least walk through the DMF code to FIND their bug.

This does not strike me as a win.

c) So now a new dev… who knows a LITTLE WDF… joins company B, which has two DMF-based drivers. Ignoring the learning and debugging burden I describe immediately above: This dev looks at the sources, and instead of saying “Oh, a WDF driver”… sees the driver AND a whole bunch of DMF framework, and wonders “What hath Gxd wrought.” She then needs to determine WHICH version of DMF is being used, WHETHER it’s been forked or whether its in its pristine state.

You may argue, correctly, that this is the state of the world in the Open Source community. And, if you’re using some library like Boost (love it or hate it) to write some random user-mode app… maybe that’s OK.

But this is driver code. It’s hard ENOUGH to figure out how Windows works, and how your hardware works, and to write a driver. NOW we introduce the variability of a random framework into the mix?

It increasingly seems to me that what (may be) good for the world of app world is not necessarily ALSO good for the kernel-mode world. Whether it’s agile, flighted features, or open source value add frameworks. I am rapidly coming to the conclusion that the world of infrastructure benefits from different rules.

Peter
OSR
@OSRDrivers

>

is accused of trying to “bate” people

and THIS is a simple misspelling of “bate” for “bait”… which would make this an otherwise perfectly proper English language phrase. One can hardly blame people for a quick misspelling… this is the Internet, and MY iPad keyboard misspells things all the time. Again, what am I missing?

Their waz un artykal ni Nychational Giogaphi dat tails most pupil understands tha soobjact form tha contaxt then two fallow the speling - Tis bout haw tha brian walks :slight_smile:

-orP


(c) So now a new dev… who knows a LITTLE WDF… joins company B,
which has two DMF-based drivers. Ignoring the learning and debugging
burden I describe immediately above: This dev looks at the sources,
and instead of saying “Oh, a WDF driver”… sees the driver AND a
whole bunch of DMF framework, and wonders “What hath Gxd wrought.”

[/quote]


Is this substantially different than encountering a WDF driver with a bunch of
locally written code to accomplish the same things as the DMF code? This is
the real comparison. What are the chances that the locally written code is
(1) more correct, (2) more readable, and/or (3) better documented than DMF?

Yes, the original author could have dragged in a whole lot of DMF while
actually needing very little of it (see also: Boost), and this will needlessly
increase the learning curve. But the comparison is what that same original
author would have done if they wrote the code themselves. The result would
probably be smaller, but would it be better? Which would you prefer, 10K of
locally written spaghetti code or 300K of DMF?

–John

> “diffusing the situation”

> This is, in fact, perfectly good English. Am I missing something?

OMG - I just don’t believe my own eyes…

If you want to make a situation less tense you have to DEFUSE and not to DIFFUSE it. Defusing is synonymous with alleviation, reduction, allaying, mitigating, softening,etc. However, DIFFUSING means “spreading all over the place” and is synonymous with dispersing, dispensing, scattering, disseminating,propagating, etc.

is accused of trying to “bate” people and THIS is a simple misspelling of “bate” for “bait”
which would make this an otherwise perfectly proper English language phrase.
One can hardly blame people for a quick misspelling… this is the Internet, and MY
iPad keyboard misspells things all the time. Again, what am I missing?

As long as you do it once this is not a big deal - after all, I accidentally wrote “hummer” instead of “hammer” once (and got a fair share of well-deserved derision on NTDEV for it, of course). However, if someone keeps on spelling it this way again and again and again…well, apparently it must be something more than just a mere coincidence, don’t you think. Probably, the guy should go back to school.

Although I am not going to tell you his name I can assure you that this particular individual is TRULY UNIQUE in his very own way - I know it may sound incredible, but he somehow managed to stay ignorant of the very concept of Fibonacci numbers despite having had programmed computers for 40+ years!!!

This is a philosophical debate, and largely depends on what “par” for quality means
to those who are debating.

Well, in this particular context we can safely define “par” for quality as a “being suitable for writing production-grade Windows drivers”. In other words, it must be comparable to WDF…

I dunno…

IIRC, you were “tired of C” not so long ago, and were dreaming about the possibility of using the managed languages like C# in the kernel. Now you seem to be agreeing with my opinion and admitting that all these"fancy" technologies/methodologies/etc, no matter how useful they may be in the userland, simply have no place in the kernel…

Anton Bassov

Precisely on point! Yes, THIS is the question.

Well stated, again… but I would make ONE slight amendment (or, for Anton, we COULD write “emendation”):

Do you prefer 10K of locally written spaghetti code, or 300K of *some version* of DMF, the genesis, age, and state of which you do not know.

I think that’s a tough call.

Given that the 300K and unknown provenance are fixed, and the level of spaghetti-ness is variable (could be 10K of code that sucks, or 10K of code that’s pretty decent)… I think I’d have to go with the 10K of locally created pasta.

Peter
OSR
@OSRDrivers

I hate that I’m coming off here as being harsh about DMF… I think it’s really great, in the right context. MY primary issue is that I’m not convinced that the overall context of the wider third party driver development community is the right context.

Spelling was never my strong suit. There, I’ve admitted it. My one failing.

That typo has a particularly funny consequence in colloquial American English.

Still am. My opinons on such topics don’t tend to be ephemeral.

Yeah… no.

Peter
OSR
@OSRDrivers

I wish I could admit to only *one* failing.
-dc

Sent from my Windows 10 phone

>As long as you do it once this is not a big deal - after all, I accidentally wrote “hummer” instead of

“hammer” once (and got a fair share of well-deserved derision on NTDEV for it, of course).

That jewel would have been lost, too, under the initial Jan. 1, 2010 date for archive migration. :wink:

http:

Gabe</http:>

Well, THERE you go. THE definitive reason for keeping the archives “back to the beginning.”

Just so we can further derail this thread, and hide important information from the unsuspecting, as of yesterday we have confirmed that we CAN migrate all the data. The early posts are pretty funny, though, because almost all the corresponding users have since been deleted (due to their leaving NTDEV, changing email addresses, or whatever). So there are thread that have no recognizable usernames associated with them.

But… we WILL have them!

Peter
OSR
@OSRDrivers