Moving from PCIe to Thuderbolt

Hello,

I developed a WDF device driver for an FPGA based on the sample: 7600.16385.1\src\general\PLX9x5x

The driver works OK.

When I changed the interface from PCIe to “Thuderbolt”, the BAR0 I got (in PLxPrepareHardware) was twice larger compared to regular PCIe.

I mapped the larger BAR using MmMapIoSpace. When I tried to read offset 0x0 in BAR0 using IOCTL requet I got invalid data.

What should I change in order to support “Thuderbolt” ?

When we tried to use an old WDM driver for the same FPGA, the driver worked OK with “Thunderbolt”.

Best regards,
Z.V

Zvi Vered wrote:

I developed a WDF device driver for an FPGA based on the sample:
7600.16385.1\src\general\PLX9x5x

The driver works OK.

When I changed the interface from PCIe to “Thuderbolt”, the BAR0 I got
(in PLxPrepareHardware) was twice larger compared to regular PCIe.

Did you expect that? If not, then you need to investigate that. It may
suggest that you are not talking to the device you think you are. Does
your device include a PCIe bridge?

What should I change in order to support “Thuderbolt” ?

Nothing. Electrically, the Thunderbolt components connect to the
PCIExpress root complex, and at that point is literally is a PCIExpress
device.

When we tried to use an old WDM driver for the same FPGA, the driver
worked OK with “Thunderbolt”.

Do you have the source code for that driver? Have you checked the
mapping code to see if it is different?


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Hi Tim,

My device includes a PCIe bridge.
But in the inf I specifiy vendor, device of the FPGA, not the bridge.
The bridge is transparent to the device driver.
It is not mentioned in the source code nor in inf.

I will the have the source code of the old WDM driver in few days.
Can you tell what part of the driver you are interested in ?

Best regards,
Z.V

-----Original Message-----
From: Tim Roberts
Sent: Saturday, May 23, 2015 02:07 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Moving from PCIe to Thuderbolt

Zvi Vered wrote:

I developed a WDF device driver for an FPGA based on the sample:
7600.16385.1\src\general\PLX9x5x

The driver works OK.

When I changed the interface from PCIe to “Thuderbolt”, the BAR0 I got
(in PLxPrepareHardware) was twice larger compared to regular PCIe.

Did you expect that? If not, then you need to investigate that. It may
suggest that you are not talking to the device you think you are. Does
your device include a PCIe bridge?

What should I change in order to support “Thuderbolt” ?

Nothing. Electrically, the Thunderbolt components connect to the
PCIExpress root complex, and at that point is literally is a PCIExpress
device.

When we tried to use an old WDM driver for the same FPGA, the driver
worked OK with “Thunderbolt”.

Do you have the source code for that driver? Have you checked the
mapping code to see if it is different?


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

I am asuming Thuderbolt is a typo and the real interface is ThuNderbolt? If not, please educate me.

Thanks,
Greg

xxxxx@gmail.com wrote:

From: “Zvi Vered”
To: “Windows System Software Devs Interest List”
Subject: Re: [ntdev] Moving from PCIe to Thuderbolt
Date: Sat, 23 May 2015 20:08:33 +0300

Hi Tim,

My device includes a PCIe bridge.
But in the inf I specifiy vendor, device of the FPGA, not the bridge.
The bridge is transparent to the device driver.
It is not mentioned in the source code nor in inf.

I will the have the source code of the old WDM driver in few days.
Can you tell what part of the driver you are interested in ?

Best regards,
Z.V

-----Original Message-----
From: Tim Roberts
Sent: Saturday, May 23, 2015 02:07 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Moving from PCIe to Thuderbolt

Zvi Vered wrote:
>
> I developed a WDF device driver for an FPGA based on the sample:
> 7600.16385.1\src\general\PLX9x5x
>
> The driver works OK.
>
> When I changed the interface from PCIe to “Thuderbolt”, the BAR0 I got
> (in PLxPrepareHardware) was twice larger compared to regular PCIe.

Did you expect that? If not, then you need to investigate that. It may
suggest that you are not talking to the device you think you are. Does
your device include a PCIe bridge?

> What should I change in order to support “Thuderbolt” ?

Nothing. Electrically, the Thunderbolt components connect to the
PCIExpress root complex, and at that point is literally is a PCIExpress
device.

> When we tried to use an old WDM driver for the same FPGA, the driver
> worked OK with “Thunderbolt”.

Do you have the source code for that driver? Have you checked the
mapping code to see if it is different?


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

You expect Thunderbolt to work on Windows? Good luck with that…

Peter
OSR
@OSRDrivers

Hi Peter,

In my case an old WDM driver works with the same FPGA.
For some reason, with Thunderbolt, the first BAR length found in the code:

for (i=0; i < WdfCmResourceListGetCount(ResourcesTranslated); i++)
{

desc = WdfCmResourceListGetDescriptor( ResourcesTranslated, i );


}

is 0x8000. But with regular PCIe the length is 0x4000.

When we tried reading offset 0x0 of this BAR we got 0x0 but we should get
the version of the FPGA.

Best regards,
Z.V

-----Original Message-----
From: xxxxx@osr.com
Sent: Sunday, May 24, 2015 06:27 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Moving from PCIe to Thuderbolt

You expect Thunderbolt to work on Windows? Good luck with that…

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Wdf doesn’t do anything with assigned resources other than to try to connect interrupts, so the window size should be the same between both instances of the driver. Are you testing both drivers on the same os and config?

d

Bent from my phone


From: Zvi Veredmailto:xxxxx
Sent: ?5/?23/?2015 10:10 PM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: Re: RE:[ntdev] Moving from PCIe to Thuderbolt

Hi Peter,

In my case an old WDM driver works with the same FPGA.
For some reason, with Thunderbolt, the first BAR length found in the code:

for (i=0; i < WdfCmResourceListGetCount(ResourcesTranslated); i++)
{

desc = WdfCmResourceListGetDescriptor( ResourcesTranslated, i );


}

is 0x8000. But with regular PCIe the length is 0x4000.

When we tried reading offset 0x0 of this BAR we got 0x0 but we should get
the version of the FPGA.

Best regards,
Z.V

-----Original Message-----
From: xxxxx@osr.com
Sent: Sunday, May 24, 2015 06:27 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Moving from PCIe to Thuderbolt

You expect Thunderbolt to work on Windows? Good luck with that…

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

Hi Doron,

Both WDF and old WDM drivers were tested on the same PC under Win7-64.

Best regards,
Z.V

From: Doron Holan
Sent: Sunday, May 24, 2015 10:07 AM
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] Moving from PCIe to Thuderbolt

Wdf doesn’t do anything with assigned resources other than to try to connect interrupts, so the window size should be the same between both instances of the driver. Are you testing both drivers on the same os and config?

d

Bent from my phone


From: Zvi Vered
Sent: ?5/?23/?2015 10:10 PM
To: Windows System Software Devs Interest List
Subject: Re: RE:[ntdev] Moving from PCIe to Thuderbolt

Hi Peter,

In my case an old WDM driver works with the same FPGA.
For some reason, with Thunderbolt, the first BAR length found in the code:

for (i=0; i < WdfCmResourceListGetCount(ResourcesTranslated); i++)
{

desc = WdfCmResourceListGetDescriptor( ResourcesTranslated, i );


}

is 0x8000. But with regular PCIe the length is 0x4000.

When we tried reading offset 0x0 of this BAR we got 0x0 but we should get
the version of the FPGA.

Best regards,
Z.V

-----Original Message-----
From: xxxxx@osr.com
Sent: Sunday, May 24, 2015 06:27 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Moving from PCIe to Thuderbolt

You expect Thunderbolt to work on Windows? Good luck with that…

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

What does the device manager say the resources look like? I?ve seen old
crufty WDM drivers that hard coded the size of the resource, which could
explain why it works with WDM but not KMDF. I have also seen FPGA code
(particularily some of the vendor supplied stuff) report the size of the
resources wrong,

Don Burn

Windows Driver Consulting

Website: http:</http:> http://www.windrvr.com

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Zvi Vered
Sent: Sunday, May 24, 2015 3:55 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Moving from PCIe to Thuderbolt

Hi Doron,

Both WDF and old WDM drivers were tested on the same PC under Win7-64.

Best regards,

Z.V

From: Doron Holan mailto:xxxxx

Sent: Sunday, May 24, 2015 10:07 AM

To: Windows System Software Devs Interest List mailto:xxxxx

Subject: RE: RE:[ntdev] Moving from PCIe to Thuderbolt

Wdf doesn’t do anything with assigned resources other than to try to connect
interrupts, so the window size should be the same between both instances of
the driver. Are you testing both drivers on the same os and config?

d

Bent from my phone

_____

From: Zvi Vered mailto:xxxxx
Sent: ?5/?23/?2015 10:10 PM
To: Windows System Software Devs Interest List mailto:xxxxx
Subject: Re: RE:[ntdev] Moving from PCIe to Thuderbolt

Hi Peter,

In my case an old WDM driver works with the same FPGA.
For some reason, with Thunderbolt, the first BAR length found in the code:

for (i=0; i < WdfCmResourceListGetCount(ResourcesTranslated); i++)
{

desc = WdfCmResourceListGetDescriptor( ResourcesTranslated, i );


}

is 0x8000. But with regular PCIe the length is 0x4000.

When we tried reading offset 0x0 of this BAR we got 0x0 but we should get
the version of the FPGA.

Best regards,
Z.V

-----Original Message-----
From: xxxxx@osr.com
Sent: Sunday, May 24, 2015 06:27 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Moving from PCIe to Thuderbolt

You expect Thunderbolt to work on Windows? Good luck with that…

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

I’m obviously missing a key piece of the puzzle here. And rather than trying to solve the symptoms of your problem, I’m more interested in the root cause.

The last time I checked, and this was prior to Win10, Windows emphatically DID NOT support Thunderbolt. There are some significant technical issues, like having a PCIe root port just appear all of a sudden, that made supporting Thunderbolt “challenging.”

There are some *very* limited and unique Thunderbolt implementations for some very specific things. But there are soooo many potential issues here that, unless you’re partnering closely with a given vendor who’s providing the Thunderbolt support, I’m not sure why you’d be trying to get Thunderbolt anything working on Windows.

Peter
OSR
@OSRDrivers

Do you have a PCIe analyzer? See what bar size the device is actually returning. Also see !pci output.

On 24-May-2015 08:10, Zvi Vered wrote:

When we tried reading offset 0x0 of this BAR we got 0x0 but we should
get the version of the FPGA.

Once we’ve seen something similar with PCIe cards behind
a MXI bridge and fiber cable extender.
I can’t recall was it a config error of the extender
or timing problem on our side, but finally it worked.
Ask your hardware folks to arrange a bus analyzer.

Good luck,
– pa

xxxxx@osr.com wrote:

You expect Thunderbolt to work on Windows? Good luck with that…

Thunderbolt is basically DisplayPort plus PCIExpress. With an
appropriate (and relatively simple) bridge, it should just Plug-and-Play.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Zvi Vered wrote:

My device includes a PCIe bridge.
But in the inf I specifiy vendor, device of the FPGA, not the bridge.
The bridge is transparent to the device driver.
It is not mentioned in the source code nor in inf.

I will the have the source code of the old WDM driver in few days.
Can you tell what part of the driver you are interested in ?

I second Don’s advice. Check to see how they are mapping the resource.
Maybe they hardcoded the length, or maybe they are only mapping a
portion of the space.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

What are you doing to read the version? Do you have all the right bits set
for accessing the FPGA? The sample driver puts the chip into software
reset when it unloads. I just finished implementing a PCI-based PLX9056
driver with WDF, and the software reset was the root of many differences
from the WDM driver it replaced. If you are using an EEPROM to load the
configuration, it will get reloaded every time the driver is started. Any
changes to the registers will be lost.

  • Phil

On Mon, May 25, 2015 at 3:33 PM, Tim Roberts wrote:

> Zvi Vered wrote:
> > My device includes a PCIe bridge.
> > But in the inf I specifiy vendor, device of the FPGA, not the bridge.
> > The bridge is transparent to the device driver.
> > It is not mentioned in the source code nor in inf.
> >
> > I will the have the source code of the old WDM driver in few days.
> > Can you tell what part of the driver you are interested in ?
>
> I second Don’s advice. Check to see how they are mapping the resource.
> Maybe they hardcoded the length, or maybe they are only mapping a
> portion of the space.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

That’s what I thought at first too, but on closer examination that is DEFINITELY NOT the case. There’s a lot more to it than that.

About a year ago, we spent *a lot* of time looking into Thunderbolt on Windows. This time was ultimately not compensated as we wound-up having to decline the project.

So, to repeat once again what I wrote above and what I know to be true as of one year ago:

Again, things might have changed in the past year or in Win10. But I know the above to be true as of one year ago. I am not guessing.

Peter
OSR
@OSRDrivers

Dear Members,

My FPGA contains only one BAR.

But with Thunderbolt I’m getting 2 BARs:

  1. 0x8000 bytes
  2. 0x400 bytes.

The second BAR contains few registers that should be in BAR0.

With regular PCIe I’m getting one BAR. Its length is 0x1000 bytes.

It seems there is a long way till Thunderbolt will be transparent to WDF
developers.
Do you think it’s a BIOS problem or Windows problem ?

Best regards,
Z.V

-----Original Message-----
From: xxxxx@osr.com
Sent: Tuesday, May 26, 2015 07:29 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Moving from PCIe to Thuderbolt

That’s what I thought at first too, but on closer examination that is
DEFINITELY NOT the case. There’s a lot more to it than that.

About a year ago, we spent *a lot* of time looking into Thunderbolt on
Windows. This time was ultimately not compensated as we wound-up having to
decline the project.

So, to repeat once again what I wrote above and what I know to be true as of
one year ago:

Again, things might have changed in the past year or in Win10. But I know
the above to be true as of one year ago. I am not guessing.

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

On 28-May-2015 20:46, Zvi Vered wrote:

It seems there is a long way till Thunderbolt will be transparent to WDF
developers.
Do you think it’s a BIOS problem or Windows problem ?

If there’s a 3rd party driver for the TB bride, the problem
is with that driver. Otherwise, the problem comes from the people who
decided to work on unsupported hardware.

– pa

GMAFB. Are my replies invisible to you for some reason, or you just deliberately ignoring what I wrote?

Peter
OSR
@OSRDrivers

Dear Members,

I owe you a huge apology.

The FPGA I used had 2 BARs.
The Thunderbolt interface did not “create” a BAR.
An older FPGA version had one BAR and I thought by mistake that no one
changed this.

So in my private case, under Win7-64, the Thuderbolt interface works fine.

Best regards,
Z.V

-----Original Message-----
From: xxxxx@osr.com
Sent: Tuesday, May 26, 2015 07:29 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Moving from PCIe to Thuderbolt

That’s what I thought at first too, but on closer examination that is
DEFINITELY NOT the case. There’s a lot more to it than that.

About a year ago, we spent *a lot* of time looking into Thunderbolt on
Windows. This time was ultimately not compensated as we wound-up having to
decline the project.

So, to repeat once again what I wrote above and what I know to be true as of
one year ago:

Again, things might have changed in the past year or in Win10. But I know
the above to be true as of one year ago. I am not guessing.

Peter
OSR
@OSRDrivers


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer