Hi,
Can I assume that all _NET_BUFFERs in a _NET_BUFFER_LIST are either all multicast/broadcast packets or all point-to-point?
Are all _NET_BUFFERs pointing to the same Destination MAC address, or are they mixed?
Regards, Calin
Interlocutor: Are all _NET_BUFFERs pointing to the same Destination MAC address, or are they mixed?
Jeffrey: All NET_BUFFERs in an NET_BUFFER_LIST must belong to the same “stream”.
Interlocutor: Ok, thanks a lot Jeffrey, you’ve just substituted the original question with a new question: What the heck is a “stream”?
Jeffrey: A stream is a sequence of packets that, as far as an NDIS driver can know, belong to the same connection and application.
Interlocutor: What? Why so vague? Can you give me specific examples & requirements?
Jeffrey: Sure. First, when you get an NBL, you can always assume that NB chain contains packets with the same layer-2 connection. For Ethernet, that means the Ethernet source & destination address, VLAN ID, EtherType (including SNAP/LLC header if present).
Secondly, when you give somebody else an NBL, the Golden Rule implies that these assumptions become requirements. That is, you are required to not create an NBL chain that mixes Ethernet addresses, VLAN IDs, or EtherTypes. The safest course of action is to always put 1 NB in to each NBL, since then you’re never accidentally mixing streams.
Interlocutor: Hmm, ok. So an NBL chain always has the same Ethernet address. That’s simple enough. Why didn’t you just say that in the first place?
Jeffrey: Depending on the EtherType, you can derive even more specific requirements. Consider EtherType = 0x0800 (IPv4). We furthermore require that IPv4 traffic is grouped by IP source & destination address. E.g., an NBL cannot contain two NBs with different IPv4 source addresses. The same is true of EtherType 0x86dd (IPv6).
Interlocutor: Oh, I see, sometimes we reach into the layer-3 headers. Do we go deeper into the packet than that?
Jeffrey: Yup. Now consider IPv4 or IPv6, with NextProtocol = 0x06 (TCP). We furthermore require that TCPv4 traffic is grouped by TCP source & destination ports. E.g., an NBL cannot contain two NBs with different TCPv4 source ports. Similar is true of NextProtocol = 0x11 (UDP).
Interlocutor: Alright, I guess TCP and UDP are special, so it’s okay to special-case them. But there’s not anything else, is there?
Jeffrey: Well… now consider encapsulated traffic (pick your technology: either riding over IP directly as in GRE or IPsec; or riding over UDP as in VXLAN). We’d like to peek into the encapsulated payload, to recursively apply our bucketization rules. So for example, we wouldn’t want to group together these two NBs: [Ethernet][IP][Gre][IP][TCP dest=80] and [Ethernet][IP][Gre][IP][TCP dest=443]. Even if all other fields are the same, the encapsulated TCP header has different destination port numbers. But, the OS can’t guarantee that every driver in the stack is aware of every possible type of encapsulation. And we wouldn’t want to, since that would be prohibitive complexity. So we don’t generally enforce tunneling.
But if your driver is specifically aware of tunneling, then you need to bucketize NBs based on the encapsulated packet. For example, if you’re sending GRE-encapsulated traffic, and you are asking for a GRE-aware RSS hash, then you need to bucket the NBs by the encapsulated IP+TCP stream, not just the outer IP header.
Interlocutor: Ok, ok, that’s a lot to remember. Can you summarize specific requirements?
Jeffrey: Sure.
For miniports: always indicate receives with 1 NB per NBL. On TX, you can assume that every NB in a packet has same Ethernet header and RSS hash (if IP traffic).
For protocols: generally you should send 1 NB per NBL. If you originate IP traffic, you can in some cases put more than 1 NB, but it’s complicated. Assume receives of 1 NB per NBL.
For LWFs: do not regroup NBs in an NBL. If you need to drop 1 NB, drop the entire NBL. If you are de-encapsulating TX traffic, you may need to allocate new NBLs, since you’re stripping off the header that previously grouped the NBs together.
See also the MSDN summaries:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff570756(v=vs.85).aspx
http://msdn.microsoft.com/en-us/library/windows/hardware/ff554851(v=vs.85).aspx
1 Like
A huge ‘thank you’, Jeffrey Tippet!
On Saturday, November 29, 2014 1:35 AM, Jeffrey Tippet wrote:
Interlocutor: Are all _NET_BUFFERs pointing to the same Destination MAC address, or are they mixed?
Jeffrey: All NET_BUFFERs in an NET_BUFFER_LIST must belong to the same “stream”.
Interlocutor: Ok, thanks a lot Jeffrey, you’ve just substituted the original question with a new question: What the heck is a “stream”?
Jeffrey: A stream is a sequence of packets that, as far as an NDIS driver can know, belong to the same connection and application.
Interlocutor: What? Why so vague? Can you give me specific examples & requirements?
Jeffrey: Sure. First, when you get an NBL, you can always assume that NB chain contains packets with the same layer-2 connection. For Ethernet, that means the Ethernet source & destination address, VLAN ID, EtherType (including SNAP/LLC header if present).
Secondly, when you give somebody else an NBL, the Golden Rule implies that these assumptions become requirements. That is, you are required to not create an NBL chain that mixes Ethernet addresses, VLAN IDs, or EtherTypes. The safest course of action is to always put 1 NB in to each NBL, since then you’re never accidentally mixing streams.
Interlocutor: Hmm, ok. So an NBL chain always has the same Ethernet address. That’s simple enough. Why didn’t you just say that in the first place?
Jeffrey: Depending on the EtherType, you can derive even more specific requirements. Consider EtherType = 0x0800 (IPv4). We furthermore require that IPv4 traffic is grouped by IP source & destination address. E.g., an NBL cannot contain two NBs with different IPv4 source addresses. The same is true of EtherType 0x86dd (IPv6).
Interlocutor: Oh, I see, sometimes we reach into the layer-3 headers. Do we go deeper into the packet than that?
Jeffrey: Yup. Now consider IPv4 or IPv6, with NextProtocol = 0x06 (TCP). We furthermore require that TCPv4 traffic is grouped by TCP source & destination ports. E.g., an NBL cannot contain two NBs with different TCPv4 source ports. Similar is true of NextProtocol = 0x11 (UDP).
Interlocutor: Alright, I guess TCP and UDP are special, so it’s okay to special-case them. But there’s not anything else, is there?
Jeffrey: Well… now consider encapsulated traffic (pick your technology: either riding over IP directly as in GRE or IPsec; or riding over UDP as in VXLAN). We’d like to peek into the encapsulated payload, to recursively apply our bucketization rules. So for example, we wouldn’t want to group together these two NBs: [Ethernet][IP][Gre][IP][TCP dest=80] and [Ethernet][IP][Gre][IP][TCP dest=443]. Even if all other fields are the same, the encapsulated TCP header has different destination port numbers. But, the OS can’t guarantee that every driver in the stack is aware of every possible type of encapsulation. And we wouldn’t want to, since that would be prohibitive complexity. So we don’t generally enforce tunneling.
But if your driver is specifically aware of tunneling, then you need to bucketize NBs based on the encapsulated packet. For example, if you’re sending GRE-encapsulated traffic, and you are asking for a GRE-aware RSS hash, then you need to bucket the NBs by the encapsulated IP+TCP stream, not just the outer IP header.
Interlocutor: Ok, ok, that’s a lot to remember. Can you summarize specific requirements?
Jeffrey: Sure.
For miniports: always indicate receives with 1 NB per NBL. On TX, you can assume that every NB in a packet has same Ethernet header and RSS hash (if IP traffic).
For protocols: generally you should send 1 NB per NBL. If you originate IP traffic, you can in some cases put more than 1 NB, but it’s complicated. Assume receives of 1 NB per NBL.
For LWFs: do not regroup NBs in an NBL. If you need to drop 1 NB, drop the entire NBL. If you are de-encapsulating TX traffic, you may need to allocate new NBLs, since you’re stripping off the header that previously grouped the NBs together.
See also the MSDN summaries:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff570756(v=vs.85).aspx
http://msdn.microsoft.com/en-us/library/windows/hardware/ff554851(v=vs.85).aspx
—
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer