Driver Problems? Questions? Issues?
Put OSR's experience to work for you! Contact us for assistance with:
  • Creating the right design for your requirements
  • Reviewing your existing driver code
  • Analyzing driver reliability/performance issues
  • Custom training mixed with consulting and focused directly on your specific areas of interest/concern.
Check us out. OSR, the Windows driver experts.

Upcoming OSR Seminars:

Writing WDF Drivers I: Core Concepts, Nashua, NH 15-19 May, 2017
Writing WDF Drivers II: Advanced Implementation Tech., Nashua, NH 23-26 May, 2017
Kernel Debugging and Crash Analysis, Dulles, VA 26-30 June, 2017
Windows Internals & Software Driver Development, Nashua, NH 24-28 July, 2017


Go Back   OSR Online Lists > ntdev
Welcome, Guest
You must login to post to this list
  Message 1 of 13  
21 Feb 17 22:16
victor mehta
xxxxxx@hotmail.com
Join Date: 28 Jul 2015
Posts To This List: 17
How does the Windows Kernel know which process is a single tcp packet being received

How does the Windows kernel know which process or application does a single tcp packet being received belongs to ? Secondly then how does it copy the tcp data to the application user's space ? does it have anything to do with the kernel code that operates within the 2GB of virtual space which is allocated to it ? https://technet.microsoft.com/en-us/library/ms189334(v=sql.105).aspx
  Message 2 of 13  
22 Feb 17 01:48
Tim Roberts
xxxxxx@probo.com
Join Date: 28 Jan 2005
Posts To This List: 11398
How does the Windows Kernel know which process is a single tcp packet being received

On Feb 21, 2017, at 7:15 PM, xxxxx@hotmail.com wrote: > > How does the Windows kernel know which process or application does a single tcp packet being received belongs to ? TCP packets all have a port number (HTTP is port 80, for example). If an app is interested in receiving packets for port 80, it creates a socket and opens a connection to port 80. When a TCP packet arrives, the data gets queued up in a buffer for that port. Sooner or later, the application will do a "read" on the socket, and the kernel returns the next chunk from the buffer. > Secondly then how does it copy the tcp data to the application user's space ? The socket "read" call becomes a ReadFile call, which crosses over to kernel space. The user's buffer gets mapped into kernel space, so when the kernel network handler copies data in, it's the same pages that the application will be reading. > does it have anything to do with the kernel code that operates within the 2GB of virtual space which is allocated to it ? You can't be serious with this question, can you?. And the 2GB virtual space only applies to 32-bit systems. It's vastly larger on the ubiquitous 64-bit systems of today. — Tim Roberts, xxxxx@probo.com Providenza & Boekelheide, Inc.
  Message 3 of 13  
23 Feb 17 00:26
victor mehta
xxxxxx@hotmail.com
Join Date: 28 Jul 2015
Posts To This List: 17
How does the Windows Kernel know which process is a single tcp packet being received

@Tim thank you for the response and for the correction on the fact that article speaks of 32 bit systems. OK then just for the sake of the argument and in keeping with 64 bit systems where the virtual space is much larger. Would there be a kernel component that resides in the virtual 8-terabyte address space that an application on a 64 bit resides ? My question as above if there is such a kernel component within this virtual address space does it play any role in communicating IO data such as TCP to the user application residing within the same virtual address space ? Such as the mapping you talked about above ? if no then what is the purpose of the kernel component which resides in the same virtual address space as the application ? Thanks again
  Message 4 of 13  
23 Feb 17 13:03
Tim Roberts
xxxxxx@probo.com
Join Date: 28 Jan 2005
Posts To This List: 11398
How does the Windows Kernel know which process is a single tcp packet being received

xxxxx@hotmail.com wrote: > thank you for the response and for the correction on the fact that article speaks of 32 bit systems. OK then just for the sake of the argument and in keeping with 64 bit systems where the virtual space is much larger. Would there be a kernel component that resides in the virtual 8-terabyte address space that an application on a 64 bit resides ? My question as above if there is such a kernel component within this virtual address space does it play any role in communicating IO data such as TCP to the user application residing within the same virtual address space ? Such as the mapping you talked about above ? if no then what is the purpose of the kernel component which resides in the same virtual address space as the application ? I don't think your mental model of the system is quite correct yet. Let's say we freeze the whole system at a given point in time. In the lower parts of the virtual address space, we have the currently running application, its DLLs, and the user-mode operating system DLLS like gdi32.dll and user32.dll. In the upper parts of the virtual address space, we have the operating system kernel, add-in kernel modules, kernel drivers, page tables, and so on. EVERYTHING you need to run the entire computer is in that address space. Let's say the application calls "recv" on a socket. The CPU starts executing in the user-mode Windows sockets DLL, ws2_32.dll. That function will eventually call ReadFile, in the user-mode DLL kernel32.dll. ReadFile then makes a syscall which shifts the CPU into kernel mode, and ends up calling into ZwReadFile in ntoskrnl.exe. Now, that's just a regular DLL, except that it lives in the kernel part of the process space. It eventually will create an I/O request and call IoCallDriver, which eventually calls into the network driver stack. The network drivers all live in that kernel part of the address space, so these are just function calls that happen to be done while the CPU is in ring 0 (kernel mode). While that's going on, the rest of the address space (including the user-mode portion) is all still there. These kernel drivers can copy to user-mode addresses just by writing to a local pointer. Eventually, the I/O request is completed, the functions return, we mode the CPU back to ring 3 (user mode), and the application continues on. When the system wants to start running a different application, it just switches the page tables for the lower half of the virtual space. The upper half (with the kernel) remains the same, always available. So, it's not like the operating system lives in some separate space. The operating system is in memory at all times. -- Tim Roberts, xxxxx@probo.com Providenza & Boekelheide, Inc.
  Message 5 of 13  
02 Mar 17 18:38
victor mehta
xxxxxx@hotmail.com
Join Date: 28 Jul 2015
Posts To This List: 17
How does the Windows Kernel know which process is a single tcp packet being received

@Thanks again Tim. I have a further question to ask how can I find out much more about the "kernel network handler" you described above ? Ideally I want to understand exactly in details as much as possible how TCP data is received and sent by an application but exactly what is happening on the kernel side of things. You given some excellent insights but I need more information or details on how the OS (kernel code) handles TCP data all the way up.down the tcp stack and the way the data gets mapped (via "kernel network handler") to and from a user mode application but from the OS (kernel) perspective. I'd like learn what is happening behind the scenes so to speak. Would there be any good online resources or chapters of any book or someone here on OSROnline who could explain more about this area. Windows Internals is a good book would it have sufficient details that I am looking for or going much further and deeper by writing a simple socket program and tracing the call under a debugger like Windbg or KD. Any comments would be appreciated. Thank you
  Message 6 of 13  
02 Mar 17 19:02
Don Burn
xxxxxx@windrvr.com
Join Date: 23 Feb 2011
Posts To This List: 1298
How does the Windows Kernel know which process is a single tcp packet being received

Take a look at https://msdn.microsoft.com/en-us/library/windows/desktop/bb451830(v=vs.85).a spx Don Burn Windows Driver Consulting Website: http://www.windrvr.com -----Original Message----- From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com Sent: Thursday, March 02, 2017 6:39 PM To: Windows System Software Devs Interest List <xxxxx@lists.osr.com> Subject: RE:[ntdev] How does the Windows Kernel know which process is a single tcp packet being received @Thanks again Tim. I have a further question to ask how can I find out much more about the "kernel network handler" you described above ? Ideally I want to understand exactly in details as much as possible how TCP data is received and sent by an application but exactly what is happening on the kernel side of things. You given some excellent insights but I need more information or details on how the OS (kernel code) handles TCP data all the way up.down the tcp stack and the way the data gets mapped (via "kernel network handler") to and from a user mode application but from the OS (kernel) perspective. I'd like learn what is happening behind the scenes so to speak. Would there be any good online resources or chapters of any book or someone here on OSROnline who could explain more about this area. Windows Internals is a good book would it have sufficient details that I am looking for or going much further and deeper by writing a simple socket program and tracing the call under a debugger like Windbg or KD. Any comments would be appreciated. Thank you --- NTDEV is sponsored by OSR Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at <http://www.osr.com/seminars> To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer>
  Message 7 of 13  
03 Mar 17 15:14
Tim Roberts
xxxxxx@probo.com
Join Date: 28 Jan 2005
Posts To This List: 11398
How does the Windows Kernel know which process is a single tcp packet being received

xxxxx@hotmail.com wrote: > @Thanks again Tim. I have a further question to ask how can I find out much more about the "kernel network handler" you described above ? Ideally I want to understand exactly in details as much as possible how TCP data is received and sent by an application but exactly what is happening on the kernel side of things. You given some excellent insights but I need more information or details on how the OS (kernel code) handles TCP data all the way up.down the tcp stack and the way the data gets mapped (via "kernel network handler") to and from a user mode application but from the OS (kernel) perspective. I'd like learn what is happening behind the scenes so to speak. You have focused on the mapping, but that is such a small part of it. Remember that every I/O request for every device in the system (including disk files) ends up mapping (or copying) data from user mode to kernel mode. That's just one of the services provided by the I/O Manager. By the time a request gets into the network stack, all of that has already been done. There are network driver samples in the WDK sample set that might show you a bit about how drivers in the network stack are constructed. The TCP/IP layers are not all that mysterious. You can pretty much guess how they have to work, with each layer adding its own packet headers and passing it down, until finally the packet gets to some network card driver, where the packets gets queued up to get sent out to a chip that will convert it to the wire. -- Tim Roberts, xxxxx@probo.com Providenza & Boekelheide, Inc.
  Message 8 of 13  
03 Mar 17 22:16
victor mehta
xxxxxx@hotmail.com
Join Date: 28 Jul 2015
Posts To This List: 17
How does the Windows Kernel know which process is a single tcp packet being received

@Thank you Don. Don I'll get back to on those return control codes as I call them. Tim thanks again. Looking at the WDK source samples on the network stack is a good idea which I'll be pursuing. So basically IO Manager is made up of several if not many services that handle the data mapping that I was asking about and the same for how data flows up.down the network (tcp) stack basically yes ?
  Message 9 of 13  
04 Mar 17 10:10
D. T.
xxxxxx@gmail.com
Join Date: 01 Apr 2017
Posts To This List: 172
How does the Windows Kernel know which process is a single tcp packet being received

Suppose that process A sends an HTTP HEAD request to someserver.org and that process B sends the exact same request to the exact same server. Now TCP/IP connections have a source and destination address, and a source and destination port. As process A and B have the same source address, the same destination address and the same destination port, the only thing left to distinguish between them is the source port.
  Message 10 of 13  
04 Mar 17 21:50
Tim Roberts
xxxxxx@probo.com
Join Date: 28 Jan 2005
Posts To This List: 11398
How does the Windows Kernel know which process is a single tcp packet being received

On Mar 3, 2017, at 7:16 PM, xxxxx@hotmail.com wrote: > > So basically IO Manager is made up of several if not many services that handle the data mapping that I was asking about and the same for how data flows up.down the network (tcp) stack basically yes ? Yes. Virtually all I/O operations in Windows happen by passing I/O Request Packets (IRPs) from driver to driver. The IRP includes a command code and a pointer to buffers, among other things. When a user-mode app makes an I/O request. the I/O Manager creates an IRP and hands it to the top driver for that device. The IRP is passed down, possibly being modified along the way, until some driver completes it. — Tim Roberts, xxxxx@probo.com Providenza & Boekelheide, Inc.
  Message 11 of 13  
07 Mar 17 14:17
victor mehta
xxxxxx@hotmail.com
Join Date: 28 Jul 2015
Posts To This List: 17
How does the Windows Kernel know which process is a single tcp packet being received

@Thank you D.T. and thank you Tim and of course thank you Don Burns. You guys have been very helpful. I'll dig a little further into the detail of the information each of you have provided above and if I have any questions come back and post again. Victor
  Message 12 of 13  
14 Mar 17 18:15
victor mehta
xxxxxx@hotmail.com
Join Date: 28 Jul 2015
Posts To This List: 17
How does the Windows Kernel know which process is a single tcp packet being received

Just a note to anyone else interested in this topic but I believe that Don Burns might be on the right track in that WFP via the ALE layers. I've discovered from someone that using the WFP and ALE layers might be the way to go.
  Message 13 of 13  
15 Mar 17 09:04
D. T.
xxxxxx@gmail.com
Join Date: 01 Apr 2017
Posts To This List: 172
How does the Windows Kernel know which process is a single tcp packet being received

>Just a note to anyone else interested in this topic but I believe that Don Burns might be on the right track in that WFP via the ALE layers. I've discovered from someone that using the WFP and ALE layers might be the way to go. TCPIP.SYS is one the few system components that register a PCREATE_PROCESS_NOTIFY_ROUTINE_EX or PCREATE_PROCESS_NOTIFY_ROUTINE notify routine. So TCPIP.SYS monitors processes creation and termination.
Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You must login to OSR Online AND be a member of the ntdev list to be able to post.

All times are GMT -5. The time now is 20:14.


Copyright ©2015, OSR Open Systems Resources, Inc.
Based on vBulletin Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Modified under license